Tuesday, March 31, 2009

Remarkable error message

Youtube search crashed for me two minutes ago.

500 Internal Server Error

Sorry, something went wrong.

A team of highly trained monkeys has been dispatched to deal with this situation.
Also, please include the following information in your error report:


Who said software has to be boring?

Monday, March 30, 2009

Friday, March 27, 2009

Shu Ha Ri

Shu Ha Ri is a Japanese martial arts concept that describes the three levels of learning.

I have always liked the concept since it provides a simple model that can help with communicating with people. The idea is to be aware of the level of your audience and match your message to their level. For example if you are an expert trying to explain a beginner how to do agile software development quoting principles only frustrates him; he needs practical advice to practical problems - not spiritual guidance.

Shu: Imitation, learning rules and individual techniques, tradition
Ha: Understanding, learning exceptions to rules, adapting techniques, breaking from tradition
Ri: Mastery, transcending the rules; flow, intuitive use of techniques

The Shu level is all about following the master and learning the rules. You don't really understand the big picture yet so you follow the book as best as you can.

At Ha level you have gained deep understanding. You understand why certain techniques work and can choose the best one for the situation. You also understand the limitations of techniques and when their use is not appropriate and when not.

By the time you reach Ri level you apply techniques naturally without thinking. You have transcended the rules.

Last year in Nääsvillen Oliopäivät Alistair Cockburn began his keynote by describing Shu Ha Ri. After his talk a member of the audience asked a longish question about a problem in applying agile in his organization. Having explained Shu Ha Ri just moments before, Alistair begun his reply:

You expect a Shu level answer to a Ri level question.

Further reading:
Shu Ha Ri by Alistair Cockburn
Three Levels of Audience in Ward's Wiki
What is Shu Ha Ri? an excellent blog entry by Kevin E. Schlabach

Saturday, March 21, 2009

Ken Schwaber on Google Tech Talk

Here is an excellent video where Ken Schwaber (co-creator of Scrum) explains what Scrum is and where it came from. He also explains how everyone can create their own design dead software and how to avoid it.

I have summarized and paraphrased some of the key points.

Scrum is not a methodology - you are on your own

Scrum isn't a methodology and as such does not have answers on how to do things. It frees us from the belief that someone else can tell us what to do in every circumstance and that it'll work. Scrum's assumption is that you are intelligent and that you will use that intelligence and your experience to come up with the best solution for whatever circumstance you are in right then.

Most Scrum implementations fail because of denial

Scrum gets all of the news visible, whether it's good or bad. It assumes that intelligent people will want the news so they can do what is best for the entire organization. Because of this only about 30-35% of organizations manage to implement Scrum successfully. Most organizations don't want to be faced with something they don't want to see.

The Core problem

Very common problem in all organizations is Core or infrastructure software that all the company's software depends on. Usually such software is fragile, does not have a test harness and there are only few developers in the company who know it and are willing to work on it. The Core software then effectively constrains all development in the company. If you don't have enough money to rebuild your Core and competition is breathing down your neck, you should shift into another market or sell your company.

You don't have to buy your Core, you can make your own. Squeezing a bit more "must have" features into your releases works in short term, but leads into cutting quality. Soon your team's capability to deliver starts dropping because they are working on a worse codebase. That leads into more squeezing, and after five years you have your own design dead product that will kill your business.

There are two main problems here. First, when developers are told to do more they cut quality without telling a soul. Second, product management beliefs in magic that to do more all they have to do is tell developers.

The solution is incremental and iterative development, and managing releases by scope rather than by time. In Scrum also every team has a Scrum Master (the prick), whose job is to make sure the team does not cut quality.

Friday, March 20, 2009

The wrong question

Ask not what agile can do to my organization?

Rather ask how can I change my organization to becoming more agile?

On a similar vein, an excellent blog from Ron Jeffries: Context my Foot.

Levels of consulting

Here is a quick categorization of consulting work that I came up with some friends some time ago.

Level 1: Introduction
Basic concepts, sales pitches, two-day training course, lectures and exercises

Level 2: Consulting
Discussing or planning how a customer should use the material in his/her context

Level 3: Coaching
Implementing changes with the daily or weekly help of a coach, may include an intensive training period

Level 4: Follow-up
Coach periodically returns to customer after a couple of months to see how things are going and adjust as necessary

What has this to do with agile? When "going agile" many companies only buy level 1, then go and run their heads to the wall. I have heard both Jeff Sutherland and Jim Coplien state that "70% of Scrum implementations fail". Wonder why.

By the way, Certified ScrumMaster training is level 1.

Thursday, March 19, 2009

Turku Agile Day 2009

Just came back from the Turku Agile Day 2009. Good to see the agile movement on the rise in Turku. Thanks for the organizers, keep up the good work, guys! Next time I am staying for the party.

A few tidbits stuck in my mind:

  • "The human execution of a process cannot be copied." said Pekka Abrahamsson when he was discussing the difficulty of duplicating successful practices in projects.

  • Agilists often add the fourth dimension of quality to the holy trinity of software projects: features, time, and resources. Ola Ellnestam added a fifth, forming the iron pentagon of software development: features, time, resources, quality, customer satisfaction.

  • Ola also suggested viewing software development as a system from which customers pull features out. Contrast this with the traditional way of pushing features into the system as requirements, which then pushes them out to the user. Pull instead of push is the starting point for Lean and all kinds of good things come from it, but I have not previously thought of pull as a way of making sure you deliver only what is needed. How silly is that?

  • Something I enjoyed was Petri Taavila's frank, down to earth presentation about Nokia Siemens Network's agile transformation with wrinkles and everything. Wonderful to have an honest presentation about the good, the bad and the ugly of turning agile. Thanks, Petri!

Wednesday, March 11, 2009

The key to test automation

The key to test automation is called software engineering.

I have met many testing teams and Quality Assurance people searching for the silver bullet that would speed their team or department on its way to test automation and all it promises. Usually these people have no software development background, and while their teams may have some scripting knowledge they often lack proper software development skills. As such, they are heavily dependent on tools, preferably those that come with a GUI or that can be extensively configured. If push comes to shove they can maybe manage with a scripting language, but that's about it.

Good tools are essential to software development, and I daresay even more so in agile software development. What is the problem with depending on tools, then?

First of all, many commercial testing tools are expensive and complex. Since they cater the needs of a wide audience they have many features, and often come with elaborate XML configuration languages or custom scripting languages. Complexity steepens the learning curve and slows down troubleshooting. While troubleshooting is may not be much of a problem for functional testing, it is a major issue in performance testing where it can take more time than executing the actual tests.

Once you have chosen and bought an expensive tool you are pretty much stuck with it. In choosing a tool you have to anticipate all your future testing needs which increases the likelihood of you choosing the most complex tool with the most features. The vicious circle is complete.

Another issue is that all tools have their limitations. No enterprise testing solution is as flexible or powerful as a good programming library or a language and you cannot iteratively and incrementally develop and improve a commercial tool. You must choose one in a big bang and then stick with it. It is impossible to start with something simple that you can evolve with time as your needs become greater.

There is an entire industry building testing solutions designed around the idea that programming skills are not needed for test automation. Hogwash! Automation of any kind implies a certain level of design skill that cannot be substituted with a tool! The meaning of "test automation" is "automated test case execution and reporting", not "automated test case design".

If you have recently started doing agile software development and (still) have a dedicated testing team that is confused about test automation my advice is this: send your testers to a programming course, and introduce a software developer or two to the team. The developers can then devise any tools your team needs, and help your testers with programming.

Adding developers to a testing team also brings testers and developers closer which you should be doing in the first place if you are trying to become agile.

Explaining things without a whiteboard

An ex-colleague of mine Samuli sent me a link to this Dilbert. Thanks!

That's me!

Monday, March 9, 2009

Agile software development summarized in Agile Dinner

Last month I organized the monthly Agile Dinner in Helsinki with the theme "what have you done right in your previous software projects?". I specifically asked people to forget literature and just focus on what they have directly experienced. The outcome could be the table of contents of any agile software development book!

Form a cross-functional team, put them in one room where they are surrounded by information radiators and whiteboards and can work in peace. Provide them direct access to business people and end users and make sure they understand the goals of the project. Maintain the big picture and provide an environment that facilitates learning and try to involve experienced developers and maybe a coach. Use metrics to measure progress. The technical environment should allow for continuous integration and automated regression testing. Consider laying down basic architecture. Be honest, deliver frequently and think a lot.

Friday, March 6, 2009

Challenge everything

Challenge everything, accept nothing at face value.

Drive this philosophy into everyone in your software organization from marketing and sales to project management and individual team members and good things will follow.

But why? Isn't challenging people's decisions and choices destructive? Isn't it mistrusting your colleagues, and slowing down work? Instead of getting things done, you spend time discussing the same issues over and over again when everybody has their round of challenging? Doesn't it set up kind of an environment where people constantly have to defend their ideas, and provoke confrontation?

That exactly is the point! Sparring ideas, working them on the whiteboard and discussing them over coffee is cheap. Spending months implementing the wrong features or bad marketing strategy is expensive.

Challenging everything is really a safety net for your organization. It prevents brainfarts from destroying your business or technology. It is also a personal safety net since it allows you to test your ideas in a safe environment. It is better to have a trusted colleague torpedo your cool idea right from the start than fail publicly six months later, is it not? And imagine the kind of confidence you can proceed with your Next Big Thing after you have had your first five sparring sessions about it.

Building an environment and culture that allows ideas to be challenged may be difficult though. It requires certain maturity from the involved, they have to understand that it's people's ideas that argue, not the people. It helps to keep in mind that the word challenge is not a negative word, it is just a request for more information or a sanity check. Challenging is not a destructive or distrustful process but a process for creating mutual understanding and confidence. It is the starting point for a conversation, not a finishing one.

Avoiding rumpling other people's feathers is tricky, but the actual act of challenging could not be easier. Just keep on asking "why" until things make sense to you.

A real-life example

Here is a (pretty convoluted) example. In a certain project in the past I was re-designing a public interface of an accounting system that, among other things, processed monetary transactions. A function of that interface was to transfer money to a user's account. The function had four parameters: transaction ID, account ID, and two monetary amounts.

Two moneys? Why?

"Because for taxation reasons - or something like that - certain transactions cannot be deposited in a lump sum" said the architect of the accounting system. OK, fair enough, but sounds like he is not completely sure. I checked with the architect of the business system using the accounting system, who told me that the accounting team demanded the feature for taxation reasons, and that the reporting team also has a stake. So I call up the reporting people, who tell me that the feature is a pain in the ass but that it is very important for the customer and the accounting team requires it. They also mentioned that a fourth team who monitor the transactional integrity of all system has a stake. So I contact the monitor team, who are a bit confused about my question and in the end could not care less. At the same time a business person - after double-checking with the customer - confirms that the feature, in fact, is not needed.

Very interesting, I thought. Everyone takes the feature of splitting lump sums for granted. After all it is an feature supported by three different systems so it must be needed by someone! But no one knows by whom.

In the end I spent two months bothering fifteen people from four teams to remove a single parameter from a function. Of course functionality considered "a pain in the ass" could now be removed from three systems which would simplify the life of four teams.

Buy why was the rather expensive feature that no one needs implemented in the first place? Certainly there is fault in the software process, but would this had happened if any of the people involved had challenged the need for the feature?

Morale of the story: it may take more effort to challenge the rationale of doing something than just doing it. But it probably pays off in the long term.

Which brings us to...

Impact on software architecture

Simple software is cheaper, runs faster, is easier to use, document, test, deploy, operate, maintain and is generally more pleasant than complex software. Common sense dictates that software should be as simple as possible (but not simpler). But how to make simple software?

Using the latest programming paradigm? Buying the fanciest Enterprise Platform on the market? Grabbing the hottest open source framework where you don't have to write code anymore, just configure XML, use annotations, or let the built in artificial intelligence run the show?

Maybe not. As a function of features complexity increases exponentially, not linearly. Therefore the key to keeping things simple (stupid), is to minimize the number of features and moving parts in your software. Disciplined processes for capturing user needs and designing software can help, but the key thing is simply to challenge the necessity of every feature and nut and bolt before adding it to your system.

In short, challenging everything

  • Is a cheap way of improving ideas, and weeding out bad ones

  • Builds mutual understanding and trust, and kills assumptions (the mothers of all fuck-ups)

  • Is a safety net that prevents your organization from doing stupid things

  • Is a personal safety net that prevents you from doing stupid things

  • Can be challenging :)

  • Is the basis for good software architecture

The lesser primate committee thinking experiment

Start with a cage containing five apes.

In the cage, hang a banana on a string and put stairs under it. Before long, an ape will go to the stairs and start to climb towards the Banana, but as soon as he touches the stairs, spray all of the apes with cold water. After a while, another ape makes an attempt with the same result-all the apes are sprayed with cold water. Turn off the cold water. If, later, another ape tries to climb the stairs, the other apes will try to prevent it even though no water sprays them.

Now, remove one ape from the cage and replace it with a new one. The New ape sees the banana and wants to climb the stairs. To his horror, all of the other apes attack him. After another attempt and attack, he knows that if he tries to climb the stairs, he will be assaulted.

Next, remove another of the original five apes and replace it with a new one. The newcomer goes to the stairs and is attacked. The previous Newcomer takes part in the punishment with enthusiasm.

Again, replace a third original ape with a new one. The new one makes it to the stairs and is attacked as well. Two of the four apes that beat him have no idea why they were not permitted to climb the stairs, or why they are participating in the beating of the newest ape. After replacing the fourth and fifth original apes, all the apes which have been sprayed with cold water have been replaced. Nevertheless, no ape ever again approaches the stairs.

Why not?

"Because that's the way it's always been around here."

Sound familiar?

Wednesday, March 4, 2009

Fundamental attribution error

Fundamental attribution error is a phenomenon in social psychology that describes a tendency in us to explain the behavior of others in terms of their personal qualities rather than circumstance. We "attribute" a person's actions to his persona rather than the immediate situation or social context.

I think this is quite a natural tendency. It is a bit like the energy minimum principle, we are always looking for simple explanations to complex issues. Much easier labeling others lazy or stupid rather than spending energy investigating the circumstances they are operating from.

And by the way, this door swings both ways. Other people are likely to think we are incompetent or not caring rather than bothering to consider that perhaps we are overworked, or having the worst hangover of our lives.

So the next time you are aggravated by your compatriots give them the benefit of the doubt. You might be missing something from the big picture that makes their (in)actions perfectly justified.

Der Führer's take on nightly builds, Scrum and REAL Agile.


Almost feel camaraderie with the man. Hilarious!