The key to test automation

The key to test automation is called software engineering.

I have met many testing teams and Quality Assurance people searching for the silver bullet that would speed their team or department on its way to test automation and all it promises. Usually these people have no software development background, and while their teams may have some scripting knowledge they often lack proper software development skills. As such, they are heavily dependent on tools, preferably those that come with a GUI or that can be extensively configured. If push comes to shove they can maybe manage with a scripting language, but that's about it.

Good tools are essential to software development, and I daresay even more so in agile software development. What is the problem with depending on tools, then?

First of all, many commercial testing tools are expensive and complex. Since they cater the needs of a wide audience they have many features, and often come with elaborate XML configuration languages or custom scripting languages. Complexity steepens the learning curve and slows down troubleshooting. While troubleshooting is may not be much of a problem for functional testing, it is a major issue in performance testing where it can take more time than executing the actual tests.

Once you have chosen and bought an expensive tool you are pretty much stuck with it. In choosing a tool you have to anticipate all your future testing needs which increases the likelihood of you choosing the most complex tool with the most features. The vicious circle is complete.

Another issue is that all tools have their limitations. No enterprise testing solution is as flexible or powerful as a good programming library or a language and you cannot iteratively and incrementally develop and improve a commercial tool. You must choose one in a big bang and then stick with it. It is impossible to start with something simple that you can evolve with time as your needs become greater.

There is an entire industry building testing solutions designed around the idea that programming skills are not needed for test automation. Hogwash! Automation of any kind implies a certain level of design skill that cannot be substituted with a tool! The meaning of "test automation" is "automated test case execution and reporting", not "automated test case design".

If you have recently started doing agile software development and (still) have a dedicated testing team that is confused about test automation my advice is this: send your testers to a programming course, and introduce a software developer or two to the team. The developers can then devise any tools your team needs, and help your testers with programming.

Adding developers to a testing team also brings testers and developers closer which you should be doing in the first place if you are trying to become agile.