When can software test automation be a mistake?

In a business environment where speed needs to be maximised and costs minimised it is all too tempting to think that increased automated testing can be a solution to these demands. We have all encountered situations where due to these pressures, the call for more automation and less manual testing has been made. But this logic is flawed. Manual testing is the cornerstone of testing on which automated testing stands.It’s true of course that automation can produce benefits with repetitive tasks and they perform consistently with no concerns about human error introducing its own problems into the testing process. But does that mean that every test should be automated? That there is no place for manual testing?

Automated Testing

Automated testing utilises a separate piece of software (or “framework”) to control a series of predefined steps on the software under test to check that the outcome of those steps is as it should be.For example we may have a script that provides a series of username and password combinations to an application. Access should not be granted should either of the username or password be incorrect. An automated script would interact with the software without human intervention to check that that is indeed the outcome.This script can be run countless times as development progresses to verify that no change to the code introduces incorrect functionality into the system. Automation though has its limits. Each check can only give a binary outcome. Is the outcome as it should be? Yes or no.Whilst this inability to react intelligently and provide context to the unexpected is a limitation, automation is still an essential part of testing. Any test where a manual script exists with step by step actions that have to be run for every release is ripe for automation and can considerably reduce the time and effort involved.

Manual Testing

A strict definition of manual testing might say “The process whereby a software tester manually interacts with a piece of software to determine if it is operating correctly.” So as in our example above for automated testing, the software tester would manually enter combinations of incorrect usernames and passwords to check that the software does not grant access to the system for those cases.It does not take an expert to see that for this scenario, automated testing has clear advantages over the manual process.What about the things that the software shouldn’t do? It is easy to define an automated process when you know what the planned outcome should be. But how do you define an automated process for unplanned outcomes? Where the outcome of the action is completely unexpected. For the scenario where the actions of the user are not planned for? The answer is you can’t. This is where manual testing comes in.Automated testing can’t make a judgement on the look and feel of a piece of software, if the interface is intuitive to use. Automated testing can’t investigate the cause of an error. Automated testing can’t judge the importance of an error. A software tester brings their years of experience, training and intuition to imaginatively challenge the functionality of the software in ways that only a human can.In conclusion there will always be a need for manual human testing of software. Without doubt there are many mandrolic processes that can be automated to save time and money, but ultimately automation is just another tool in the software testers toolkit