Static testing will identify defects earlier in the development lifecycle but also reduce the number of unwelcome change requests.
Why Static Testing? We hear a daily push within the software testing world of ‘agility’, ‘deliver at pace’, ‘speed of delivery’, ‘teams must be more agile’, and of course we are all for that. But with this drive we must not forget those disciplines that got us here in the first place. Some of the key techniques should not be lost in the ‘frenzy’ for agility, speed to market. One of these practices we continually recommend is the one area we feel can really add value to the testing process and project delivery. That is ‘Static Testing’.
“key techniques and disciplines should not be lost in the ‘frenzy’ for agility”
Possibly not the most exciting subject within the software testing community but when we know these techniques improve the output, and reduces the cost of defect resolution then why not spend more time doing it? Static test analysis is the review or inspection of key artefacts. These are the very artefacts that define the test planning and dynamic test delivery processes. These techniques can be executed early as not a single line of code is required. Static testing also applies to static code reviews with an aim of improving the overall quality of the solution and again identifying defects as early as possible within the software quality assurance process.
There are many articles attempting to calculate the true cost of a defect and of course there are varying opinions on this subject. However, we can say categorically that the majority of defects are introduced during the earlier stages of a project. We will also try to demonstrate why static testing will have a positive impact or any project.
The graph below demonstrates a typical defect profile for a project delivering using a Waterfall or V-model methodology. It shows defect introduction (when in the project defects are introduced), defect identification (typically when the defect is uncovered), and the cost of defect resolution (an exponential view demonstrating that the cost of resolution increases significantly the later it is discovered).
The graph demonstrates how we must ‘shift-left’ the defect found curve (the identification). By shifting the identification towards the earlier stages of project delivery you can see that this will automatically reduce the cost of defect resolution. The next graph demonstrates that if we introduce static testing techniques alongside a more agile delivery then the defect profile would look more like this.
Here we see a notable shift of the curve peak, with identification of defects primarily discovered during our project definition phase. The introduction of the Agile methodology also smooths the identification curve. This significantly reduces our defect resolution cost, along with smoothing the resource profiles throughout the project lifecycle.
So we’ve concluded that static testing and analysis is a huge benefit. However, as with all of these processes, in order to maximise their value then they must be performed consistently and correctly. But how?
Documentation Reviews & Walkthroughs
Reviews and walkthroughs will identify and eliminate defects early within the lifecycle. If completed correctly they are an excellent tool within the testing toolbox. Depending on your methodology and approach they can apply to a range of artefacts within a project. For example, requirements, designs, functional specifications, and process documents can all benefit from reviews, walkthroughs and inspections.
Code Reviews
Peer reviews of code and configuration will also add value to the process. Ensure developers take responsibility for the quality of their code, but also use the software testing process to identify training needs, or process improvement requirements. Continue to adapt, monitor, and improve and ensure your teams have the opportunity to do the same.
Static Analysis Tools
There are a huge range of tools out there, they are inexpensive, and we know we must strive towards automation as much as possible. Use these tools, automate them through trigger points within the development process and use the data and information they provide to ensure code quality can be measured very early within the lifecycle. For a comprehensive list of tools then see this wikipedia article. We recommend an excellent tool by Kiuwan.
Integration
Ensure static testing is as much part of the process as the dynamic testing phases. Provide formal feedback in templates, use your defect management process to track prioritise, allocate, and track static testing defects to resolution, but also integrate with your methodology. We integrate static code review tools into our CI/CD pipelines. We integrate workflow to ensure specific gates cannot be passed through without static testing completing. We integrate the static test results within our release process, and we track static testing defects to resolution via our defect management process.
“We Integrate static code review tools into our CI/CD pipelines”
These are proven disciplines and techniques. We have demonstrated here that static testing can add value if done well. We all know that finding defects early significantly reduces the cost of development and defect resolution, but we also see the strive for agility and pace of delivery. The benefits of automation can sometimes lead to us underestimating the benefits we know these static testing techniques will provide.
We must strive to improve our software testing processes, increase the pace at which we deliver quality, and ensure alignment with the methodologies under which we all operate but in all of this don’t forget to be just a little ‘static’.
