Continuous Improvement and the Root Cause of Defects

Continuous Improvement - the benefits of root cause analysing defects to death!

When projects come under pressure to deliver to plan, budget or quality experience shows there is rarely an appetite for learning the lessons of failure.  Project teams drive to overcome the challenges of poor quality, inaccurate estimates, delayed delivery, and cost escalation. They work through the issues, resolve them, the project is deployed and then the knowledge and experience disappears into the sunset. We are inquisitive QA specialists and therefore we really want to learn from experience. We want to promote an ethos and culture of continuous improvement.  Why did we uncover a high volumes of defects?  How come our estimates were not accurate? Why were so many change requests raised? Most importantly what can we do to make it better next time?Also, we recognise that through a haste to classify business challenges/issues as IT challenges/issues there can be a tendency to miss the finer but important distinction between systems, behaviour, process and data issues. If we better understand the distinction, we will better understand the root cause of any given problem we are considering. If we better understand root cause, we are more likely to establish accountability and ownership for resolution, and develop a culture for continuous improvement.

"Continuous improvement alongside CI and CD, sounds like the holy grail to us"

Often projects will go from a 'green' status to 'red' during the QA phases whether that be during an agile or waterfall delivery.  We won't analyse the reasons behind this, but what we really want to focus on is how important it is for organisations and the testing community to consider the benefits of root cause analysis.We consider this as a critical activity throughout the testing process for every project and every operation.  RCA defects to death and benefit from an effective continuous improvement culture, alongside continuous integration and continuous deployment - sounds like the holy grail to us.

"Don't blame the technology"

In our experience defects are labelled generically as “system problems” and by doing so assume the issue has been introduced by the technology and can only be resolved the by an IT team.   We know that this will not always be the case. Agreeing the problem categories can be the first step in understanding how projects can improve and we can recommend the following as a starting point:

• Systems – refers specifically to technology – a system defect is when technology breaks, or is slow, does not function as expected or is incapable of delivering current or new benefits.
• Behaviour – refers to people, culture and perceptions, all of which impact on how we use technology.  This is a people or cultural issue around how a system is used, not a system issue itself.
• Process – refers to the processes, procedures and workflow which provide the framework to how a system is used. There can be a tendency to confuse procedure shortfalls with system shortfalls.
• Data – refers to the information we put into a system. Accurate data is key to successful system usage. A system doesn’t make data bad, people do.  “Garbage In Garbage Out” applies here.

Now we understand these key categories we can work to identify the root cause for each and more importantly strive to improve through actions assigned to accountable owners.  Here are some ideas on how:

Culture and Behaviours

Create a culture of rigour, of ensuring that you never work on assumption, that you dig deep into root cause so that there is an understanding of why things went wrong and then take steps to ensure they won’t go wrong again.It is not about blame, it is about continuous analysis and continuous improvement. Ensure it is never felt that the analysis is personal to an individual, it should be about the process however it may also identify training needs for team and/or individual so don't shy away from that.

Continuous Improvement of Processes

Ensure root cause analysis is a mandatory activity as part of your defect management process. Don't close a defect without it, make it a defect manager responsibility, discuss the analysis during your post implementation review.  The aim is to create an ongoing culture of immediate improvement. Don't just recognise the potential improvements once a project completes.More importantly action any root cause.  Key actions should be assigned to process owners and tracked to completion.  If necessary improve your 3rd party contracts and associated KPI's and SLA's. To do nothing with this information will ensure one thing, the symptoms will be experienced again.  Again, prevention before cure.

Report, Measure & Communicate

Far too often an RCA will be performed but then not made available for future projects. Standardise your measurements and reporting across projects and across the organisation.  A company wide view can really add value.  Improve quickly, regularly, fail fast and learn from failure faster. The only way to improve is to have the knowledge and experience from the past. Then manage those lessons to effective change and keep striving for continuous improvement.

Promote the QA Team

Use the data from the analysis to promote your QA teams and measure their value objectively.  This data can be hugely beneficial when used to promote the value of the QA and testing teams.  Calculate the cost of defect discovery. Calculate the benefit of discovery of the defect early in the process. Articulate the cost and impact if the defect had made it into a production environment.  This will promote the value of testing within your organisation, motivate your teams, and can be very useful when discussing your next pay rise!If you'd like to discuss this topic then feel free to get in touch, we would love to hear from you.