Quality is the level of excellence something achieves. To apply that concept to a software product in a way that can be understood by everyone involved in the product’s development is difficult. In this blog post, I’ll discuss how I created a definition of quality for my product that helped us to align on what excellent was, to understand our current product, and to direct our quality-improvement efforts effectively.
Quality is completely dependent on context; what might be excellent to one company or user-base will be unsatisfactory to another. Therefore, it’s important that you use a definition tailored to your product, your business and your users, if you’re trying to understand the quality of your product.
My team created our definition of quality by first embracing the idea that “quality is value to someone who matters” (originally by Jerry Weinberg, extended by Michael Boulton). We mapped out the set of people who matter to us and what they care about, and from there, identified eleven quality aspects that appeared frequently in relation to important stakeholders. These were:
Notice that we then categorised these into three areas: ‘capabilities’ in pink, ‘user-visible’ aspects in green, and ‘internally-facing’ aspects in blue. This categorisation has proven useful in helping stakeholders to understand the different perspectives this definition takes into account, to avoid them being intimidated or alienated by its size. It has also enabled us to enhance our communication about quality by summarising our performance in each category, as well as in each individual aspect; this is particularly useful when communicating with stakeholders who are interested in only one or two of these categories.
After deciding on the quality aspects to include in our definition, we needed to communicate what we meant by them clearly. To do this, we created a table which explains:
- What the aspect means in the context of our product.
- Why it’s important to us and what the impact of succeeding or failing is.
- Where we’d like to be regarding this aspect of quality.
Here are some examples from this table. Note that I’ve removed the detail that made them specific to our product, although they may still reflect elements of our product, industry and business model.
Security and Privacy
How well the product protects data and resources of value to ourselves and our users.
Keeping our product secure and maintaining privacy is the right thing to do, for both our users and for our company. If we fail, as well as the implications of any lost assets, there is a risk of fines and legal consequences, and we will suffer a loss of reputation.
Our product must be compliant with our internal security policy.
Usability
The overall experience that a user has with the service, including: how discoverable and intuitive it is, how much they enjoy using it, and how seamlessly it fits into their workflows.
People will like using the product if it has good usability. This will reduce user churn and also increase sales, as usability will create a good impression during demos and trials. It will also help to build a relationship between the user and the product; they will work with us, giving us feedback when they don’t like something and leeway to resolve these issues.
Users who need our service do not stop using our product because of poor usability.
Developability and Testability
The ease with which engineers can develop and test the product. How complex, maintainable and extensible the code is.
This improves velocity, reduces ramp-up time and increases our confidence in the overall quality of our deliveries. It also makes our engineers happy!
Well-architected code with sufficient test infrastructure to make development and testing fun.
Once you have filled in this table for all the quality aspects you selected, you have a complete definition of quality. You can now use it everywhere! Initially, it helps to educate and align people on what you’re trying to achieve with your product, and later, it can be used as a checklist throughout every stage of product development. We use ours to shape conversations about quality in whiteboards, in specification and design documents, in our test strategies and our quality evaluations. To have our whole team working from the same definition of quality makes our product development immeasurably more efficient and effective.
We also use our quality definition as the framework for our ongoing quality reporting; for each aspect, we’ve selected metrics that indicate how we’re doing with respect to that area of quality and we regularly report and reflect on these. I’ll expand on this in my next post.
Have more questions? Enjoyed this post? Let me know!
Error: ‘Quality’ is not defined
In April 2021, I gave a Masterclass at the Ministry of Testing, called “Error: ‘Quality’ is not defined”. In this Masterclass, I discussed how to:
- Start a productive conversation about quality with your team.
- Create a definition of quality tailored to your product.
- Choose and implement metrics to measure the quality of your deliveries.
- Keep the conversation about quality at the top of the agenda.
The Masterclass can be watched back by Ministry of Testing Pro users here.
After the Masterclass, I was contacted by a few people asking for further details of how I went about my own journey to define quality, and so here I am: this is the first of two blog posts answering some of these questions. The other blog post is available here.
Georgia Bloyce 
Thanks for the really informative post. Looking forward to the next part on the metrics used for Quality definitions
LikeLike