Some Thoughts on Bugs

The proper application of test-driven development yields in fewer bugs. This experience is share by quite a few people in the agile community. Still, there are bugs. This is not because your engineers have relaxed their quality standards. I think it is because expectations are higher. Customers can be more picky.

A few years ago people didn't dare to demonstrate software if it was only a few days old. After all it didn't go through a proper testing cycle, right? And to a degree - in particular in more traditional organizations - they were right. Quality was poor in some cases. Crashes could enrage even a mild mannered reviewer. But even if it wasn't a crash, bugs tended to be quite substantial in their impact. So showing the software created more uncertainty rather than excitement.

In a newer environment it is possible to show "brand new" software. Depending on your environment "new" means it's just a few hours or days old. Or just take the latest successful build.

When I look at some of the bugs raised today I see that they fall into the category "aesthetics". In the past people wouldn't have bothered raising them because compared to others they would have been negligable in terms of severity.

Does that mean that we should be happy about this? I don't think so. A bug is a bug is a bug. We shouldn't be proud of any bug. Each bug is an indication of a piece of unfinished work. If some bugs are a question of taste then it might be we will never be able to reach zero defects. In that case we should at least see this as an ideal towards which we work. Toyota - despite it's recent hickups (recalls of vehicles, even Lexus) - is seen as the well-recognized quality leader in the automative industry. Does this mean their products are defect free? No, not even close. They are subject to Murphy as well. But they have their unique way of dealing with them. They fix them one at a time. And not only that. They also try to identify ways of how to avoid the same defect from reappearing.

So where from here?

We should take Toyota's behavior as an example. We should still strive for reducing defects wherever we can. And we should do so by avoiding them in the first place. Communicating and collaborating closely with the customer is one of the first and most important steps. Writing your tests first and use them to drive your implementation helps, too. Keep your code simple, readable and maintainable.

And with bugs: Each time one slips through, follow these guidelines:

1. Try to understand the root cause for the bug. Prove your understanding by writing an automated test that fails because of the bug.
2. Make the test pass.
3. Refactor mercilessly.
4. Identify similar scenarios in your code base. Check whether they contain the same bug. If they do, chances are that you have identified another opportunity for refactoring.

Here is another aspect of bugs. Compare a bug to an ordinary story in your backlog. What is the major difference? Generally speaking, a story tends to be easier to estimate in particular if it is similar to a past one. A bug can be anything from "user misunderstanding" to "several weeks" of work. The point is that the amount of uncertainty tends to be higher for bugs.

So if you compile a single, consolidated list of all your work items, you will end up with a mixture of stories and bugs. The larger the proportion of bugs the higher the uncertainty. The higher the uncertainty the bigger the impact if one or more of the bugs require more than the "average" fix time.

So if you are interested to become more reliable (read predictable) then the suggestion is to keep the bug level as low as possible.

Does this mean you can't have crash parties any more? Under all circumstances, if the crash party helps you to assess the quality of your product and helps you to find more bugs, and you if you see value in this, then you definitely should have them. Not doing them because it keeps the bug numbers low, is cheating. You would be lying to yourself.

Also, if you don't fix bugs but keep piling them up, while you at the same time your are "completing" story after story, you are actually reporting a velocity (in stories per week) that is too high. Expectations for the next release cycle will be higher, and it will be even harder to negotiate time for fixing bugs or doing other items.

And more, if your approach results in a similar number of bugs your stakeholders may start asking the question whether your approach is really better than the traditional one.

I'm sure you don't want to let it come to that. So regardless how you look at it: Not fixing bugs and finding ways for avoiding them in the future doesn't help at all. Instead it can be the start of a vicious cycle. And by keeping the number of bugs low, the question of how to deal with bugs becomes a no-brainer!