Some Thoughts on Bugs

The proper application of test-driven development yields in fewer bugs. This experience is share by quite a few people in the agile community. Still, there are bugs. This is not because your engineers have relaxed their quality standards. I think it is because expectations are higher. Customers can be more picky.

A few years ago people didn't dare to demonstrate software if it was only a few days old. After all it didn't go through a proper testing cycle, right? And to a degree - in particular in more traditional organizations - they were right. Quality was poor in some cases. Crashes could enrage even a mild mannered reviewer. But even if it wasn't a crash, bugs tended to be quite substantial in their impact. So showing the software created more uncertainty rather than excitement.

In a newer environment it is possible to show "brand new" software. Depending on your environment "new" means it's just a few hours or days old. Or just take the latest successful build.

When I look at some of the bugs raised today I see that they fall into the category "aesthetics". In the past people wouldn't have bothered raising them because compared to others they would have been negligable in terms of severity.

Does that mean that we should be happy about this? I don't think so. A bug is a bug is a bug. We shouldn't be proud of any bug. Each bug is an indication of a piece of unfinished work. If some bugs are a question of taste then it might be we will never be able to reach zero defects. In that case we should at least see this as an ideal towards which we work. Toyota - despite it's recent hickups (recalls of vehicles, even Lexus) - is seen as the well-recognized quality leader in the automative industry. Does this mean their products are defect free? No, not even close. They are subject to Murphy as well. But they have their unique way of dealing with them. They fix them one at a time. And not only that. They also try to identify ways of how to avoid the same defect from reappearing.

So where from here?

We should take Toyota's behavior as an example. We should still strive for reducing defects wherever we can. And we should do so by avoiding them in the first place. Communicating and collaborating closely with the customer is one of the first and most important steps. Writing your tests first and use them to drive your implementation helps, too. Keep your code simple, readable and maintainable.

And with bugs: Each time one slips through, follow these guidelines:

1. Try to understand the root cause for the bug. Prove your understanding by writing an automated test that fails because of the bug.
2. Make the test pass.
3. Refactor mercilessly.
4. Identify similar scenarios in your code base. Check whether they contain the same bug. If they do, chances are that you have identified another opportunity for refactoring.

Here is another aspect of bugs. Compare a bug to an ordinary story in your backlog. What is the major difference? Generally speaking, a story tends to be easier to estimate in particular if it is similar to a past one. A bug can be anything from "user misunderstanding" to "several weeks" of work. The point is that the amount of uncertainty tends to be higher for bugs.

So if you compile a single, consolidated list of all your work items, you will end up with a mixture of stories and bugs. The larger the proportion of bugs the higher the uncertainty. The higher the uncertainty the bigger the impact if one or more of the bugs require more than the "average" fix time.

So if you are interested to become more reliable (read predictable) then the suggestion is to keep the bug level as low as possible.

Does this mean you can't have crash parties any more? Under all circumstances, if the crash party helps you to assess the quality of your product and helps you to find more bugs, and you if you see value in this, then you definitely should have them. Not doing them because it keeps the bug numbers low, is cheating. You would be lying to yourself.

Also, if you don't fix bugs but keep piling them up, while you at the same time your are "completing" story after story, you are actually reporting a velocity (in stories per week) that is too high. Expectations for the next release cycle will be higher, and it will be even harder to negotiate time for fixing bugs or doing other items.

And more, if your approach results in a similar number of bugs your stakeholders may start asking the question whether your approach is really better than the traditional one.

I'm sure you don't want to let it come to that. So regardless how you look at it: Not fixing bugs and finding ways for avoiding them in the future doesn't help at all. Instead it can be the start of a vicious cycle. And by keeping the number of bugs low, the question of how to deal with bugs becomes a no-brainer!

Tools

No engineer can exist without tools. Tools come in many shapes and sizes. This is no different for software engineers. And there are tools for agile project leaders as well.

For a software engineer most tools are pretty obvious. For instance a compiler, an editor, or a test framework (e.g. csUnit, JUnit, etc.), etc. But then there are other tools that are less obvious. What about the index card that you use for writing your story? What about the cards you use for planning poker (TM)? They are all tools in a wider sense.

An agile project leader has tools as well. Again some are more obvious like a reporting tool that gives him/her the latest financials for his/her projects. But there are others such as if you detect that a group of people is not on the same page in terms of information sharing. Then your tool of choice might be to get these people into one room and get them talking. Communication and collaboration are agile principles and so getting people to do both of this is a tool that an agile project leader has at his/her disposal.

It is important to use a broader definition of the term "tools". If you do you will discover many more tools you are using. Experiment with them. Try out new ones. Drop the ones that don't work for you. Listen to your team members. Often they have the best ideas anyways. Don't put tools away completely that didn't work at some stage as maybe at a later stage their time might actually have come.

Equally important is to look for tools that are simple and easy to use. Keep them as simple as you can get them. No need for an elaborate version control system if you can get away with a simple one like CVS or Subversion which have essentially little more features beyond update, commit, label, or branch.

Simple and small tools are much more flexible. They are easier to learn. They are easier to integrate with your existing environment. They tend to be easier to operate. And you can combine them forming more powerful solutions. If you select a good set of light weight yet efficient tools you can almost use them like Lego (tm) blocks to build the environment you need. And still you stay flexible and can adapt as needed by recombining or reconfiguring your toolset.

An Example

As an example for a simple yet efficient tool let's look at how you evaluate the engineers on your team. Many options exist. In large companies I have found systems (probably running on expensive hardware) which the managers are supposed to use for their evaluations. But there are simpler options. First of all, are you as the manager in the best (or only?) position to evaluate your team? What if you would delegate that task to your team and perform a peer review? What if you could use a simple tool to do that, e.g. a spread sheet that you send to all your engineers to evaluate each other?

You can find such a tool for free here. And here are a few things you can try out:

• Before you send out the actual sheet ask you team to review the categories or the descriptions of them.
• Try out different cycles, e.g. once a year, at the end of each project, every three months, etc.
• Try to expand it to non-engineering roles
• Ask people to also fill in the column with their own name. Is there a big difference between their self-perception and how they are seen by the team?

I'm sure you can find more factors to play with. Let me know how it goes and whether you find this type of tool useful. And always remember:

"A fool with a tool is still a fool." (anonymous)

So always THINK before you start using a tool.

And finally: The agile community would love to hear about your experiences. So I'd like to invite you to participate in the tools stage at the Agile 2008 conference.

To Pair or Not To Pair

Although I have been using pair programming as a standard practice with my teams since 2001 once in a while I still run into the question - usually asked by an uninformed outsider - whether pairing really makes any sense. The naive view is that it appears that two people are doing the work of one so it doubles the cost for most of the items.

Initially I was extremely adament about pair programming. It's called EXTREME programming after all! Quite some time ago I changed my position slightly. Today, when I start working with a team I make it clear that the default for doing any software engineering related work is to pair with a second person.

Does this mean that the people in the team I work with pair all the time? No, not according to my observations. There are quite a few occasions when people work by themselves.

One such case is if people would like to explore a new technology or tool. Different people prefer different learning styles. And before they present their findings to the team for further discussions most people want to understand enough about a topic before using their colleagues time. This gathering of information could be just surfing the web or reading a book on the subject. But it could equally be running a few experiments on a development workstation.

Then there are the usual other activities that most people prefer to do alone, e.g. email, filling in time sheets (yes, there are still a lot of companies who have them), etc. Also, you might want to call your doctor or spouse or log in to your broker without a colleague sitting next to you.

Unfortunately there seems to be only few studies out there on the subject. Laurie Williams' work is available in different versions, e.g. here and here, and also as a book. But some people would like to see additional researchers/authors who come to similar conclusions. Some studies have been conducted in an academic environment, e.g. with students. These studies are easily challenged when you try to use them in an industrial context. (Please note that I'm not saying these studies are useless.)

More generally I believe that reasoning about pair programming using a linear approach doesn't help. System thinking should be employed. Pairing (or not pairing) affects too many other factors. A simple linear cause-effect thinking tends to exclude critical factors.

Based on my observations and experience I would describe pair programming as an activity that has a positive impact on quality, team learning, soft skills, communication, productivity, innovation, creativity, collaboration, training, and many more. To pick one, let's look at productivity.

While many people believe that productivity goes done (two doing the work of one?), I believe that productivity actually goes up (or at least stays the same). My reasoning for that is - again - manyfold.

First of all, some people confuse body time with brain time. Have you ever walked through a development department where people work solo? Run a little experiment. Peek at the screens of the people and take a note of the percentage of people not currently looking at your IDE (or other work related tool). Now, do the same experiment with a team that uses pair programming as a standard practice. Or even better work with a partner and try to track your stock quotes at the same time... I think, you my point. With pairing there is no way to back out or to just doze off. You're in it full time. The ratio of brain time versus body time is much higher than with solo programming.

Next, a pair can and will be much more adament about the quality of the change set they deliver if they are members of a properly trained agile team. They will merciless refactor trying to use the simplest possible design and implementation to get the job done. They will be creative and innovative in doing so. They will have automated tests in place preventing other people from accidentally breaking their code. And once they have committed their change set they usually can focus entirely on the next task without having to go back to some past work.

There are other factors that are worth exploring, but I'll save them for now. I'd like to come back to them in a future post in this blog. As always, I would be very interested in hearing about your experiences and also about your feedback to this post. Thank you!

More on MindManager

Thank you for the comments on my previous MindManager blog entry!

Granted, performance issues can certainly depend on many different factors. According to my information the majority of the users don't seem to have performance issues. But there also seem to be a non-negligible number of users who did experience those issues.

However it is no help to the latter group of users if the former group of user have performance issues.

In my particular case when I observed the performance problems

• There was enough free memory (hundreds of MBytes)
• There was enough free hard disk space
• CPU usage was less than 10%
• No network usage
• No disk I/O

That basically indicates that probably more than one thread is running and a lock conflict is happening with one thread waiting for the other. Eventually a time-out will occur and the software continues to run. This is just a guess based on my observations, and the real issue might be a completely different one.

If you have used and developed professionally many different software products over 15 years then you start to get a sense for when a software has an issue. If a promise is given several times ("Update and the performance issues are resolved...") then I simply stop believing it. Instead of asking me to pay for an upgrade that solves an issues I would prefer to get a properly working version in the first place.

Anyways. Too many words already spent on the subject. I will consider giving version 7 a try.

MindManager, Performance, and Other Issues

For gathering new, complex, and broad areas of knowledge I found mind maps to be a very efficient tool. Although drawing on paper is often faster, an electronic version has it's advantages, too. As a product I use MindManager.

I have been using MindManager from MindJet for a several years now. Performance has always been an issue, but admittedly the featureset has improved over time. Each time there was a new version the company promised that the performance issue would be addressed.

I can't remember which version I started with. I upgraded several times, and currently the version number is 6.2.399 Service Pack 2. The performance with a map of reasonable size is poor. My laptop has no CPU usage, no I/O, plenty of free memory, and no network activity. When pressing the "Insert" key to add a new topic, it is as if the product doesn't do anything at all. It can take many, many seconds until it does something.

Of course I did complain to technical support. The people were very responsive. They suggested to switch off hardware acceleration for the graphics adapter. I tried that, and it helps. Sometimes.

Some observations:

• Suggesting to switch off hardware acceleration is basically like saying "Yes, sure did you buy the Ferrari, but adding our add-on allows for driving in first gear only." I didn't pay for a laptop with graphics hardware acceleration just to hear from a software vendor that I shouldn't use parts of it. That's not what I would call protection of my existing investments.
• Why is it that other software vendors are able to write software that has no issue with hardware acceleration?
• When I disconnect from the network and/or run the computer with batteries, the performance becomes even worse. I have had instances where MindManager didn't respond to user input for 15 minutes or more. By then I decided to kill the program. What has the network to do with graphics hardware acceleration?

A more "minor" issue is the function "Balance Map". If I could only figure out, what it is good for... When I click it, it rearranges the topics, but I can't find that the map is more balanced... Usually I have to print it on larger sheets, or I have to use a zoom of 38% instead of 50% or so.

Now, MindJet has made available version 7, and they encourage me to upgrade. Of course I have to pay for the upgrade. Why should I trust them that the tool has really been improved? Why should I pay for an upgrade if I cannot even get the existing version in a status that allows me to fully use it?

Given my experience I might just go for an open source tool or try a competitor. Maybe there are competitive cross-update offerings... Maybe I try FreeMind. They even claim that their software is faster to use than MindManager.

And another story

Let's see who thinks this time that I'm telling his/her story...

Again, I met a friend, let's call him Peter. Does he really exist? I don't care as that is completely besides the point. It is more about the story which, again, I have heard more than once from different people. In this case the first and the last account are several years apart.

The story goes like this: Peter is working for a consulting company. They developed and delivered a system that collects certain information across large geographical area. Part of the solution is using mobile devices for this, and once per day the data is transferred back to the head quarter to a central server.

The problems started when they started to work on the part of the software that deals with the transfer of the data to the central location. The firewall settings and the security policy of the company didn't allow for a direct connection, not using SSL (or any other secure means) and not even to a server in the DMZ (Demilitarized Zone). Then Peter asked whether they could sent the data via email, e.g. as an attachment which could then be picked up by a server. This solution wasn't acceptable either. Peter's client started internal discussions, and the went on and on.

At some point the delivery date of the project was at risk, and Peter couldn't wait any longer for a decision to be made. He had to come up with a solution.

Peter discussed the problem with his team. After some lengthy discussion, back and forth, they found the solution:

• Print the data
• Fax the data
• Enter the data

Yes, you are right! Faxing was ok. And then some people in the headquarter of Peter's customer would take the received faxes and enter the data manually! Crazy, but it saved the delivery date for the overall system.

Update: In the meantime the customer has settled on a much better solution and the data is now transferred via SSL to the DMZ, where it is picked up and transferred to the server within the firewall.

Morale: Sometimes you have to be very creative and adaptive to make the deadline. I think Peter and his team were very agile!

Where my stories come from

Once in a while people ask me where I get my stories from. Well, there are many different sources but most of the time they develop as I talk to friends.

All the stories I tell in this blog are inspired by true facts and data. However, in order to protect people and companies, I never tell a story that is based on a single account only. I know people working for over 25 different companies from all over the world. Only when I hear a similar story at least twice from different companies I assume a pattern. Then I take the information and use it as an inspiration for my blog.

Sometimes there is also a large difference in terms of when I have heard the different stories. One story I might have heard many years ago, while a different one I just came across recently. Together they may suggest a pattern. One story might have been from a very small compay, and a different one may come from a very large one. One story might be from Europe, and the other from India or Australia. These stories are particularly interesting as they may reveal cross-cultural patterns.

I ask you for your understanding that to protect my sources I will not reveal any names, locations, times, or other specifics that can be used to identify a company or a person or a source. Only where I have the expressed permission from a person or company I can give more specific details. Generally the names I use are not the real names.

Or as they say at the end of the movies: all my stories are inspired by real facts and data, but any resemblance with actual people, companies or events is pure coincidence.

The ripple effects of moving in a release date

When a project is behind schedule several options are typically considered. The schedule is adjusted, the number of people working on the project is increased, or the scope is reduced.

A few days ago I met Susan, an old friend of mine. She is working for a very small software company, and she told me about a project that she used to work on until a few months ago. The project was an online reservation system for hotels, and as the hotel owner decided to go into business sooner than originally planned the schedule for the project had to be shortened by a few weeks as well.

This decision caused a number of ripple effects, mostly undesired side effects.

First it turned out that the staffing level wasn't high enough to achieve a sufficient velocity for the shortened time line. Additional engineers were required. As the company is only a very small outfit, they had reassign engineers from a different team to this project team thus sacrificing valueable work in the other team.

This in turn not only reduced the velocity of the other team. As a consequence the scope for the other team's deliveries had to be reduced.

Also, it turned out that in order to make the shortened deadline the project team had to take a number of short cuts, e.g. by using a less elegant design in some places or by not refactoring code to the degree it deserved.

There were many more impacts, but so far we have:

1. Need to increase staffing level in the project team: additional project cost
2. Need to reassign engineers from the other team: cost in the form of delayed deliveries in the other team
3. Need to clean up the "short cuts" caused rework after the release

Bottom line: Moving in a release date had not only the immediate impact but also cause a number of side effects, each of them causing a number of additional problems and/or cost.

The software has been released a few months ago, and the project team is onto a new project. My friend reckons that it probably will take a few more months until the effects of the changes have been resolved.

On the positive side, Susan's team delivered on time and with functionality and quality as specified. In addition the customer is very happy with her company's ability to adapt to the changed plans. Susan and her colleagues are, however, very aware of the price they had to pay.

Apologies from Agile 2007 Conference Program Committee

In case you have submitted a paper, tutorial, etc. for the Agile 2007 conference, you most likely have noticed that the submission system had a system error. Here is a message from the conference chair on that regard:

I am finding as many opportunities as I can to express sincere apology for an error in the Agile 2007 submission system which caused multiple
accept/reject messages to be sent out. While all of us have experienced errors in our software projects, this is one that inconvenienced many people... for this, I apologize.

We can point accusations at the software developers and testers, but it will not accomplish anything. Instead, it is more effective to correct the error and move forward. We have sent revised accept/reject notices (with an apology) and will make every effort to learn from this experience.

We hope that this mistake will not cause you to overlook the countless other things our hard-working team of volunteers are doing so well in order to make the Agile 2007 conference fabulous in many, many ways.

Mary Lynn Manns
Conference Chair, Agile 2007

Being a member of the program committee myself, there is nothing I can add.

Fingerpointing, however, will not help. Instead, it is important to look at the situation, make the best out it, and move forward. I think Mary Lynn and the other members of the committee are doing this. We all want the Agile 2007 conference to be a success!

A Really Short Release Cycle and An Impressive Quality

Just came across a blog entry by David J. Anderson. His company releases every 9.5 business days and with zero defects since January 2007! Impressive!

Agile Certification

In a recent post of a position paper on "Agile Certification" the chair of the Agile Alliance describes the various aspects of whether a certification for agile practitioners should be introduced or provide value.

I'm convinced that there will be people who believe that certification provides value to them for recruitment purposes. With the teams I work we use different means to determine whether a candidate fits the team. That by itself is already a statement: We don't look for certifications, we look for "good fit". Someone can have all the certifications on the planet, we will not hire that person if he/she is not a good fit for the particular team.

Just the other day we had an application by a candidate who used "throw new NullPointerException()" in the programming task. In this particular case we tried to imagine what would happen if we had that bit of code in a deployed system, and the exception fired off... No certification in the world can prevent that, and in the particular case we didn't consider the candidate any longer.

Here are some other things we do when we assess candidates:

• Pair programming session
• Programming task
• Design session
• Google search (if the candidate is claiming to be an active member of the community)
• Peer interviewing by members of the future team
  

I'm sure that this approach is not perfect, but so far the results speak for themselves. For us, I don't believe certification would make a big difference.

In the teams I have worked with I have met accountants, biologists, former HR people, and many more different professions. All of them had degrees and certifications, most of them in areas other than software engineering. And still, all of them were excellent software engineers!

And then, there are the companies that get ISO9000 certified. Sometime people I wonder what that is good for except for becoming supplier for certain companies. Does it mean that the quality improves or the productivity? It might, it might not. The ISO9000 certification certainly doesn't guarantee that. There are companies that are ISO9000 certified and still deliver crab, and there are companies that are not, and they deliver outstanding quality.

Hey! I tried to lean towards the more provocative end of the spectrum in this post. So if you, my dear reader, have a different perspective, I would love to hear from you!

Embracing Change: What Stays Constant?

My experience is that many people like to have something that doesn't change. Sure people understand that to stay in the game, we need to adapt as quickly to new situations as possible. Only if we are fast enough, the ability to adapt can be turned into a competitive advantage.

But then I observe the people I work with, I observe myself. And I discover that they as well as myself, we are still looking for something that stays a little bit more constant.

And when we take a closer look there are things that stay constant. We change the tools, we change the architecture, the designs of systems, and retire tools in favor of better tools. We change the techniques and processes. We create a flow of all these elements of software engineering. And yet, there are things that stay the same: It is the values and principles that we apply. The people I work with, and myself, we have chosen the agile values and principles as a guideline, as a foundation for how we work. They provide us with the foundation, with that sense of fixture.

Some of the more traditional projects and teams that I have seen in the past, they, too, have things that stay constant and things that change. They often have chosen the process to stay the same, for instance with all the artifacts that people create, but which sometimes don't provide as much value as they require effort to create and maintain them. And the thing that changes are the values. Sometimes they claim that they respect the human individual, their people's dignity. But then, during the execution of the projects, these are the things that go overboard first. With pressing deadlines people are overloaded, with a shortage of people, contractors are parachuted in, with problems popping up here and there, people are moved around (ever heard of a "Move Meeting"?). Should we as leaders really be surprised, if no hand shows up when we ask the question, whether people feel treated as being human individuals?

There is better ways, and one of the questions we should continuously ask and answer is which elements we want to keep constant, and which elements are the ones where we want our teams to be adaptable. If we decide to keep our values and principles constant, and want to change the way we work, the technology, the design, the tools, the processes, then we are usually in a much better position for success! And all people on our team and the stakeholders will have more fun, too!

P.S. I had a very good discussion today on this subject. Thank you Raymond, Mark, and Curt!