Two More Thoughts About Selecting Tools

In a previous post I shared a thought about selecting tools, in particular whether you should decided for a tool just because you happen to be a partner of the vendor.

Here are two more thoughts about selecting tools.

Driving Towards A Decision

The first one deals with ensuring that the decision making doesn't take forever. Sometimes teams discuss about tools for weeks or months without coming to a conclusion. As a result they don't have the tool. And as a consequence they don't enjoy the benefits.

For instance you are considering adding a particular type of test tool to your tool set. Of course there is the danger that you may select the wrong tool (I'll discuss below how to mitigate that risk). On the other hand even a tool that is not a perfect fit but at least does a perfect job is better than no tool at all. No testing tool is even worse than a testing tool that may be usable only for very specific test cases.

Therefore it is important to drive to a decision. And the best technique is time boxing. Give your time a time box and insist on a decision by a specific time. Only extend the time box if an excellent reason is given. That way as a leader you can steer the team towards a decision.

And if the the team can't come to a conclusion? Well, you can always use the threat to make the decision yourself....

Homeworks

While it is important to reach a conclusion you also want to make sure that the team makes a proper assessment of the different options. I usually expect that the major player are being looked at and compared. As a tool I recommend a decision making matrix by which the columns are the different products and the rows are the selection criteria.

To get the team started I describe to them how the tool works. Don't assume they already know. Make sure they do. Next I also tell them that I won't provide them with the full list of selection criteria but that I expect them to choose those as well before they start the assessment. I guess the management speak for that would be "empowerment".

Usually I make a small number of selection criteria mandatory. For tools these are:

• Ability to be launched from a command line
• Ability to deliver the result as an XML file (or other standard format)
• Price for chosen configuration, e.g. based on number of license required

The first two items are important from my perspective since a tool needs to be easy to integrate into automated processes. By using command line and standard-format based files (XML preferred) only little 'glue code' is required if at all.

Why the decision making matrix? First, you will notice that the team will then have a recipe for their assessment. They know what questions to ask and what to find out about the tool. Second, the team will gain an overview of the major players for each tool. Often team members have different backgrounds and experience with different tools. By working together the decision will become data-driven and facts-based as opposed to 'I like tool A more than tool B'. Also the team will have a basis on which they can have a meaningful discussion. The result is much more detailed and specific.

And finally, you will observe that all the sudden all major vendors are looked at and a better decision is made. You can take the outcome including the decision making matrix to create the business case that you may need for the purchasing process anyways. And even if such a business case is not required it still is an excellent exercise to go through some number crunching to determine by when a particular tool is fully paid off and what ROI it creates.