Not too long ago I had a conversation with a friend and we discussed productivity in software engineering. In particular we got hung up on the question how to measure it.
It’s not that nobody had ever tried to measure it. I have probably about 30 to 40 books just on metrics alone. So far, however, I haven’t found a method that would meet my requirements. Let me explain.
Let’s assume you have a team that is working frantically and has a high productivity. Let’s further assume that you had a metric that can measure the productivity reliably. At some point the team ships the new system, not only on time but with a record breaking productivity maybe even ahead of time.
Enter the customer or even better the actual user. They sit in front of your brand new system. The initial comment: “This is not what I wanted. I don’t need a system for making hotel reservations I need a system that helps me control my production plan.” (OK, not a real case but I have to protect the innocent and I think it makes the point.)
In this scenario, what good is it if you have the best productivity on the planet if as a result you deliver the wrong features? Wouldn’t this indicate that in the end it is the customer who judges whether they get enough value for money? And wouldn’t that be a good metric for productivity? In this case, because the wrong system was delivered the “productivity” in terms of features with business value was zero. At least from this customers perspective.
In the discussion with my friend we really go stuck at some point. We had already agreed that counting lines of code (LOC) doesn’t help, or counting the number of classes, methods, function points, statements, screens, etc. Equally counting the hours doesn’t work.
My friend and I agreed on all of this. We couldn’t find a metric that we believed would work. Then I asked him how he measures productivity and his answer was: “gut feeling”. I’m not quite sure about his answer.
As for me I have decided that if several customers tell me that they believe they get good value for their money, e.g. for their maintenance fees, then I take that as a sign for a good productivity. Productivity measured as in business value perceived by the customer. Does that mean we now lean back and have a good time? No. We still use a continuous improvement process within our team to find even better ways of working, even more waste we can eliminate. And we continue to have a good time on top of the hard work!
2 comments:
Counting code is definitely the wrong answer - code is just a side-effect of solving problems, which can ideally be solved without changing a single line. And on a good day, I've solved problems by *deleting* a few thousand lines of code - I've no idea how you'd feed something like that into any sensible metric.
That said, I can't see a better way. "Gut feeling" seems as good an answer as any - you might not be able to measure productivity, but you can at least judge whether the process is working well or not. And as you say, the ultimate measure is whether the customers are happy, and how much it's costing you to achieve that.
I can follow your conclusions pretty well - the key question in measuring productivity seems what to measure and how to do it (ok, so it's actually two questions;). After all what is code worth that does not serve a business value? In theory one could measure the entire value chain from customer to validated code. That would surmount in an impressive bulk of data to be collected, understood and acted accordingly. A full-fledged CMMI level 5 approach would lead you there. But does that really make you more productive in terms of business value?
By coincidence we researched the question of productivity with respect to agile teams on the platform for agile management at the same time as you, coming to similar conclusions: http://p-a-m.org/2011/12/the-legend-of-hyper-performing-teams/
Looking forward to reading more from you
Thomas Spielhofer
Post a Comment