Comments on: The politics of software metrics

By: Arun Kumar

Arun Kumar — Tue, 11 Mar 2008 11:27:55 +0000

I am not clever.

I fail to understand what they mean by data-collection when it comes to the IT companies. Aren’t they supposed to be technology drivers! I believe that if you implement a good process with appropriate sophisticated tools that seamlessly tracks the process so that you don’t do explicit data collection for the sake of metrics analysis, but simply allow it to be inferred by the tool we use, then atleast some of the grudging and misunderstanding could be avoided.

By: ed

ed — Tue, 04 Mar 2008 13:09:10 +0000

Interesting comments! For some reason, I decided to do a google search for “Dilbert” and “metrics”. Found this site, with a reference to the “Dilbert Barometer”: http://tabletumlnews.powerblogs.com/posts/1178255423.shtml

By: Ram

Ram — Tue, 04 Mar 2008 08:06:02 +0000

I don’t fully agree with Alfonso. Creating a very good software application is an art. (It involves deep thought, creativity, doing things differently, analysis). Organizations commoditize it. You want to treat it as an Engineering process, to ensure you meet the deadlines and meet the requirements and the budget. So it is like… ditch the creativity.. Except for say things like, be creative in creating reusable components to increase productivity and reduce defects.. “Creativity” part will be demonstraated as the value addition. From my experience, every project may need to tune the metrics to be effective. There is a need to identify what parameters are different in your project as compared to the standard – skill levels, technology, team structure, requirements stability/details, user skill level etc. You may need to create additional measurements. Well, most of the projects that we do need to meet the client requirements. So you need to keenly observe if you have the right metrics. Success will depend on the answering the following questions… What is the accuracy levels of metrics in your organization? (Do you have history data of successful metrics used for projects’ success for a few years?). Is the data collection automated or is it an overhead? Do you collect data as real time as possible for your analysis or do you do it “to comply” ? Who gets the credit for a successful metrics driven project in your organization and who gets the blame for a project in trouble? Is your senior mangement willing to listen to you?

By: Alfonso Guerra

Alfonso Guerra — Sun, 02 Mar 2008 15:59:30 +0000

The fears echoed by your list, and the backlash they engender against metrics, are the second biggest problem I have with the extreme programming religion: the lack of dispassionate measurements of the process to prove its value. The lack of metrics means the emperor has no clothes. And any claims to it’s superiority with, “It feels better.” is mere hand-waving.

Going with your gut doesn’t fly for improving performance in software, nor does it fly in improving the software development process. You can’t fix what you refuse to measure. How can you determine the better practices from different XP shops without measuring them? How can you determine the benefits gained from making changes at cost X without measuring them?

The claim that software metrics are too hard because developers are human and development is art not science, is a cop-out. Practically every other human endeavor has a metric available for it, including sports and arts. Practically every other field has seen tremendous leaps forward in productivity and quality except software development. Why? Because developers are the last shadow of the old school data processing priesthood, luddites who fear change while promoting it everywhere else: the last ones with control.

Anyone who claims IDEs provide no more than 8% productivity gains over punched cards would face a lot of criticism by the anti-metrics crowd, but they wouldn’t be able to disprove it. In fact, without measuring for results, the stand-up sessions and pair-programming sessions favored by XPers can be proven to reduce productivity due to the loss in programming time.

The potential for metrics to be misused is no guarantee they will be. If you feel the current state of metrics lacking, then develop more accurate ones and demonstrate their proper use. But promoting “feel-good programmery” and mystic coding ability without practical, reproducible measurements is just promoting another religion.

By: Joe Cascio

Joe Cascio — Sun, 02 Mar 2008 03:17:24 +0000

The only serious metrics project I was in any way involved in was among the 90% that didn’t make it. The reason, as I recall (it was many years ago), was that it didn’t seem to yield anything particularly useful. It’s like it was telling barbers how many hairs on someone’s head got cut and ended up on the floor. Perhaps accurate but not particularly useful for giving stylish cuts.

And that brings me to really the most serious objection I’d make about metrics. The only thing that counts in the end is user satisfaction, and how on earth do you measure that, or even assuming you could, how then to attribute it to individual coders?