Metrics Mayhem

Buried in the middle of a paper I wrote for the upcoming STAR East conference is this (forgive lack of full context):

There’s a funny thing about measuring – if you measure something, it will usually change. You just need to make sure it’s changing for the right reasons. If you choose to measure something about your process or product, it’s probably for a good reason, but despite your best intentions, in the end, that metric may not tell you what you were looking for in the first place. If you’re not careful, this can quickly get out of hand and lead to full scale metrics abuse. I have heard of some test teams so frustrated with metric abuse that they have stopped using metrics entirely! .

The bold sentence is something that scares me – an example of “worst practices”. I was planning to write more about it some day, but as it seems to happen every time I’m about to write something, somebody said what I wanted to say – only more eloquently and convincing. Jason Gorman over at parlezuml.com wrote Being Skeptical About Metrics != Dismissing All Metrics. If you’re afraid of metrics, go read it.

It’s sort of like saying that because there’s a chance you’ll get in an accident, you should never leave home. It’s silly, but people do it. Denying measurement just because it can be abused is silly too.

Thanks for writing the article Jason.

Comments

• Anonymous
  April 11, 2009
  PingBack from http://microsoft-sharepoint.simplynetdev.com/metrics-mayhem/
• Anonymous
  April 15, 2009
  >>There’s a funny thing about measuring – if you measure something, it will usually change.True ... that is "uncertainity principle of software measurements". There is no such thing as "non intrusive" measurement.But the question is who cares about the impact, the side effect?Shrini
• Anonymous
  April 16, 2009
  The comment has been removed