CA Community






This Blog

Why most ITIL Metrics are not worth the trees they kill!

Published: February 16 2009, 11:54 PM
by Peter Doherty

So how do you measure your ITIL Service Management Program? 

The first trouble with measuring ITIL is that you need a certain level of maturity to gather baseline metrics and a good deal of organisations are not in a position to capture those metrics. The second trouble is that most of the metrics they capture are worthless - as good as marks on a chart.

Where the true value is from metrics that: 

  • Inform
  • Support Decisions
  • Prompt Action

Sadly we are lucky to see ones that mainly fit into the top category. After all, metrics are just the dashboard of the car, they are neither the car nor the destination. 

So how do you measure Service Management? Well we really do need to take a balanced score card approach. 

There is no doubt that you need to run your process metrics with a high level of governance as without this your processes could lose their effectiveness. For example; failed Changes which have lead to causing Incidents, etc. These will often only have a quantitative component to them and should roll up to an overall process conformance score.   

You also need to monitor the efficiency of the process by comparing the effort or elapsed time it took to perform core functions prior to the Service Management Program and comparing to what they take after the program. This should be rolled up as an efficiency score but you can also start doing cost comparisons based on this. These will have a qualitative component to them. 

Thirdly you need to monitor effectiveness which will be the outputs of the processes and generally defined in terms of cost savings and service availability improvements. This effectiveness metric is absolutely crucial for measuring the results of a Service Management Program. Why? Just about nearly all other metrics will give you soft savings, but you can finally put dollar figures around these metrics. These are the metrics that should be measured against the projected outcomes that were put forward in the business case. For example, if you are an ITIL V3 shop doing Service Portfolio Management these metrics will contribute to show the value to the business of the Service. 

I often blog and write about how most organisations forget about the people side of Service Management and I have a number of ideas on how we can address this. A balanced scorecard must include customer satisfaction - is it the be all and end all? Of course not but as technologists we often get caught up with delivering technically oriented Service Levels which reflect ‘reality' - at least in our minds. But guess what, most of our customers perceive value in the expectation of the service experience so you need to track their perceptions. A happy customer is  more engaged and able to be more productive! 

In ITIL we sometimes misuse metrics for our own benefit and one of my pet hates is ‘Closed on First Call'. It really tells us nothing, unless it is really high in which case it is rarely true. This is a typical example of a metric that has a quantitative but no qualitative component. So what if you close 70% of Calls on First Contact? Have they been within SLA? Is the customer happy? What about the other 30% that's probably harder and more important to the organisation? That metric simply informs you that something is happening but with no insight into how well it is being done. What decisions or actions could that possibly prompt? 

If you are asked how would you measure an ITIL Service Management Program you need to ask yourself a much more fundamental question; what are the business drivers for this Program? Have you defined those business drivers? Once you know the business drivers it is a simple matter of translating them to the business initiatives and collecting metrics to support this. And guess what? These are the metrics that I'd want to see on my Service Management balanced Score Card.  

Share this post:  

 

By: Peter Doherty
Peter Doherty is an ITILv3 contributing author and a Principal Consultant for CA. With 25 years IT experience in Service Management as well as Enterprise Network and Systems Management, Peter Doherty is CA’s foremost Service Management evangelist in the Asia Pacific region. His day-to-day responsibility...
Read More..

1 person has left a comment:

When measuring time lines can you discuss the relative meanings value of Average vs Median.   I have discussed this with management to influence the analysis to no avail.  I submit that the average can be severely skewed by a fairly small number of extremely long incidents, whereas a median is a more representative indication.   For example:  An analysis of time to resolve cases was desired to be 1 hour.  However 12 incidents out of nearly 400 exceeded 60 hours -- unfairly skewing the averages fairly dramatically.   Median is less affected.  To illustrate my point, if you drop out those 12 highest length incident resolution times, the Average drops to 78 mins from 128 mins.   This represents 3% of the total incidents, but can skew the Average by 33% !   Now if I have made any sense whatsover, how do you think the "median" is affected dropping out those same incident times in the calculations?   Correct, it remained @ 68 Mins!!  Yet, median continues to be ignored in favor of arithmetic mean or average.

Posted by: al | September 26, 2009 2:18 AM

Leave a Comment

* An asterisk indicates a required field

* :  

:

* :  

 Submit