I occasionally re-post from my other blog called Manage By Walking Around. This one discusses the challenges of setting KPIs:
-------
At a recent performance management workshop, I asked an attendee for an example metric that he was using to track performance. The attendee, who worked for a city’s public works department, immediately replied “# of miles of streets cleaned”. Before I could ask any questions, he proudly added “and we cleaned more than 1000 miles last quarter; up 10% since the previous quarter”.
I’m no maintenance worker but 1000 miles and up 10% both seemed like good performance. Therein, however, lies the problem with our old friend: context. The attendee (we’ll call him Joe) had sat through most of my workshop and was trying to provide additional information about the actual miles of streets cleaned by comparing it to the previous quarter’s actual value. Clearly, an increase of 10% is easier for me – as a non-expert – to understand than the raw number.
However, because this is an activity measure, all I really know is that output increased from last quarter. What if a new subdivision opened in the last 90 days so that the total number of streets that could have been cleaned is 15% higher now? This means that the 10% increase in actual output is further away from our target of cleaning all of the streets. Output is up but performance is down.
Immediately seeing my point, Joe suggested that they switch to “% of street miles cleaned each month” with a target value of 100% and letter grade scoring so that greater than 90% would be an ‘A’, 80-90% a ‘B’, etc. This was a definite improvement and might have ended the discussion until one of the other attendees pointed out that there is an incremental cost to cleaning a higher percentage of streets. Do we really need to set a target of 100% so that every street is cleaned monthly? Is there sufficient value to increase the percentage from 80 to 90? Perhaps the right answer is to set the target slightly higher than the benchmark from our sister communities. That will make sure that we don’t spend too much money but that we continue winning those awards for the best places to live.
That discussion brought us back to one of my favorite topics. The problem with this and other output measures is that it’s not easily tied back to an outcome. What goal are we trying to achieve? After scratching his head for a few minutes, Joe remembered that one of the overall objectives for the Public Works department was “to increase citizen satisfaction”. Presumably, if the streets were clean, people would complain less and they would be happier with the department.
Logical, but worth going the next step. If the goal is to increase satisfaction, we might establish an outcome measure such as “# complaints about street cleanliness reported every quarter” with the target of reducing it by 5% every quarter for two years. If we are hitting the target for the output measure of street miles cleaned but not our target for the outcome measure of complaints, then we’re probably tracking the wrong activity and unlikely to meet our citizen satisfaction objective. Perhaps people care about more than just clean streets.
Speaking of which, Joe, there’s this pothole in front of my house I would like to talk to you about.
----
If you enjoyed this story, there's a follow on here: http://alignment.wordpress.com/2006/11/29/dirty-kpis-revisited
