Measuring the worth

Finding the right thing to measure and the right way to do it is common sense but it's the interpretation of the data that matters most
Measuring is a human construct, it's interpreting the data that matters most. Photograph: David Levene

The Audit Commission's report on the lack of evidence-based decision-making in local councils highlights a problem that is by no means confined to the public sector.

Finding the right things to measure, finding the right way to measure them and then taking the right decisions based on that information may seem like common sense, but is in fact extremely hard, particularly in large, complex organisations.

A fascinating report written for the Audit Commission last year by academics Mike Kennerley and Steve Mason at the Cranfield School of Management on the use of information in decision-making emphasises that measuring anything is a human construct: "When measure is identified with the very essence of reality, this is illusion". It's the interpretation of data that matters most.

But in both the private and public sectors, a whole industry has grown up around metrics and measurement and the resulting obsession with jargon-laden concepts, such as key performance indicators (KPIs), balanced scorecards and the like, and have left many senior managers baffled while failing to come to terms with the evidence they need to make effective decisions.

That is clear from both the commission's most recent report and its previous report on the subject, which pointed out the correlation between good use of information and better public services.

That correlation is what you would expect, according to David Waltho, head of public sector at software firm SAS and a member of the external advisory group that provided expert input into the report, but it has taken the public sector a while to catch up with the best private sector attitudes to information management. "It is only in the past year or 18 months that the government has woken up to the fact that information management is critical to transforming government," he said.

Waltho would like to see a more coordinated approach to information management across government, rather than the "blizzard" of individual reports on the subject, but the idea of centralised edicts on information is firmly rebuffed by John Kirkpatrick, director of studies at the Audit Commission and author of its recent report.

"There are a lot of people happily working round different corners on this agenda," he acknowledged, but said the whole point of his own report was to emphasise that it is councillors and decision-makers in local authorities who need to decide for themselves what information they need. Any centralised approach would risk "spawning a sort of compliance culture that is exactly the opposite of what we want," he said.

Kirkpatrick said the use of technical measurements like balanced scorecards, which measure different aspects of an organisation's performance, have been developed because of the difficulty in picking specific indicators of performance. "A further complexity that bites the public sector in particular is that you don't need the same information for accountability as you do for managing the business," he said.

The Audit Commission has developed a number of online decision guides to accompany its report, as a checklist for decision-makers. SAS's Waltho welcomed a more positive approach to information management in the public sector. "If information is seen as a toxic liability, rather than as the most underused asset in an organisation, it is no surprise if local authorities do not appreciate its worth," he commented. "If you make information a positive asset, then you would naturally take care of it in the same way as you take care of your other assets, such as people or property."

Public leaders jobs

Today in pictures