Voluntary organisations are under increasing pressure to measure and demonstrate the positive effects of their work: government sources and private funders want to know how to get the most from their money and organisations want to be able to demonstrate their value, especially as economic conditions get tighter.
Many tools have been developed to provide a basis for measurement but, among these, Social Return on Investment (SROI) has risen to prominence, promoted by a number of government departments and voluntary sector organisations.
SROI produces a ratio that compares money spent with social outcomes achieved. So, for example, a ratio of 1:3 means that every £1 spent produces £3 of social benefit.
One challenge, of course, is that social benefits are not always easy to identify or quantify, let alone to translate into monetary value. It relies on the judgement, and to some extent the imagination, of those conducting the SROI.
So, the increased wellbeing of 30 people might be translated into a monetary value using the annual NHS cost of treating 30 people with depression. But we know that happier people are also physically healthier, so we might add on an increased benefit of X times less GP visits, or Y times less sick days.
SROI is a great way for organisations to demonstrate their benefit in a universal language that is immediately easy to conceptualise and, arguably, contains some of the force that narrative descriptions alone often lack. But its growing popularity affords it special need for analysis. The aim behind the Third Sector Research Centre's research is to help make sure new approaches like this can become as workable as possible.
One of the biggest challenges identified by our research is that while those who have promoted SROI warn against its use as a comparative tool, our research shows that those using it to produce a comprehensive social impact assessment often do so to strengthen their competitive advantage.
We may therefore assume that SROI is, and will be, used in a comparative context. But the ratio itself is so dependent on the individual assumptions and processes used to create it that it cannot be compared between organisations.
One type of standardisation is to create an index of commonly used indicators that organisations can turn to. A second is for organisations working in similar fields, for example in mental health, to use very similar frameworks. This could mean bypassing the stakehoder consultation to identify indicators, targets, values, that different stakeholders see in different ways.
The framework would need to be well informed and take all possible types of stakeholder groups' ideas into account but would ignore the organisation-specific exercise of bringing stakeholders together in the consultation process.
We are at an early stage of planning further work, but one possibility would be to look into a group/network of organistions, all working in the same field and for the same main funder, that would be required to undertake a standardised SROI. It would be very interesting to look into what this means in terms of gained rigour, but perhaps loss in stakeholder engagement.
While the SROI ratio provides a powerful illustration of impact, it is, in reality, a metaphor: an example of what the social benefits they create might look like if we thought of them in monetary terms.
It does not necessarily represent real monetary savings (although many of them may do), and does not always involve standardised measurements or values; real meaning is provided by using the ratio in conjunction with the corresponding narrative, which explains how the assumptions and figures were made.
What's more, research shows that once completed it is very easy for the SROI ratio to become divorced from that narrative and become the sole focus of the evaluation results. This makes it hard to assess the claims being made, but also places the focus solely on proving impact rather than evaluating success.
Certainly one tool cannot do everything – how we value social outcomes represents a problem inherent in all types of impact measurement – but the task of our research is not simply to point out the shortcomings of SROI but to help strengthen it. Most importantly, careful attention needs to be paid to how SROI and its results are employed but it may prove most useful once we explicitly recognise its limitations.
This article is based on the research report, The ambitions and Challenges of SROI, by Dr Malin Arvidson, Professor Fergus Lyon, Professor Stephen McKay and Dr Domenico Moro.
Malin Arvidson is a research fellow at the Third Sector Research Centre