What do the Peterborough SIB interim results tell us?

Update: actual results are now out.

First cohort results from the Peterborough SIB were released August 7 2014. The Social Finance UK press release on the results has lots of great information, with a quote below.

“Results for the first group (cohort) of 1000 prisoners on the Peterborough Social Bond (SIB) were announced today, demonstrating an 8.4% reduction in reconviction events relative to the comparable national baseline. The project is on course to receive outcome payments in 2016. Based on the trend in performance demonstrated in the first cohort, investors can look forward to a positive return, including the return of capital, on the funds they have invested.”

Outdated information below:

Peterborough Interim ResultsOn the 13th of June, the Ministry of Justice released interim results from the Peterborough Pilot SIB. The results were seen as very encouraging, although Social Finance stressed that the results “do not measure  reoffending behaviour over as long a period as the Social Impact Bond will be judged and are not compiled on precisely the same basis as will be used by the Independent Assessor during the course of 2014 to determine whether a payment is due.”

What the results do tell us

The results tell us that the reoffending events have improved in the Peterborough cohort as the national average has worsened. We can be fairly confident that the reduction in reoffending is due to the Peterborough SIB, or more specifically the One Service. This in itself is quite an achievement for Government policy.

These results are also an excellent demonstration of the need to have a contemporary comparison, in this case the Police National Computer control group, rather than a historical baseline. If the historical re-conviction rates at Peterborough had been used as the only comparison, it would appear that a 6% reduction had been produced. Using the national comparison group shows that the programme also counters an increasing trend in re-convictions to produce a relative reduction of 23%. The inability of historical baselines to represent the fluctuating environment affecting reoffending is further illustrated when we look at results leading up to the 2010 launch of the One Service. The graph below shows that reoffending by both Peterborough inmates and prisoners across the nation was increasing until SIB services began in 2010, but then reversed in comparison to the national rates.

Peterborough Interim Results 2

What the results don’t tell us

The results do not tease out for us which aspects of the One Service might be more or less responsible for success. So, the results do not tell us if the reduction in reoffending is due to the:

  • use of volunteer mentors
  • voluntary, rather than mandatory, participation
  • long-term nature of the SIB funding
  • flexibility of funding
  • ability to innovate programme delivery to optimise outcomes
  • focus on a single outcome
  • extraordinary skills of the people involved in managing and delivering services
  • continuous evaluation and improvement of the One Service
  • discipline of reporting to external investors
  • alignment of financial and social returns.

The April 2014 evaluation commissioned by the Ministry of Justice from Rand Europe sheds some light on the perceived benefits of the SIB model, including the way the flexibility of funding allows for the service to improve in response to performance management data.

Update April 2014

See 24 April results and press release from the Ministry of Justice stating “Before the pilot, for every 100 prisoners released from Peterborough there were 159 reconviction events annually. Under the scheme this figure has fallen to 141 — a fall of 11 per cent. Nationally that figure has risen by 10 per cent over the same period.”

Toby Eccles blog analyses how well the Peterborough SIB achieved its objectives to:

  • Enable innovation
  • Enable flexibility and focus on outcomes
  • Bring rigour to prevention
  • Better alignment
  • Investment in social change.

And a good analysis of the Peterborough journey and what was learnt is given by the Social Spider here. I have a slightly broader view of SIBs in the context of policy reform, but I like the discussion.


Ministry of Justice, Statistical Notice: Interim re-conviction figures for the Peterborough and Doncaster Payment by Results pilots12 June 2013.

Social Finance, Interim Re-Conviction Figures for Peterborough Social Impact Bond Pilot, press release 13 June 2013.

Ministry of Justice, Mentoring Scheme Reduces Reoffending, press release 13 June 2013.

Vibeka Mair, Peterborough social impact bond has slashed reoffending rates says MoJ, Civil Society Finance, 13 June 2013.

Alan Travis, Pilot schemes to cut reoffending show mixed results, The Guardian, 13 June 2013.

BBC, Prison payment-by-results schemes see reoffending cut, 13 June 2013.

Nicholls, A. & Tomkinson,E. Case Study – The Peterborough Pilot Social Impact Bond – Oct 2013, Oct 2013.

A second social impact bond for NSW Australia

Benevolent Society Social Benefit Bond(Source: The Benevolent Society. “First Charity” refers to the Benevolent Society being the first charity established in Australia.)

On June 14 2013 it was announced that the terms of a second social benefit bond have been agreed in New South Wales, Australia. It will provide services to strengthen 400 families and reduce the need for out-of-home care over five years, beginning October 2013. Press releases were issued by the charity service provider, The Benevolent Society and one of their partners in the SIB, Westpac Institutional Bank. The other partner was a second bank, the Commonwealth Bank of Australia and the involvement in this deal of two of the four large Australian banks has captured media attention. The banks were involved in the construction of the bond as pro bono advisors and are also investors. The Benevolent Society is also investing in the Social Benefit Bond, a move promote confidence in their ability to deliver outcomes. Other investors include institutional investors NRMA Motoring & Services and Australian Ethical Investments.

Investors will provide working capital for the services up-front and the NSW Government will cover all repayments once outcomes have been produced and measured. The social benefit bond also involves financial services firm Perpetual as trustee, a move expected to give investors further confidence in the deal. The role of the intermediary as played by Social Finance in Peterborough and MDRC in New York is not played by any organisation in this arrangement in a comparable way. 

Benevolent Society SBB tiers

The offering closed on Friday October 4th having raised $10m. 40 investors make up the low-risk tier and 17 investors make up the high-risk tier. In contrast to expectations that social impact bond risk needs to be reduced to attract investors, the high-risk tier proved more attractive and sold out much faster.

Once again, we see the NSW Government offering a guarantee, which means that the Government will pay even if the service does not produce the desired outcomes for recipients. In the previous NSW social benefit bond this guarantee was for between 50% and 75% of investor capital. This time it’s 100% of investor capital guaranteed, but only for the low-risk tier of investments, worth $7.5m or 75% of total investment. This may be an investment by the Government in establishing a market and track record for social impact bonds, rather than a model we can expect to see replicated in future years. The $7.5m of the bond with protected capital will also accrue variable returns up to 10%, dependent on outcomes. An additional $2.5m high-risk tier of investment will involve capital at risk, but will offer outcome-dependent returns up to 30%.

Media Coverage

Date Author Publisher Title
14/06/2013 Benevolent Society First charity issues Social Benefit Bond with Westpac and CBA
14/06/2013 Bela Moore Super Review Westpac Institutional and Commonwealth team up to launch social benefit bond
14/06/2013 James Fernyhough Financial Standard Big banks move into SBB space
14/06/2013 Australian Financial Review NSW strikes deal for $10m social bond
14/06/2013 Westpac First of its kind Social Benefit Bond supports efforts to keep families together – paves the way for socially responsible investors
15/06/2013 Clancy Yeates The Age / Brisbane Times Lenders back bond to keep children out of foster care
18/06/2013 Pro Bono Charity Issues Bank-Backed Social Benefit Bond
19/06/2013 Rick Morton The Australian Banks pitch in for $10m bond
4/10/2013 Rachel Alembakis The Sustainability Report Westpac, CBA raise $10 million for Benevolent Society SBB
5/10/2013 Sally Rose The Australian Financial Review NRMA and Australian Ethical buy Benevolent Society bond

This is the second of three social benefit bonds that are the product of a request for proposal issued by the NSW Government in September 2011 and awarded March 2012. The first was the Newpin social benefit bond announced in March and the final bond to reduce adult reoffending is still under development by the Government and Mission Australia, a charity service provider.

(Updated 8 October 2013.)

Making the economic case to government


Public agencies commissioning social impact bonds or other payment by results programmes want to see some kind of cost benefit analysis. But they might not always be so willing to provide the information an external organisation needs accurately estimate the benefit side. Different commissioners also have different requirements for cashable savings – for some it’s a key driver and for others it’s not a consideration.

I suggest that collecting benefits into the following five categories makes the information for the commissioner much clearer and, for UK commissioners, also explicitly addresses the requirements of the Social Value Act. All estimated amounts should be itemised and if an external applicant is unsure of the savings or benefits to public service agencies, then they should present a best estimate that prompts the commissioner to provide a more accurate figure. The Global Value Exchange is a database of proxy values that will be helpful for this.

Estimated amount
Cashable savings to commissioner(s)
Cashable savings to other public agencies
Non-cashable benefits / efficiency savings to commissioner(s)
Non-cashable benefits / efficiency savings to other public agencies
Savings/benefit to other stakeholders (social value)
Total economic benefit  

NSW Newpin social benefit bond – returns to investors


(Social Ventures Australia: Newpin Social Benefit Bond)

The NSW social benefit bond (social impact bond) for UnitingCare Burnside’s Newpin programme is attracting interest from a range of investors, including NGS Super – the first time we’ve seen a pension/superannuation fund sign up to a social impact bond. The Newpin programme is for families with children aged 0-5 and results are measured on the proportion of children in out-of-home care (which includes foster care, institutional care and placement with extended family) that are returned to their families by the courts. This is called the restoration rate.

Eureka Report’s A high-yield bond with social benefits recently revealed the social and financial returns over the seven year social benefit bond as follows:

Restoration rate (r) Return to investor (IRR)
≥ 70% 15%
65% ≤ r < 70% 12%
60% ≤ r < 65% 7.5%
55% ≤ r < 60% 3%
•minimum 5% yield over first three years
•no minimum yield after three years
•75% of capital returned if bond redeemed at four years
•50% capital returned if redeemed after four years

The Newpin restoration rates were 74.5% last year. Their approach to this social benefit bond, to measure their results for a similar cohort for the year prior to the bond, gives social investors information with which to judge the social and financial risk of the investors. Providing this information attracts investors beyond the die-hard philanthropists who have backed the programme from the start.

Funds were raised from 59 investors by Social Ventures Australia (SVA) with a minimum investment of AU$50,000. The SIB was oversubscribed. Below is a breakdown of investment by investor categories as presented by Ian Learmonth, Executive Director, Social Ventures Australia at the 2013 Social Finance Forum in Sydney/Australia.

NewPin SIB investors

Three examples of public engagement in policy development

(from www.socialbrite.org)

The Social Impact Bond Knowledge Box I produced for the Centre for Social Impact Bonds at the Cabinet Office (UK) includes contributions from many external individual contributors. It provides opportunity for users to comment on or suggest edits for each page and submit links to their own work in order to keep the resource up-to-date. It was inspired by these three examples of policy interactions with the public:

1. The White House has a petition site called ‘We the People’ that promises a response to petitions that raise over 25,000 signatures. In 2012, a petition to “Secure resources and funding, and begin construction of a Death Star by 2016” received the requisite signatures, and the White House used their humorous response to promote domestic innovations and the study of science, technology, engineering and maths.

2. In November last year the Cabinet Office (UK) launched an on-line consultation seeking public views on the new datasets code of practice that will be issued under Section 45 of FOIA. It set out the amendment with a comment box under each clause, inviting the public to make comments or specific edits. While the consultation helped the team refine the amendment in light of users’ responses, it was also intended as an educational exercise, to “make public authorities aware of their new responsibilities under the new FOIA ‘datasets’ sections.” The consultation is now closed.

3. The London Assembly encourages the public to take part in their processes by:

  • Participating in consultations
  • Suggesting something to investigate
  • Putting a question to the Mayor
  • Asking an Assembly Member to present a petition on their behalf
  • Subscribing to their monthly ezine.

Check out Govloop’s Citizen Engagement Hub that’s packed with resource links and their guide Innovating at the Point of Citizen Engagement: Making Every Moment Count which includes seven inspirational stories.

What other examples are there of government crowd-sourcing their policy development?

Randomised controlled trials (RCTs) in public policy


The basic design of a randomised controlled trial (RCT), illustrated with a test of a new ʻback to workʼ programme (Haynes et. al, 2012, p.4).

In 2012, Laura Haynes, Owain Service, Ben Goldacre & David Torgerson wrote the fantastic paper Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. They begin the paper by making the case for RCTs with the following four points.

1.We don’t necessarily know ‘what works’ – “confident predictions about policy made by experts often turn out to be incorrect. RCTs have demonstrated that interventions which were designed to be effective were in fact not”

2. RCTs don’t have to cost a lot of money – “The costs of an RCT depend on how it is designed: with planning, they can be cheaper than other forms of evaluation.”

3. There are ethical advantages to using RCTs – “Sometimes people object to RCTs in public policy on the grounds that it is unethical to withhold a new intervention from people who could benefit from it.” “If anything, a phased introduction in the context of an RCT is more ethical, because it generates new high quality information that may help to  demonstrate that an intervention is cost effective.”

4. RCTs do not have to be complicated or difficult to run – “It is much more efficient to put a smaller amount of effort [than a post-intervention impact evaluation] into the design of an RCT before a policy is implemented.”

Laura and her team are making a huge difference to the way the UK Government perceives and implements RCTs.

The World Bank has also published some fantastic guidance in their  Impact Evaluation OverviewThis includes information abou their  Development Impact Evaluation (DIME) initiative that has the following objectives:

  • “To increase the number of Bank projects with impact evaluation components;
  • To increase staff capacity to design and carry out such evaluations;
  • To build a process of systematic learning based on effective development interventions with lessons learned from completed evaluations.”

I’ve popped both these resources on the Social Impact Bond Knowledge Box page Comparisons and the counterfactual, but thought they were so valuable it was worth expanding on them here.

Start a mistakes log

mistakes“No one is exempt from the rule that learning occurs through recognition of error.” Alexander Lowen, Bioenergetics

There’s too many lessons we’re missing out on because of our tendency to only publish good results. It’s perfectly understandable to want to promote wins, but publishing mistakes and what’s been learned from the them may be even more valuable.

Ben Goldacre is crusading against publication bias in evidence based medicine. He is one of the forces behind http://www.alltrials.net/, an online petition to get all medical trials registered and subsequently all results reported. This is important stuff.

But apart from medicine, those of us involved in designing and delivering social programmes continue to repeat the mistakes of the past, because we simply don’t know enough about what has happened. I’m a strong believer in evidence-based policy, but evidence of policy history and why things failed is rarely captured and shared. Might it be possible for us to value mistakes enough to create incentives for their publication?

Curt Rosengren writes in his blog, the genius of mistakes:

You might even try keeping a mistake genius journal. Not a place for you to berate yourself for how many mistakes you make, but a place for you to actively learn from what has happened. Explore the mistake, explore what insights you’ve gained as a result, and summarize those insights into key points.

One organisation that’s created a ‘mistakes genius journal’ is Givewell in the US, with a section on their website, Our Shortcomings, logging their mistakes and what they’ve done in response. My opinion of the organisation was heightened by this discovery and I thought that this honest recognition and promotion of continuous improvement might have had the opposite effect most would expect from publishing their mistakes. Yes, we’re all worried about tabloid headlines, but wouldn’t it be a little less exciting when it’s not a secret ‘uncovered’, but a quote from the source straight off their public website. Imagine how wonderful it would be if governments and service providers kept similar logs!

As we try to design new services and financial products to address entrenched problems in this emerging social investment market, it would be really valuable to know what didn’t work out for others and most importantly, what they changed in response.

Allia recently showed an exemplary commitment to learning following the closure of their Future for Children Bond, which was the first opportunity for retail investors to invest a proportion of funds in a social impact bond, but failed to raise sufficient capital.

As a first pilot product, the Future for Children Bond has nevertheless been hugely valuable in assessing the retail market for social investment and generating learning about the steps needed to enable it to grow. These lessons will be used to inform the development of future Allia products and will be shared with the sector, together with policy recommendations, in a report by NPC to be published in May.

So here’s to seeing a whole lot more mistakes logs and lessons learned appearing in the public domain – great PR and enhanced social impact – what is there not to like?

Did you get what you came for? Realising social impact bond objectives

shopping basketIn a previous blog, Do SIBs Work?, I listed 22 objectives that have been given for pursuing SIBs as they have been announced around the world. Now I want to examine the achievability (is that even a word) of each one. Some objectives are achieved just by signing the contract to deliver a SIB, but some objectives rest on longer-term outcomes and these are the more interesting. But the conditions of achieving outcomes need to be built into SIBs from the start. If we’re after transparency, then we need to publish terms and results. If we want to shift funding to prevention, there must be a plan to continue funding the SIB preventative service after the SIB is over.

Better programs and better results for the people who participate in them
1. Improving results for beneficiaries by focusing on outcomes rather than outputs

I’d say this is the objective most likely to be achieved. Peterborough doesn’t have its results in, but word on the ground is that providers feel the focus on outcomes has made them deliver better services and participants enjoy the holistic and seamless approach. This may be an objective more easily achieved where payments are based on a single outcome, like Peterborough or Essex, than multiple outcomes like the DWP Innovation Pilots.

2. Improve the likelihood of delivering real and sustainable solutions to important social challenges

For this to happen you need a funding to be sustained – there hasn’t been a whole lot of talk so far of what happens after the SIB, but if the solutions are to be sustained, they’re going to need some continuity of funding. I think the services we’re seeing are real solutions, but we can’t think that solving a problem means it goes away. Solving social problems mean they go away for some of the people for some or all of the time – bettering a situation is noble – thinking you’re going to eradicate is is naive.

3. Making effective interventions available to far more people in need than the number that can be reached through traditional state contracts and philanthropy

This will be achieved as long as SIBs are delivering additional services to those who would otherwise have received little to nothing.

4. Harnessing the innovation capacity of both investors and service providers for publicly funded services

We’re unlikely to get any innovation if the same providers are delivering the same proven programmes – the objective will be achieved if there is room to improve and real changes are made on the ground. Peterborough is certainly delivering on this- not in any of the components of service for participants, but in the consistent and collaborative way they are delivered, so that the impact on their lives is real and sustained.

5. Adding discipline to measuring outcomes for government programs because there is an upfront agreement on how to measure success

This depends how much discipline you had about measuring outcomes before and how much political will there is to improve measurement in government – this may be more applicable to an Australian context where increased use of evidence-based policy is very much on the agenda – Peterborough has certainly introduced a measurement system that was then used in a number of other pilots, but it looks like the new Ministry of Justice probation reforms will revert to the less informative binary measure as it’s perceived as more favourable to Ministers and the public.

6. Improving the evidence base for social services, by mandating measurement and publication of outcomes

Only achieved if the results of the SIB are published along with full evaluations of the programmes that achieved them, ideally including follow-up measurement after the SIB is over. 

7. Accelerating the adoption and implementation of promising programs

Dependent on the programme chosen – was it promising? In all SIBs so far I would say yes.

8. Accelerating the expansion of evidence-based programs delivered by effective nonprofits

This is very much the objective that seems to have been focussed on in the US. You’d have to look at the expansion of programmes like Multi-Systemic Therapy (Essex SIB) and Moral Reconation Therapy (New York SIB) to see how the inclusion of the programmes in these SIBs impacted their expansion rates.

A social finance market
9. Unlocking funds to tackle social issues

We’ve certainly seen new funds committed by Government, and the involvement of an investor like Goldman Sachs suggests new funds (unless it comes from their corporate social responsibility budget).

10. Growing the social finance and social business sector

SIBs have certainly grown interest in the social finance sector and have become a bit of a poster-child. This has occurred at the same time as rapid growth in the social finance and social business sector, so I think it’s fair to assume an element of causal relationship.

11. Providing new financial instruments to harness private investment for the benefit of the community

This is achieved the minute the contract is signed. But in all honesty, if investors don’t receive their capital back, there won’t be any more and the instrument will no longer be viable.

12. Enabling investors to achieve financial returns and social impact

SIBs will either achieve both financial returns and social impact or neither, so this objective is pretty achievable. Some might argue that the measurement of impact is a little on the weak side for some SIBs, but if the measurement system is assumed to be a good indicator of social impact, then this is highly achievable.

13. Increasing funding for prevention and early intervention programs in a sustainable manner

This objective will only be achieved if funding is sustained, but we should all be asking the question of commissioners “What happens when your SIB is finished?”

14. Improving accountability and transparency for publicly funded services

Depends on publication of terms and results – some current SIBs are achieving this better than others – if it’s an objective stated at the start, the public should expect and demand publication of information

15. Allowing governments to accept and measure new ideas from external providers, only paying for the ones that deliver

This is difficult and unlikely to be achieved in the current market. Government’s ability to accept new ideas from the market is often limited by procurement rules, but the DWP innovation SIBs certainly allow 10 new ideas to be tested and compared. Only paying for the ones that deliver? This occurs in the UK, but not so much in Australia where investors stand to receive between 50% and 75% of their funds if services deliver no results at all.

16. Saving Government money

There is no doubt that there is a multitude of benefits to government from being involved in a SIB, but money in the hand is one of the least achievable, unless the services that will experience reduced demand are spot-purchased (i.e. purchased as units of service in direct response to demand). Steve Goldberg defines this as an accounting problem, rather than a savings problem, but governments are constrained within their accounting systems and are thus unlikely to cut anyone’s budget to pay for a SIB. The only SIB we’ve seen where savings resulting from a SIB intervention are used to repay investors is the Essex SIB , although these savings will not cover the cost of the SIB.

17. Lowering risk for government

Financial risk may be lower, but due to media attention, the reputational risk of programme failure is extraordinarily high. As a result, the due diligence undertaken by governments mitigates most of the risks of SIBs before they enter.

18. Savings can be recaptured and reinvested into a permanent funding stream for the program

Highly unlikely – see ‘Saving Government money’ above.

Service providers
19. Increasing accessibility of payment by results contracts

This is a UK objective – in the US many not-for-profits are on these kinds of contracts and in Australia there is a dominance of not-for-profits delivering government contracts for social services, despite there being few payment by results contracts. In the UK, it seems that this objective has been achieved for the SIBs introduced, but the test of whether it opens up broader payment by results contracts will be the upcoming rehabilitation reforms.

20. Giving nonprofit providers a committed, long-term funding stream not subject to budget cuts

This is achieved when contracts are signed and seems a significant contribution of SIBs for providers.

21. Aligning the interests of beneficiaries, nonprofit service providers, private investors, and governments

The SIB contract development process certainly achieves this. The only risk is that some voices around the table are louder than others and some interests are therefore maligned, but that consideration would hold over the development of any contracting arrangement.

22. Facilitated coordination with organisations working on overlapping problems

We’ve seen this demonstrated very well in Peterborough, but it will depend on the delivery structure. In New South Wales, Australia, the government has contracted for results directly with UnitingCare Burnside, a service provider, so they are not mandated to work with other delivery organisations.

A world of SIBs (social impact bonds)

In the tropicsworld
I was a bit late picking up this post in the Stanford Social Innovation Review by Michael Belinsky & Sebastian Chaskel last November, Designing a Social Impact Bond in the Tropics. They are working with the government of the state of Antioquia in Colombia and could bring us the first SIB from Latin America. They highlight the following:

  1. The ups and downs of legal innovation – the differences in regulatory environments that SIBs are being implemented in
  2. Savings are unlikely to significantly offset the cost of the program – this is becoming more widely accepted now. The concept of savings from prevention flowing back into a system that can pay for itself makes sense in abstract, but it’s more likely governments will be interested in purchasing better outcomes than reconfiguring their accounting systems to pick up ‘savings’
  3. Significant opportunity for development agencies – something we’re hearing more of and certainly development has a stronger history of innovative financial mechanisms than public services – in fact the SIB idea was seeded by Arthur Wood’s work in finance mechanisms for sanitation programmes
  4. Opportunity to shake up service providers – this is much more attractive in some places than others – the article talks about transferring responsibility to intermediaries, but some governments are considering a SIB as a way to reinvigorate and challenge their long-term providers to keen innovating
  5. The necessity of multitasking – this is about the multitude of skills needed to develop a SIB. The legal, statistical, financial and operational demands of developing these arrangements are immense, but I think that’s why everyone gets so excited about SIBs – they’re really challenging to work on and demand the building of capacity from all those involved.

In Mozambique

Interesting article Social impact bonds gear up for Mozambique from Dave Mayers recently. The most interesting variant of this idea is the use of a non-government commissioner, mining companies, to pay for improvements in malaria rates. The argument is that the benefit to their workforce from effective malaria programmes justifies the investment – keep watching!

In South Korea

Park Ki-yong writes about a “social impact bond-inspired incentive” programme of Public-private sector partnership to prevent suicide in over 65-yr-olds. The programme doesn’t fit with my definition of a social impact bond because it doesn’t have external investors, so I’d just call it a performance-based contract. It’s interesting because:

  • it uses competition between providers as a measure of results,
  • it has a 10% bonus or 5% penalty based on results – not a whole lot, but certainly a logical way to drive performance
  • the justification for not involving external investors was the inability of government to guarantee payments – I’m not sure what this means exactly, but it would be interesting to hear the expanded story about this.