On a recent trip to the US, I noticed that the discussions around ‘Pay for Success’ were a little different to those I’d been having on ‘Social Impact Bonds (SIBs)’ with other countries. Particularly in the measurement community, there was an idea that Pay for Success took measurement of social programs to a new level: that ‘Pay for Success’ meant paying for an effect size (by comparison to a control group), rather than ‘Pay for Performance’ which paid for the number of times something occurred. The media release for the recent Social Impact Bond Act mentioned both Pay for Success and Pay for Performance. I have not seen a definition of social impact bond that specifies a type of measurement. A classification of measurement types can be seen in the diagram below:

Some social impact bonds use one type of measurement to trigger payment from government and another to trigger payment to investors (e.g. Newpin Social Benefit Bond, NSW Australia). In this discussion, we focus on the measurement that triggers payment from government (referred to as ‘the commissioner’ in the UK).

The diagram below classifies existing social impact bonds into pay for performance and pay for success. Comparable language used in the UK and Australia is also suggested.

Some situations will suite some types of measurement more than others. The key considerations are:

1. how well a measurement system is able to capture the change that can be attributed to the intervention. Attribution means how much of the outcome was caused by the intervention and how much was caused by the contribution of other organisations or people.
2. what incentives are created by the measurement system. A good measurement system will encourage service provision in the manner desired by the government/commissioner.

For example, in the ‘It’s All About Me’ adoption SIB, councils refer children (and make a registration payment) when they deem them too difficult to place with their own local system. This means they have already judged the children unlikely to be adopted otherwise. The council pays at four performance milestones as they happen, rather in comparison to some other child. There is a milestone payment at one and two years, with the two year payment worth about 25% of the total possible payments. This creates an incentive for providers to place children appropriately and support families and children after placement.

Some social impact bonds don’t fit into the above diagram

• Essex children at risk of going into care – results are compared to a historical baseline
• London homeless people – two outcomes are compared to a historical baseline and three have no comparison
• New South Wales Newpin families with children in care – the comparison outcome is fixed for the first three years as a control group is established, and then the control group is used
• Rotterdam NEETs – targets are set based on historical figures

Other considerations for measurement systems

• A historical baseline that has been flat for decades is a stronger comparison than a volatile baseline, and in some situations may be sufficient
• More rigorous evaluation might occur after all payments are made e.g. the Department of Work and Pensions Innovation Fund
• Some programs come with more of an evidence base and some are more innovative
• Some programs require that a service be delivered to everyone that is referred – these are more likely to fall in the ‘pay for performance’ category. For example, in the UK adoption SIB, local authorities refer individual children to the service, so expect that some effort is made to deliver each of them an outcome, which is reflected in a payment that is made at the time of referral. In contrast, in the Peterborough SIB, all male, short-sentenced prisoners released from HMP Peterborough are eligible, but it is the responsibility of the service provider to engage those that they think are more likely to reoffend, and focus their services towards the most frequent offenders.

Why is there more of an emphasis on this split in the US?

The US has a longer and more extensive history of performance-based contracting than most countries, with employment services paid on a performance basis since 1982. It is now estimated that over 80% of public human service funding is delivered through not-for-profits and in child welfare services, over half of this is performance-based.

In the UK and Australia, the idea of performance-based payments in employment services is fairly well established, but in other human services it is relatively novel. The idea of payments based on measures of performance (and subsequent incentives for the provider) has been innovative and difficult to develop in both countries.