Bolshy Divas – gathering grass roots voices and stories to effect policy change

August 21, 2012 Emma Tomkinson Evidence-based policy

Nothing communicates a policy change or program evaluation like a story of someone whose life was changed, however when it comes to making decisions based on the evidence, one anecdote isn’t enough. But what if you have lots and lots of anecdotes? Then you have a very strong picture of what life is like for the beneficiaries of policies and services.

The Bolshy Divas are an anonymous grassroots collective formed in Western Australia and have become a national voice on disability reform. They campaign for genuine consultation, jargon-free documents and transparent processes.

My favourite part of Karen Soldatic and Terence Love’s article New forms of disability activism: who on earth are the Bolshy Divas? from Ramp Up on the ABC reads:

Faced with resistance from the WA Government to disability reform, in 2010, they amassed forces and gathered 100 stories of unmet need virtually overnight. These stories were presented to Premiers and the Prime Minister at the COAG meeting when the WA Government wavered on their commitment to a National Disability Insurance Scheme. The WA Premier, Colin Barnett, backflipped and slipped into the ranks of NDIS supporters, leaving WA Government staff and advisors perplexed…

I love the humour and realism of the Bolshy Divas’ campaigns, also their respect for the human qualities of the politicians they target.

Governments in Australia find it difficult to consult effectively with the diverse range of beneficiaries of their services – working with groups like the Bolshy Divas to access community voices would strengthen the evidence available to make and illustrate policy.

12 points of interest in the New York (Rikers) Social Impact Bond

August 19, 2012August 1, 2015 Emma Tomkinson Rikers Island, Social Impact Bonds, Social Innovation, Social Investment & Finance Bloomberg, Goldman Sachs, New York, social impact bonds, US

There was no hype before the Social Impact Bond (SIB) was announced – this is in contrast to other Governments who have announced their intention well before finalising details. Perhaps the deal was developed quickly, which may well be the case with Bloomberg controlling both the philanthropic guarantee and political appetite. Or perhaps the parties didn’t want to announce before they had signed the contracts, avoiding the pressure of having to live up to an announcement of intention.
In addition to a press release, the City of New York released a briefing pack outlining the justification, partners, program and payment terms of the SIB. The terms of Government payment are clearly set out, unlike in the Peterborough SIB. For a new funding mechanism, this certainly promotes understanding.
It’s called a Social Impact Bond- Obama’s administration announced funding for “pay for success” bonds and the NSW Government has branded them Social Benefit Bonds. Using the same term as Peterborough makes it clear that they follow the same model and helps keep related literature together.
Bloomberg Philanthropies guarantees it – they’ve only guaranteed $7.2m of the $9.6m Goldman Sachs investment, but having the guarantee reduces investor risk and may be one of the key factors in getting this deal finalised. It’s not clear under what terms the guarantee is paid or what happens to it if it’s not required. The fact that it’s a grant seems to suggest that MDRC would repurpose it. This also means that like Peterborough, if outcomes are not achieved, there is no payment of public monies.
The Goldman Sachs payment to MDRC is described as a loan – people often ask of SIBs “Are they debt or equity?”. In fact, they are neither, they are multi-party contracts. Payments from Government are due to an outcomes-based contract. The private investor is really buying a futures option. Classification as debt sheds light on how Goldman Sachs will account for this investment. It also may suggest that Goldman Sachs will take a hands-off approach. If this is so, the benefit of investor skills and incentive are not transferred through to the service providers. This transfer has historically been referred to as one of the reasons for involving external investors, but we’ve yet to see a SIB where investors have a relationship with service providers.
Similar to Peterborough, there is a third-party, independent evaluator. This may prove to be an essential feature of all SIBs as they emerge.
The intermediary is MDRC, a research centre – the strengths of MDRC are in program evaluation and design. Intermediaries in other SIBs bring financial investment or service delivery experience to the SIB. In NSW, the intermediary role will be played in part by banks and not-for-profits, as well as by the intermediary Social Finance (no relation to the UK or US Social Finance).
The payment terms state that a 10% reduction in recidivism is the threshold for investors to break even, although when cost of capital is factored in, this will represent a loss. It is not clear what the 10% reduction is in relation to i.e. what the comparison group is. The SIB also uses stepped payment terms, rather than a sliding scale, which is simple and unambiguous.
There is value in being a first mover – the press releases announces the “Nation’s First Social Impact Bond Program”. There is a conflict between the kudos received for being a first-mover and the confidence gained by waiting to see how other SIBs turn out. It will be interesting to see whether SIBs become more popular and mainstream, or whether development of this model will slow once its novelty wears off.
Benefits to nonprofit providers are described in the briefing pack as “a committed funding stream not subject to budget cuts”. The literature that exists on SIBs describes the major benefit to be providers being the flexibility afforded by focusing on outcome, rather than current prescriptive Government contracts. This could be a sign that MDRC will be prescriptive in what it asks of the two providers, or that prescriptive contracts are less of an issue in this jurisdiction.
Is there conflict between innovation and evidence? The briefing pack justifies the SIB as “encouraging innovation in a time of fiscal constraints”. The idea that SIBs encourage innovation by allowing Government to pay only if outcomes are achieved might be one for the long-term market only. At the moment, the reputational risk of failure to all parties and intense media scrutiny has resulted in SIBs providing services with a strong history and evidence-base that may require strict adherence to service models. These services are so safe they would be ideal candidates for a direct outcomes-based contract with a provider. The real innovation occurring here might be in how the delivery partners have to work together towards agreed outcomes.
They’re creating a pipeline – the press release refers to an August 2 request for expression of interest for additional social impact investment projects by the City of New York. This would suggest that a pipeline of proposals will be established, but I haven’t been able to find the request anywhere. It would be interesting to see whether outcomes are suggested and priced, in the manner of the NSW Government, or whether this will be up to the market.

The launch of Accounting for Value is timely for discussions on Australian charities’ financial reporting

August 1, 2012August 1, 2012 Emma Tomkinson Australian Not-for-profit Reform ACNC, charities, Jeremy Nicholls, not-for-profit regulation, SROI Network

The SROI Network has just launched it’s site Accounting for Value where Jeremy Nicholls challenges us to take principles-based accounting back to basics and ask ourselves what we want to account for and how we might agree on the answer.

While the discussion may seem idealistic and somewhat hypothetical, it’s a great basis for charities in Australia to start from when thinking about the financial reports they’ll soon be submitting to the Australian Charities and Not-for-profits Commission. What story do we want to tell, and well how do our current accounting principles and practices deliver it?

Health correlation fail – misinterpreting association as cause!

July 26, 2012July 27, 2012 Emma Tomkinson Evidence-based policy attribution, correlation, health

Gary Taubes had a great rant in March on misinterpreting correlation in health research. He describes the compliance effect, where the “Girl Scouts” in longitudinal health studies do everything they’re told is good for their health. So they are very healthy, but this doesn’t mean that every single thing they do is good for your health. It’s a great point to keep in the back of our minds when we attempt to attribute an effect, particularly in longitudinal studies.

How do you know when outcome change can be attributed to your intervention?

July 25, 2012July 25, 2012 Emma Tomkinson Evidence-based policy, Social Impact Measurement 3ie, international development, payment-by-results, performance-based contracting, Social Benefit Bonds, social impact bonds, statistical significance

It was exciting to read the new working paper Addressing attribution of cause and effect in small n impact evaluations: towards an integrated framework by Howard White and Daniel Phillips. While it would be ideal that all the interventions we design would have the number of participants (n) and impact that would give us a statistically significant result at a high level of confidence, there are many reasons that this doesn’t happen. For payment-by-results contracts, in particular social impact bonds, attributing an impact to an intervention is a pre-requisite for the transfer of public funds. Also, funders the world over are attempting to identify the impact they are making across their portfolios, to increase the effectiveness of their investments. White and Phillips produce a fantastic summary of methods and examples that seek to attribute change to a cause. While their framework of small n methods is useful, it’s their up-to-date literature review that I find most useful.

The paper is published by 3ie: International Initiative for Impact Evaluation. 3ie have developed a database of policy briefs, impact evaluations and systematic reviews. They’re governed and staffed by a trans-global team, and while focussed on international development, their evaluation work is certainly relevant for interventions that alleviate disadvantage at a local or national level.