The Justice Data Lab – an overview

MoJ Data LabWhat is the Justice Data Lab?

The Justice Data Lab allows non-government organisations to compare the reoffending of the participants in their programmes with the reoffending of other similar ex-offenders. It “will allow them to understand their specific impact in reducing re-offending… providing easy access to high-quality re-offending information” (Ministry of Justice, Justice Data Lab User Journey p.10). There is no charge to organisations that use the Justice Data Lab.

The Justice Data Lab is a pilot run by the Ministry of Justice. The pilot began in April 2013. Each month, summaries of results and data are published, including Forest plots of all results so far.

Who might use it?

The Justice Data Lab can be used by “organisations that genuinely work with offenders” (Justice Data Lab User Journey p.11). One request will provide evidence of a programme’s effect on its service users’ reoffending. Several requests could compare services within an organisation or over time to answer more sophisticated questions about what is more effective.

This information could be used by non-government organisation for internal programme improvements, to report impact to stakeholders or to bid for contracts. It was set up at the time the Ministry of Justice’s Transforming Rehabilitation Programme was encouraging bids from voluntary and community sector organisations to deliver services to reduce reoffending.

What are the inputs?

Input data are required to identify the service users from a specific program and match them with a comparison group. Information on at least 60 service users is required and the organisation must have worked with the offender between 2002 and 2010.

Essential:

  • Surname
  • Forename
  • Date of Birth
  • Gender

At least one of the following:

  • Index Date
  • Conviction Date
  • Intervention Start Date [note: feedback from applicants is that this is required]
  • Intervention End Date [note:feedback from applicants is that this is required]

Highly Desirable: PNC ID and/or Prison Number

Optional: User Reference Fields

What are the outputs?

The one year proven re‐offending rate –  defined as the proportion of offenders in a cohort who commit an offence in a one year follow‐up period which received a court conviction, caution, reprimand or warning during the one year follow‐up or in a further six month waiting period. The one year follow‐up period begins when offenders leave custody or start their probation sentence. A fictional example of the output provided by the Ministry of Justice is quoted below:

The analysis assessed the impact of the Granville Literacy Project (GLP) on re‐ offending. The one year proven re‐offending rate for 72 offenders on the GLP was 35%, compared with 41% for a matched control group of similar offenders. The best estimate for the reduction in re‐offending is 6 percentage points, and we can be confident that the reduction in re‐offending is between 2 and 10 percentage points.
What you can say: The evidence indicates that the GLP reduced re‐offending by between 2 and 10 percentage points.

Publication
Applicants should note the following requirement: “an organisation requesting data through the Justice Data Lab must publish the final report, in full, on the organisation’s website within four months of receiving the final report.”

I’d be very interested in the opinions of applicants on this requirement. Is it an issue? Does it create perverse incentives?

What are the implications?

The implications are huge. Prior to the Justice Data Lab it was very difficult for non-government organisations to establish a comparison group against which to measure their effect. Evaluations of effect are expensive and thus prohibitive, particularly for smaller organisations. In addition, the differences in their methods and definitions meant that evidence was more difficult to interpret and compare.

This is exactly the type of evidence that developers of social impact bonds find so difficult to establish and will be essential to constructing social impact bonds to deliver  Transforming Rehabilitation services. It is a measure of outcome, which is desirable, but often more difficult to quantify than input (e.g. how much money went into the programme), activity (e.g. what services were delivered) or output (e.g. how many people completed the programme).

New Philanthropy Capital (NPC) were involved in designing the Justice Data Lab and their Data for Impact Manager, Tracey Gyateng, is specifically thinking about applications to other policy areas.

How is it going?

See my November 2014 post on information coming out of the Justice Data Lab.

Also note the announcement of an Employment Data Lab by NPC and the Department of Work and Pensions.

More information

Information on the Justice Data Lab home page includes links to a series of useful documents:

  • User journey document – information on what the justice data lab is, and how to use its services.
  • Data upload template – use this template to supply data to the justice data lab. Further descriptions of the variables requested are given, and there are key areas which must be filled in on the specific activities of the organisation in relation to offenders.
  • Methodology paper – this document gives details of the specific methodology used by justice data lab to generate the analysis
  • Privacy impact assessment – this is a detailed analysis of how an organisations’ data will be protected at all stages of a request to the justice data lab
  • Example report template – two examples of a standard report, completed for two fictional organisations showing what will be provided.

Criminal justice service providers might also benefit from getting involved in the Improving Your Evidence project, a partnership between Clinks, NPC and Project Oracle. The project will produce resources and support, so follow the link and let them know what would be of most use. The page also links to an introduction to the Justice Data Lab – a useful explanation of the service.

The bulk of this post has been copied directly from the Ministry of Justice documents listed above. It is intended to act as a summary of these documents for quick digestion by potential users of the Justice Data Lab. The author is not affiliated with the Ministry of Justice and does not claim to represent them.

4 thoughts on “The Justice Data Lab – an overview

  1. Thanks Emma really helpful- been sitting around contemplating this one, just felt like ‘a mountain to climb’ in terms of getting motivated, gonna grasp the nettle now and get on with it.
    Well a member of my team will anyway ! pat:)

  2. Hi Emma,
    Thanks, this is terrific. My only concern on the Justice Data Lab has been the risk of biased input samples. In other words, what is to stop entities simply providing programme completers as their input group, rather than programme starters or people to whom the programme was available but chose not to even start?

    Is it up to discerning funders and other stakeholders to ensure that social organisations do this properly, or the role of the Justice Data Lab to ask?

    Hope you’re trip back down under went ok.

    All best
    Toby

    • Hi Toby!

      Always great to be back on home soil.

      I don’t think it’s the role of the Justice Data Lab to ask how the input sample was defined,but they should communicate the data in a way that encourages users to interpret them correctly. The information they provide is only part of the picture of a service, but I don’t think it’s their responsibility to paint the whole picture. (Although the publication requirement and format of this publication might encourage a comparative league-table interpretation…)

      It’s the age-old argument about whether data should be released when it has the potential to be misinterpreted or misused. I really think it’s a job of those who are communicating data to present them clearly and those who are using them to ask the right questions. There are lots of questions about a service that are important to interpreting the data coming out of the Lab. Perhaps we could do with a brief user’s guide for this kind of data including vital questions like “How have you defined your input group?”. NPC would be great at that – shall we ask them?

      Emma

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s