Social Impact Bonds /
Pay for Success:
Pros, Cons, Promises
and Risks
Kelly Walsh, PhD
Justice Policy Center – Urban
Institute
October 7, 2014
1:00PM ET
John K. Roman
Justice Policy Center – Urban
Institute
J U S T I C E
P O L I C Y
Kyle McKay
Texas Legislative Budget Board
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
• Leads a national movement
• State-based juvenile justice coalitions
and organizations
• Laws, policies and practices that are
fair, equitable and developmentally
appropriate for all children, youth and
families
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Photo: Moriza
Pay for Success:
Funding the Infrastructure for Evidence-Based Change
Kelly Walsh, PhD
J U S T I C E
P O L I C Y
C E N T E R
Washington, DC
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
What is Pay for Success?
•
New financial instrument to support discretionary social programming
o Brings evidence-based programs to the necessary scale
•
Can support programs that
o Involve significant start-up costs, but provide long-term savings
o Serve large populations
o Involve political or programmatic risks
•
PFS addresses these programs with private capital
o Offers an investment return if the program meets performance goals
o Government only pays for successful outcomes
•
Sometimes called “social impact bonds”
o PFS does not operate like a bond
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
4
How Does Pay for Success Work?
•
An intermediary selects an evidence-based program
•
The program is supported with private capital solicited by the
intermediary
•
The intermediary
o oversees program implementation and
o ensures model fidelity
•
An independent evaluator determines if the program has met
performance targets
•
Government pays the intermediary for successful outcome
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
5
The Structure of Pay for Success
Intermediary
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
6
Advantages and Disadvantages
Advantages
Drawbacks
•
Transfers risk away from the
government
•
Requires significant expertise from
multiple fields
•
Develops or scales local service
networks
•
Complicates execution
•
Could limit nonprofit innovation by
focusing on programs with a proven
track record
•
May reallocate existing impact and
philanthropic capital rather than
drawing new capital
•
Addresses the “wrong pockets”
problem
•
Allows agencies to pool resources
and knowledge
•
Provides significant flexibility to
deploy programs across
jurisdictions or regions
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
7
Building PFS in 5 Steps
• PFS projects should target justice system
“inefficiencies” (e.g., prison population
drivers)
• Once drivers are identified, PFS is
developed in 5 steps
Price the
product
Strategic
Planning
1
Develop
infrastructure
2
3
Make the
deal
J U S T I C E
P O L I C Y
Evaluate the
program
4
5
Deliver service
and TTA
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
8
PFS in the US
The US PFS market:
almost $100 million in total investment
Youth and the CJS:
• $ 9.6 million invested in New York City Rikers Island (2012)
• Adolescent Behavioral Learning Experience (ABLE) Program
• $27 million invested in Massachusetts’s Roca, Inc. (2014)
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
9
Pay for Success:
The Promise of Scale Finance
John K. Roman, PhD
J U S T I C E
P O L I C Y
C E N T E R
Washington, DC
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Strategic Planning
Identify target
population
1
2
Assess PFS
Suitability
Price the
Product
4
1
3
Find
evidencebased
solutions
J U S T I C E
Develop
Infrastructure
2
Make the
Deal
P O L I C Y
3
4
5
Product Development
Strategic Planning
Identify
population
and cost
drivers
Evaluate
the
Program
Deliver Service
and TTA
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
11
Strategic Planning
Identify target
population
1
2
Assess PFS
Suitability
Price the
Product
4
1
3
Find
evidencebased
solutions
J U S T I C E
Develop
Infrastructure
2
Make the
Deal
P O L I C Y
3
4
5
Product Development
Strategic Planning
Identify
population
and cost
drivers
Evaluate
the
Program
Deliver Service
and TTA
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
12
Strategic Planning
• PFS relies on good strategic planning to maximize cost effectiveness
• Before starting a program, governments should do the following:
o
o
o
o
Identify population and cost drivers
Identify the problems that generate these drivers
Find evidence-based solutions to the problems
Determine if the evidence-based solutions are PFS-compatible
• Governments should identify a research partner to assist
with strategic planning
• Justice program databases can inform implementation:
o
o
o
o
o
o
Office of Justice Programs’ Crime Solutions
Washington State Institute for Public Policy (WSIPP)
Vanderbilt University Peabody Research Institute
Blueprints for Violence Prevention
The Urban Institute’s Meta cost-benefit analyses for District of Columbia Crime Policy
Institute (DCPI)
National Reentry Resource Center’s What Works in Reentry Clearinghouse
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
13
PFS Opportunities in Juvenile Justice
• Juvenile justice
o Family-based intervention and prevention programs can
prevent delinquent conduct and reduce recidivism
 Implementing these programs requires start-up investment to
train local staff
• Adolescent diversion
o Adolescent diversion from secure confinement has been found
to have a large cost-benefit ratio
 Training and project implementation impose significant
upfront costs
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
14
Scale of the Problem
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
15
Strategic Planning – Likely Results
From Steven H. Goldberg, Caffeinated Capital
• Drivers of costs and populations: states and counties spend
– $5.7 billion every year taking legal and physical custody of more than
60,000 juveniles adjudicated delinquent
• Early-intervention programs are under-utilized:
– Could reduce juvenile incarcerations by 70%
– At a cost that is 80% less than confinement
– Fewer than 5% of eligible families benefit from these programs.
• Potential for enormous cost savings:
– Family-Based therapies cost less than $15,000 per family
– Compared to an average out-of-home placement cost of $88,000 per
youth per year.
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
16
Florida Redirection as a National
Model for Scaled Implementation
Florida Redirection was managed by EvidenceBased Associates (EBA). From 2004-2013:
 Florida paid EBA a total of $65.5 million to
manage the delivery of three intensive family
therapy programs:
 Multisystemic Systemic Therapy® (MST),
 Functional Family Therapy (FFT), and
 Brief Strategic Family Therapy (BSFT)
 More than 10,000 families in 18 of the state’s 20
judicial districts we served.
Source:
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
17
How Cost-Effective are MST FamilyBased Therapies?
Source: Washington State Institute for Public Policy
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
18
PFS Opportunities in Juvenile Justice
Source: National Center for Justice Planning
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
19
PFS Opportunities in Juvenile
Justice: from Caffeinated Capital
Source: Social Finance, UK
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
20
PFS Opportunities in Juvenile Justice
Source: National Center for Justice Planning
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
21
Future of PFS
• PFS efficacy depends on maintaining a
research emphasis
• Broad adoption of PFS will be a
learning process
o Governments, for-profits, and nonprofits learn
how to collaborate
o PFS offers a new instrument for sharing
resources and benefits
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
22
Comments?
Questions?
Email:
Jroman@Urban.org
Twitter:
@JohnKRoman
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Social Impact Bonds:
The cost of complexity
Kyle McKay
Presentation to National
Juvenile Justice Network,
October 7, 2014
All opinions expressed below are solely the author’s and should not be attributed to any of the
individuals or organizations with which McKay is associated.
How complex are social impact bonds?
The idea of paying for success sounds simple.
However, even basic performance-based contracts
without investors can become unwieldy.
Social impact bonds (SIBs) are particularly complex.
• There is “uncertainty arising from defining outcomes,
estimating the potential impact of interventions, measuring
and attributing change, valuing benefits, demonstrating a fiscal
return and getting interventions to scale. …The weakness of the
chain arises not from its weakest link but from the sum total of
the weakness of every link.” Fox and Albertson, Journal of
Criminology and Criminal Justice
How complex are social impact bonds?
“Complexity in some instances meant that the actual transfer of risk is not clear”
– RAND Europe
How complex are social impact bonds?

In Peterborough, “key staff who deliver services to cohort
members include:
 6 full-time caseworkers provided by St. Giles Trust
 Fifty lay volunteers recruited, trained and managed by
Sova.
 Up to six St Giles Trust volunteers (individuals who
may be currently on probation or have personal
experience of the criminal justice system).
 Two, part-time family specialist practitioners from
Ormiston Children and Families Trust.
 One part-time recovery worker from Mind.
 One trainer from John Laing Training who delivers a
construction skills course.”
Source: RAND Europe
Can SIB borrowing grow the pie?
Commercial investors likely to seek secured source of income
State law and fiscal policy can create non-appropriation risks
Appropriations for payments were made in advance of any
potential payments in:
• Massachusetts
• Utah
• Peterborough
In NYC, Bloomberg Philanthropies assumed the liability of the
government by providing $7.2 million to “be held by MDRC in
a guarantee fund to back the loan”
Can SIB borrowing grow the pie?
Legislative budget offices:
• California Legislative Office analysis of Assembly Bill 1920
found “Unknown, major future cost pressure potentially in the
millions of dollars to fund [SIB] contracts”
• Maryland Department of Legislative Services fiscal note on
House Bill 517: “It is assumed that the state will at least
initially have to provide upfront funding for some or all of the
SIB contract costs” and “general fund expenditures may
increase.”
• NJ fiscal note on Assembly 3289 “If there are no savings
realized by the departments and no repayments made to the
lenders, the authority may be responsible for the full
$15,000,000”
Can SIB program outcomes grow the pie?
In Peterborough, payment amounts were set “on the basis
that the SIB was innovative.” –RAND Europe
“The Peterborough SIB is too small to deliver substantial
‘cashable’ savings (monetised benefits)” –RAND Europe
“There is no guarantee that a reduction in re-offending
will reduce demand for prison places.” –Fox and
Albertson, Journal of Criminology and Criminal Justice
“This tool is a more expensive way to scale programs than
if government simply contracted directly with a service
provider.” –McKinsey & Company
What kind of innovation can we expect
from investor selected program?
“Profit-seeking investors will be most interested in social
programs or models that are proven.” –MDRC Staff
“Investors appear to be sticking to models that have already
been extensively evaluated.” –CLASP
Surveys of impact investors:
• “Lack of successful investments” was the most commonly cited
challenge to impact investment –2011 JP Morgan Survey
• 62% of respondents would sacrifice financial returns for greater
impact, but 60% thought this was not necessary –2011 JP Morgan
Survey
• “65 percent principally target ‘market rate returns.’” There was
also a “clear client interest in risk-mitigating features.” –2013 JP
Morgan Survey
How confident can we be in the results?
“Interventions have to be delivered within systems...
When a program achieves positive results, the success
may be attributable to a wide range of factors, and it is
often difficult to identify exactly why the program
worked.” –MDRC staff
“The political imperative to demonstrate the success of
this new financing scheme can create incentives for
weaker evaluation designs that are more likely to show
positive results, but that are spurious.” –MDRC staff
How confident can we be in the results?
“Overall, the results suggested that those released from HMP
Peterborough were of lower risk of reoffending than those released
from other prisons.” –QinetiQ, University of Leicester
“It is debatable whether the use of PSM in the current context
violates the CIA assumption. This is because data held on the Police
National Computer (PNC) are not very useful in controlling for
selection of being released from HMP Peterborough.” –QinetiQ,
University of Leicester
The Peterborough SIB “does not have a defined intervention model
or theory of change. While this was perceived to facilitate
individualised services, it poses some challenges for evaluation.” –
RAND Europe
Evaluations always have caveats. Value of research accrues over time
with additional evidence.
Summary
“There is currently no conclusive evidence substantiating the success
of this model.” –Austin/Travis County Reentry Roundtable, Feb
2014
Extensive evidence in history of performance-based contracting
suggests a number of significant challenges and risks.
Governments and stakeholders should carefully evaluate not only the
intended benefits but the risks and opportunity cost of the model
before starting a pilot.
• Comparison should be between direct financing and operations via public
agencies or contracts, rather than “no intervention.” SIBs are a means to
implementing a program and should be evaluated compared to other
means of implementation.
Important to remember that no program is a policy unto itself.
Resources
“Social Impact Bonds: Overview and Considerations,” Elizabeth
Lower-Basch, CLASP
Massachusetts Contracts & SIB RFIs maintained by the Harvard
Social Impact Bond Lab
“Evaluating Social Impact Bonds as a New Reentry Financing
Mechanism: A Case Study on Reentry Programming in Maryland, “
Maryland Department of Legislative Services
“Payment by results and social impact bonds in the criminal justice
sector: New challenges for the concept of evidence-based policy?”
Chris Fox and Kevin Albertson, Journal of Criminology and Criminal
Justice
Kelly Walsh, PhD
kwalsh@urban.org
John K. Roman
jroman@urban.org
Kyle McKay
kyle.mckay@lbb.state.tx.us
Keshia Cheeks
Cheeks@njjn.org
J U S T I C E
P O L I C Y
C E N T E R
The views expressed are those of the authors and should not be attributed to the Urban Institute, its trustees, or its funders.
Photo: Moriza