Cost-Benefit Analysis (CBA)
Illustrated by the California
GAIN Random Assignment
Experiment
Plan of Discussion
• What is Cost-Benefit Analysis?
• How is it done for social programs?
• How are the costs of social programs
estimated?
• (If time permits) what are some alternatives
to CBA?
What Is Cost-Benefit Analysis (CBA)?
• The key objective is to determine whether the
benefits of a government policy or program or
project exceed its costs
• How does this differ from private sector
investment analysis?
– A broader perspective is used—society as a whole
– It is often more difficult to put values on outcomes
Types of CBAs
• Ex ante
• Ex post
• In Medias Res
Basic Steps in CBA
1. Decide whose benefits and costs count
2. Select the portfolio of alternatives
3. Catalogue (potential) impacts and select
measurement indicators
4. Determine impact quantities over the life
of the project
Basic Steps in CBA (Continued)
5.
6.
7.
8.
9.
Monetize impacts
Chose the social discount rate
Add up the benefits and costs
Perform sensitivity analysis
Recommend the alternative with the
largest net social benefits
An Illustration of the Steps
• GAIN: An innovative welfare-to-work program
for AFDC clients in California during the 1980s
• Evaluated through random assignment in 6
counties by MDRC
• The 6 counties varied considerably in the degree
to which they emphasized a training first (human
capital approach) or a work first approach
• Alameda County was at one end of this spectrum
and Riverside County at the other; but all 6
provided structured job search and training
(1) Decide Whose Benefits and Costs Count
• This is called the “standing” issue
• Who should have standing in the GAIN
CBA—the AFDC beneficiaries, the
government (‘taxpayers’), or both?
• Analysis performed from 3 perspectives
(see handout)
– AFDC Recipients
– Taxpayers (or Government)
– Society (AFDC Recipients + Taxpayers)
(2) Select the Portfolio of Alternatives
• A change in policy (‘the counterfactual’)
must be compared with the existing policy
(‘the status quo’)
• The GAIN CBA compared one alternative to
the status quo in each county by randomly
assigning AFDC clients to 2 groups: a
program and a control group
• However, findings for the 6 counties could
be (and were) compared non-experimentally
(3) Catalogue Potential (Physical) Impacts
and Select Measurement Indicators
• ‘Impacts’ refers to both inputs (costs) and outputs
(benefits)
• Important to list all non-trivial impacts, which isn’t done
in Table 7.5 of handout, which is all in terms of effects
on income
• Beneficial impacts examined in the GAIN CBA included
reductions in transfer payments and increases in tax
payments, increases in earnings and fringe benefits
• Cost impacts included staff time and materials
• Measurement indicators and impacts specified
simultaneously
(4) Determine Impact Quantities
Over the Life of the Project
• This step usually requires predictions of how
long impacts will last and how they vary
over time
• This is often very difficult to do accurately
• EXAMPLES from the GAIN CBA
– earnings
– transfer payments
(5)Attach Money Values to All Impacts
• Very important step
• Must be done to make all impacts
“commensurable,” thereby allowing them to be
summed
• Example from GAIN evaluation
– Staff time
• Sometimes difficult to do
– Value of time spent at work experience job
– Value of scenery
– Value of life
(6) Choose the Social Discount Rate
• Needed because some benefits and costs
occur in the future
• Use of a discount rate makes benefits and
costs that occur in different time periods
commensurable
• It is used to compute present values
• The appropriate rate to use is controversial
• MDRC used 5% for the GAIN CBA
Formula for Net Present Value (NPV)
Bt  Ct
NPV  
t
(
1

r
)
Where
t0

Bt = benefits over time
Ct = costs over time
r = the social discount rate
t = a time period
(GAIN was discounted over 5 years, not infinity)
(7) Sum the Benefits and Costs
• Decision rule if all benefits and costs occur in
the same time period:
– If net benefits = B – C > 0, adopt new program
– If net benefits = B – C < 0, stay with status quo
• Decision rule if benefits and/or costs occur in
different time periods, as in the GAIN
evaluation:
– If NPV = PV(B) – PV(C) > 0, adopt new program
– If NPV = PV(B) – PV(C) < 0, stay with status quo
Bottom Line in GAIN CBA
• In Butte, Riverside, San Diego, the programs
passed the cost-benefit test from all 3 perspectives
• In Los Angeles, the program failed the costbenefit test from all 3 perspectives
• In Alameda and Tulare, the gains to participants
were more than offset by the losses to the
government
• But in Tulare, it was a fairly close call (What
should one make of a close call?)
(8) Perform Sensitivity Tests
• All CBAs require assumptions
• Sensitivity tests involve attempts to see if the
results are robust to alternative assumptions
• This reduces the risks of a faulty assumption
• For example, MDRC examined whether their
results changed much when alternative
assumptions were made about how earnings
and transfer payments change over time (they
didn’t)
(9) Recommendations
• The counter-factual option with the largest net
social benefits should be adopted
• But only if net social benefits are positive
• In GAIN, Riverside clearly had the largest net
social benefits, and they were strongly positive
• Counties throughout California adopted the
Riverside model
Cost Analysis
• Involves measuring expenditures on the
resources required to operate a program
• Uses
– Essential for CBA
– Needed for planning program roll-outs
– Helpful in monitoring and modifying on-going
programs
• Some of the concepts are again illustrated with
the GAIN evaluation
• However, depending on circumstances, there are
a variety of ways of determining costs
A Few Principles
• The goal in a CBA is to estimate net costs, not
just gross costs (This requires separate
calculations of costs for program groups and
control groups.)
• It is usually more useful for CBAs of peopleorientated programs to measure costs per case,
rather than aggregate cost
– Facilitates comparisons with other programs
– Facilitates comparisons with benefits
• The distinctions among ex ante, ex post, and in
medias res remain important
Data Items Needed for Cost Analysis
•
•
•
•
Time spent by clients in each program component
Payments to clients
Vendor payments on behalf of clients
Salary and other costs of employing the staff
operating the program and the counterfactual
program
• Special purchases made for program use
• Office overhead rates or the cost of each item that
comprises overhead
Steps in Estimating Net Costs
1. Enumerate program components (see Table 3.5)
2. Compute unit costs for each component for the
program and control groups
3. Determine participation rates for each component for
the program and control groups
4. Determine the average length of participation in each
program component for the program and control
groups
5. Compute the gross cost of each program component
for the program and control group
6. Compute the net cost of each program component
(2) Compute Unit Costs
• Compute the average cost of providing a unit of each
program component (e.g., an hour, a week, a session)
• In GAIN, total expenditures on many components
were available from administrative data
• These totals were divided by the total number of
“participant months”
• Note: ideally, total expenditures should include all
costs (i.e., staff and overhead costs, costs engendered
by both participants and “no shows”)
• Note: both program group members and other may
have participated in some of the same activities
(3) Determine Participation Rates
• Participation rates are the fraction of clients that
participate in each program component
• It is essential to derive separate participation
rates for the program group and the control group
• In GAIN, participation rates were derived from a
combination of data collected from hard-copy
case files and survey data
• Combining two sources of data can improve
accuracy
(4) Determine Average Length of
Participation
• The average length of time clients who
participated in a program component remained
active (i.e., the number of units received by
component participants)
• If the average length of stay differs substantially
between the program and control groups they must
be measured separately
• Again, in GAIN, participation rates were derived
from a combination of data collected from hardcopy case files and survey data
(5) Compute Gross Costs
• This step is completely mechanical
• Gross costs per client =
unit costs x participation rate x average length
of participation
• Note: Computing the cost of a program
component in this way means that the costs are
averaged over those who actually participate in
the component and those who do not. If this was
not done, the different cost components could not
be summed to determine total costs.
(6) Compute Net Costs
• This step is also mechanical
• Net costs per program participant =
gross cost for the program group minus
gross cost for the control group
Alternatives to ‘Pure’ CBA
• ‘Pure’ CBA is only possible
– If all the benefits and costs can be measured
– If all the benefits and costs can be monetized
• If this isn’t possible, an alternative to pure
CBA must be used
• Two possible alternatives are:
– Qualitative CBA
– Cost-effectiveness analysis
Qualitative CBA
• Probably much more frequently used than pure CBA
• It works as follows:
– The evaluator first obtains monetary measures for
all benefits and costs for which this is possible
– Then, the remaining benefits and costs are assessed
as well as possible
– For example, if feasible, they are quantified
• GAIN has a qualitative aspect; non-monetary
outcomes are discussed and sometimes measured (e.g.,
high school graduation, implications of increased
employment, substitution effects)
Cost-Effectiveness Analysis (CEA)
• CEA can be used if all benefits and costs are
quantified, but some aren’t monetized
• It is an attempt to get around putting a monetary
value on all project impacts
• Often used in the health and educations fields,
where it is difficult to put a monetary value on
some impacts
Example of CEA: Health and Safety
Programs that Save Lives
• In CBA, a monetary value would be put on each
life saved
B/C = Value of a life saved in dollars / Cost in dollars
= Dollars of benefits per dollar of expenditure
• In CEA, a monetary value is not placed on lives
saved
C/E = Cost in dollars / Number of lives saved
= Number of dollars expended per life saved
Limitations of CEA
1. Mainly useful for comparing alternative
policies or programs, but doesn’t indicate
whether any of the alternatives should be
adopted
2. Comparisons of alternatives are often subject
to scale problems
3. Difficult to use when more than one benefit is
not valued in monetary terms
Cost-Utility Analysis
• To get around the last problem in the health area,
techniques have been developed for combining two
effects of health policies
• Doing this is called cost-utility analysis
• Specifically, the effects of health care on the length of
life and on the quality of life are combined into
something called QALYs—Quality-Adjusted Life-Years
• Idea is to allow a policy that provides people with 10
years of enjoyable life to be ranked higher than a policy
that keeps people alive for 10 years as vegetables, even
if the first policy costs more