Program Evaluation from an
Economic Perspective
Phaedra Corso, Ph.D.
Associate Professor
College of Public Health
University of Georgia
Why Care About Economics in
the Context of Prevention?

Maximizing outcomes is important.

Minimizing costs is important too.

Resources are limited, so hard (resource
allocation) decisions must be made.

Demonstrates the value provided from the
resources expended (return on
investment).
The PH Model for Prevention –
Economics?
Risk and
Protective Factor
Identification
Problem
Identification
Economic
Impact - COI
Program and
Policy
Evaluation
Program
and Policy
Development
Economic
Evaluation
Implementation
and
Dissemination
EE Methods

Partial evaluation – costs only
Cost of illness (COI) analysis
 Cost analysis (program costs)


Full evaluation – costs and outcomes

Cost-benefit analysis (CBA)

Cost-utility analysis (CUA)

Cost-effectiveness analysis (CEA)
COI Analysis

Estimates total costs incurred because of a
disease or condition


(i.e., medical costs, non-medical costs, productivity
losses)
Generally reported as
annual total cost
 average per person lifetime cost


Used to show potential benefits of prevention
efforts
Costs of Violence in
the United States
Corso et al., AJPM 2007
Cost Parameters

Direct costs

Medical care

ED visits
Hospitalizations
Ambulance/paramedic
MD visits
Dental
Physical Therapy
Prescription Drugs

Mental health care







Productivity losses
Work losses
 Household productivity losses

Incidence

~2.2. million medically-treated injuries
associated with violence occurred in 2000

~17,000 homicides, ~30,000 suicides

People aged 15 to 44 years comprise 44 percent
of the population, but account for nearly 75
percent of violent injuries
Costs

The total cost associated with nonfatal injuries
and deaths due to violence in 2000 was more
than $70 billion.
$37 billion for interpersonal violence
 $33 billion for self-inflicted violence




The average cost per homicide was $1.3 million
in lost productivity and $4,906 in medical costs.
The average cost per case for a non-fatal assault
resulting in hospitalization was $57,209 in lost
productivity and $24,353 in medical costs.
The average cost per case of suicide is $1
million lost productivity and $2,596 in medical
So What?

The incidence and economic burden of
injuries in the US is substantial

This information can be used to lobby for more
prevention resources

Implementation of effective interventions
could reduce this burden

The cost to implement effective interventions
less the economic burden prevented –
represents the potential returns on investment for
Cost Analysis (CA)

Estimates total costs of running a program


Costs are the value of the resources (people, building, equipment and
supplies) used to produce a good or a service
Important for realizing costs from varying
perspectives

e.g., incurred by program, incurred by participant

Includes not just financial, but also economic
costs.

Important for budget justification, decision
Cost Analysis
of a national replication of a
child maltreatment program
Corso et al., CDC, OCAN (in progress)
Define Cost Categories
**Type of Activity
(D) Direct: Client-focused, face-to-face activity
(I) Indirect: Collateral activities on behalf of client
systems
(AC) Administrative-Client: Related to client
activities
(AP) Administrative-Program: Related to
programmatic/management activities
***Activity Description
a. Advocate
b. Assess
c. Counseling/support
d. Court representation
e. Assist/provide
f. Plan
g. Refer
h. Schedule
i. Teach
j. Transport
a. Advocate
b. Clinical documentation
c. Research
d. Preparation for court
e. Testify in court
f. Consult/Collaborate
g. Locate resources
h. Team meeting
i. Risk management meeting
j. Clinical Interdisciplinary team mtng
k. I&R referral
a. Gives supervision
b. Receives supervision
c. etc
Etc.
Preliminary Results at end of Year 1

The average cost per family referral ranged from
$2,319 to $8,906.

The average cost per family receiving services
ranged from $4,238 to $33,742.

At the end of the first year of implementation,
pre-implementation costs as a percentage of
total costs ranged from 23% to 42% of the total
costs of the program.
So What?

Provides information for Agency X who might want to
implement the program in the future.

Provides the cost component of a full EE.

Lessons learned on how to conduct a programmatic
CA:
 Prospective data collection
 Input from site implementers
 Technical assistance throughout data collection
 Revisions of cost collection templates along the way
Reference: Applying Cost Analysis to Public Health Programs
(at www.phf.org)
What is Economic Evaluation (EE)?
Applied analytic methods to:
Identify,
Measure,
Value, and
Compare
the costs and consequences of
treatment* and prevention**
strategies.
* Done a lot
** Done “not so much”
Cost-benefit Analysis - CBA

A method used to compare costs and benefits
of an intervention


Provides a list of all costs and benefits over
time:
•
•

where all the costs and benefits are standardized
or valued in monetary terms.
Can have different time lines
Can have different amounts at different times
Provides a single value:
•
Net Benefits: NB (Benefits – Costs)
When is CBA Used?

To decide whether to implement specific
programs


When choosing among competing options


If NB > 0, implement
Implement program with highest NB
For setting priorities on options given
resource constraints
Quantify Benefits - CBA

Cost-of-Illness (COI) approach

Willingness-to-Pay (WTP) or
Contingent-valuation surveys
•
(e.g., how much is society willing to pay
to reduce the annual morbidity and
mortality risk associated with a
disease or injury)
Corso Survey (Fall 2007, Georgia)
“Based
on national
2 out
of every
100,000
“Now
we
haddata,
a nationally-sponsored
child maltreatment
“If thisimagine
program
were
available
to your
state, would
youchildren
be willing to
prevention
thatper
wasof
available
to your
state
and this
annually,
or
an taxes
average
4 to
children
every
day
are this
killed
pay $150 inprogram
extra
year
sponsor
this
program?”
program
wasofproven
reduce the riskbyofparents
a child being
killed due
as
a result
child to
maltreatment
or caretakers.”
to child maltreatment by 50%. This means that the number of
YES
– “Would
you
willingevery
to pay
children
killed
onbeaverage
day$225?”
in the U.S. by child
NOmaltreatment
– “Would youisbe
willingfrom
to pay
$75?”
reduced
4 per
day to 2 per day.”
Cost-utility Analysis - CUA




A method used to compare costs and benefits of
interventions where benefits are expressed as the number
of life years saved adjusted to account for loss of quality.
Combines
• Length of life (survival), and
• Quality of life
Compares disparate outcomes in terms of utility
• Quality-adjusted life years (QALYs)
• Disability-adjusted life years (DALYs)
Derives a ratio of cost per health outcome
• $/QALY or $/DALY
When is CUA Used?

When quality of life is the important
outcome.

When the program affects both morbidity
and mortality.

When the programs being compared have a
wide range of different outcomes.

When the program is being compared with a
program that has already been evaluated using
CUA.
Quantify Benefits - CUA
 Utilities
are:
•
A “preference-based” measure of health,
that relies on choice and uncertainty to
elicit preferences
•
Typically based on a 0 (death) to 1 (perfect
health) scale
Example of Tool to Elicit Utilities:
Time Trade-Off (TTO)
Quality of
Life
Which life do you prefer?
Short and fun
Long and dull
Length of Life
Example: TTO
Utility
healthy
U(healthy) = 1.0
blind both eyes
U (blind both eyes) = ?
Dead
0
Years
12
20
Combining Quality of Life with
Length of Life
Utility
without prevention
with prevention
1.0
0.7
0
30
QALYS (with prevention)
QALYS (without prevention)
75
= 1.0*75 = 75
= 0.7*30 = 21
Years
Cost-effectiveness Analysis - CEA

Estimates costs and outcomes of interventions

Expresses outcomes in natural units
 e.g., cases prevented, lives saved

Compares results with other interventions affecting the
same outcome

Summary measure: cost-effectiveness ratio
 Cost per some outcome achieved
 e.g., cost per case prevented, cost per life saved
When is CEA Used?


Used to identify
•
most cost-effective strategy from options that
produce a common effect
•
practices that are not “worth” their costs
Used for empirical support for under-funded
programs
Quantify Outcomes – CEA of
parenting intervention

Intermediate outcomes
•
•
•

Increased child self-esteem and mental
health status
Increased family cohesiveness/coping
skills
Decreased depression in parents
Final outcomes
•
QOL improvements in parents and
children
CEA Caveat

Outcomes cannot be combined, so one or two of
the most important effectiveness measures should
be considered (separately) for the CEA.

The number of summary measures depends on
the number of outcomes chosen.

If 2 outcomes, A and B, are considered the most
important for evaluation, then



Cost/outcome A
Cost/outcome B
This makes translation for policy makers difficult!!
Example: CM Prevention Program
Average CE Ratios for depressive
symptoms
Program Baseline Ending
score
score
3-mo.
22.7
18.7
Difference Program Avg. CE
cost
Ratio*
4
$984
$246
9-mo.
8.3
$2,304
$278
21.1
12.8
* Compared to baseline
Final Comments

Economic evaluation (EE) is valuable to
decision making and for setting health
policy.

For new researchers in PH, this is an
important specialization to consider –
because the demand for these skills is
growing.
Coming soon…..
Center for Economic Evaluation
Institute for Behavioral Research
and
College of Public Health
pcorso@uga.edu