Weighing Benefits, Harms and Costs:
What is the “Right” Number Needed to Treat?
George F. Sawaya, MD
Professor
Department of Obstetrics, Gynecology and
Reproductive Sciences
Department of Epidemiology and Biostatistics
University of California, San Francisco
Objectives
• Understand population effects of interventions (e.g.,
screening tests) in terms of benefits (effectiveness) and
harms (costs and patient morbidity) and how these might be
weighed
• Appreciate the need to systematically synthesize evidence
about benefit and harms using decision analyses (e.g.,
outcomes tables) and cost-effectiveness analyses
• Know the difference between cost-benefit, costeffectiveness and cost-utility analyses
• Understand efficiency curves in cost-effectiveness analyses
An example: recent changes in cervical cancer screening
In November 2009, the American College of Obstetricians and Gynecologists (ACOG,
Practice Bulletin 109) changed the recommendation from screening every 1-3 years
among women aged 21 and older to the following:
Sawaya GF N Engl J Med 2009 361;26 2503-2505
Possible reasons why less
screening was recommended
How to determine the optimal
balance of benefits and harms?
Consider…
• Two currently available treatments: Treatment A and B
Treatment A has a 90% cure rate for Disease X.
Treatment B has a 95% cure rate for Disease X.
• What would you want to know to decide to offer B over A?
• Treatment A costs $1,000 and Treatment B costs $10,000
• Treatment B cost $100,000 or $10 million?
• What if the cure rate with Treatment B were 90.1% and Disease
X was cancer?
• Should an insurance plan cover Treatment B?
Weighing benefits and harms:
possible approaches
• Shared decision making (example: amniocentesis
benefits and harms)
• Decision analyses: outcomes tables of both benefits
and harms
• Cost analyses: assigning values to both benefits and
harms in terms of both quantity and quality of life
(life years, quality-adjusted life years)
What is a life year?
What is a quality-adjusted life year?
Life year
• a measure of the quantity of life lived
• used in decision analyses to compare benefits of various health
intervention strategies
• may be expressed as “life years expected per 1000 people” for
each intervention strategy
Quality-adjusted life year (QALY)
• a measure of both the quantity and quality of life lived (i.e., a
year of life adjusted for its quality or value).
• A year in perfect health is equal to 1.0 QALY; a year in ill
health would be “discounted” (e.g., a year bedridden might be
equal to 0.5 QALY). “Perfect health” is highly subjective; some
individuals may believe that certain health states are worse than
death.
Decision analysis
• Estimates the outcomes of different clinical decisions
• Breaks down problem into components: treatment
options, outcome probabilities with each option (both
benefits and harms); uses systematic reviews and
meta-analyses
• Applies to large, theoretic cohorts of individuals going
forward in time (effectiveness)
Example: chemoprophylaxis of breast cancer (tamoxifen)
1) what are the benefits and harms specific to this question?
2) how likely are they to occur?
3) how to weigh these and make recommendations?
1,000 45-year-old women: estimated 5-year
benefits and harms of tamoxifen
5-year risk
Invasive breast CA
- FHx
+FHx
0.7%
1.6%
3-4 avoided
8 avoided
Non-invasive breast CA 1-2 avoided
2-3 avoided
Hip fracture
<1 avoided
<1 avoided
Endometrial CA
1-2 caused
1-2 caused
Stroke, PE, DVT
3-5 caused
3-5 caused
USPSTF Evidence Report, 2001
1,000 65-year-old women: estimated 5-year
benefits and harms of tamoxifen
- FHx
+FHx
1.5%
3.2%
7-8 avoided
16 avoided
Non-invasive breast CA 2-3 avoided
4-5 avoided
5-year risk
Invasive breast CA
Hip fracture
5 avoided
5 avoided
Endometrial CA
21 caused
21 caused
Stroke, PE, DVT
21 caused
21 caused
USPSTF Evidence Report, 2001
Current USPSTF Recommendation
• The USPSTF recommends against… use of tamoxifen or
raloxifene for the primary prevention of breast cancer in
women at low or average risk for breast cancer.
• D recommendation
• The USPSTF recommends that clinicians discuss
chemoprevention with women at high risk for breast cancer
and at low risk for adverse effects of chemoprevention.
• B recommendation
Cost analyses: assigning values to
both benefits and harms
• Cost-benefit
• Cost-effectiveness
• Cost-utility
Cost-benefit analyses
assigns a monetary value to both interventions
and outcomes for each decision option
($ for $)
Example: Is it cost-beneficial to have
publicly-funded contraception programs in
the State of California?
Impact of publicly funded
contraceptive services on unintended
pregnancies and implications for
Medicaid expenditures
“for every dollar spent to provide publicly-funded
contraceptive services, an average of $3 was
saved in Medicaid costs for pregnancy-related
health care and medical care for newborns.”
Family Planning Perspectives, 28:188-195, 1996
Cost-effectiveness analyses
assigns a monetary value only to the intervention; the
outcome is expressed on another scale (e.g., life years
saved).
allows quantitative comparisons of various medical
decisions (a measure of value or “bang for the buck”)
Example:
Should we continue to screen women annually for
cervical cancer once they have had 3+ normal Pap
tests?
A real life example
CDC has a national program that screens lowincome women for cervical and breast cancer
Resources are constrained but were expended in
recalling women for additional Pap tests after
normal tests have been documented.
What is the most cost-effective screening strategy
for cervical cancer in women with 0, 1, 2 and 3+
prior normal Pap tests (annual, biennial, triennial
testing)?
Sawaya et al NEJM 2003:349;1501-9
Kulasingam et al Obstet Gynecol 2006 Feb;107(2):321-328
Analysis: effectiveness
• Interventions that extend life an average of
30 days are thought to be “effective.”
Effectiveness: Results, long-term
(weighted average of all ages, 100,000 women with ≥3 prior
normal Pap tests, screened to age 65)
Strategy
Expected lifeyears
(3% discount)
Incremental lifeyears
No screening
18.08965
-
Pap q 3 years
18.09247
1 day, 42 minutes
Pap q 2 years
18.09277
2 hours, 36 minutes
Pap q 1 year
18.09310
2 hours, 53 minutes
Adding in costs
Cost-effectiveness analyses
• Threshold values of $50,000 - $100,000 per year of life
saved have been described as being “cost-effective”
but…
• Greatest value is in comparing interventions
Cost-effectiveness: Results, long-term
(weighted average of all ages, 100,000 women with ≥3 prior
normal Pap tests, screened to age 65 and followed to 85)
Strategy
Expected lifeyears
(3% discount)
Incremental costeffectiveness
No screening
18.08965
-
Pap q 3 years
18.09247
$115,429
Pap q 2 years
18.09277
$460,422
Pap q 1 year
18.09310
$1,192,770
Costs vs. Benefits:
The Big Picture
Marginal (or
incremental) benefit
Benefits
Day 1
Day 2
Costs
Efficiency curves: women aged 30-44,
by prior Pap test results
Diamond=0 prior nl tests
Circle=3+ prior nl tests
Kulasingam et al Obstet Gynecol 2006 Feb;107(2):321-328
How to best balance these benefits
and harms for a population?
Reasoned judgment?
Cost-based benchmarks and precedence?
All of the above?
But it’s about more than just
costs…
Cost-utility analyses
like cost-effectiveness analyses, but adjusts years of life
saved for quality of life (quality-adjusted life-years-how long someone will live, with an adjustment for
quality of life -- healthy years count more, sick or
disabled years, less).
Unit of measure is a “utility”
Example: Should we recommend screening for prostate
cancer?
Prostate cancer screening:
a decision analysis
• Optimal decision dependent on utilities of most common adverse
outcomes: impotence and urethral stricture (incontinence)
• When adverse outcomes of treatment ignored, screening favored
(24.86 vs 24.22 life year expectancy)
• No screening preferred to screening when patients' utilities
considered (24.14 vs 23.47 quality-adjusted life years expected)
• When quality-of-life preferences of men considered, prostate
cancer screening not recommended
J Fam Pract 1995 Jul;41(1):33-41
Note the differences
• Cost-benefit
$ spent per $ saved
• Cost-effectiveness
$ spent per life-year saved
• Cost-utility
$ spent per qualityadjusted life-year saved
Conflict: how should we optimally weigh
benefits and harms for screening/prevention
recommendations?
• Outcomes tables
Pros: transparent
Cons: heavy reliance on judgment
• Cost analyses
Pros: levels the playing field to allow comparisons; can
quantify and project population effects of beneficial and
harmful outcomes over a lifetime
Cons: opaque and complex, monetary basis
• Shared decision making
Summary
• Technological advances in medicine make tests more
sensitive, often less specific and certainly more costly
• Large-scale trials (comparative effectiveness) may
prove small benefits of statistical significance, but of
unclear clinical value
• You must be aware of harms incurred in pursuit of
marginal benefits
• Many forces are at play
• DAs and CEAs can be useful tools to judge the balance
of benefits and harms for population-based
recommendations, but neither is perfect
• Judgment = controversy (stay tuned for breast cancer
screening)