Methods for Comparative
Effectiveness and HSR
John B Wong
Wong, MD
Tufts Medical Centere
Academy Health
Boston, MA
June 27, 2010
IOM Definition of CER
• “CER is the generation and synthesis of
evidence that compares the benefits and
harms of alternative methods to prevent,
diagnose treat and monitor a clinical
diagnose,
condition, or to improve the delivery of care.
purpose
p
of CER is to assist consumers,,
The p
clinicians, purchasers, and policy makers to
make informed decisions that will improve
health care at both the individual and
population levels.”
Sox HC, Greenfield S Ann Intern Med 2009;203-5
Information Sources
Tunis SR et al Ann Intern Med 1994;120:956-63
Lau J et al N Engl J Med 1992;327:248-54
Antman EM et al JAMA 1992;268:240-8
Cumulative Meta
Meta-analysis
analysis
Evidence
• Ineffective and possibly harmful care
(routine lidocaine) was recommended
from 1970 through
g at least 1990 in
textbooks and review articles
• Effective care (streptokinase) was not
recommended until 15 years later in
textbooks and review articles
Antman EM et al JAMA 1992;268:240-8
CER Solutions
• Systematic reviews and meta-analysis of RCTs
– Direct,
Direct indirect and network
• Observational data sets
– Covariate adjusted, propensity, inverse probability weighted,
instrument variables
• “Decision makers need to assess and appraise all the
available evidence irrespective as to whether it has
been derived from RCTs or observational studies,
and the strengths and weaknesses of each need to
b understood
be
d t d if reasonable
bl and
d reliable
li bl conclusions
l i
are to be drawn.” - Sir M Rawlins
Rawlins M Lancet 2008;372:2152-61
Unique and Complex
• The dilemma articulated by Bernard in
1865 still haunts the clinician: the
response
p
of the “average”
g p
patient to
therapy is not necessarily the response
of the patient being treated
treated.
Yusuf S et al JAMA 1991;266:93-8
No Average Patient
• Average results of clinical trials do not
apply to all patients in the trial
• Even with well-defined
well defined inclusion
exclusion criteria, variation in outcomerisk and (therefore) treatment-benefit.
treatment benefit
Kent DM et al Am J Med 2002;113:104-11
No Average Patient
• Risk models, which look at multiple
variables simultaneously, can be
applied
pp
to clinical trials to disaggregate
gg g
treatment-effects into clinically
meaningful sub-groups
Kent DM et al Am J Med 2002;113:104-11
“Medicine is a science of
uncertainty and an art of
probability
probability”
-Sir William Osler
GUSTO Trial
tPA
Streptokinase
Mortalityy
6.3%
7.3%
Brain
hemorrhage
0 72%
0.72%
0 52%
0.52%
Cost
$2,750
$320
Distribution of Predicted Incremental Mortality Benefit in the
GUSTO Population for t-PA vs.
vs Streptokinase
Absolutte Morta
ality Ben
nefit
(in pe
ercentag
ge points
s)
6%
5%
4%
3%
2%
1%
0%
0
10
20
30
40
50
60
Percentile Mortality Benefit
70
80
90
100
Including Bleeding Risk
• 60% of benefit in 25% of patients and 85% of
benefit in 50% of patients and some harmed
Kent DM et al Am J Med 2002;113:104-11
Incremental CostCost
effectiveness of tPA vs SK
• Average ICER $32,800
• Quartile of benefit
– Highest Quartile 1 = $13,900
$13 900
– Quartile 2 = $36,500
– Quartile 3 = $88,100
$
– Quartile 4 = No benefit
Fundamental Principles of
Evidence-based
Evidence
based Medicine
• Evidence is never enough for clinical
decision making
Guyatt G Users’ Guides to the Medical Literature 2002
What Do PCI/CABG Patients
Believe?
•
•
•
•
88% substantially or greatly ↓ risks
75% prevent MI & 71-83% prolong life
Expect life expectancy gain 10 years
But CABG vs Medical therapy actually
suggest
– 19 months gain for left main disease
– 6 months
th ffor 3
3-vessell di
disease
– 2 months for 1- or 2-vessel disease
Holmboe ES et al J Gen Intern Med 2000;15:632-7; Kee F et al Quality in Health Care 1997;6:131-9;
Yusuf S et al Lancet 1994;344:563-70; Whittle J et al Am Heart J 2007;154:662-8
What do Patients Know?
• Physician-patient agreement regarding
survival benefit no better than chance
• Poor agreement regarding symptom
improvement
• 52% correctly
tl answered
d ≤1
1 outt off 5
questions (29% mean)
Whittle J et al Am Heart J 2007;154:662-8; Wong JB et al J Am Coll Cardiol 2009;53(suppl 1):A368-9
Decision Aids or Decision
Support
• Purpose is not to persuade but to
• Comprehend clinical information
– Disease,
Disease options
options, risks,
risks benefits
• Clarify and communicate preferences
(values) for outcomes
• Arrive at decision consistent with values
Preferences about Angina
Severity
y
• Guidelines base treatment on angina severity
• Patients with similarly severe angina have
very different attitudes about their angina
– 1/3rd of patients with functional class II angina had
lower quality of life than someone with more
severe class
l
III/IV angina
i
• Treatment guidelines should be based on
preferences of the individual patient and not
on symptom severity
Nease RF et al JAMA 1995;273:1185-90
Randomized Trial of Shared
Decision Program (SDP)
• SDP vs. usual care at Toronto Hospital
– Increased knowledge 75% vs. 62%
(p
(p=0.001)
)
– Decreased revascularization choice by
23% (p
(p=0.01,
0.01, 58% vs. 75%)
– Deceased actual revascularization by 21%
(p=0 06 52% vs
(p=0.06,
vs. 66%)
• Note Ontario 9-times less likely to have
CABG for 1- or 2-vessel than in NY
Morgan MW et al J Gen Intern Med 2000;15:685-93; Tu JV et al Ann Intern Med 1997;126:13-9
Should where you live affect your
likelihood of CABG?
4 f ld variation
4-fold
i ti in
i likelihood
lik lih d off CABG
www.dartmouthatlas.org
Shared Decision Making
• The patient. . .contributes her expertise about
her own goals,
goals attitudes towards risk
risk, and the
value she places on various outcomes
• Then,
Then combining their perspectives
perspectives, they can
negotiate a solution that is uniquely suited for
this patient
patient.
Quill TE, Suchman AL Humane Medicine 1993;9:109-20
A Rational Framework for Health
Policy
y Decision Making
g
• Donepazil for AD
• 68% probability
b bilit that
th t
drug is cost-effective
• Is
I additional
dditi
l
research potentially
cost effective?
cost-effective?
• Which model inputs
would be most
valuable?
Claxton K et al Lancet 2002;360:711-15
A Rational Framework for Health
Policy
y Decision Making
g
• Expected Value of
P f t Information
Perfect
I f
ti
= $339 million
• Partial EVPI for
Efficacy duration =
$270 million
• Research costs
likely <$270 million
Claxton K et al Lancet 2002;360:711-15
Conclusions
• Evidence synthesis from RCTs and
observational
b
ti
l studies
t di regarding
di risks
i k
and benefits
• Patient-centered individualized risks
and benefits and shared decision
making
Cost-effectiveness
effectiveness analysis,
analysis EVPI,
EVPI
• Cost
PEVPI for policy making and priorities