Niteesh Choudhy, M.D., Ph.D.
Evaluating the Medical Evidence
A TOOLKIT FOR THE INTERPRETING THE EFFECTIVENESS OF
INTERVENTIONS
Take home points
1
• Evidence-based medicine has revolutionized the way
health care is delivered
2
• There is “evidence” to support whatever you believe!
3
• Academic detailers are ambassadors of the evidence
and need to know how to embrace its strengths and
limitations
Evidence matters




Evidence-based medicine aims
to apply the best evidence
gained from the scientific
method to clinical decision
making
Gained prominence in the early
1990’s
De-emphasizes intuition,
unsystematic clinical experience
and pathophysiologic rationale
Application of evidence in
patient care has resulted in
substantial reductions in
morbidity and mortality
Contributors to cardiovascular death rates
SOURCES: Guyatt et al. JAMA 1992;268:2420-2425; Ford et al. NEJM 2007; 356:2388-98
The volume of “evidence” is overwhelming
NOT ALL EVIDENCE IS OF EQUAL QUALTY



In 1992, internists needed to
read an estimated 17 articles
every day of the year in order to
“keep up” with the literature
The volume of published articles
since then has increased
exponentially
Made more difficult because not
all evidence is of equal quality
(i.e. difficult to identify those
studies that are particularly
important)
Creates a virtually impossible
problem for practicing
physicians
3,500,000
3,000,000
2,500,000
Articles in Medline

2,000,000
1,500,000
1,000,000
500,000
0
SOURCES: Davidoff et al BMJ 1995; 310: 1085; http://www.nlm.nih.gov/bsd/medline_lang_distr.html
A hypothetical example


A new cholesterol lowering pill, nolipid,
has been synthesized and developed
into tablet form for oral consumption
In a prospective study, nolipid:
 significantly reduced LDL cholesterol levels
by 50% (p<0.0001)
 was well tolerated
 had no adverse effects
WOULD YOU RECOMMEND THE USE OF NOLIPID
FOR PATIENTS WITH ELEVATED CHOLESTEROL?
The questions we should be asking:
Did they choose
the right
comparator?
Did they choose
the right
outcome?
Absolute or
relative
changes?
Overall or
subgroup
results?
Choosing the right comparator
SPARCL



4731 patients who had stroke or
TIA one to six months before
study entry and NO CAD
Randomized to atorvastatin 80
mg daily or placebo
Significant reduction in primary
end-point (fatal or non-fatal
stroke)
 Placebo: 13.1%
 Atorvastatin 11.2%
SOURCE: SPARCL investigators. NEJM 2006; 355: 549-59
Choosing the right comparator
SPARCL
BUT…
 Many patients in SPARCL would
already be on a statin according
to current treatment guidelines
 SPARCL should have compared
high and lower intensity statin
therapy
 More generally, to get FDA
approval, drugs generally only
need to demonstrate superiority
over placebo but in reality,
clinicians and decision makers
want information about
comparative efficacy/safety
SOURCE: SPARCL investigators. NEJM 2006; 355: 549-59
Current NCEP/ATPIII cholesterol treatment
guidelines
Risk Category
CHD or CHD
Risk
Equivalents
(10-year risk
>20%)
LDL Goal
(mg/dL)
<100
LDL Level at
Which
to Consider
Drug Therapy
(mg/dL)
130
(100–129: drug
optional)
Evaluating the right outcome
EZETIMIBE AND THE ENHANCE TRIAL

720 patients with familial
hypercholesterolemia

Randomized to simvastatin +
ezetimibe or simvastatin alone

Substantial reductions in LDL from
combination therapy


A widely used “surrogate” outcome in
cardiovascular trials
However, there was no change in
atherosclerosis (carotid-artery
intima-media thickness)

Thankfully, this was the trial’s
“primary” outcome although many
other trials that preceded it only
evaluated LDL
SOURCE: Kastelin et al. NEJM 2008; 1431-43
Surrogate end-points

A surrogate end-point is “a
laboratory measurement or
physical sign used as a
substitute for a clinically
meaningful end-point that
measures directly how a
patient feels, functions, or
survives”


Use of surrogate end-points may
lead to rapid and appropriate
dissemination of new
treatments (e.g. HIV)
However, may also lead to
excess morbidity/mortality (e.g.
inotropes may improve
hemodynamics but some may
cause excess mortality)
The majority of clinical trials
focus on these outcomes
Looking at the “right” results
THE SUB-GROUPS OF CHARISMA
Enrolled 15,603 patients with
established cardiovascular
disease or multiple risk
factors
 Randomized to clopidogrel 75
mg or placebo added to
aspirin 75-162 mg daily
(median follow-up duration
28 months)
 The published conclusion:

HR: 0.93 (95% CI 0.83 to 1.05)
Overall v. subgroup results
CHARISMA



Numerous pre-specified subgroup analysis some of which
reached (borderline) statistical
significance
When analyzing multiple
subgroups, some will reach
statistical significance by chance
alone
While there are statistical
methods to deal with this, we
should ideally focus on the
overall trial results (or the
results of a limited set of
prespecified subgroups)
RR 0.88 (95%CI: 0.77-0.998)
Absolute v. relative risks
JUPITER TRIAL NEJM 2008; 359: 2195-207
Absolute v relative reductions
JUPITER TRIAL NEJM 2008; 359: 2195-207
Event rate per 100 person years
5
4.5
4
3.5
RELATIVE REDUCTION = 44%
ABSOLUTE REDUCTION = 0.59 events per 100 person years
3
2.5
2
1.5
1
0.5
0
Rosuvastatin
Placebo
An EBM toolkit
Right
comparator?
• Something is often better than nothing
• We often care about how well a new treatment compares with the
existing standard of practice (not placebo)
Right endpoint?
• Surrogate end-points are easier to measure and are often sufficient for a
new drug to be approved
• If available, we should really focus on “hard” outcomes
Overall v.
subgroup
results?
• Often possible to find subgroups that derive less or more benefit from a
treatment
• Should focus on the overall trial results (entire cohort, primary outcome)
Right effect
measure?
• Absolute and relative effects can lead to very different assessments of
the benefit/safety of a treatment
• Should use both when weighing the significance of a therapy
Take home points
1
• Evidence-based medicine has revolutionized the way
health care is delivered
2
• There is “evidence” to support whatever you believe!
3
• Academic detailers are ambassadors of the evidence
and need to know how to embrace its strengths and
limitations