Applying Trials and
Systematic Reviews to
Individual Patients
Paul Glasziou & Sharon Sanders
University of Queensland
for
Cochrane Applicability & Recommendations
Methods Group
www.sph.uq.edu.au/CGP/training/CochraneMethodsGroup.html
The problem of applying trial results
The Trial
patients
The Trial
report
The actual
patients
Who does this trial apply to?
 Yes, the trial showed it worked
1. But will it work as well in THIS patient?
•
A 30% relative risk reduction (RRR) means
 It “worked” in 30%
 It didn’t work in 70%
(and they were at risk of adverse outcomes)
2. And what is the importance of it
“working” for THIS patient?
Should Mr RM buy an
electric toothbrush?
 72 year old pensioner with Parkinson’s
Disease
• Has gingivitis and frequent caries
 Trials in young healthy folk showing
improvements in gingivitis scores but not
caries.
• What should he do?
Possible approaches to
who a trial applies to?
1. Effect size (larger will apply more)
2. Look at biological mechanisms to predict
who it might or might not work
1. Is there study evidence to support this?
3. How do trial population compare to
patient characteristics
1. How DOES THE Risk COMPARE?
Transferability and
applicability of results
A. TRANSFERABILITY (across groups)
1. What are the benefits and harms?
2. Is there predictable variation in the effects?
3. How does effect vary with predicted risk?
B. APPLICATION (to individual)
4. What are the predicted absolute risk
reductions for individuals?
5. Do the benefits outweigh the harms?
Example of Toothbrush
5-steps
 See in your papers
Rofecoxib, Celecoxib, and Paracetamol in
Osteoarthritis of the Knee
•
•
•
Design: Randomized, parallel-group, double-blind trial,
Patients: 382 patients 40+ yrs with OA of the knee
Interventions:
 Rofecoxib, 12.5 mg/d OR 25 mg/d;
 Celecoxib, 200 mg/d; OR
 acetaminophen, 4,000 mg/d
•
Results: Over 6 weeks, rofecoxib, 25 mg/d, better for
 night pain (P<.002 vs celecoxib and P=0.006 vs acetaminophen
and P=0.02 vs rofecoxib, 12.5 mg/d),
 composite pain subscale (P.03 vs all other treatments),
 stiffness subscale (P=0.04 vs celecoxib and acetaminophen),
 physical function subscale (P=0.001 vs acetaminophen).
 Global responses over 6 weeks showed a similar pattern.
Geba GP, et al JAMA. 2002 ;287:64-71.
All or some responders?
I. Everyone gets small benefit?
II. A few get a larger benefit?
Osteoarthritis N-of-1s
 Comparison of
• 1,000mg paracetamol tds
• 400mg ibuprofen tds
 Two weeks x 6
• Outcome diary of pain and
stiffness of target joint
Paracetamol
NSAID
Pair 1
NSAID
Paracetamol
Pair 2
NSAID
Paracetamol
Pair 3
N-of-1: overall & examples
AVERAGE PAIN
PAIN SCORE (MEAN + 95% CI)
8
6
4
2
0
Panadol
NSAID non-responder
Actiprofen
DRUG
NSAID responder
Interventions:
Levels of Evidence




N-of-1 Trial
Systematic review of randomised trials
A single randomised trial
Controlled, non-randomised
• Parallel control
• Historical control
• Case-control
 Case-series
Guyatt, JAMA, 2000
When n-of-1 not possible:
The benefit-harm model (Lubsen, Tijssen*)
 When does benefit outweigh harm?
 Assumptions
• Benefit (rate difference) proportional to event rate
• Harm constant over event rate
 Net benefit = benefit - harm
8
6
4
2
0
Benefit
Harm
0
10
20
*Controlled Clinical Trials; 10: 151S-160S.
Transferability and
applicability of results
A. TRANSFERABILITY (across groups)
1. What are the benefits and harms?
2. Is there predictable variation in the effects?
3. How does effect vary with predicted risk?
B. APPLICATION (to individual)
4. What are the predicted absolute risk
reductions for individuals?
5. Do the benefits outweigh the harms?
Tutorial
 Look at the systematic review
• Do you have any medical questions?
 Don’t read it all !!
 Try to extract the information to complete
the first step transferability checklist
1. What are the benefits
and harms?
 List all potential benefits
 List all potential harms
 Estimate from (meta-analysis) of best
available of evidence
• Relative effect
• Absolute effect
Step 1: benefits & harms
STEP 1: benefits and harms
 Were all benefit outcomes (short & long-term) considered?
Outcome
Relative Effect
(RRR/…………..)
Absolute Effect
(NNT/…………….)
Comment
Outcomes not Considered: _______________________________________________
 Were all harm outcomes considered?
Outcome
Relative Effect
(RRI/…………..)
Absolute Effect
(NNH/…………….)
Comment
Outcomes not Considered: _______________________________________________
“Safety” in systematic reviews
Ernst, Pittler, BMJ 2001; 323:546
Transferability and
applicability of results
A. TRANSFERABILITY (across groups)
1. What are the benefits and harms?
2. Is there predictable variation in the effects?
3. How does effect vary with predicted risk?
B. APPLICATION (to individual)
4. What are the predicted absolute risk
reductions for individuals?
5. Do the benefits outweigh the harms?
The problem of applying trial results
The Trial
patients
The Trial
report
The actual
patients
2. Are there predictable
variations in the effects?
 Does effect vary by (PICO)
• Patient features, e.g., comorbidity or disease
features, e.g., stage
• Intervention features e.g., dose/intensity/timing?
• Comparator, e.g., placebo, add-on, or active
• Outcome measures, e.g., reliability, duration
 But beware of artefactual causes
• Differences in followup, compliance, measures , …
Tutorial
 Look at the systematic review again
 Try to extract the information to complete
the second step transferability checklist
•
•
•
•
P
I
C
O
CHD Death + Non-fatal MI*
LIPID: Major Subgroups of Patient Features
Placebo p-value for
better heterogeneity
Pravastatin
better
Placebo Pravastatin
%
%
Hypertension
No
16
15
14
11
0.07
Diabetes
No
22
15
19
12
0.60
Smoker
Ex-smoker
Non-smoker
21
16
13
15
12
12
0.29
Total
16
12
0.5
*The prespecified outcome for
subgroup analysis
0.6
0.7
0.8
0.9
Relative Risk
1
1.1
1.2
Are the groups different?
DO
Test for
difference
DON’T
Test each
separately
Group 1
Group 2
Combined
0.5
1.0
Relative Risk
1.5
Step 2: effect modifiers
STEP 2: Effect modifiers (Patients, Intervention, Comparison, Time)
 What potential effect modifiers were considered?
Considered:
Not Considered:
 What potential harm modifiers considered?
Considered:
Not Considered:
 Are the results robust across different subgroups of patients? Yes/Uncertain/No
3. How does effect vary
with predicted risk?
 Is Relative Risk constant across low to
high risk groups?
• Relative Risk is most often constant
• Need to check using:
Plots
Heterogeneity statistics
When n-of-1 not possible:
The benefit-harm model (Lubsen, Tijssen*)
 When does benefit outweigh harm?
 Assumptions
• Benefit (rate difference) proportional to event rate
• Harm constant over event rate
 Net benefit = benefit - harm
8
6
4
2
0
Benefit
Harm
0
10
20
*Controlled Clinical Trials; 10: 151S-160S.
Warfarin in non-valvular Atrial
Fibrillation: the trial evidence
3. How does effect vary
with predicted risk?
Trials of Warfarin in Atrial Fibrillation
Rate versus rate plots
Treatment group rate
L’Abbe plot of trials of Warfarin in Atrial Fibrillation
Line of equality
Constant relative reduction
Control group rate
Rate versus rate plots
Treatment group rate
L’Abbe plot of trials of Warfarin in Atrial Fibrillation
Line of equality
Constant absolute risk reduction
Constant relative reduction
Control group rate
Which risk measure is
most constant?
Analysis of the effect of control rate in 115 meta-analysis
Schmid et al Stats in Med 1998: 1923-42.
Measure
Odds Ratio
% varying with control
group risk
13%
Relative Risk
14%
Risk Difference
31%
4. How do absolute benefits and
harms vary with risk/severity?
Baseline
Risk
20%
Relative
Absolute Number
Risk
Risk
needed to
Reduction Reduction Treat
75%
15%
7
8%
75%
6%
16
Trial patients
4%
75%
3%
33
Typical patients
1%
75%
.75%
133
For biological effect &
For clinical
transferability
decision making
Benefit versus Harm
Clinical predictors of stroke
Stroke Equivalents
0.18
0.16
Benefit
0.14
= 73% RRR
0.12
0.1
0.08
0.06
0.04
1 ICH death = 4 strokes
1 ICH death = 1 stroke
0.02
0
0
0.05
0.1
0.15
Harm
= 0.01 deaths
0.2
Stroke Risk/Yr
Risk Factors*
Frequency
0
42%
1
46%
2 or 3
12%
*hypertension, recent CCF, previous thromboembolism,
Guidelines for Atrial Fibrillation:
Evidence + Recommendations
Recommendations
Risk/
Yr
NNT
1%
133
3%
50
>7%
<20
No Risk Factors* - use aspirin only
One Risk Factor - individualise treatment
Two or more Risk Factors - anticoagulation
strongly recommended
* The 5 Risk Factors are: hypertension, recent CCF,
previous thromboembolism, LV dysfunction, atrial size
Guidelines: proportion of patients
with AF needing anti-coagulation
120
100
No of patients
100
83
79 81
80
61
60
65 66 66
46 48
33 33 35
40
20
88 91
17 18 20
16
13
0
Guidelines
Thomson R BMJ 1998;316:509-13
Trial inclusion/exclusion criteria
An appropriate basis for transferability?
 Inclusion/exclusion criteria not (usually)
aimed at transferability but at:
• improving study power
choose high risk groups
minimise death from other causes
ensure good compliance
• maximising safety
exclude if any possible adverse effects
Some excluded sub-groups may have net benefit
Some included sub-groups may have net harm
SUMMARY: data for applying
the results of controlled trials
Relative Risk
Values of Benefits, Harms
Your patient, Trials patients
Benefits, Harms
systematic review of RCTs
Individual Clinical Decision
Patient’s Expected Event Rate
Predictive model from Cohort study
sun21.imbi.uni-freiburg.de/mailman/listinfo/applicabilitygroup
 GOOD
• Toothbrush example nice
• (but maybe fill out form!!)
 Do DIFFERENTLY
•
•
•
•
Do reading before session
More time; more coffee
Interested in steps 4 & 5
Stream groups