Heterogeneity of Treatment Effect and
Implications for Comparative
Effectiveness
David Kent, MD, MS
Institute for Clinical Research and Health Policy Studies
Tufts Medical Center
Comparative Effectiveness
Research
• “The core question of comparative effectiveness
research (is) which treatment works best, for whom,
and under what circumstances.”
CAPRIE Trial
• Included 19,185 patients with atherosclerotic
vascular disease manifested as either recent
ischaemic stroke (IS), recent MI, or symptomatic
peripheral arterial disease (PAD).
• Randomised to:
– clopidogrel (75mg once daily)
– aspirin (325mg once daily)
• Outcome: composite outcome cluster of ischaemic
stroke, MI, or vascular death
CAPRIE Steering Committee. Lancet 1996;348(9038):1329-1339.
Results
Hasford J, et al. J Clin Epimemiol 2010:63;1298-1304.
CAPRIE Steering Committee. Lancet 1996;348(9038):1329-1339.
For Whom Should Clopidogrel Use Be
Approved/Recommended/Funded?
• Approve for patients with:
– Either recent IS, recent MI, or
symptomatic PAD.
– Recent IS or symptomatic PAD, but not
recent MI
– PAD only
– None of the above.
NICE vs IQWiG
• National Institute for Health and Clinical
Excellence (NICE) UK
– Benefit for all
– However not cost-effective
• Institute for Quality and Efficiency in Health Care
(IQWiG) Germany
– Acknowledged superiority only in subgroup with
PAD
– Performed no CEA (by law)
Subgroup analyses that have shown clinically
important heterogeneity of treatment effect, which
has subsequently shown to be false
Rothwell PM. Lancet 2005;365(9454):176-86.
• Subgroup analysis of clinical trials can be
misleading.
• Subgroup analysis of clinical trials can be
misleading.
• Overall (summary) results of clinical trials can be
misleading.
• Average results of clinical trials do not apply to
all patients in the trial.
• Even in trials with well-defined
inclusion/exclusion criteria, there is often
extreme variation in outcome-risk and (therefore)
treatment-benefit.
• Aggregating results across heterogenous
patients can lead to distortions or misperceptions
of the treatment effect.
• Even “typical” patients included in the trial may
not be likely to get the average benefits in the
summary results.
• Conventional (“one-variable-at-a-time”) subgroup
analyses will be inadequate to detect differences
in treatment-effect across different patient
groups.
• Risk models, which look at multiple variables
simultaneously, can be applied to clinical trials to
disaggregate subjects into subgroups likely to
have clinically meaningful differences in their
treatment effect.
Outcome Risk with Treatment
35%
30%
25%
20%
15%
10%
5%
0%
0%
5%
10%
15%
20%
Outcome Risk
Placebo
Treatment without Harm
Treatment with Harm
25%
30%
Outcome Risk with Treatment
35%
30%
Median “typical” patient
25%
20%
Average (overall)
result
15%
10%
5%
0%
0%
5%
10%
15%
20%
Outcome Risk
Placebo
Treatment without Harm
Treatment with Harm
25%
30%
25%
Relative Risk Reduction
20%
15%
10%
5%
0%
-5%
-10%
-15%
-20%
0%
5%
10%
15%
Outcome Risk
Relative Risk Reduction
20%
25%
30%
25%
Relative Risk Reduction
20%
15%
10%
5%
0%
-5%
“one-variable-at-time”
subgroup analysis
-10%
-15%
-20%
0%
5%
10%
15%
Outcome Risk
Relative Risk Reduction
20%
25%
30%
25%
Relative Risk Reduction
20%
15%
10%
5%
0%
-5%
“one-variable-at-time”
subgroup analysis
Multivariate risk-based analysis
-10%
-15%
-20%
0%
5%
10%
15%
Outcome Risk
Relative Risk Reduction
20%
25%
30%
Kent DM, et al. J Gen Intern Med 2002;
17:887-94.
Kent DM, et al. J Gen Intern Med 2002;
17:887-94.
16.3%
1.0%
Kent DM, et al. J Gen Intern Med 2002;
17:887-94.
DANAMI-2
Thune JJ, et al. Circulation 2005,112:2017-2021.
DANAMI-2
Thune JJ, et al. Circulation 2005,112:2017-2021.
Clinical Conditions where Outcome
Risk is Major Determinant
Clinical Condition
Treatment
Symptomatic carotid stenosis
Carotid endarterectomy17
Non-valvular atrial fibrillation
Anticoagulation for primary prevention of stroke44
Coronary artery disease
Coronary artery bypass grafting45
Primary prevention of coronary
artery disease
Blood pressure lowering46
Aspirin47
Lipid lowering48
Acute coronary syndromes
Early invasive strategy (versus conservative)
Clopidogrel (versus placebo)49
Enaxparin (versus unfractionated heparin)50, 51
ST-Elevation acute myocardial
infarction
tPA (versus streptokinase)12, 52
Percutaneous coronary intervention (versus
thrombolytic therapy)53, 54
Severe sepsis
Drotrecogin alfa (activated protein C)55
Kent DM, et al. Trials 2010;11:85.
Interim Summary
• Heterogeneity of outcome risk is ubiquitous
(other examples)
Interim Summary
• Heterogeneity of outcome risk is ubiquitous
(other examples)
• Heterogeneity of outcome risk gives rise to
heterogeneity of treatment effect.
Interim Summary
• Heterogeneity of outcome risk is ubiquitous
(other examples)
• Heterogeneity of outcome risk gives rise to
heterogeneity of treatment effect.
• If there is any treatment-related harm, there
will be both relative and absolute treatment
effect heterogeneity.
Interim Summary
• Heterogeneity of outcome risk is ubiquitous
(other examples)
• Heterogeneity of outcome risk gives rise to
heterogeneity of treatment effect.
• If there is any treatment-related harm, there
will be both relative and absolute treatment
effect heterogeneity.
• One variable at a time subgroup analyses are
inadequate (and prone to spurious false
positive results).
Interim Summary
• Heterogeneity of outcome risk is ubiquitous (other
examples)
• Heterogeneity of outcome risk gives rise to
heterogeneity of treatment effect.
• If there is any treatment-related harm, there will be
both relative and absolute treatment effect
heterogeneity.
• One variable at a time subgroup analyses are
inadequate (and prone to spurious false positive
results).
• Risk-based analyses will usually be adequately
powered.
Checklist for Reporting on Subgroup Analyses
and Heterogeneity in Treatment Effects
1.
2.
3.
4.
5.
Evaluate and report on the distribution of risk in the overall study
population and in the separate treatment arms of the study by
using a risk prediction model or index.
Primary subgroup analyses should include reporting how relative
and absolute risk reduction varies in a risk-stratified analysis.
Any additional primary subgroup analysis should be pre-specified
and limited to patient attributes with strong a prior
pathophysiological or empirical justification.
Conduct and report on secondary (exploratory) subgroup analyses
separate from primary subgroup comparisons.
All analyses conducted must be reported and statistical testing of
HTE should be done using appropriate methods (such as
interaction terms) and avoiding over-interpretation.
Kent DM, et al. Trials 2010;11:85.
Dimensions of Risk Heterogeneity
•
•
•
•
Outcome Risk
Treatment-Related Harm
Competing Risk / Attributable Fraction
Treatment Responsiveness
Thank you!