Using Observational Data to Extend the
Results of a Randomized Controlled
Trial: an Application to the HOPE Trial
Motivation
Gold Standard = RCT but…
„
Randomized Controlled Trials (RCT) can never address
all research questions
‰
‰
„
Paul L. Hebert, PhD
Mary Ann McLaughlin, MD, MPH
Jodi Casabianca, MS
Anu Lala, MD
Jason Wang, PhD
„
Certain patient groups under-represented
Head-to-head drug trials are rare
Observational studies subject to selection bias
Causal statistical models rely on untestable assumptions
‰
‰
Instrumental variables: Uncorrelated with unobserved factors
Propensity score modeling, Inverse Probability of Treatment
Weighting: “Ignorability” assumption
Funding provided by NIDDK R21DK64322 (Hebert)
Concept
„
„
„
Find an RCT that was widely accepted, but left
some questions unanswered
Use statistical models on observational data to
“recreate” the results of the RCT
If models worked, then apply the same models to
‰
‰
patients who were excluded from original RCT, or
treatments that were not considered in the RCT
Unanswered questions from HOPE
Application: The Heart Outcomes PrEvention
(HOPE) Trial (2000)
„
„
‰
‰
„
Does ramipril work just as effectively in non-white
patients?
„
Are other ACEIs as effective as ramipril (i.e., is there a
class effect for ACEIs)?
„
„
„
„
„
„
Ramipril $53.03/ 30 day supply
Benazepril $23.99
Captopril $12.99
Enalapril $8.19
22% lower rate of death/stroke/AMI in ramipril group
Important: similar benefit found in subset of patients with
hypertension
Limitations of HOPE findings
‰
95% of patients in HOPE were white
‰
HOPE used ramipril--a newer, more expensive ACEI
Data
„
„
9297 patients with risk factors for heart disease
randomized to ramipril vs placebo
Results: Ramipril is effective in patients at high risk of
cardiovascular events
Dually-eligible Medicare+Medicaid beneficiaries from CA, 19961999. During time when HOPE was conducted.
Medicaid SMRF/MAX files (inpat, outpat, and rx) + Medicare
MedPAR (inpatient) and Physician/Supplier (outpatient) claims
Inclusion criteria
‰
‰
„
Filled an Rx for antihypertensive in 1996 and 1997
Eligible for Medicare because of age, surviving spouse, or disability
Exclusion criteria
‰
HOPE-specific
‰
Other reasonable exclusions
„
„
‰
Age 55+, No CHF, stroke/MI in last 4 weeks, overt nephropathy
Charlson score>5, age>=85, ESRD, taking antihypertensive for < 9 months
Ever in an HMO, 1996-99
1
Sensitivity
Estimation Strategy
„
Recreate HOPE:
‰
Logit model for White patients
„
„
„
„
„
Instrumental variable estimates of ramipril vs. other antihypertensives
„
‰
Outcome: combined outcome of death or hospitalization for
stroke or MI from 1997 to 1999
Key Independent Variable: filled RXs for ramipril in 1997
Control variables: Age-squared, gender, Long-term care,
disabled, indicators of conditions in Charlson, log(per capita
income in ZIP), other medications in 1996 (diuretics, calcium
channel blockers, beta blockers, statins, NSAIDs, antiarythmias)
Repeat on non-white patients
Repeat using other ACEIs
ACE Inibitors
Non-ACEI Ramipril Captopril Enalapril Benazepril
Total
100,508
4,436
3,656
16,329
9,504 134,433
71.6
69%
52%
12%
6%
20%
24%
8%
11%
12%
70.4
66%
55%
12%
10%
17%
26%
4%
17%
9%
Are patients dieing of things that ramipril could prevent?
‰
71.6
69%
59%
13%
8%
14%
31%
12%
23%
11%
71.5
68%
56%
13%
7%
17%
26%
9%
19%
11%
70.8
70%
54%
14%
7%
18%
27%
6%
18%
10%
71.5
69%
53%
13%
6%
19%
25%
8%
13%
11%
Estimate time to stroke/MI, COPD by ACEI used
„
„
‰
„
Stroke/MI should be preventable by better ACEI
No evidence that better ACEI should prevent COPD
Rare event so use Propensity Score Matching
Could compliance with medications explain improved
outcomes for ramipril vs other ACEIs?
‰
Results: Table 1
N
Demographics
Age
Female
White
Black
Hispanic
Other/Unk race
Disabled
Long-term Care
Diabetes
COPD
Medications
Number Medications
Beta Blocker
Calcium Channel Blocker
Diuretics
„
Instruments= % ramipril use in ZIP code, county HMO market
penetration
Generate compliance-adjusted estimates using stratified Cox
models with time-varying indicator for filled ACEI prescriptions on
propensity score matched samples
Yit=b0+b1ACEIi+b2Compliantit+uit
Results: Can we match HOPE in NH white patients (n=
54,970)? Is ramipril just as effective in non-white patients
(n= 39,476)?
White
Black, Hispanic or Unknown
Lipid Lo we ring Ag ent
Di uretic
Ca lcium Ch ann el Blocker
Beta Blocker
Nu mber of Med s
Ho sp ita lized 19 96
Tu mor
Di abetes
CO PD
De mentia
CV D
Prior MI
Di sa bled
Fe male
7.6
23%
54%
36%
10.3
13%
37%
32%
13.9
18%
48%
49%
12.6
19%
52%
42%
11.1
21%
54%
43%
8.7
22%
53%
38%
Age- Squa red
RA MIPRIL
.5
1
1.5
2
2.5
.5
1
1 .5
2
2.5
Odds Ratio
Graphs by racegrp
Other control variables: liver disease, rheumatic disease, ulcers, long-term care
residence, log(per capita income in ZIP code), NSAIDs, anti-arythmias
Results: Are other ACEIs as effective as ramipril?
White
Sensitivity Analysis: Instrumental Variables
Correlation with instruments
Black, Hispanic or Unknown
% Ramipril use by ZIP code, Los Angeles
Lipid Lo we ring Ag ent
Di uretic
Ca lcium Ch ann el Blocker
Beta Blocker
LPM of ramipril use. F(2,54970)=596
Coef (s.e.)
P-value
Constant
0.021
(0.003)
<0.001
% Ramipril in 1
mile radius of
ZIP code
0.8718
(0.0274)
<0.001
HMO market
Penetration in
County
-0.0246
(0.0054)
<0.001
Nu mber of Med s
Ho sp ita lized 19 96
Tu mor
Di abetes
CO PD
De mentia
CV D
Prior MI
Di sa bled
Fe male
Age- Squa red
BENA ZE PRIL
ENALA PRIL
CA PTOPRIL
RA MIPRIL
.5
1
1.5
2
.5
1
1.5
2
Odds Ratio
Other control variables: liver disease, rheumatic disease, ulcers, long-term care
residence, log(per capita income in ZIP code), NSAIDs, anti-arythmias
2
Instruments uncorrelated with unobserved risk factors?
IV results: Effect of ramipril on death/stroke MI
Coefficient on ramipril use
Ramipril users versus users of
other Antihypetensives
Other
HTNs
Age
Female
COPD
Diabetes
Hospitalizations prior year
Medications
Number of Medications
Beta Blockers
Calcium Channel Blockers
Diuretics
Lipid Lowering Agents
Long term Care
High ramipril ZIP codes versus
low ramipril ZIP codes
High
Low Ramipril
Ramipril
ZIP
Ramipril Difference ZIP codes
Codes Difference
71.8
72%
13%
11%
0.2
70.7
70%
10%
17%
0.1
-1.2
-2%
-3%
6%
-0.1
71.9
73%
13%
12%
0.2
71.5
72%
13%
11%
0.2
-0.4
-1%
-1%
0%
0.0
8.0
24%
49%
41%
18%
11%
11.0
14%
36%
33%
24%
5%
3.0
-10%
-13%
-8%
5%
-6%
8.1
24%
48%
41%
18%
12%
8.3
23%
48%
40%
19%
9%
0.1
-1%
0%
-1%
1%
-3%
White
(n= 54,973)
Black, Hispanic
Unknown
(n=39,476)
Linear Probability Model
-0.0596
(p<0.001)
-0.014
(p=0.084)
Instruments: % Ramipril use within own
ZIP code, county HMO penetration
-0.1076
(p=0.056)
0.011
(p=0.843)
Instruments: % Ramipril use within 1 mile
of ZIP code , county HMO penetration
-0.1134
(p=0.041)
0.0171
(p=0.760)
Instruments: % Ramipril within 3 miles of
ZIP code, county HMO penetration
-0.0784
(p=0.162)
0.0261
(p=0.653)
* NOTE: Estimates are not stable. Sensitive to alternative specifications.
Sensitivity: Effects of noncompliance
Summary
Effectiveness of other ACEIs compared to ramipril at preventing death,
stroke/MI, and COPD in propensity score matched samples; unadjusted and
compliance adjusted
„
Cox proportional hazard models
with time-varying indicators of
compliance with ACEI.
Estimated on propensity-score
matched samples.
Death/stroke/MI
Stroke/MI
Attempts to match results of HOPE had mixed success
‰
COPD
B ena ze pril
‰
„
Same statistical procedures applied to Non-white patients
suggest no benefit of ramipril
„
Same procedure applied to ramipril versus other ACEI
suggests ramipril is better than enalapril and captopril, similar
to benazepril
„
With better data, this could be a good strategy for using
observational data to extend the results of randomized trials
E nala pril
Benazapril vs. ramipril
1006 matched pairs
Ca pto pril
Enalapril vs. ramipril
1136 matched pairs
.5
1
1.5
2
.5
1
1.5
2
.5
1
1.5
Logistic worked
IV Unstable
2
Hazard Ratio
Captopril vs. ramipril
859 matched pairs
Unadjusted
Compliance Adjusted
Note: Ramipril is the reference drug
Are other ACEI’s as effective as ramipril?
Odds ratios from logistic regression of death, stroke or MI for ACEI users versus
Patients taking other antihypertensives
White Only (n=43,572)
Antihypertensive in
1997
Non-ACEI
Ramipril
Captopril
Enalapril
Benazepril
Reference
0.80 [0.64,1.00]
1.06 [0.90,1.26]
1.03 [0.93,1.14]
0.81 [0.71,0.92]
Black, Hispanic,
Oth/Unk Race
(n=31,211)
Reference
1.03 [0.78,1.35]
1.25 [1.00,1.57]
1.15 [1.01,1.32]
1.09 [0.92,1.28]
All Races (n=83,474)
Reference
0.87 [0.73,1.03]
1.13 [0.99,1.29]
1.09 [1.01,1.18]
0.93 [0.84,1.03]
Control variables: age-squared, demographics, other drug use, comorbidities,
Long term care residence, log(per capita income)
3