INNOVATIVE STATISTICAL
APPROACHES IN HSR:
BAYESIAN, MULTIPLE INFORMANTS,
& PROPENSITY SCORES
Thomas R. Belin, UCLA
Nicholas J. Horton, Smith College
Sharon-Lise T. Normand, Harvard University
1
Sharon@hcp.med.harvard.edu (ARM 2004)
A REVIEW OF CONCEPTS &
APPROACHES FOR CAUSAL
INFERENCE:
PROPENSITY SCORES
Sharon-Lise T. Normand
(sharon@hcp.med.harvard.edu)
Harvard Medical School & Harvard School of Public Health
Thanks: Laura A. Petersen
2
Sharon@hcp.med.harvard.edu (ARM 2004)
REGIONALIZING CARDIAC
SERVICES
Under FFS, incentive for hospitals to
duplicate profitable cardiac service
capabilities.
Under non-FFS, incentive to regionalize
costly services to referral institutions.
Non-FFS systems provide lower rates of
cardiac procedures than FFS systems.
Availability of on-site cardiac
procedures affects the use of such
procedures.
Sharon@hcp.med.harvard.edu (ARM 2004)
3
Should we regionalize invasive
cardiac services?
Examine use of clinically needed
angiography between AMI patients
treated within a regionalized system
(VA) and those treated within a nonregionalized system (FFS).
(Petersen, Normand, Leape, McNeil, NEJM: 2002)
4
Sharon@hcp.med.harvard.edu (ARM 2004)
OUTLINE OF TALK
• What is a causal effect?
– Role of Randomization.
• Statistical Adjustments.
– Regression
– Stratification
– Both
• Concluding remarks.
5
Sharon@hcp.med.harvard.edu (ARM 2004)
WHAT IS A CAUSAL EFFECT?
Let Yi denote the “outcome” for the ith
patient. The causal effect is:
i = Yi(treatment) – Yi(control)
Receipt of Angiography if
treated in a regionalized
system (VA)
Receipt of Angiography if
treated in a nonregionalized system (FFS)
Outcomes under treatments not assigned are missing 6
Sharon@hcp.med.harvard.edu (ARM 2004)
ROLE OF RANDOMIZATION
Properties of an RCT:
1. Experimenter determines the assignment
of treatments to patients using a known
mechanism.
2. Every participant has a non-zero “chance”
of being assigned the treatment.
3. Outcome and treatment assignment are
independent given the covariates.
7
Sharon@hcp.med.harvard.edu (ARM 2004)
ROLE OF RANDOMIZATION
So what? Randomization (treatment
assignment mechanism) permits:
i = Yi(treatment) –
Treatment
=
Effect
Y for those
assigned to treatment
Yi(control)
Y for those
assigned to
control
8
Sharon@hcp.med.harvard.edu (ARM 2004)
OBSERVATIONAL STUDY
Properties of an Observational Study:
1. Investigator has no control over the
assignment of treatments to patients so
that the mechanism is unknown.
2. No longer true that every participant has a
non-zero “chance” of being assigned the
treatment.
3. Outcome and treatment assignment may be
dependent given the covariates.
9
Sharon@hcp.med.harvard.edu (ARM 2004)
TYPES OF BIASES
1. Non-representative sample of target
population.
2. Treatment assignment is non-ignorable,
can not assume treatment assignment
depends only on observed covariates.
3. All Measured Confounders:
•
•
Inexact matching on basis of covariates.
Incomplete matches (discarding treated).
10
Sharon@hcp.med.harvard.edu (ARM 2004)
TYPES OF BIASES
RCT has random assignment of treatments
but not random selection of subjects
from the target population.
Observational study has random sampling of
target population but not random
assignment of treatments.
11
Sharon@hcp.med.harvard.edu (ARM 2004)
ANALYTIC ADJUSTMENTS
1. Model-Based (Statistical) Adjustment:
regress outcome on the confounders and
estimate adjusted outcomes.
2. Matched Sampling: create categories
from the confounding variables (X) such
that within each category, there are both
treated and control.
3. Adjust & Stratify: Replace confounders
by a single summary, e(X), and construct
strata/matches/weight. Estimate treatment
differences within each stratum/pair etc.
12
Sharon@hcp.med.harvard.edu (ARM 2004)
CHARACTERISTICS OF ACC/AHA
CLASS I HOSPITALIZED AMI PATIENTS
VA
FFS
Stand.
Characteristic
Diff.
(N = 1104)
(N = 10464)
Hx Hypertension
67%
59%
18%
Black
Prior MI
Asthma/COPD
ST-Elevation
13%
38%
31%
49%
4%
35%
23%
43%
35%
7%
17%
10%
Ischemia
29%
12%
44%
Angiography
44%
51%
13
Sharon@hcp.med.harvard.edu (ARM 2004)
SO THEY ARE DIFFERENT!
(USUAL) REGRESSION
logit(P(Y=1|X)) = 0 + (VA) + TX
Performs poorly when the variances of the
confounders in the treated and control
groups are unequal.
Unequal variances of confounders are not
uncommon in observational studies. .
14
Sharon@hcp.med.harvard.edu (ARM 2004)
REGRESSION
Difficult to assess whether treated and
control groups overlap enough on X to
allow sensible estimation of an effect.
• If treatment groups have different covariate
distributions, model-based adjustments depend
on the form of model.
• But, distn of X may differ for treated and control
 must impose linearity and extrapolate
over different regions of covariates.
15
Sharon@hcp.med.harvard.edu (ARM 2004)
HOW TO MAKE VALID CASUAL INFERENCE?
Infer the assignment
mechanism.
Ensure everyone has a
chance of being assigned
the treatment.
Estimate:
e(X) = P(VA = 1| X)
0 < e(X) < 1
Guarantee we are
comparing similar patients.
Want comparable e(X)
between treatment
groups.
Estimate effect of treatment
(VA) on Y (angiography).
E(Y| e(X),VA = 1) –
E(Y| e(X),VA = 0)
16
Sharon@hcp.med.harvard.edu (ARM 2004)
PROPENSITY SCORE: TREATMENT
ASSIGNMENT MECHANISM
P(VA = 1 | X) = e(X) =
demographics (age, race)
+
severity on entry (ST-Elevation,
chestpain duration, etc)
+
comorbidity (Hx of AMI, CHF, Diabetes,
etc).
17
Sharon@hcp.med.harvard.edu (ARM 2004)
200
TREATMENT ASSIGNMENT MECHANISM
0
50
100
150
CLASS I VA PATIENTS (N = 1104)
0.0
0.2
0.4
0.6
0.8
1.0
4000
6000
Estimated Propensity Score: e(X) = P(VA)
0
2000
CLASS I FFS PATIENTS (N = 10464)
0.0
0.2
0.4
0.6
0.8
Estimated Propensity Score: e(X) = P(VA)
18
REGRESS & STRATIFY (OR
REGRESS & MATCH)
Stratify (or match) patients based on the
estimated propensity score.
Within each stratum, distribution of
observed confounders differs only
randomly between treated and control
subjects.
Regardless: matching towards a
population who looks like those who
received the treatment.
19
Sharon@hcp.med.harvard.edu (ARM 2004)
200
300
1090 MATCHED PAIRS
0
100
CLASS I VA (N = 1090)
0.0
0.2
0.4
0.6
0.8
1.0
300
Estimated Propensity Score: P(VA = 1)
0
100
200
CLASS I FFS (N = 1090)
0.0
0.2
Sharon@hcp.med.harvard.edu (ARM 2004)
0.4
0.6
0.8
Estimated Propensity Score: P(VA = 1)
1.0
20
CHARACTERISTICS OF ACC/AHA
CLASS I POST MATCHING (1090 PAIRS)
Characteristic
VA
FFS
Stand.
Diff.
Hx Hypertension
67%
66%
2%
Black
Prior MI
Asthma/COPD
ST-Elevation
12%
38%
30%
49%
12%
42%
30%
47%
1%
-7%
1%
4%
Ischemia
27%
26%
2%
ROC Area = 0.80
Sharon@hcp.med.harvard.edu (ARM 2004)
21
ADJUSTED COMPARISON USING 1090
MATCHED PAIRS OF CLASS I PATIENTS
Adjusted for Patient Characteristics Only
No. (%) of
Discordant
Pairs
No. (%) of Discordant
Pairs in which VA
Received Angiography
553 (51)
242 (44)
P-Value
0.0038
Not adjusted for hospital clustering. OR from regression
adjustment [95% CI] using all patients: 0.77 [0.66, 0.88].
22
Sharon@hcp.med.harvard.edu (ARM 2004)
ADJUSTED COMPARISON USING 1085
MATCHED PAIRS OF CLASS I PATIENTS:
Adjusted for Patient Characteristics and Hospital
Availability of Angiography
No. (%) of
Discordant
Pairs
No. (%) of Discordant
Pairs in which VA
Received Angiography
505 (47)
255 (50)
P-Value
0.86
Not adjusted for hospital clustering. OR from regression
adjustment [95% CI] using all patients: 1.04 [0.89, 1.21].
23
Sharon@hcp.med.harvard.edu (ARM 2004)
CONCLUSIONS FROM EXAMPLE
Under-use of angiography regardless of
system of care.
Under-use is higher within the
regionalized system [VA].
Differences in under-use between VA and
FFS associated with on-site availability
of cardiac procedure technology.
24
Sharon@hcp.med.harvard.edu (ARM 2004)
CONCLUDING REMARKS
• Role for propensity score approaches in
HSR.
• Provides a principled and practical
approach for assessing appropriateness
and robustness of assumptions.
• No Panacea – doesn’t overcome hidden
bias problems.
• Should you always use?
25
Sharon@hcp.med.harvard.edu (ARM 2004)