S-L Normand (FDA Non-Tech: 2004)
A REVIEW OF CONCEPTS &
APPROACHES FOR CAUSAL
INFERENCE:
PROPENSITY SCORES
INNOVATIVE STATISTICAL
APPROACHES IN HSR:
BAYESIAN, MULTIPLE INFORMANTS,
& PROPENSITY SCORES
Sharon-Lise T. Normand
Thomas R. Belin, UCLA
Nicholas J. Horton, Smith College
Sharon-Lise T. Normand, Harvard University
(sharon@hcp.med.harvard.edu)
Harvard Medical School & Harvard School of Public Health
Thanks: Laura A. Petersen
1
Sharon@hcp.med.harvard.edu (ARM 2004)
REGIONALIZING CARDIAC
SERVICES
Under FFS, incentive for hospitals to
duplicate profitable cardiac service
capabilities.
Under non-FFS, incentive to regionalize
costly services to referral institutions.
Non-FFS systems provide lower rates of
cardiac procedures than FFS systems.
Availability of on-site cardiac
procedures affects the use of such
procedures.
Should we regionalize invasive
cardiac services?
Examine use of clinically needed
angiography between AMI patients
treated within a regionalized system
(VA) and those treated within a nonregionalized system (FFS).
(Petersen, Normand, Leape, McNeil, NEJM: 2002)
3
Sharon@hcp.med.harvard.edu (ARM 2004)
4
Sharon@hcp.med.harvard.edu (ARM 2004)
OUTLINE OF TALK
WHAT IS A CAUSAL EFFECT?
• What is a causal effect?
Let Yi denote the “outcome” for the ith
patient. The causal effect is:
δi = Yi(treatment) – Yi(control)
– Role of Randomization.
• Statistical Adjustments.
– Regression
– Stratification
– Both
Receipt of Angiography if
treated in a regionalized
system (VA)
• Concluding remarks.
5
Sharon@hcp.med.harvard.edu (ARM 2004)
2
Sharon@hcp.med.harvard.edu (ARM 2004)
Receipt of Angiography if
treated in a nonregionalized system (FFS)
Outcomes under treatments not assigned are missing
6
Sharon@hcp.med.harvard.edu (ARM 2004)
1
S-L Normand (FDA Non-Tech: 2004)
ROLE OF RANDOMIZATION
ROLE OF RANDOMIZATION
Properties of an RCT:
1. Experimenter determines the assignment
of treatments to patients using a known
mechanism.
2. Every participant has a non-zero “chance”
of being assigned the treatment.
3. Outcome and treatment assignment are
independent given the covariates.
So what? Randomization (treatment
assignment mechanism) permits:
δi = Yi(treatment) –
Treatment
=
Effect
Y for those
assigned to treatment
Yi(control)
Y for those
assigned to
control
7
Sharon@hcp.med.harvard.edu (ARM 2004)
8
Sharon@hcp.med.harvard.edu (ARM 2004)
TYPES OF BIASES
OBSERVATIONAL STUDY
1. Non-representative sample of target
population.
2. Treatment assignment is non-ignorable,
can not assume treatment assignment
depends only on observed covariates.
3. All Measured Confounders:
Properties of an Observational Study:
1. Investigator has no control over the
assignment of treatments to patients so
that the mechanism is unknown.
2. No longer true that every participant has a
non-zero “chance” of being assigned the
treatment.
3. Outcome and treatment assignment may be
dependent given the covariates.
•
•
Inexact matching on basis of covariates.
Incomplete matches (discarding treated).
9
Sharon@hcp.med.harvard.edu (ARM 2004)
10
Sharon@hcp.med.harvard.edu (ARM 2004)
ANALYTIC ADJUSTMENTS
TYPES OF BIASES
1. Model-Based (Statistical) Adjustment:
regress outcome on the confounders and
estimate adjusted outcomes.
2. Matched Sampling: create categories
from the confounding variables (X) such
that within each category, there are both
treated and control.
3. Adjust & Stratify: Replace confounders
by a single summary, e(X), and construct
strata/matches/weight. Estimate treatment
differences within each stratum/pair etc.
RCT has random assignment of treatments
but not random selection of subjects
from the target population.
Observational study has random sampling of
target population but not random
assignment of treatments.
11
Sharon@hcp.med.harvard.edu (ARM 2004)
12
Sharon@hcp.med.harvard.edu (ARM 2004)
2
S-L Normand (FDA Non-Tech: 2004)
SO THEY ARE DIFFERENT!
(USUAL) REGRESSION
CHARACTERISTICS OF ACC/AHA
CLASS I HOSPITALIZED AMI PATIENTS
VA
FFS
Stand.
Characteristic
Diff.
(N = 1104)
(N = 10464)
Hx Hypertension
67%
59%
18%
Black
Prior MI
Asthma/COPD
ST-Elevation
13%
38%
31%
49%
4%
35%
23%
43%
35%
7%
17%
10%
Ischemia
29%
12%
44%
Angiography
44%
51%
logit(P(Y=1|X)) = β0 + α(VA) + βTX
Performs poorly when the variances of the
confounders in the treated and control
groups are unequal.
Unequal variances of confounders are not
uncommon in observational studies. .
13
Sharon@hcp.med.harvard.edu (ARM 2004)
14
Sharon@hcp.med.harvard.edu (ARM 2004)
HOW TO MAKE VALID CASUAL INFERENCE?
REGRESSION
Estimate:
e(X) = P(VA = 1| X)
Infer the assignment
mechanism.
Difficult to assess whether treated and
control groups overlap enough on X to
allow sensible estimation of an effect.
Ensure everyone has a
chance of being assigned
the treatment.
• If treatment groups have different covariate
distributions, model-based adjustments depend
on the form of model.
• But, distn of X may differ for treated and control
⇒ must impose linearity and extrapolate
over different regions of covariates.
0 < e(X) < 1
Guarantee we are
comparing similar patients.
Want comparable e(X)
between treatment
groups.
Estimate effect of treatment
(VA) on Y (angiography).
E(Y| e(X),VA = 1) –
E(Y| e(X),VA = 0)
15
Sharon@hcp.med.harvard.edu (ARM 2004)
16
Sharon@hcp.med.harvard.edu (ARM 2004)
200
TREATMENT ASSIGNMENT MECHANISM
CLASS I VA PATIENTS (N = 1104)
50
100
150
PROPENSITY SCORE: TREATMENT
ASSIGNMENT MECHANISM
0
P(VA = 1 | X) = e(X) =
demographics (age, race)
+
severity on entry (ST-Elevation,
chestpain duration, etc)
+
comorbidity (Hx of AMI, CHF, Diabetes,
etc).
0.0
0.2
0.4
0.6
0.8
1.0
4000
6000
Estimated Propensity Score: e(X) = P(VA)
0
2000
CLASS I FFS PATIENTS (N = 10464)
17
Sharon@hcp.med.harvard.edu (ARM 2004)
0.0
0.2
0.4
0.6
0.8
Estimated Propensity Score: e(X) = P(VA)
18
3
S-L Normand (FDA Non-Tech: 2004)
1090 MATCHED PAIRS
200
300
REGRESS & STRATIFY (OR
REGRESS & MATCH)
0
0.0
0.4
FFS
Stand.
Diff.
Hx Hypertension
67%
66%
2%
Black
Prior MI
Asthma/COPD
ST-Elevation
12%
38%
30%
49%
12%
42%
30%
47%
1%
-7%
1%
4%
Ischemia
27%
26%
2%
ROC Area = 0.80
300
200
0.0
0.2
0.4
0.6
0.8
1.0
Estimated Propensity Score: P(VA = 1)
20
ADJUSTED COMPARISON USING 1090
MATCHED PAIRS OF CLASS I PATIENTS
Adjusted for Patient Characteristics Only
No. (%) of
Discordant
Pairs
No. (%) of Discordant
Pairs in which VA
Received Angiography
553 (51)
242 (44)
P-Value
0.0038
Not adjusted for hospital clustering. OR from regression
adjustment [95% CI] using all patients: 0.77 [0.66, 0.88].
21
Sharon@hcp.med.harvard.edu (ARM 2004)
22
Sharon@hcp.med.harvard.edu (ARM 2004)
ADJUSTED COMPARISON USING 1085
MATCHED PAIRS OF CLASS I PATIENTS:
CONCLUSIONS FROM EXAMPLE
Adjusted for Patient Characteristics and Hospital
Availability of Angiography
No. (%) of Discordant
Pairs in which VA
Received Angiography
255 (50)
Under-use of angiography regardless of
system of care.
Under-use is higher within the
regionalized system [VA].
Differences in under-use between VA and
FFS associated with on-site availability
of cardiac procedure technology.
P-Value
0.86
Not adjusted for hospital clustering. OR from regression
adjustment [95% CI] using all patients: 1.04 [0.89, 1.21].
23
Sharon@hcp.med.harvard.edu (ARM 2004)
1.0
100
CHARACTERISTICS OF ACC/AHA
CLASS I POST MATCHING (1090 PAIRS)
VA
0.8
CLASS I FFS (N = 1090)
Sharon@hcp.med.harvard.edu (ARM 2004)
Characteristic
0.6
0
19
505 (47)
0.2
Estimated Propensity Score: P(VA = 1)
Sharon@hcp.med.harvard.edu (ARM 2004)
No. (%) of
Discordant
Pairs
CLASS I VA (N = 1090)
100
Stratify (or match) patients based on the
estimated propensity score.
Within each stratum, distribution of
observed confounders differs only
randomly between treated and control
subjects.
Regardless: matching towards a
population who looks like those who
received the treatment.
24
Sharon@hcp.med.harvard.edu (ARM 2004)
4
S-L Normand (FDA Non-Tech: 2004)
CONCLUDING REMARKS
• Role for propensity score approaches in
HSR.
• Provides a principled and practical
approach for assessing appropriateness
and robustness of assumptions.
• No Panacea – doesn’t overcome hidden
bias problems.
• Should you always use?
25
Sharon@hcp.med.harvard.edu (ARM 2004)
5