Observational
Ob
ti
lD
Data
t A
Analysis
l i ffor
Comparative
p
Effectiveness
Research
Sebastian Schneeweiss, MD, ScD
J
Jeremy
R
Rassen, S
ScD
D
Division of Pharmacoepidemiology and Pharmacoeconomics,
Department of Medicine, Harvard Medical School
Schneeweiss S, Clin Pharm & Ther 2007
1
Potential conflicts of interest
 PI of the Brigham & Women’s Hospital DEcIDE
Research Center on Comparative Effectiveness Res
Res.
 Co-investigator of the Mini Sentinel System funded by
FDA
 No paid consulting or speaker fees from pharmaceutical
manufacturers
 Consulting/ board membership in past year:
 HealthCore;
H lthC
Th
The LLewin
i Group;
G
RTI;
RTI ii4sm;
ii4
WHISCON
 Investigator-initiated research grants from Pfizer,
H lthC
HealthCore
 Multiple grants from NIH to study all sorts of things
2
Objective of Comparative Effectiveness Research
Placebo
comparison
Efficacy
y
Effectiveness*
Effectiveness
(Can it work?)
(Does it work in
routine care?)
Most RCTs for
drug
g approval
pp
(or usual care)
Active
comparison
(head-to-head)
Goal of CER
* Cochrane A. Nuffield Provincial Trust, 1972
3
As much as we all love randomized
effectiveness trials
 It is an unrealistic expectation that we will have
head-to-head randomized trails




ffor every intervention and
its combinations
i every patient
in
ti t subgroup
b
that exactly mimic routine care
 We need Effectiveness evidence in a timely manner
manner.
Randomized studies may take some time to conduct
 About
Ab t 85% off the
th CER evidence
id
is
i from
f
nonexperimental data!*
* Academy Health Report June 2009
Opportunities
pp
for non-randomized CE
research with electronic healthcare data:
 Representative of routine care




Spectrum of disease severity
Spectrum of co-morbidities
Co-medications
R l world
Real
ld adherence
dh
 Very large size
 Infrequent
f
exposure, recently
l marketed
k d medications
d
 Many subgroups to study treatment effect heterogeneity
 Long
L
follow-up
f ll
 With hard clinical endpoints
 Produce
P d
results
l fast
f
5
A Preferred Data Structure for fast,
improved CER
Example: Older adults using Medicare data
2006
2007
2008
Medicare Part A: Hospitalizations
Medicare Part B: Medical services
Medicare Part D: Pharmacy dispensings
Laboratory results data
Medicare Current Beneficiary Survey plus+
Ongoing disease registries, e.g. cancer registry, RA registry
New study‐specific registries
2009
Claims data describe
the sociology of
health care and its
recording practice in
light of economic
interests
Schneeweiss, J Clin Epi 2005
7
Electronic health care information in each Center
Constant flow of data with little delay and at low cost
Millions of patients with defined person–time denominator
Data reflect routine care
Generalizable to large population segments
HIPAA compliance protects patient privacy
Claims Data
•
•
•
•
•
Member ID
Plan
Gender
A
Age
Dates of Eligibility
Administrative
Data
• Member ID
• Prescribing
physician
h i i
• Drug dispensed
(NDC)
• Quantity and
date dispensed
• Drug strength
• Days supply
• Dollar amounts
Pharmacy
Cl i
Claims
Data
• Member ID
• Physician or Facility
identifier
• Procedures (CPT-4,
revenue
codes, ICD-9)
• Diagnosis (ICD-9CM, DRG)
• Admission and
discharge dates
• Date and place of
service
• Dollar amounts
Physician and
F ilit Claims
Facility
Cl i
Data
Supplemental Data
• Member ID
• Lab Test Name
• Result
Lab Test
R
Results
lt
Data
•
•
•
•
•
•
•
Member ID
Income
Net Worth
Education
Race & Ethnicity
Life Stage
Life Style
St le
Indicators
Consumer
Elements
•
•
•
•
•
•
Member ID
Subspecialty notes
Endoscopy reports
Histology reports
Radiology reports
Free text notes
Electronic
M di l
Medical
Records
Computerized Linked Longitudinal Dataset
8
Challenges that come to mind
 Can we handle confounding by indication?
 Can we reliably assess the relevant outcomes?
 Can we identify relevant subgroups?
 Can we really study long-term outcomes?
…and don’t forget the basics
9
What to do when RCTs find different
treatment effects than nonexperimental studies?
10
The Mantra
of Trialists:
Non-randomized studies are
inherently biased due to patient
selection into treatment groups
11
Confounding
Patient factors become confounders (C) if they are
associated with treatment choice and are also
independent predictors of the outcome:
C
Randomization
Trt
Severity
Prognosis
C
Comorbidity
bidit
Outcome
12
A spectrum of effectiveness research
C
Coxib
Potential for
confounding
by indication
GI
event
e.g. Coxibs and
gastric toxicity
“Effectiveness Research”
e.g.
e
g Coxibs and
cardiac events
C
Coxib
MI
Unintended
effects
Intended
effects
Intentionality of treatment effect by the prescriber
“Safety research”
13
A causal experiment
p
P ti t A:
Patient
A
Drug
I
Improvement
t
R i d titime
Rewind
Patient A: No Drug
No improvement
Watterson B. The Authoritative Calvin and Hobbes, p67-2
14
Design choice by source of exposure variation
Exposure variation
within patients
yes
no
Case-crossover
study
Exposure variation
between patients
yes
no
Crossover trial
Cohort study
Exposure variation
between providers
yes
Randomized
controlled trial
Instrumental
variable analysis
Cluster
randomized trial
Dealing with confounding
Confounding
Measured
C f
Confounders
d
Design
Unmeasured
Confounders
Analysis
•Restriction
•Standardization
•Matching
•Stratification
•Regression
P
Propensity
it scores
•Marginal
Structural Models
16
Schneeweiss, PDS 2006
0 30
0.30
only hdd‐PS
0.60
Mos
st
+ hhd‐PS
adjustm
adment
just.
Mocified
re
+ spec
adjustm
ment
covaars
Age‐s
age‐sex‐
‐sex‐
race‐y
yearyear
‐race
adjustm
adjument
sted
Unadjus
sted
Unadjuusted
log
elative risk)
risk)
log (re
(reelative
Adjustment in non-randomized research
Clopidogrel ‐ MI (2)
C ib ‐ GI bleed
Coxib
bl d
Statin ‐ death
TCA suicide (1)
0.00
‐0.30
‐0.60
Dealing with confounding
Confounding
Measured
C f
Confounders
d
Design
Analysis
Unmeasured
Confounders
Unmeasured, but
measurable in
substudy
•Restriction
•Standardization
•Matching
•Stratification
•2-stage sampl.
•Regression
P
Propensity
it scores
•Marginal
Structural Models
Unmeasurable
Design
Analysis
•Ext. adjustment
•Cross-over
•Imputation
•Active
comparator
(restriction)
•Instrumental
variable
•Proxy
analysis
•Sensitivity
analysis
18
Schneeweiss, PDS 2006
Foot-in-Mouth Award (Economist ‘04):
“… there are known knowns; there
are things we know we know. We
also know that there are known
unknowns; that is to say we know
that there are some things we do not
k
know.
But
B t there
th
are also
l unknown
k
unknowns – the ones we don’t know
we don’t
don t know.
know …, it is the latter
category that tend to be the difficult
ones.”
(Wisely unknowing) Donald Rumsfeld
19
Case-crossover studies
For studying transient drug effects on acute outcomes
Time of
index
event
24 hours prior
to index event
-25
-24
0
-1
X
Control time
period
Patient
periods
time p
Ctrl.
Case
0
1
Exposed
1
0
Non-exp.
Case time
period
- SSRIs and risk of hip fracture: *






X
X
16,341 cases of hip Fx in GPRD
Case-control study: RR= 6.1
1
0
0
1
1
1
0
0
Case-crossover: RR = 1.9
- Viagra and MI
ORMH or
ORCond. log. reg.
*Hubbard et al. Am J Epi 2003
X
= Case-defining event
= Shaded case or control periods represent periods exposed to the study drug
20
Case-crossover studies
 Why is the CCS design not more frequently used in
PE?




Requires rapid onset outcomes
Requires time-varying
time varying exposure (treatment x-over)
x over)
Requires transient drug effects
Is subject
j
to within-person
p
(between-time)
(
) confounding:
g
Decreasing health status correlates with increasing
drug use
 Can be expanded to the case-time-control design
21
Case-time-control analysis
Beta-2-agonist inhaler use
and risk of fatal or near
near-fatal
fatal asthma:
Cases
Current B2A use
Discordant use (case crossover)
Discordant use (control crossover)
Case time control
Controls
High
93
Low
36
29
9
29
9
High
241
65
65
Adjusted
j
Low
414
OR
4.4
25
25
33.22
2.6
1.2
OR
3.1
95%CI
1.8-5.4
11.5-6.8
5-6 8
1.6-4.1
0.5-3.0
22
Restriction
23
Example study (basic design)
 Cohort study of PA Medicare beneficiaries 65+ with
drug insurance through the PACE program
 Medicare Part A, B, and D
 1995 - 2002
 Exposure: statin use
 Outcome: 1-year
1 year all-cause
all cause mortality
 Covariates: Cardiovascular co-morbidities, comedications, other proxies for comorbidity
 Censoring: Death and at 365 days
 Analysis: CoxCox regression w/ EPS adjustment
24
0) Incident and prevalent drug users vs. non-users (matched by exact date)
1a) Incident drug users vs. non-users (matched by exact date)
1b) IIncident
id t d
drug users vs. non-users (matched
( t h db
by d
date
t and
d system
t
use))
2) Incident drug users vs. incident comparison drug users
3) Incident drug users vs. incident comparison
drug users without contraindications
4) Adherent incident drug users v. adherent incident
comparison drug users without contraindications
Restrict to
incident
drug users
Match
non-users
on system
use
Restrict to
incident
comparison
drug users
Restrict to Restrict to
pats w/o
adherent
contrapatients
indications
Restrict to
RCT
inclusion
criteria
RCT population
25
Summarized results of restriction
Cohort
Unadjusted RR
Fully adjusted RR
0) Incident and prevalent statin users versus non-users
0.32 (0.30, 0.33)
0.62 (0.58, 0.66)
1a)) Incident
d
statin users versus non-users
0.39 (0.36,
(
0.42))
0.65 (0.59,
(
0.72))
1b) Incident statin users versus non-users matched on
Rx or visit date
0.41 (0.38, 0.44)
0.68 (0.61, 0.75)
2) Incident statin users versus incident glaucoma
medication users
0.56 (0.51, 0.62)
0.79 (0.70, 0.88)
3) PS trimmed incident statin users versus incident
glaucoma medication users
0.62 (0.56, 0.69)
0.78 (0.69, 0.89)
4) Incident and adherent statin users versus incident
and adherent glaucoma medication users
0 64 (0
0.64
(0.55,
55 0.74)
0 74)
0 80 (0
0.80
(0.67,
67 0.95)
0 95)
5) PROSPER Eligible
0 72 (0.57,
0.72
(0 57 0.92)
0 92)
0 79 (0.60,
0.79
(0 60 1.03)
1 03)
26
Schneeweiss et al Med Care 2007
Results* in comparison to RCTs
1.2
PROSPER
M
Mortality
Rate Ra
atio
1
(70-82, prim +sec
prevention)
P
Pravastatin
t ti
0.8
(pooling
65+,
4S
LIPID, CARE)
(65+, secondary
prevention)
0.6
0.4
0.2
0
0
1a
1b
1c
2
3
4
5
Cohort restriction
* Unadjusted mortality rate ratios
27
1st decision point:
Prevalent and/or incident users?
 Problems with mixed prevalent/incident user cohorts
 Under
Under-ascertainment
ascertainment of events related to drug initiation
 In prevalent users covariates will be assessed after initial
p
and mayy be the consequence
q
of treatment
exposure
 Prevalent users are by definition adherent to index drug
(=survivor cohort)
 Duration of use needs to be adjusted
28
Time-varying hazards
Hazard fu
unction
(instanta
aneous ris
sk)
EXAMPLES:
- HRT and CHD/MI
- Statins and muscle pain
p y
p
- CYP polymorphisms
Time since start of exposure
 Reasons?
 Biology
g in cohort composition
p
 Change
29
Depletion of Susceptibles
 Past experience modifies current risk
 P
Persons who
h tend
t d tto remain
i on d
drug are those
th
who
h can
tolerate it
Susceptibles select themselves out of exposure
 “Susceptibles”
 Changes the population at risk
 Form of informative censoring
Askling J et al., Ann Rheum Dis 2007
30
Ray AJE 2004
31
A basic cohort design in longitudinal
healthcare claims data
Fixed covariate
assessment period
Follow-up
pp
period
Time
Initiation of exposure with study
and comparison drugs and start
of follow-up
32
Dealing with switchers: comparing
starters with starters and switchers with
switchers
First line use drug A
First-line
N d
No
drug use
S
First-line use drug B
Common 1st line use drug A
Switch to 2nd line drug C
S
Switch to 2nd line drug D
Common 1st line use drug A
Add drug C to A
S
Add drug D to A
33
Proportion of ne
ew users in
ation
popula
New drug
g on the block
100%
N l marketed
Newly
k t dd
drug
Calendar time
Old drug
Time off new drug
Ti
d
approval
Schneeweiss PDS 2010
34
Summary of restriction
 Powerful way to control bias
 “see” the amount of bias control achieved
 Need to start with a large source population
 Generalizable to routine care?
35
Back to confounding
36
The power of proxies
Measured confounders (C) may serve as redundant proxies
for unmeasured confounders (U):
Comorbidity
U
Age
C
Trt
Outcome
The more proxies the better…
37
Limited clinical information in admin databases
---------- ID=********** dob=**/**/1948 sex=M eligdt=1/2000 indexdt=6/2001
-------------------
Service Site of
___________Drug or Procedure________ ________Diagnosis_____
Date
Service Prov Type
Code
Description
* Code Description
---------------------------------------------------------------------------------------------10/01/00 OFFICE
Family Practice 90658 INFLUENZA VIRUS VACC/SPLIT
V048 VACC FOR INFLUEN
10/01/00 Rx
Pharmacy
CIPROFLOXACIN 500MG TABLETS
10
11/05/00 OFFICE
Family Practice 17110 DESTRUCT OF FLAT WARTS, UP
0781 VIRAL WARTS
11/07/00 R
Rx
Pharmac
Pharmacy
CIPROFLOXACIN 500MG TABLETS
10
01/15/01 Rx
Pharmacy
CIPROFLOXACIN 500MG TABLETS
10
06/25/01 OFFICE
Emerg Clinic
99070 SPECIAL SUPPLIES
* 84509 SPRAIN OF ANKLE
E927 ACC OVEREXERTION
06/30/01 OFFICE
Orthopedist
99204 OV,NEW PT.,DETAILED H&P,LOW * 72767 RUPT ACHILL TEND
06/30/01 OFFICE
Internist/Gener 99202 OV,NEW PT.,EXPD.PROB-FOCSD
* 84509 SPRAIN OF ANKLE
OUTPT HP Anesthesiologis 01472 REPAIR OF RUPTURED ACHILLES * 84509 SPRAIN OF ANKLE
Hospital
27650 REPAIR ACHILLES TENDON
* 84509 SPRAIN OF ANKLE
85018 BLOOD COUNT; HEMOGLOBIN
* 84509 SPRAIN OF ANKLE
Orthopedist
p
27650 REPAIR ACHILLES TENDON
* 84509 SPRAIN OF ANKLE
06/30/01 OFFICE
Orthopedist
29405 APPLY SHORT LEG CAST
* 72767 RUPT ACHILL TEND
07/30/01 OFFICE
Orthopedist
29405 APPLY SHORT LEG CAST
* 72767 RUPT ACHILL TEND
08/13/01 OFFICE
Orthopedist
L2116 AFO TIBIAL FRACTURE RIGID
* 72767 RUPT ACHILL TEND
Can we make better use
of this information ?
38
Multivariate adjustment
10 covariates
100 covariates
1000 covariates
39
MI Outcome (Unadjusted)
Cumulative
e Incide
ence
HR=2.11 (1.46-3.04)
111% (46%-204%)
Risk Increase
Statin Initiators
Statin Non-Initiators
Non Initiators
Months of Follow-Up
Seeger et al. PDS 2005
Propensity score analysis
 Goal: To identify patients with the identical likelihood of
receiving treatment but some will actually receive treatment
others will not.
not
 Estimation:
 St
Step 1
1: Estimate
E ti t the
th propensity
it ffor ttreatment
t
t as a
function of observed covariates:
- mimic the p
prescribers decision process
p
for treatment
- if exposure is prevalent then little limitations to modeling
- Predicted value is each patient’s “propensity score”
 St
Step 2
2: U
Use th
the estimated
ti t d propensity
it score tto adjust
dj t
treatment model:
- quintiles,
qu t es, deciles
dec es o
of p
propensity
ope s ty sco
score,
e, ttrimming
g
- PS matching
- Model adjustment
41
42
Fig. A: Propensity score matching
Patients
always treated
with study
drug
Patients
never treated
with study
drug
% off
subjects
0
0
05
0.5
1
Exposure propensity score
= treated with study drug
= treated with comparison drug
43
Fig. B: After matching
Patients
always treated
with study
drug
Patients
never treated
with study
drug
% off
subjects
0
0
05
0.5
1
Exposure propensity score
= treated with study drug
= treated with comparison drug
44
45
MI Outcome (After Propensity Score Matching)
31% (7%-48%)
Risk Reduction
Cumulative
e Incide
ence
HR=0.69 (0.52-0.93)
Statin Non-Initiators
Statin Initiators
Months of Follow-Up
PS matching provides results in
actionable and transparent ways
Relative
risk
Risk
difference
Number needed
to treat
Intended
treatment effect
Adverse effect 1
Adverse effect 2
Subgroup 1
Subgroup 2
Benefit-risk balance:
Values and
utilities
3
10
Investigator-specified
covariates
Empirically
Empiricallyspecified covariates
101
Confounding
Factors
New
Therapeutic
Health
Outcome
48
1,000,000-s of data
items and patterns
Mining codes,
t t patterns
texts,
tt
with new and
existing tools
The hd-PS SAS macro.
The hd-PS SAS macro can be downloaded at www.drugepi.org … links … downloads.
EXAMPLE CODE
th/t /
/di
t
/hd
%i l d "/
%include
"/path/to/macro/directory/hdps.mcr";
"
10,000-s of potential
confounding factors
Prioritization
according to
potential for
confounding
1000-s of
confounders
f
d
Statistical analysis using e.g.
propensity score
methods or
shrinkage
estimators
Title1 'High-dimensional propensity score adjustment';
Title2 '(study description)';
%RunHighDimPropScore (
var patient id
var_patient_id
var_exposure
var_outcome
vars_demographic
vars_force_categorical
top_n
k
trim_mode
percent_trim
input_cohort
input_dim1
input_dim2
input dim3
input_dim3
input_dim4
input_dim5
output_scored_cohort
output_detailed
results_estimates
results diagnostic
results_diagnostic
);
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
id,
id
exposed,
outcome,
age sex race,
year,
200,
500,
500
BOTH,
5,
master_file,
drug_claims
outpatient_diagnoses
inpatient_diagnoses
inpatient diagnoses
inpatient_procedures
outpatient_procedures
scored_cohort,
detailed_cohort,
estimates,
variable info
variable_info
generic_name,
icd9_dx,
icd9_dx,
icd9 dx
icd9_proc,
cpt,
www.drugepi.org
49
Flow chart for basic high-dimensional propensity score algorithm.
1.
Specify data sources
Define
D
fi p ddata
t dimensions;
di
i
use ddata
t stream
t
off 180 days
d
up to
t the
th initiation
i iti ti off study
t d exposure. This
Thi
includes diagnoses on the day of initiation but no drugs or procedures on the day of initiation.
Exclude selected codes from covariate adjustment
Base case: p = 8
1
2
3
4
5
6
7
8
MC Part A* MC Part A MC Part A MC Part A
MC Part A
Part B
Part B
Drugs
Hosp Dx
Hosp proc
Amb Dx
Amb proc Nurse Home
Dx
Proc
generic
(ICD-9)
(ICD-9)
(ICD-9)
(ICD-9)
(ICD-9)
(ICD-9) (CPT-4) entities
* MC = Medicare, Hosp = hospital, Amb = ambulatory, Dx = diagnosis, proc = procedure, Rx = prescription dispensings
2.
Investigator specified covariates
Demographics
Age, sex, race,
yyear
Predefined
Hx, Dx, Rx, Procs
Identify empirical candidate covariates
Within each data dimension sort by prevalence of codes. Identify the n most prevalent codes.
Base case: n= 200; Granularity = 3 digit ICD-9, 5 digit CPT, generic drug name.
3.
Walker. Ch. 9,
Observation
and Inference
1991
Assess recurrence
For each identified code create three covariates:
CovX once = 1 if that code appeared at least once within 180 days
CovX_once
CovX_sporadic = 1 if code appeared at least twice
CovX_frequent = 1 if code appeared at least three times.
4.
Prioritize covariates
Within each data dimension calculate for each covariate the possible amount of confounding it could adjust
in a multiplicative model given a binary exposure and outcome after adjusting for demographic covariates:
1
Biasmult = PC1 ( RRCD  1)  1 if RRCD 1, PC1 ( RR  1)  1 otherwise. Sort in descending order.
Bross. J Chron
Dis 1966
CD
PC 0 ( RRCD  1)  1
5.
Select covariates
PC 0 ( RR1CD  1)  1
Schneeweiss et al, Epidemiol 2009
Add d demographic covariates from step 1 and l predefined covariates in the top positions. Select top k
50
_
pp
CovX_sporadic = 1 if code appeared at least twice
CovX_frequent = 1 if code appeared at least three times.
4
4.
y
Prioriti e covariates
Prioritize
co ariates
Within each data dimension calculate for each covariate the possible amount of confounding it could adjust
in a multiplicative model given a binary exposure and outcome after adjusting for demographic covariates:
1
Biasmult = PC1 ( RRCD  1)  1 if RRCD 1, PC1 ( RR  1)  1 otherwise. Sort in descending order.
CD
PC 0 ( RRCD  1)  1
5.
PC 0 ( RR1CD  1)  1
Select covariates
Add d demographic covariates from step 1 and l predefined covariates in the top positions. Select top k
empirical covariates from step 4.
4 Optional,
Optional include multiplicative 2-way
2 way interactions for d demographic and l
predefined covariates with the top 20 empirical covariates.
Base case: d = 4 (age, sex, race, year); l = 14; k= 500
6
6.
Estimate exposure propensity score
Estimate propensity score using multivariate logistic regression, including all d + l + k covariates.
Truncate 5% of patients on either end of PS distribution and form deciles.
7.
Estimate outcome model
Estimate exposure-outcome association adjusted for propensity score deciles as well as PS weighted.
Schneeweiss et al., Epidemiol 2009
51
Table 1: Characteristics of 49,653 initiators of selective COX-2 inhibitors or
Example:
non-selective (ns) NSAIDs as defined during 6 months prior to first medication
use.
Coxibs vs.
nsNSAIDs and risk
of GI complications
p
in 180 days
Initiators of
Cox-2 selective
NSAIDs
N
%
Initiators of
nsNSAIDs
N
%
OR*
95% CI
N
32,042
Age75 years or older
24,079
75%
11,496
65%
1.61
1.545-1.674
Sex, % female
27,528
86%
14,293
81%
1.42
1.348-1.487
Race: white
30,583
95%
15,808
90%
2.39
2.23-2.57
black
1,133
4%
1,580
9%
0.37
0.34-0.403
other
326
1%
223
1%
0.80
0.68-0.95
Charlson comorbidity score >= 1
24,343
76%
12,521
71%
1.29
1.233-1.340
Use of >4 distinct drugs in prior year
24,120
75%
11,852
67%
1.48
1.421-1.541
>4 physician visits in prior year
22,919
72%
11,363
65%
1.38
1.328-1.437
Hospitalized in prior year
9,804
31%
4,591
26%
1.25
1.200-1.303
Nursing home resident
2,671
8%
996
6%
1.52
1.407-1.635
Prior use of gastroprotective drugs
8,785
27%
3,600
20%
1.47
1.407-1.536
1.407
1.536
Prior use of warfarin
4,252
13%
1,153
7%
2.18
2.041-2.337
Prior use of oral steroids
2,800
9%
1,373
8%
1.13
1.059-1.211
History of OA
15,549
49%
5,898
33%
1.87
1.802-1.945
History of RA
1 602
1,602
5%
476
3%
1 90
1.90
1 707 2 102
1.707-2.102
History of peptic ulcer disease
1,189
4%
426
2%
1.55
1.389-1.739
551
2%
196
1%
1.55
1.319-1.831
23,332
76%
12,363
70%
1.14
1.092-1.184
History of congestive heart failure
9,727
30%
4,328
25%
1.34
1.283-1.395
History of coronary artery disease
5,266
16%
2,603
15%
1.13
1.078-1.193
History of gastrointestinal hemorrhage
History of hypertension
* OR = odds ratio; CI = confidence interval
17,611
52
Table 3: Variations in covariate adjustment and relative risk estimates for the association of selective cox-2 inhibitors
Model #
p
within 180 days
y of first medication use.
and GI complications
Covariates included in
propensity
p
p
y score model
Number of
covariates
adjusted
j
Variables
tested
per data
source
Data source
granularity
g
y
Covariate
prioritization
algorithm
g
cstatistic
of PS
model
Outcome
model
Relative
risk
95% CI
N = 49,653
1
Unadjusted
-
1.09
0.91-1.30
2
Age, sex, race, year**
d 4
d=4
0.61
1.01
0.84-1.21
3
+ predefined covars (Tab1)
d=4; l=14
0.66
0.94
0.78-1.12
4
+ empirical covariates
d=4;l=14;k=200
n=200
3-digit ICD
Biasmult
0.69
0.86
0.72-1.04
5*
+ empirical covariates
d=4;l=14;k=500
n=200
3-digit ICD
Biasmult
0.71
0.88
0.73-1.06
Bootstrapped 95% CIs:
5b
Only demographics +
empirical
i i l covariates
i t
d=4;; k=500
n=200
3-digit
g ICD
Biasmult
0.71
0.87
0.73-1.06
0.72-1.05
53
Small sample performance
Example:
Coxibs vs
vs. nsNSAIDs and GI
complications in 180 days
Confounder prioritization now
with 0
0-cell
cell correction (+0.1)
hd-PS2, SAS 9.2 or higher,
substantially improved speed
20mins -> 2mins
27
56
83
110
277 events
54
55
56
Clopidogrel ‐ MI (2)
C ib ‐ GI bleed
Coxib
bl d
Statin ‐ death
TCA suicide (1)
0.60
0 30
0.30
0.00
‐0.30
only hdd‐PS
+ hhd‐PS
ad just.
+ speccified
covaars
Age‐s
age‐sex‐
‐sex‐
race‐y
yearyear
‐race
adjustm
adjument
sted
‐0.60
Unadjus
sted
Unadjuusted
log
elative
risk)
log (re
elative risk)
Performance of different adjustment procedures,
including hd
hd-PS
PS adjustment
57
0 30
0.30
only hdd‐PS
0.60
+ hhd‐PS
ad just.
+ speccified
covaars
Age‐s
age‐sex‐
‐sex‐
race‐y
yearyear
‐race
adjustm
adjument
sted
Unadjus
sted
Unadjuusted
log
elative
risk)
log (re
elative risk)
… and hd-PS adjustment alone
Clopidogrel ‐ MI (2)
C ib ‐ GI bleed
Coxib
bl d
Statin ‐ death
TCA suicide (1)
0.00
‐0.30
‐0.60
58
Kitchen sink models and the risk of
collider-stratification bias
U
U
C
Trt
IV
U
Trt
Outcome
Outcome
 M-bias confounding is usually considered weak
 Z-bias is a bit more likely: conditioning on treatment
will open a back-door path and an IV-like variable
will
ill become
b
a confounder
f
d
 Do we need variable un-selection?
Greenland, Epidemiol 2003
Brookhart, Schneeweiss et al. AJE 2006
59
Summary of Propensity score analyses
 Models treatment choice
 If exposure is frequent then very large numbers of
covariates can be adjusted
 It fits nicely with the incident user cohort design
 hdPS further improves confounding adjustment and
p
easyy and less prone
p
to
makes implementation
investigator error
p
analysis
y since the
 It is a veryy transparent
improvement in balance can be demonstrated
 Provides difference measures for fair benefit-risk
assessment
60
Sensitivity analyses
61
Basic design sensitivity analyses
(1) Varying wash
wash-out
out
periods for incident user
definition
(2) Follow-up until
discontinuation
vs. fixed follow-up
Time
(3) Varying length of
exposure risk window
Initiation of exposure with study
and comparison drugs and start
of follow-up
62
Exposure risk window
Allergic reaction?
Cancer?
Bacterial infections?
Cancer?
There is no right and no wrong.
rong Yo
You need to arg
argue
e your
o r
case based on event of interest, the biology, PK and PD
63
Expl: Coronary revascularization
Bare metal vs.
vs drug-eluting
drug eluting stents
Death
MI
“Landmark analysis”
64
Sensitivity
y analysis
y (array
(
y approach))
6.0
50
5.0
RRadjusted
4.0
Fixed:
ARR = 2.0
PC0 = 0.5
3.0
20
2.0
4.5
1.0
25
2.5
0.8
0.9
1.0
0.6
0.7
0.4
0.3
0
0.2
0
0.1
0.8
0.5
0.0
0. 0
RRCD
PC1
65
Schneeweiss PDS 2006
Sensitivity
y analysis
y (rule
(
out approach))
* Plotted is the strength of the associations between an unmeasured confounder and treatment choice
(OREC) and the association between an unmeasured confounder and disease outcome (RRCD) that are
required to fully explain the observed association (ARR = 3.38) or its lower 95% confidence limit (ARR =
1 88) Any factor (a single factor or combination of multiple factors) that has a combination of RRCD and
1.88).
OREC values resulting in points higher than and to the right of the plotted lines will be able to fully explain
our observed results.
Conclusions on methods
 Case-crossover analyses are valuable but there are
only few applications in CER
 Propensity score analyses are very valuable in
claims data analyses for a wide range of CER
 Attention to basic epidemiology principles is
paramount
 Sensitivity analyses will help you put results in
perspective
67
Th k you very much
Thank
h
68
Recommended reading
 Overview:


Schneeweiss S. Developments in Comparative effectiveness research. Clin Pharm &
Th 2007.
Ther
2007
Rubin DB. Estimating causal effects from large data sets using propensity scores. Ann
Intern Med 1997
 Propensity scores:



Seeger JD et al. Analytic strategies to adjust confounding bias using exposure
propensity scores and disease risk scores. PDS 2005
D’Agostino. Propensity score methods for bias reduction in the comparison of a
treatment to a non
non-randomized
randomized control group.
group Stat Med 1998
Sturmer T et al. Analytic strategies to adjust confounding bias using exposure
propensity scores and disease risk scores. AJE 2005
 Instrumental variables



Brookhart MA et al. Evaluating short-term drug effects in claims databases using
prescribing preferences as an instrumental variable. Epidemiology 2006
Brooks J et al. Heterogeneity and the interpretation of treatment effect estimates
from risk adjustment and instrumental variable method.
method Med Care 2007
Schneeweiss et al. Simultaneous assessment of short-term gastrointestinal benefits
and cardiovascular risks of selective COX-2 inhibitors and non-selective NSAIDs: an
instrumental variable analysis Arth & Rheum 2006
69