Web Appendix to:
Prospective cohort studies of newly marketed medications:
Using covariate data to design large-scale studies
August 5, 2013
Derivation of bias
The derivation of bias due to inappropriate model extrapolation is due directly to the
derivations presented in Drake & McQuarrie1. Specifically, we assume that the true
underlying outcome model is given by
log{𝐸(𝑌𝑖 |𝑇𝑖 , 𝑋𝑖 )} = 𝛽0 + 𝛽𝑇 𝑇𝑖 + 𝛽𝑋 𝑋𝑖 + 𝛽𝐻 𝑇𝑖 𝑋𝑖
so that the true average conditional rate ratio (RR) treatment effect on the log scale is given
by the mean of 𝛽𝑇 + 𝛽𝐻 𝑋̅ where 𝑋̅ is the mean of X in the population; for estimation
purposes we substitute with the mean in the sample. We assume that the treatment effect
parameter is estimated via a log-linear model that does not account for treatment effect
heterogeneity:
log{𝐸(𝑌𝑖 |𝑇𝑖 , 𝑋𝑖 )} = 𝛽̂0 + 𝛽̂𝑇 𝑇𝑖 + 𝛽̂𝑋 𝑋𝑖
Therefore, the bias in 𝛽̂𝑇 for 𝛽𝑇 + 𝛽𝐻 𝑋̅ is
Bias(𝛽̂𝑇 ) = 𝐸(𝛽̂𝑇 ) − (𝛽𝑇 + 𝛽𝐻 𝑋̅)
Drake & McQuarrie provide a calculation for the bias due to an omitted confounder (in our
case, the omitted interaction 𝑇𝑖 𝑋𝑖 ) in a generalized linear model:
Drake C. and McQuarrie A. A note on the bias due to omitted confounders. Biometrika.
1995; 82(3): 633—8.
1
Bias(𝛽̂𝑇 ) = 𝐸(𝛽̂𝑇 ) − 𝛽𝑇
≈ 𝛽𝐻 [𝐸(𝑋𝑇|𝑇 = 1) − 𝐸(𝑋𝑇|𝑇 = 0) − {𝐸(𝑋|𝑇 = 1) − 𝐸(𝑋|𝑇 = 0)}
×
𝑝ℎ′ (𝛽0 + 𝛽𝑇 )cov(𝑋, 𝑋𝑇|𝑇 = 1) + 𝑞ℎ′ (𝛽0 )cov(𝑋, 𝑋𝑇|𝑇 = 0)
]
𝑝ℎ′ (𝛽0 + 𝛽𝑇 )var(𝑋|𝑇 = 1) + 𝑞ℎ′ (𝛽0 )var(𝑋|𝑇 = 0)
where 𝑝 = Pr⁡(𝑇 = 1), 𝑞 = 1 − 𝑝, and ℎ′ (𝛽0 + 𝛽𝑇 ) indicates the first derivative of the
inverse link function, evaluated at 𝛽0 + 𝛽𝑇 . Note that we use different notation than what
was originally used by Drake & McQuarrie; their 𝜇, 𝛼, 𝛽1, and 𝛽2 correspond to our 𝛽0, 𝛽𝑇 ,
𝛽𝑋 , and 𝛽𝐻 . In addition, their 𝑋1 and 𝑋2 correspond to our X and XT, and we have adjusted
the above equation to reflect our use of T in {0,1} rather than {-1,1} to indicate treatment
assignment. To arrive at the equation for bias presented in the main body of the paper, we
simplify the above expression and subtract off the remaining term for the true average
treatment effect:
Bias(𝛽̂𝑇 ) = 𝐸(𝛽̂𝑇 ) − (𝛽𝑇 + 𝛽𝐻 𝑋̅)
= [𝐸(𝛽̂𝑇 ) − 𝛽𝑇 ] − 𝛽𝐻 𝑋̅
= 𝛽𝐻 [𝑋̅1 − 0 − {𝑋̅1 − 𝑋̅0 } ×
= 𝛽𝐻 [𝑋̅1 − 𝑋̅ − (𝑋̅1 − 𝑋̅0 )
𝑝𝑒 𝛽0 +𝛽𝑇 𝜎̂12 + 𝑞𝑒 𝛽0 ∗ 0
] − 𝛽𝐻 𝑋̅
𝑝𝑒 𝛽0+𝛽𝑇 𝜎̂12 + 𝑞𝑒𝛽0 𝜎̂02
𝑝𝑒 𝛽𝑇 𝜎̂12
]
𝑝𝑒𝛽𝑇 𝜎̂12 + (1 − 𝑝)𝜎̂02
Simulation study
To evaluate the validity of this approximation in the context of ommitted interaction
effects, we performed a small simulation study. In this study, we set 𝛽0 = −2 and randomly
selected 20 true parameter values for each of the other parameters to define 20 datagenerating scenarios:
𝛽𝑇 ~𝑈𝑛𝑖𝑓(0, .25)
𝛽𝑋 ~𝑈𝑛𝑖𝑓(0, .1)
𝛽𝐻 ~𝑈𝑛𝑖𝑓(−.15, .15)
For each of the 20 randomly-selected data-generating scenarios, we simulated 500 datasets
with the following variables:
𝑇~𝐵𝑒𝑟𝑛𝑜𝑢𝑙𝑙𝑖(. 5)
𝑋𝑇=1 ~𝑁(mean = 2,⁡sd = ⁡ .5);⁡𝑋𝑇=0 ~𝐸𝑥𝑝(rate = 2)
𝑌~𝑃𝑜𝑖𝑠𝑠𝑜𝑛(exp{𝛽0 + 𝛽𝑇 𝑇𝑖 + 𝛽𝑋 𝑋𝑖 + 𝛽𝐻 𝑇𝑖 𝑋𝑖 })
Treatment effect was estimated as the coefficient on treatment in each dataset using a
generalized log-linear Poisson model: log{𝐸(𝑌𝑖 |𝑇𝑖 , 𝑋𝑖 )} = 𝛽̂0 + 𝛽̂𝑇 𝑇𝑖 + 𝛽̂𝑋 𝑋𝑖 . The true bias
for each data-generating scenario was found by taking the difference between the
estimated treatment effect and the true log RR,⁡(𝛽𝑇 + 𝛽𝐻 𝑋̅), and averaging across the 500
simulated datasets. Estimated bias was calculated in each dataset using the approximation
above, and then averaged across the simulated datasets within each scenario. Appendix
Figure 1 shows the comparison of the true bias in each simulation scenario and the average
estimated bias, which indicated a strong correlation between true bias and estimated bias
with some random variation.
Appendix Figure 1: The true bias in each simulation scenario on the x-axis versus the
average estimated bias for each scenario on the y-axis.
Details of Example Study
We implemented methods in a cohort study of the short-term effects of Cox-2 inhibitor
(celecoxib, rofecoxib, or valdecoxib) use on gastrointestinal (GI) toxicity and myocardial
infarctions (MI) versus ns-NSAIDs. Our study population was pulled retrospectively from
patients enrolled in Medicare and either the Pharmaceutical Assistance Contract for the
Elderly (PACE) provided by the state of Pennsylvania or the Pharmaceutical Assistance to
the Aged and Disabled (PAAD) provided by New Jersey. However, we mimicked a
prospective pilot phase study design by evaluating patients sequentially, as they initiated
treatment, beginning in the early post-marketing period of each Cox-2 inhibitor. Celecoxib
was approved by the United States Food and Drug Administration (FDA) on December 31,
1998. Rofecoxib was approved on May 20, 1999, and was pulled from the market by the
manufacturer on September 30, 2004. Valdecoxib was approved on November 20, 2001,
and was pulled from the market on April 7, 2005.
We included new oral NSAID users age 65 years and older that filled a prescription
for an oral preparation of a ns-NSAID or selective Cox-2 inhibitor between January 1, 1999,
and December 31, 2005. A new user is defined as a patient with no use of the index drug in
the 365 days prior to their index fill. Therefore, patients initiating celecoxib could have
prior use of other study drugs. In order to assess past NSAID use and covariate status,
patients were required to have continuous Medicare and PACE/PAAD enrollment during
the 365 days prior to treatment initiation and to display continuous health system service
use, defined as at least one prescription drug claim and one healthcare claim in each of two
180-day periods preceding NSAID initiation. Exposure was classified as ns-NSAID,
celecoxib, rofecoxib, or valdecoxib, based on the index prescription. Covariates were
created to capture known risk factors of NSAID-associated gastrotoxicity and acute MI and
were assessed based on healthcare and prescription claims in the 365 days prior to the
index prescription.
Additional Figures
Appendix Figure 2: Example covariate distributions that display balance and overlap. The
covariate distribution for treated and untreated patients is plotted in black and red,
respectively. In the left panel, the dashed lines indicate the region of overlap (the area
between the lowest value among the exposed and the highest value among the
untexposed).
Appendix Figure 3: Estimated percent of patients lying in the region of overlap on the
propensity score after restricting the cohort to patients with no prior NSAID use, separately
for patients at each interim analysis and within each state. Nonselective-NSAID patients are
the reference in all analyses.