Evaluating the effect of change on change: a different viewpoint
Eyal Shahar, MD, MPH
Address:
Eyal Shahar, MD, MPH
Professor
Division of Epidemiology and Biostatistics
Mel and Enid Zuckerman College of Public Health
University of Arizona
1295 N. Martin Ave.
Tucson, AZ 85724
Email: Shahar@email.arizona.edu
Phone: 520-626-8025
Fax: 520-626-2767
1
Abstract
When a causal variable and its presumed effect are measured at two time points in a
cohort study, most researchers prefer to fit some type of a change model. Many of them
believe that such an analysis is superior to a cross-sectional analysis “because change
models estimate the effect of change on change”, which sounds much stronger
epistemologically than “estimating a cross-sectional association”. In this article I trace
two commonly used regression models of change to their cross-sectional origin and
explain that these models do not test anything conceptually different from cross-sectional
models. A change model is superior to a cross-sectional model mainly because it
corresponds to a self-matched design. Therefore, the main advantage of regressing
“change on change” is complete control of time-stable confounders, a key concern in
observational studies. Many analysts fail to realize that that important advantage is lost if
a random effect model is used.
2
Introduction
The most rudimentary example of longitudinal data analysis may be a cohort study where
the putative cause and its putative effect are measured twice: at baseline and again some
time during follow up. In such cases researchers welcome the opportunity to study “the
effect of change on change”, intuitively assuming that longitudinal changes, rather than a
cross-sectional association, is a different kind of effect. After all, causal effects are
intuitively grasped as “the change in my value of Y, if my value of X were changed”. In
this article I suggest that intuition leads us astray. A model of changes between two time
points is not testing anything conceptually different from a cross-sectional model, and its
key advantage operates in the domain of confounding. Other views are presented as well.
To provide a concrete example, rather than use generic notation, I will consider variables
from a question of interest to sleep researchers: what is the effect of disordered breathing
during sleep (of which sleep apnea is the most severe manifestation) on systolic or
diastolic blood pressure (BP)? Although measuring sleep-disordered breathing is a
challenge, a commonly used variable is called the respiratory disturbance index (RDI)—
the average number of abnormal breathing episodes per hour of sleep. The data structure
is simple. In a typical cohort study the RDI and BP are measured twice, several years
apart.
Time 1
RDI1
BP1
Time 2
RDI2
BP2
3
In the interest of simplified notation, the subscript “i” (indicating the i-th person) is
omitted throughout.
The first type of a change model
To study the effect of the RDI on BP, we may fit two cross-sectional linear regression
models, namely, regress BP1 on RDI1 and regress BP2 on RDI2. Most researchers,
however, will prefer to fit some type of a change model, the simplest of which is
BP2–BP1 = β0 + β (RDI2–RDI1) + ε
(Model 1)
This change model, however, could be traced to simple subtraction of two cross-sectional
models—provided that the effect of the RDI on BP is forced to be identical at both
times:[1, page 10]
BP2 = μ2 + β RDI2 + γZ + α + ε2
(Model 2)
BP1 = μ1 + β RDI1 + γZ + α + ε1
(Model 3)
BP2–BP1 = β0 + β (RDI2–RDI1) + ε
(Model 1)
Where,

Z is a vector of confounders whose effect on BP is identical at the two time points.

α = all differences between persons that remain constant over time (and are not
captured by γZ)

β0 = μ2 – μ1

ε = ε2 – ε1
4
Comparing model 1 (the change model) to models 2 and 3, we realize that we haven’t
really studied any unique causal idea of change that was missing from the two crosssectional models. The coefficient of the change variable (β)—the so-called effect of
change from model 1—is not different from the coefficient of the cross-sectional effect.
Notice, again, that we had assumed identical cross-sectional effects of the RDI on BP,
which implies no effect modification by time on an additive scale, where “time” is a
surrogate for time-dependent variables.
So why fit the “change model” instead of fitting two cross-sectional models?
As shown above, the key advantage is control of all time-stable confounders (both
measured and unmeasured Z variables), which are eliminated by subtracting the crosssectional models.[1, page 10] No cross-sectional model alone offers that important
advantage!
Interestingly, we may reach the very same conclusion about the control of confounding
from the viewpoint of study design, too. The change model is essentially derived from a
self-matched cohort design, because we estimate the RDI effect on BP within each person
(and then take the average over all people.) Of course, by matching on personal identity,
the person-specific estimates of the RDI effect cannot be confounded by any
characteristic that has not changed between the two time points, such as gender, ethnicity,
and genes. Confounders of the within-subject association (i.e., time-varying
5
confounders) are not eliminated by subtraction. Nonetheless, it may be argued (but
cannot be tested) that unknown between-subject confounders usually outnumber, and are
perhaps more important, than unknown within-subject confounders. The differences
between people at any time are usually more striking than the differences within a person
over time.
It is perhaps easier to recognize the matching property of the model if the continuous
causal variable, RDI, is replaced by a binary (0, 1) variable, say, taking an antihypertensive drug (DRUG). Then, the difference BP2–BP1 for a person who is observed
at both DRUG=1 and at DRUG=0 is derived from two BP values that such a person has
contributed: once as exposed (DRUG=1) and once as unexposed (DRUG=0). We also
see that people who took the drug at both time points, or at neither time point, do not
contribute anything to the estimate. Stated differently: the change model in a cohort
study may be viewed as the observational counterpart of the randomized crossover trial.
In both designs people alternate between at least two exposure values.
Finally, notice that the change model has no built-in conditioning. We do not condition
the association of RDI2 with BP on RDI1, because both coefficients are forced to have the
same absolute value (BP2 – BP1 = β0 + β RDI2 – β RDI1 + ε). From the perspective of
causal diagrams,[2] the model simply assumes that RDItBPt (where t=1, 2).
6
The second type of a change model
So far we have forced homogeneity of the RDI effect on BP at both times. From a
theoretical standpoint, we may often accept that assumption: Why would the effect of the
RDI be different at two arbitrary time points, especially when no relevant causal
reasoning has dictated the choice of the time points at which these variables were
measured? Furthermore, each measured RDI presumably serves as a surrogate for a
summary of historical RDI values. Sleep researchers do not assume that the value on a
single night determines a contemporaneous blood pressure value.
Nonetheless, it is easy to relax the homogeneity assumption and derive a different kind of
a “change model”:[1, page 15]
BP2 = μ2 + β2 RDI2 + γZ + α + ε2
BP1 = μ1 + β1 RDI1 + γZ + α + ε1
After subtraction: BP2–BP1 = β0 + β2 RDI2 – β1 RDI1 + ε
(Model 4)
With reorganization of the last equation, we may model BP2 alone as the dependent
variable and move BP1 to the right hand side (as part of a person-specific intercept or as
an offset variable):
BP2 = [BP1 + β0] + β2 RDI2 – β1 RDI1 + ε
7
(Model 5)
Again, as we see above the key advantage of this model is control of all time-stable
confounders, both measured and unmeasured. But there is more to be said: Since we
allowed the effect of RDI2 to differ from that of RDI1, we have also conditioned the
associations. This model has four key properties:
1) The association of RDI2 with BP2 is conditional on RDI1
2) The association of RDI1 with BP2 is conditional on RDI2
3) Both conditional associations control all time-stable confounders
4) The coefficient of RDI1 (namely, β1) may be biased! In particular, if RDI1 affects
RDI2
A causal diagram perspective of model 5
To understand the fourth property of model 5, we should recall two principles of directed
acyclic graphs (causal diagrams):[2, 3]
1) The marginal association between two variables reflects causal paths between them
(the effect of interest) and back-door paths (confounding).
2) Conditioning on a common effect of two variables (a collider) creates an association,
or contributes to the association, between the colliding variables.
Figure 1 shows a possible, realistic, causal diagram according to which RD1 affects RDI2
whereas RDI1, RDI2 and BP2 share a common cause, U (a confounder of the effect of RDI
on BP2.) A causal chain from RDI1 to BP2 via BP1 is also depicted.
8
U
RDI1
RDI2
BP2
BP1
Figure 1. A theoretical causal diagram for RDI and BP, when measured at two time
points.
There are two reasons why we should not condition on RDI2 when we try to estimate the
effect of RDI1 on BP2: First, part of the effect of RDI1 on BP2 is mediated via RDI2.
Conditioning on an intermediary variable on a causal pathway eliminates some of the
total effect, which is unwise. Second, since RDI2 is a collider on a path from RDI1 to BP2
(via U), conditioning on RDI2 violates a key rule of directed acyclic graphs for estimating
effects,[3] because it opens a non-causal associational path between RDI1 and BP2
(RDI1—UBP2).
Therefore, when heterogeneity of the two cross-sectional associations is allowed, we
should ignore the coefficient of RDI1. Only the coefficient of RDI2 is informative
(assuming no important within-person confounding). So what do we gain from allowing
such heterogeneity? Or in other words: what do we gain from conditioning the
association of RDI2 with BP2 on RDI1? We may gain the blocking of confounding paths
via RDI1, such as the back-door path (thick arrow) that is shown in Figure 2. If a high
value of BP1 has triggered the prescription of an antihypertensive medication (MED),
9
which in turn has affected BP2, conditioning the association between RDI2 and BP2 on
RDI1 should block this confounding path. When we regress BP2 on RDI2 alone (a crosssectional association at time 2), this confounding path remains open.
RDI1
RDI2
BP2
MED
BP1
Figure 2. A theoretical causal diagram according to which the cross-sectional
association of RDI2 and BP2 contains confounding by RDI1
Other views
Textbooks on longitudinal data analysis recognize, of course, the benefit of change
models for the controlling of time-stable confounders, but they apparently follow the
assumption that “a longitudinal effect” is a different causal idea from “a cross-sectional
effect”. For example, the authors of one book [4] seem to suggest conceptual difference
between what they call βC (the cross-sectional effect; my β1) and what they call βL (the
effect of longitudinal change according to their parameterization; my β2). The following
citations illustrate their viewpoint:
“…the prime advantage of a longitudinal study is its effectiveness for studying
change.”[4, page 17]
10
“…the major advantage of a longitudinal study is its ability to distinguish the crosssectional and longitudinal relationships between the explanatory variables and the
response.”[4, page 23]
Like others, these authors acknowledge the benefit of controlling time-stable confounders
and recommend conditioning on the baseline causal variable: “…one may simply view
Xi1 [my RDI1] as a confounding variable whose absence may bias our estimate of the true
longitudinal effect [my β2, the coefficient of RDI2].”[4, page 24]
It appears that the effect that is confounded in their mind is “the true longitudinal effect”.
As shown here, however, that longitudinal effect is simply the cross-sectional effect at
time 2, after controlling for all time stable covariates and the baseline causal variable.
The opportunity to partition the total variance into within-person and between-person
components has led statisticians to propose another parameterization of longitudinal
data.[5] The pertinent expression (for the i-th subject) takes the following form:
BPt = βt + βwithin (RDIt – mean RDI) + βbetween (mean RDI) + γZ + α + εt
where t=1, 2
Those who proposed that parameterization tell researchers to estimate two kinds of
exposure effects in longitudinal studies:[5] “an overall effect of age [here, RDI] on
response [here, BP]” as estimated by the coefficient βbetween, and “the effects of deviations
11
from the average age [here, RDI]” as estimated by the coefficient βwithin. Furthermore,
they argue that researchers typically fail to distinguish between these two kinds of effects,
and that models that incorrectly assume βbetween = βwithin “can lead to very misleading
assessments of the association of covariates with response”.[5]
Indeed, one of these associations, or both, could be misleading, but “association” is not
synonymous with “causation”, and causation is not partitioned into causation by “mean
RDI” and causation by “deviation from mean RDI”. If the idea of different causal
effects were valid, we should have never compared, for example, the estimated effect of a
drug from a parallel group trial (“between-patient effect”) to the estimated effect of that
drug from a cross-over trial (“within-patient effect”). Moreover, if the idea were valid,
analysts of cross-over trials would have regularly reported “both kinds of effects”
(because the “between-patient effect” can also be estimated from that design). Evidently,
they don’t.
In the absence of time-dependent confounders (i.e., within-subject confounders), the
difference between βbetween and βwithin in a non-randomized study may be easily explained
by unknown between-subject confounders, although competing explanations are always
possible.[6] Moreover, typical multivariable methods to estimate these coefficients, such
as a random-effect model and GEE, often sacrifice the controlling of all time-stable
confounders for the return of a smaller standard error of β—a classical trade-off between
bias and variance.[7] But epidemiologists who realize the last point would rarely be
sympathetic to the trade-off. In the absence of within-subject confounding, they don’t
12
usually care about optimal use of a covariance matrix that will minimize the standard
error—of a confounded estimator of effect.
Finally, in another textbook on applied longitudinal data analysis, the authors dismiss
altogether the studying of change between two time points “because two-wave
studies…confound true change with measurement error.”[8, page 10] Apparently, the
key benefit of two wave studies—controlling of all time-stable confounders—is not
deemed that important to these authors. As we see, statisticians speak in many voices
about the meaning and interpretation of change models.
Conclusion
The two models of the effect of change in RDI on change in BP simply estimate
coefficients of cross-sectional associations. They offer control of time-stable
confounders, which is not offered by any isolated cross-sectional association, and that’s
the main reason for preferring these models—whenever we assume that unknown withinsubject confounding is a much smaller threat than unknown between-subject confounding
(unfortunately, an untestable assumption). That important advantage is usually lost if a
random effect model is used.
If heterogeneity of the cross-sectional associations is allowed, then only the coefficient of
RDI2 should be reported, and in that case, we are also controlling all confounding paths
13
via RDI1. Most important, and contrary to prevailing thought, a model of changes
between two time points does not estimate any special causal idea called “longitudinal
effect”.
14
References
1.
Allison PD. Fixed effects regression methods for longitudinal data using SAS.
Cary: SAS Institute Inc. 2005.
2.
Greenland S, Pearl J, Robins JM. Causal diagrams for epidemiologic research.
Epidemiology 1999;10:37-81
3.
Hernán MA, Hernández-Diaz S, Robins JM. A structural approach to selection
bias. Epidemiology 2004:15:615-625.
4. Diggle PJ, Liang K-Y, Zeger SL. Analysis of longitudinal data. New York: Oxford
University Press, 1995.
5. Neuhaus JM, Kalbfleisch JD. Between- and within-cluster covariate effects in the
analysis of clustered data. Biometrics 1998;54:638-645.
6. Mancl LA, Leroux BG, DeRouen TA. Between-subject and within-subject statistical
information in dental research. J Dent Res 2000;79:1778-81.
7. Palta M, Yao T-J. Analysis of longitudinal data with unmeasured confounders.
Biometrics 1991;47:1355-69.
15
8. Singer JD, Willett JB. Applied longitudinal data analysis: modeling change and event
occurrence. New York: Oxford University Press, 2003.
16