Ron Heck, Fall 2011
EDEP 768E: Seminar in Multilevel Modeling
1
Nov. 13, 2011
Week 13:
Extending the Examination of Growth
In this week’s notes, I’ll expand on the introduction to methods that can be used to examine
changes in individuals presented in Chapter 5 of the text. As noted previously, time is often a key
factor in understanding how developmental processes unfold, as well in observing their impact
on outcomes. Because of the challenges that longitudinal data present (e.g., strong temporal
dependencies that can be present in repeated observations), in the past there were fewer options
available for investigating change. Again, we’ll consider the situation where we wish to examine
development in students’ reading scores over time, which might be related to background
variables such as socioeconomic status, a specific treatment variable, or perhaps differences in
motivation. This latter type of variables could be specified as a time-varying covariate. Timevarying covariates provide a way of accounting for temporal variation that may increase (or
decrease) the value of an outcome predicted by the individual’s growth trajectory (Raudenbush
& Bryk, 2002). Alternatively, we could also consider motivation as a “static” variables (i.e., the
average motivation level), which requires only one value for the covariate. In this latter case, we
would then enter the covariate at the between-subjects level (i.e., level 2 in the multilevel
formulation). One of the advantages time-varying formulation, however, is that the effect of
motivation levels on student achievement can then be modeled as a random parameter at level 2
(between individuals) or level 3 (between groups).
In studying individual development, we assume that individuals have been sampled and
measured on several occasions on one or more time-varying variables. Because multilevel
modeling does not require balanced data, it is not a problem if all measurements are not available
on all participants. This can be beneficial if there are subjects who drop out during a longitudinal
study. This type of multilevel formulation can be extended to include situations where the timing
and spacing of the measurements differ across individuals. As noted previously, in randomcoefficients growth modeling, within- and between-individual changes are typically represented
through a two-level univariate model. We can extend this basic univariate model to include more
than one growth process, and we can also add group processes above level 2.
Examining the Shape of the Change
As Hox (2010) notes, when a categorical variable represents an increasing level of a dosage or
successive measurements, the trend is often examined using polynomial curves. Polynomial
curves are often used for describing individual developmental processes because they can be
estimated using standard linear modeling procedures, and they are very adaptable to a variety of
different growth trajectories. The polynomial equation is not nonlinear; in fact, mathematically it
is linear. If there are k measurements there is a k-1 polynomial that can be used to fit the model.
So, for example, if we have four measurements, the highest polynomial would be cubic. We
often only include up to the highest significant component (e.g., cubic, quadratic) in the model.
Although many nonlinear functions can be approximated using a polynomial function, in some
cases, the nonlinear function may not be well approximated in this manner (Hox, 2010). For
(EDEP 768) Week 13: Extending the Examination of Growth
2
example, the logistic curve (which cannot be transformed to a linear model) may be better in
approximating a process that is slower in the beginning, speeds up in the middle, and then slows
at the end. Often, however, logistic and exponential functions can be well approximated by using
a cubic polynomial (Hox, 2010). It is also important to note that in the random-coefficients
approach, the time-related variable describing the shape of the growth trajectory (e.g., linear,
quadratic, or cubic polynomial) is entered into the data set as a variable, with the successive
measurements and polynomial components structured vertically for each subject. The number of
lines for each individual is defined by the number of measurement occasions.
This specification of change is in contrast to the repeated-measures ANOVA approach or the
structural equation modeling (SEM) approach (which is often referred to as latent change or
latent curve analysis), where the successive measurements in the data set are structured
horizontally (i.e., where there is typically one line per subject). In contrast to the randomcoefficients approach, where the polynomial components describing the shape of the growth
trajectory are entered as data for each individual, in the SEM approach, the shape of the growth
trajectory is defined by the pattern of factor loadings on a growth latent factor. The latent change
approach can actually facilitate investigating a number of different nonlinear growth models
where the change over time may not follow a typical polynomial growth model.
Specifying the Level-1 Model
There are a number of different ways to notate individual developmental processes. Following
Raudenbush and Bryk’s notation (2002), the systematic growth for each individual in reading
can be represented as a polynomial of degree P, with the level-1 model at time t for individual i
written as
ytj = 0i + 1j ati +2ia2tj + ... + ipaPtj + εtj,
(1)
where 0i is the intercept parameter (i.e., defined as the level of the individual’s “true” math
achievement status at some relevant point in the series of measurement occasions), ati is a timevarying variable of interest, with individual i measured on t occasions, and ip is the growth
trajectory parameter p for subject i associated with the polynomial of degree P (i.e., p = 0,...,P).
One or more time-varying covariates (Xt) can also be added to the level-1 model as needed.
Most commonly, a simple error structure is assumed for εti, represented as
ε tj ~ N(0, σ2ε),
(2)
which suggests the errors are independent and normally distributed (N), with a mean of zero and
constant variance across time (Raudenbush & Bryk, 2002). The residual variance parameter (σ2ε)
represents the variation in residuals around each individual’s true change trajectory.
Restrictions about the within-individual residuals can be relaxed, however. Sometimes it is
useful to define different error structures (e.g., diagonal, compound symmetry, autocorrelated, or
unstructured), depending on the nature of the repeated measurements per subject. More complex
error structures, however, can make it more difficult to arrive at a solution that converges (i.e.,
provides reasonable estimates for all proposed parameters).
(EDEP 768) Week 13: Extending the Examination of Growth
3
The growth trajectory (slope) parameter is the most important parameter in a longitudinal model
because it represents the rate at which individual i changes over time. The model facilitates the
representation of several different trajectories. In terms of interpretation, however, if one
considers that the linear trajectory model represents a constant rate of growth (or decline) over
time and the quadratic component represents change (accelerating or decelerating) in the rate of
growth over time, it becomes increasingly more difficult to interpret models of higher
polynomial degrees. Trajectories with significant linear and quadratic components are curvilinear
in shape. For example, a significant cubic component would suggest an accelerating or
decelerating change in the changing rate of growth over time. In is important to note, however,
that the parameters of the polynomial model have no direct interpretation in the growth process,
such that interpretation is best made by looking at average plots of the growth or a subset of
typical growth curves (Hox, 2010).
The specification of the general level-1 model implicitly assumes that all the change trajectories
have a common algebraic form, but not every individual student has the same trajectory (Singer
& Willett, 2003). Each individual, therefore, draws her or his parameter values (i.e., intercept,
slope) from an unknown (underlying) bivariate distribution of intercepts and slopes. In Figure 1,
I have plotted the growth in an academic outcome Y (solid line) over four intervals against what
the growth might look like if it were linear, quadratic, or cubic (dotted lines). At the bottom of
the figure, linear growth rises (or declines) constantly over time. Quadratic growth tends to rise
more quickly (or decline) relative to the constant rate of growth represented in the linear
trajectory. In contrast, the cubic growth takes off much more rapidly over the four time periods.
The actual growth trajectory plotted in Figure 1 for Y may have elements of linear, quadratic,
and cubic growth. It appears to change linearly over interval 1 to 2, flatten between intervals 2
and 3(quadratic trajectory), and then accelerate more rapidly between interval 3 and 4 than the
linear growth occurring between interval 1 and 2. The cubic polynomial actually combines the
effects of the four coefficients (i.e., linear, quadratic, cubic coefficients, and the intercept).
Different values of each coefficient will move it up or down with respect to the horizontal axis
(i.e., changing the intercept), alter the steepness of the cubic curve (changing the cubic
coefficient), its slope (changing the linear component) or the curvature of the parabolic element
(changing the quadratic coefficient). When a cubic element is present in a trajectory, it suggests a
focus on the whole growth trend and not just any particular interval.
(EDEP 768) Week 13: Extending the Examination of Growth
4
Figure 1. Examining several different individual growth curves
30
Achievement
25
20
Y
linear
15
quadratic
cubic
10
5
0
1
2
3
4
Time
Coding the Time-Related Polynomials
There are different ways that we can code the time-related variables preliminarily. If the
measurements are evenly spaced, we can code the time-related variables to examine change per
each interval. For example, the linear component would be coded 0, 1, 2, 3. The corresponding
quadratic component would be coded 0, 1, 4, 9. The corresponding cubic component would be
coded 0, 1, 8, 27. Using COMPUTE, the linear coefficients can first be squared (linear*linear)
and then cubed (linear*linear*linear) and saved in the data set for the quadratic and cubic
components, respectively. Since in this example there are four measurements, the cubic
polynomial would be sufficient to capture the shape of the estimated change occurring over the
entire trend between time 1 and time 4. We can also accommodate unequal spacing between
measurements by adjusting this basic coding scheme.
One disadvantage of polynomial functions, however, is that they typically have high correlations
between the components comprising the function. With more complex growth trajectories this is
likely to become a problem. Because they must be defined as data in the data set, the linear,
quadratic, and cubic components are almost perfectly correlated. This can produce
multicollinearity problems, which can affect model estimation (Hox, 2010). I note in passing that
since in the SEM approach these variables do not have to be created and saved in the data set,
this problem does not arise. This is because the latent change trajectory is defined by factor
loadings. More specifically, the shape of the change is specified by the pattern of factor loadings,
and then this specification can be tested against the data and evaluated according to several
(EDEP 768) Week 13: Extending the Examination of Growth
5
model fit indices. One can easily use this type of level and shape latent curve formulation to
capture a wide variety of nonlinear growth processes (Raykov & Marcoulides, 2006).
Below we can observe the relatively strong correlations on these time-related parameters. The
table suggests the various time-related variables are quite highly correlated (with coefficients
equal to 0.91 or above). The number of individuals in the table reflects that there are 4
measurements on 9,363 individuals (9,363*4 = 37,452) from the sample data. The number of
individuals does not change the correlations.
If the occasions are equally spaced (as in our example with yearly measurements) and there is no
missing data, transforming the polynomials to orthogonal polynomials is a viable solution (Hox,
2010). As I show subsequently, this approach is typically used in repeated measures ANOVA.
Even in situations where the measurements may not be exactly balanced, using orthogonal
polynomials will reduce the multicollinearity problem (Hox, 2010). I used the following
polynomial contrasts to recode the time-related polynomials so that they would be orthogonal.
Because of the way the orthogonal polynomials are defined in the following table, the intercept
for the transformed model will be centered near the middle of the growth sequence (i.e., the
grand mean of the set of estimates). This is in contrast to the intercept being initial status (i.e.,
coded 0), if we do not transform the time-related variables.
Polynomial Contrasts
Lin
Quad
-3
-1
1
3
1
-1
-1
1
Cubic
-1
3
-3
1
(EDEP 768) Week 13: Extending the Examination of Growth
6
Once we apply these transformations the new correlations are now zero, as shown in the
following table:
Although not required, if desired, we can then standardize these orthogonal estimates so that they
will be on the same scale of measurement (Hox, 2010).
How Does Transforming the Polynomials Affect Model Estimation?
When the data are coded such that the initial measurement is 0, the multilevel-intercept will
reflect school achievement at the first measurement interval. The fixed-effect table suggests the
“true” initial math achievement level for schools, is 589.26. Moreover, the linear, quadratic, and
cubic coefficients are all significant (p < .001). This suggests all three are needed in defining
individuals’ growth in math over time. Because the time-related polynomials have not been recentered using orthogonal polynomials, the intercept can be interpreted as the Time 1 (or initial
status) estimate of the mean reading level for individuals. If we examine the initial estimates
from the Week 12 handout (Table 6), we can see the following estimates for reading.
(EDEP 768) Week 13: Extending the Examination of Growth
7
We might try to add the time-related estimates in the above table (106.07, -58.01, 10.24) to
obtain an estimate of the amount of growth taking place over time. When we do so, we get an
average growth per interval of approximately 58.3. In the table below, for the interval between
Time 1 (589.26) and Time 2 (647.56), the growth is 58.3 (647.56 – 589.26). The estimated
growth is quite accurate, therefore, for the initial interval. Over the rest of the study, however,
that estimate of growth over each interval seems too high. Between interval 2 and 3, for example,
we would estimate the growth as another 58.3. The estimated mean for Time 3 (647.56 + 58.3 =
705.86) would be much higher, however, than the actual mean for Time 3 in the table below
(651.30).
We can conclude from this initial model that the actual coefficients associated with each timerelated variable may not be directly interpretable in terms of the proposed developmental model
when there are higher polynomials present. The estimates are only accurate in describing the
initial interval. If the higher order polynomial terms were not significant, that is, if there were
only a linear trend, the rate of change would remain the same over each successive interval
comprising the trend.
We will next run the previous model with the transformed (orthogonal) polynomials.
(EDEP 768) Week 13: Extending the Examination of Growth
8
We can see that the intercept (637.52) now corresponds to the grand mean for the trend (which is
somewhere between interval 1 and interval 2). We can see that all three polynomial components
are significant (p < .001), again suggesting they should be retained in subsequent analyses. The
time-related estimates, however, no longer correspond to any particular time interval in the
study. It should be noted that transforming the polynomials does not affect the variance
components at level 1 or 2.
Examining Which Polynomial Components Vary Randomly Across Groups
Before proceeding to build a model, we need to decide which of the polynomial coefficients
should be considered as randomly varying across individuals. This concerns which polynomial
component (or components) we should build our successive models concerning differences in
growth trajectories by student background factors (e.g., gender, socioeconomic status). The best
advice is that it may depend on the specifics of the data set. I suggest trying to build a model that
is parsimonious but does justice to the particular features of the data set. This will usually require
some preliminary analyses. This part of developing the model can be challenging, since it is not
immediately clear which components may be randomly varying, and which components we
should actually build the explanatory model. Specifying all the components as random (with a
large data set such as this) may result in a message that there is “insufficient” memory to
estimate the model. So this can take a bit of exploration and some patience. Typically, I suggest
starting with the highest order polynomial that is significant (i.e., in this case, the cubic effect).
Specifying the cubic component only as random resulted in a model that would not converge, so
the linear component was added.
Since both are significant, we would likely build the growth part on the random cubic
component, since it is the highest polynomial in the model. Building a model on both the linear
and cubic components would likely create some overlap and make interpretation more difficult
(EDEP 768) Week 13: Extending the Examination of Growth
9
(keeping in mind the significant cubic component suggests focusing on the entire trajectory). Our
next step would likely be to specify a model examining differences in initial status and grow due
to background factors such as gender.
References
Hox, J. (2010). Applied multilevel analysis. NY: Routledge.
Raudenbush, S. & Bryk, A. (2002). Hierarchical linear models (2nd Edition). Newbury Park,
CA: Sage.
Singer, J. & Willet, J. (2003). Applied longitudinal data analysis: Modeling change and event
occurrence. New York: Oxford University Press.