Technical Appendix for:
Dynamic Effects Among Patients’ Treatment Needs, Beliefs, and Utilization: A Prospective
Study of Adolescents in Drug Treatment.
Terry L. Schell, Maria Orlando, Andrew R. Morral
Drug Policy Research Center
RAND, Santa Monca.
Initial Model Selection
The cross-lagged modeling technique is widely used to assess causal associations in data derived from non-experimental, longitudinal research designs. In this modeling approach, each variable in the model is regressed on all of the variables that precede it in time. In the current study, some variables have a clear sequential relationship even when measured in the same wave
(e.g., perceived need for treatment at the time of the interview is preceded by the treatment actually received in the 90-days prior to the interview). In other cases, the preceding measurement occurred in the prior wave of data collection. A two-wave representation of the initial cross-lagged model is shown in Figure 1. In this model, SPI scores and TX dose at each wave were predicted by the previous wave’s SPI, TX dose, LSE and PNT measures; while
“Current LSE” and “Current PNT” scores were predicted by the TX dose in the preceding 90 days, the SPI in the preceding 30 days, as well as the LSE and PNT measures from the previous wave. Variables that have synchronous or overlapping time frames were allowed to correlate, i.e., measures of SPI over the previous 30 days and TX dose of the previous 90 days were correlated, and current LSE and current PNT were correlated within each wave.

In addition to the four primary outcomes of interest, the two models included several covariates to control for possible confounds. Specifically, the individual study site was included as a covariate using dichotomous, dummy-coded variables. This effect was included as a predictor for every dependent variable at every wave to account for possible differences in the treatment regimen or treatment components at the different sites. In addition, gender, ethnicity, and age were included as covariates for all variables. This cross-lagged model was fit to five waves of data simultaneously within each treatment sample.
Model Development
An inspection of modification indices for these models showed that the largest source of model misspecification was from positive autoregressive association that occurred between some variables and their counterparts from the measurement wave 6 months earlier. For example,
Low Drug Resistant Self-Efficacy measured at time T was independently predicted by LSE measured at time T-3months and at T-6 months. This indicates that some of the wave-to-wave changes in attitudes or substance use problems were transitory in nature. The adolescents studied tended to move somewhat toward their earlier values on these variables, rather than simply regress to the mean. To account for these highly significant associations, additional autoregressive paths were added to the initial model. Specifically, the SPI, LSE, and PNT measures were regressed on their respective values from the two prior waves in both the residential and outpatient samples. This model results in adequate model fit in both samples,
SRMR’s < .08 and CFI > .95.
Although these models were an adequate summary of the covariance matrices, they were not very parsimonious. As a result, they are difficult to interpret and are not very efficient from a statistical standpoint. One contributor to this problem was the fact that each relationship of

conceptual interest (e.g., between PNT and subsequent TX dose) was replicated several times in each model. Thus, the primary conceptual hypotheses of interest were being spread across several different statistical tests. To improve these models, we constrained conceptually equivalent parameters to be equal when predicting 6, 9, and 12-month outcomes within each of the treatment samples. These simplified models assume that the process which created the data can be approximated by a series of constant prospective effects that repeat over time. For example, the predicted unit change in LSE per unit of TX dose in the prior 90-days is assumed to be constant when predicting 6-, 9-, and 12-month LSE. Imposing this parameter constraint did not significantly reduce model fit for either treatment sample,
 2 ’s (32) < 46, p
’s>.05. The observed data are not inconsistent with this simplified, constant effect model.
Path coefficients predicting 3-month outcomes were allowed to be different from conceptually equivalent parameters predicting 6, 9, and 12-months outcomes for three reasons:
(a) study eligibility requirements included receiving drug treatment in the first 3-months which may attenuates the ability of any baseline measures to predict 3-month TX dose; (b) many of the baseline measures used as predictors of 3-month outcomes may have different interpretations prior to initiation of the intended treatment program (e.g., PNT, TX dose); and, (c) the significant auto-regressive paths that cross two intervals (discussed above) cannot be estimated prior to the
6-month wave.
In addition to simplifying the multi-interval longitudinal effects in the model, we also investigated the role of the covariates in these models (study site, age, gender, and ethnicity).
These variables were included as controls and did not represent hypothesized relationships, however these post-hoc controls add many parameters to the model because they are all included when predicting each of the model’s 16 dependent variables. To increase efficiency in the

estimation of the parameters of primary interest, we pruned unneeded covariates from the model
(e.g., Cohen & Cohen, 1983). To remove unneeded covariates, the individual effects of a covariate on an outcome were set to zero when this constraint resulted in a more parsimonious model. A Minimum Akaike Information Criterion (Akaike, 1987) was used as the measure of parsimony for model pruning. This method retains all significant effects as well as some effects that do not reach conventional levels of significance. In the current case, it resulted in all effects with p >.2 being removed from the model.
Computational Methods
All reported analyses were conducted in Mplus 2.13 (Muthén and Muthén, 2002) using
Expectation Maximization (EM) algorithm parameter estimates to make use of partially complete data (McLachlan & Krishnan, 1996). This technique for handling missing data results in unbiased parameter estimates when the missing data are missing and random or missing completely at random (Little & Rubin, 1987). The current study has a small amount of planned missing data (33% of the Residential participants did not have a 9-month assessment); data missing by design is considered “ missing and random”
and is appropriately handled by the EM algorithm (Schafer, 1997).
To insure that our inferential statistics were robust to violations of the assumption of normality, we estimated standard errors using a sandwich estimator and tested the significance of coefficients with the Yuan-Bentler T2* test statistic (See Yuan, Marshall, & Bentler, 2002). This method adjusts the expected distribution of covariances to account for non-normality in the distributions of the individual variables, and gives unbiased estimates of standard errors even with substantial deviations from normality.

Overall model fit was evaluated using the comparative fit index (CFI, Bentler, 1990), and the standardized root mean-squared residual (SRMR, Bentler, 1995). As recommended by Hu &
Bentler (1999), CFI values greater than .95 and SRMR’s less than .08 are used as the criteria for a well-specified model. A modified chi-squared statistic was used to test the relative fit of nested models. This modified statistic is based on the Yuan-Bentler test statistic and incorporates the robust standard error estimates (see Satorra & Bentler, 2001).
Figure 1.
Initial Cross-Lagged Model
Note: Model applied simultaneously to 0-, 3-, 6-, 9-, and 12-month data. Correlations between measures with overlapping time frames are not shown. Paths from baseline covariates are not shown.

References
Akaike, H. (1987). Factor analysis and AIC. Psychometrica, 52 , 317-332.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin,
107, 238-246.
Bentler, P. M. (1995). EQS structural equations program manual.
Encino, CA: Multivariate
Software.
Cohen, J. & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences.
Hillsdale, N.J.: Lawrence Erlbaum Associates.
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis:
Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
Little, R. J. A. & Rubin, D. B. (1987). Statistical analysis with missing data. New York: Wiley.
McLachlan, G.J., & Krishnan, T. (1996). The EM algorithm and extensions . NY: Wiley-
Interscience.
Muthén, L. K. & Muthén, B. O. (2002).
Mplus: The comprehensive modeling program for applied researchers [Computer program]. Los Angeles: Muthen & Muthen.
Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66 , 507-514.
Schafer, J.L. (1997). Analysis of Incomplete Multivariate Data. London:Chapman & Hall.
Yuan, K.H., Marshall, L. L. & Bentler, P. M. (2002). A unified approach to exploratory factor analysis with missing data, nonnormal data, and in the presence of outliers. Psychometrika,
67, 95-122.

Model Specification

The data wave (in months from baseline) for time varying constructs is indicated by a number following an underscore

Variables followed by “@0” have regression coefficients set to zero in the final model as a result of pruning

Variables followed by a number in parentheses are constrained to have the same regression coefficients as other variables followed by that same number

Variables than do not serve as dependent variables in the model are allowed to correlate with one another
Residential Sample:
LSE_0 is regressed on
SPI_0
TXDOSE_0
RESSITE2
RESSITE3 black hisp female age
PNT_0 is regressed on
SPI_0
TXDOSE_0
RESSITE2
RESSITE3 black hisp female age
SPI_3 is regressed on
SPI_0
TXDOSE_0
LSE_0
PNT_0
RESSITE2
RESSITE3 black hisp female age
TXDOSE_3 is regressed on
SPI_0
TXDOSE_0
LSE_0
PNT_0
RESSITE2
@0
@0
@0
@0
@0
@0
@0

RESSITE3 black hisp female age
LSE_3 is regressed on
SPI_3
TXDOSE_3
LSE_0
TX_0
RESSITE2
RESSITE3
@0
@0
@0 black hisp female age
PNT_3 is regressed on
SPI_3
TXDOSE_3
@0
@0
@0
@0
@0
SE_0
PNT_0
RESSITE2
RESSITE3 black hisp female age
@0
@0
@0
@0
@0
SPI_6 is regressed on
SPI_3 (1)
TXDOSE_3 (2)
LSE_3
PNT_3
SPI_0
RESSITE2
RESSITE3 black hisp female
(4)
(5)
(6)
@0
(51)
(52)
@0
(55) age
TXDOSE_6 is regressed on
SPI_3 (7)
TXDOSE_3 (8)
LSE_3 (10)
PNT_3
RESSITE2
(11)
RESSITE3 black hisp female age
LSE_6 is regressed on
@0
@0
@0
@0

SPI_6 (17)
TXDOSE_6 (18)
LSE_3 (20)
PNT_3
LSE_0
(21)
(22)
RESSITE2
RESSITE3 black hisp female age
PNT_6 is regressed on
@0
@0
(64)
@0
SPI_6 (23)
TXDOSE_6 (24)
LSE_3
PNT_3
(26)
(27)
PNT_0
RESSITE2
RESSITE3 black
(28)
@0
@0
@0 hisp female age
@0
@0
@0
SPI_9 is regressed on
SPI_6 (1)
TXDOSE_6 (2)
LSE_6 (4)
PNT_6
SPI_3
RESSITE2
RESSITE3 black hisp female age
TXDOSE_9 is regressed on
SPI_6 (7)
TXDOSE_6 (8)
LSE_6
PNT_6
RESSITE2
RESSITE3
(10)
(11)
@0
(5)
(6)
@0
@0
(51)
(52)
@0
(55) black hisp female age
@0
@0
@0
@0
LSE_9 is regressed on
SPI_9 (17)
TXDOSE_9 (18)
LSE_6
PNT_6
LSE_3
(20)
(21)
(22)

RESSITE2
RESSITE3 black hisp female
@0
@0
@0
@0
(64) age @0
PNT_9 is regressed on
SPI_9 (23)
TXDOSE_9 (24)
LSE_6
PNT_6
PNT_3
(26)
(27)
(28)
RESSITE2
RESSITE3 black hisp female age
SPI_12 is regressed on
@0
@0
@0
@0
@0
SPI_9 (1)
TXDOSE_9 (2)
LSE_9
PNT_9
(4)
(5)
SPI_6
RESSITE2
RESSITE3 black
(6)
@0
(51) hisp female age
(52)
@0
(55)
TXDOSE_12 is regressed on
SPI_9 (7)
TXDOSE_9 (8)
LSE_9 (10)
(11) PNT_9
RESSITE2
RESSITE3 black @0 hisp female age
@0
@0
@0
LSE_12 is regressed on
SPI_12 (17)
TXDOSE_12 (18)
LSE_9 (20)
PNT_9
LSE_6
RESSITE2
RESSITE3 black hisp female
(21)
(22)
@0
@0
@0
(64)

age @0
PNT_12 is regressed on
SPI_12 (23)
TXDOSE_12 (24)
LSE_9
PNT_9
PNT_6
RESSITE2
RESSITE3 black hisp female
(26)
(27)
(28)
@0
@0
@0
@0
@0 age @0
SPI_0 is correlated with TXDOSE_0
SPI_3 is correlated with TXDOSE_3
SPI_6 is correlated with TXDOSE_6
SPI_9 is correlated with TXDOSE_9
SPI_12 is correlated with TXDOSE_12
LSE_0 is correlated with PNT_0
LSE_3 is correlated with PNT_3
LSE_6 is correlated with PNT_6
LSE_9 is correlated with PNT_9
LSE_12 is correlated with PNT_12
Outpatient sample:
LSE_0 is regressed on
SPI_0
TXDOSE_0
OPSITE2
OPSITE3
OPSITE4 black hisp female age
PNT_0 is regressed on
SPI_0
TXDOSE_0
OPSITE2
OPSITE3
OPSITE4 black hisp female age
SPI_3 is regressed on
SPI_0
TXDOSE_0
LSE_0
@0
@0
@0
@0
@0
@0
@0

PNT_0
OPSITE2
OPSITE3
OPSITE4 black hisp female age
TXDOSE_3 is regressed on
SPI_0
TXDOSE_0
LSE_0
PNT_0
OPSITE2
OPSITE3
OPSITE4 black hisp female age
LSE_3 is regressed on
SPI_3
TXDOSE_3
LSE_0
PNT_0
OPSITE2
OPSITE3
@0
@0
@0
@0
@0
@0
@0
@0
@0
@0
@0
@0
OPSITE4 black hisp female age
PNT_3 is regressed on
SPI_3
@0
@0
@0
@0
TXDOSE_3
LSE_0
PNT_0
OPSITE2
OPSITE3
OPSITE4 black hisp female age
SPI_6 is regressed on
SPI_3 (1)
TXDOSE_3 (2)
LSE_3
PNT_3
(4)
(5)
@0
@0
@0
@0
@0
@0
@0
SPI_0
OPSITE2
OPSITE3
(6)
@0
@0

OPSITE4 black hisp female age
@0
@0
@0
@0
(55)
TXDOSE_6 is regressed on
SPI_3 (7)
TXDOSE_3 (8)
LSE_3 (10)
PNT_3
OPSITE2
OPSITE3
(11)
@0
OPSITE4 black hisp female age
LSE_6 is regressed on
SPI_6
@0
@0
@0
(60)
TXDOSE_6 (18)
LSE_3 (20)
PNT_3
LSE_0
(21)
(22)
OPSITE2
OPSITE3
OPSITE4 black
(17)
@0
@0
@0 hisp female age
@0
(64)
@0
PNT_6 is regressed on
SPI_6 (23)
TXDOSE_6 (24)
LSE_3 (26)
PNT_3
PNT_0
OPSITE2
OPSITE3
OPSITE4 black hisp female
(27)
(28)
@0
@0
(66)
@0
(69)
@0 age
SPI_9 is regressed on
SPI_6 (1)
TXDOSE_6 (2)
LSE_6 (4)
PNT_6
SPI_3
(5)
(6)
OPSITE2
OPSITE3
OPSITE4 @0

black hisp female age
@0
@0
@0
(55)
TXDOSE_9 is regressed on
SPI_6 (7)
TXDOSE_6 (8)
LSE_6
PNT_6
(10)
(11)
OPSITE2
OPSITE3
OPSITE4
@0 black hisp female age
@0
@0
@0
(60)
LSE_9 is regressed on
SPI_9 (17)
TXDOSE_9 (18)
LSE_6
PNT_6
LSE_3
OPSITE2
OPSITE3
OPSITE4 black hisp
(20)
(21)
(22) female age
PNT_9 is regressed on
SPI_9 (23)
TXDOSE_9 (24)
LSE_6
PNT_6
(26)
(27)
(28) PNT_3
OPSITE2
OPSITE3
OPSITE4
@0
@0
@0
@0
(64)
@0
@0 black hisp female age
(66)
@0
(69)
@0
SPI_12 is regressed on
SPI_9 (1)
TXDOSE_9 (2)
LSE_9
PNT_9
SPI_6
OPSITE2
OPSITE3
OPSITE4 black
(4)
(5)
(6)
@0
@0
@0

hisp female age
TXDOSE_12 is regressed on
@0
@0
(55)
SPI_9 (7)
TXDOSE_9 (8)
LSE_9
PNT_9
OPSITE2
(10)
(11)
OPSITE3
OPSITE4 black @0 hisp female age
@0
@0
(60)
LSE_12 is regressed on
SPI_12 (17)
TXDOSE_12 (18)
LSE_9 (20)
PNT_9
LSE_6
OPSITE2
OPSITE3
OPSITE4 black hisp female
(21)
(22)
@0
@0
@0
@0
(64)
@0 age
PNT_12 is regressed on
SPI_12 (23)
TXDOSE_12 (24)
LSE_9 (26)
PNT_9
PNT_6
(27)
(28)
@0 OPSITE2
OPSITE3
OPSITE4 black
@0
(66) hisp female age
SPI_0 is correlated with TXDOSE_0
SPI_3 is correlated with TXDOSE_3
SPI_6 is correlated with TXDOSE_6
SPI_9 is correlated with TXDOSE_9
@0
(69)
@0
SPI_12 is correlated with TXDOSE_12
LSE_0 is correlated with PNT_0
LSE_3 is correlated with PNT_3
LSE_6 is correlated with PNT_6
LSE_9 is correlated with PNT_9
LSE_12 is correlated with PNT_12