Power and Bias in Three-Form Planned Missing Designs
for Longitudinal Mediation
Richard M. Kinai, Terrence D. Jorgensen, Graham G. Rifenbark,
Fan Jia, Alexander M. Schoemann, Todd D. Little, Wei Wu
PLANNED MISSING DATA DESIGNS
Modern missing data methods minimize most problems
associated with missing data (e.g., power loss and biased
parameter estimates) and they present new opportunities such
as research designs that purposefully incorporate missing data
(Palmer & Royall, 2010).
Planned missing designs can minimize participant fatigue,
practice effects, and dropout rates, while also reducing
research costs. This has important implications for designing
longitudinal studies where cost and dropout are major
concerns. One popular planned missing data design with
potential benefits for longitudinal studies is the 3-form design.
In the 3-form design the sample is randomly split into 3
subgroups, each with their own pattern of missing data.
SIMULATION
We simulated complete data from a 3-latent variable 3timepoint mediation model with N = 500, 200 datasets per
condition.
We varied:
• factor loadings (from .7 to .85),
• autoregressive paths (from .4 to .9),
• cross-lagged effects (from 0 to .4),
• within-time covariances (from .2 to .5), and
• numbers of variables with no missingness (1-6 per time
point; all other variables had 1/3 missing data)
MCAR missingness was imposed according to a 3-form design
(Graham et al., 2006), where the “X” set (items that all
participants receive, with no missingness) contained from 1-6
items.
Models were fit using FIML estimation
Form
1
2
3
Variable
Set A
X
X
--
β4,1
η1
ψ2,1
β5,1
β6,1
ψ2,2
ψ3,1
β4,2
β5,2
η2
Variable
Set B
X
-X
Variable
Set C
-X
X
ψ7,7
β7,4
η4
ψ5,4
ψ3,2
β4,3
ψ3,3
β8,5
η3
β5,3
β6,3
η6
ψ8,8
η8
ψ9,7
β9,5
β7,6
ψ6,6
ψ8,7
β7,5
ψ5,5
η5
ψ6,5
η7
β8,4
β6,2
ψ9,8
β8,6
β9,4
β9,6
η9
ψ9,9
Note: To save space, the indicator variance, error covariance, and loadings are excluded from this DISCUSSION
RESULTS
illustration,
but they follow the same conventions from Model 1, extended to 3 factors at each time.
• With between 11
*** Due to lack of space, a curve was omitted at Time 2, but still estimate ψ6,4
Parameter Estimates
• As expected, parameter estimates were unbiased, because
the missingness mechanism is MCAR
Standard Errors
• Standard errors were also unbiased (compared to empirical
standard errors computed as the standard deviation of the
parameter estimates across replications)
• Standard errors increased as the number of items in the X
block decreased (i.e., as the % missing data increased)
Power
• As the % missing data increased, power decreased very
slightly (corresponding to slight increase in SEs)
• e.g., the drop in power to detect a cross-lagged path of .1
decreased from 54% with 11% missing data to 52% with
30% missing data
No.
Percent
Pop.
Average
items in missing cross-lag
Est.
X block
(%)
path
THREE-FORM DESIGN
Common
Set X
X
X
X
ψ4,4
ψ1,1
1
2
3
4
5
6
11.1
14.8
18.5
22.2
25.9
29.6
.10
.10
.10
.10
.10
.10
.10
.10
.10
.10
.10
.10
Empirical Average
SE
Est. SE
.05
.05
.05
.05
.05
.05
.05
.05
.05
.05
.05
.05
Bias
.02
.02
.02
.02
.03
.03
SE Bias Power
.03
.05
.05
.05
.08
.07
.52
.54
.54
.53
.53
.54
and 30% MCAR missing through a planned
missing design, there was little noticeable effect of missing
data on power
• Parameter estimates and standard errors showed very little
bias when FIML was used for SEM estimation
• With N = 500, the only parameters to show less than perfect
power were those valued less than .40; future investigations
should be done with smaller samples
• Initial investigations suggest that smaller samples may be
subject to convergence failures with planned missing
designs
REFERENCES
Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned
missing data designs in psychological research. Psychological Methods, 11, 323–
343.
Graham, J. W., Taylor, B. J.,& Cumsille, P. E. (2001). Planned missing data designs in
the analysis of change. In L. M. Collins & A. G. Sayer (Eds.), New methods for the
analysis of change (pp.335–353). Washington, DC: American Psychological
Association.
Thomas, N., Raghunathan, T. E., Schenker, N., Katzoff, M. J., Johnson, C. L. (2006).
An evaluation of matrix sampling methods using data from the national health and
nutrition examination survey. Survey Methodology, 32, 217–231.
Palmer R. F. & Royall D. R. (2010) Missing data? Plan on it! Journal of the American
Geriatrics Society, 58, S343-S348
Funding generously
provided by:
July 11, 2012 • The 77th Annual Meeting of the Psychometric Society