Power of Planned Missing Designs in Longitudinal Panel Designs
Kimberly Gibson, Alexander M. Schoemann, Fan Jia, Graham G. Rifenbark,
Terrence D. Jorgensen, Mijke Rhemtulla, Wei Wu, Todd D. Little
Planned Missing Data Designs
• Participants are randomly assigned to conditions in which
they only answer a subset of questions, or participate in a
subset of measurement occasions
• The planned missing data are missing completely at
random (MCAR), so they can be estimated without bias
Why use Planned Missing Designs?
• Increase Data Quality (Validity)
When participants have fewer questions to answer or
fewer repeated measurements, they are less likely to be
fatigued, and more likely to offer high-quality data
Simulation
Model
• 200 datasets per condition (N = 500) were simulated
according to a 3-timepoint panel design
• MCAR missingness on 1-3 variables per timepoint was
imposed according to a 3-form design (Graham et al., 2006)
• Cross-lag path values varied from .1 to .9 across
conditions
• We investigated power to detect cross-lagged paths as the
amount of planned missingness increased
Three Form Design
(Raghunathan & Grizzle, 1995).
• Reduce Cost
Planned missing designs require less data collection than
complete data designs, reducing the cost of measurement.
For example, studying cortisol trends requires multiple
split-sample measures per day across several days (Graham
et al., 2006).
• Reduce Unplanned Missing Data
Fatigued participants are more likely to skip questions or
assessment occasions, or drop out entirely. Unplanned
missingness can lead to biased results.
• Quantify Practice Effects
Participants who skip a measurement occasion will not be
contaminated on the next occasion, making it possible to
disentangle growth effects from practice effects (e.g., McArdle
Form
Common
Set X
Variable
Set A
Variable
Set B
Variable
Set C
1, ⅓ of N
X
missing
X
X
2, ⅓ of N
X
X
missing
X
3, ⅓ of N
X
X
X
missing
Variable
Set A
Variable
Set B
Variable
Set C
Common
Condition
Set X
High miss
X1
X2, X5
X3, X6
X4
Med miss
X1, X4
X2
X3, X5
X4
X3
X5
X6
Low miss X1, X2, X4
& Woodcock, 1997).
Missing Data Estimation
Results
Full Information Maximum Likelihood (FIML)
• FIML estimates a model in the presence of missing data,
using all available data to determine the most likely
parameter estimates
• As long as the model is correctly specified (e.g., relevant
interaction terms are included, multilevel structure is
specified if it exists), and missingness is MAR or MCAR,
parameter estimates and standard errors will be unbiased
Percent
Condition
Missing
Hi miss
1000
 41
ˆ
41
 (ˆ
i 1
ˆ )


41
41
1000  1
SE 41
̂ 41   41 SE   SE Power
SE
 41
41
41
41
.1
.091
.050
.054
-.090
.083
.365
Med miss 22.22
.1
.091
.050
.054
-.089
.083
.375
Low miss
.1
.092
.050
.053
-.081
.064
.385
Note: Convergence is 100% for all models with N = 500. Results are for one cross-lagged
path. Results for other parameters and other models are similar.
Funding generously
provided by:
July 11, 2012 • The
th
77
• Parameter estimates resulting from the PM design were
unbiased and efficient
• Power suffers little whether 1/6 or 3/6 variables per timepoint
are missing
• With N = 500, power to detect cross-lagged paths increases
to 1.0 as the parameter value reaches .4 or higher,
regardless of missingness condition
• With lower N, convergence may become a problem
References
27.78
16.67
Conclusions
Enders, C. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
Graham, J. W., Hofer, S. M., & MacKinnon, D. P. (1996). Maximizing the usefulness of data
obtained with planned missing value patterns: An application of maximum likelihood
procedures. Multivariate Behavioral Research, 31, 197–218.
Graham, J. W., Hofer, S. M., & Piccinin, A. M. (1994). Analysis with missing data in drug
prevention research. In L. M. Collins & L. Seitz (Eds.), National Institute on Drug Abuse
Research Monograph Series (pp. 13–62). Washington, DC: National Institute on Drug Abuse.
Graham, J. W., Taylor, B. J.,& Cumsille, P. E. (2001). Planned missing data designs in the
analysis of change. In L. M. Collins & A. G. Sayer (Eds.), New methods for the analysis of
change (pp. 335–353). Washington, DC: American Psychological Association.
Graham, J. W., Taylor, B. J., Olchowski, A. E., & Cumsille, P. E. (2006). Planned missing data
designs in psychological research. Psychological Methods, 11, 323–343.
McArdle, J. J., & Woodcock, R. W. (1997). Expanding test–retest designs to include
developmental time-lag components. Psychological Methods, 2, 403–435.
Raghunathan, T. E., & Grizzle, J. E. (1995). A split questionnaire survey design. Journal of the
American Statistical Association, 90, 54–63.
Annual Meeting of the Psychometric Society