Appendix: Monte Carlo simulation results
We conducted a Monte Carlo simulation study addressing the following questions:
(1) Are 3 observations per person sufficient in a multilevel model (MLM) to yield an unbiased
estimate of the parameter addressing age-differences in the within-person relationship between
happiness and sadness?
(2) Is the standard error of the parameter unbiased in this case (i.e., are statistical significance
tests accurate)?
(3) How does the statistical power of our study compare to the power of prior research with more
observations per person but lower sample size?
The model analyzed in the Monte Carlo study corresponds to those used to examine “age
differences in mixed emotions using the covariation approach” in the manuscript.
Specifically, the following MLM was tested:
Within-subject equation:
Yij = β0j + β1jXij+ rij
(1)
Between-subject equations:
β0j = γ00 + γ01Wj + u0j
(2)
β1j = γ10 + γ11Wj + u1j
(3)
In this model, Yij is the happiness rating of person j for episode i, Xij is the sadness rating of
person j for episode I, and Wj is the age of person j. In the within-subject equation (1), β0j and β1j
represent intercept and slope of the regression of sadness on happiness for subject j, and rij is a
within-subject residual term. In the between-subject equation (2), a subject’s intercept is a
function of a common (fixed) intercept for the population (γ00), a common (fixed) effect of age
(γ01), and a residual intercept variance term (u0j). In the between-subject equation (3), a subject’s
slope is a function of a common slope effect (γ10), a common (fixed) effect of age on the
regression slope (γ11), and a residual slope variance term (u1j).
For the simulations, the parameter values used in the data generation were taken from the results
from the PATS sample reported in the manuscript. Analyses were conducted using the Mplus
version 7.3 Monte Carlo simulation facility, using 1000 replications for each simulation analysis
(Mplus example code is given below). Two conditions were varied in the simulation: (a) sample
size and (b) number observations per person. Specifically, in one series of models, the sample
size was held constant at N=180 with varying numbers of observations per person (3, 15, 25, 35,
45, 55, 67, 700); In a second series of models, the sample size was varied (N=180, 900, 1500,
2100, 2700, 3300, 4000) and the number of observations per person was held constant at 3. The
total number of observations (N times number of observations) was matched in each series (i.e.,
180*15 = 900*3; 180*25 = 1500*3, and so on). The combination of N =180 with 35
observations per person corresponds with the design by Carstensen et al. (2000), and the
combination of N=4000 with 3 observations corresponds with the design of the PATS sample in
the current study.
Of particular interest in the present context is the performance of parameter γ11, which represents
the estimated effect of age on the relationship between happiness and sadness. The results (%
parameter estimates bias, % standard error bias, power) for this parameter are shown in the tables
below.
Bias of parameter estimates and standard errors: Percent bias of the parameter estimate and
standard errors were well within acceptable limits (for focal parameters, bias should be below
±5%, (Wang, 2012)) for all tested conditions. For instance, for a sample size of 180 and 35
observations per person – corresponding with the design in Carstensen et al. (2000)-- parameter
bias was -2.90% and SE bias was -2.96%. For a sample size of 4000 and 3 observations per
person -- corresponding to the design for the PATS data in the current manuscript -- parameter
bias was 0.65% and SE bias was 1.75% (see Table).
Given that some prior studies in this literature relied on small samples with fewer than 50
participants (Ong & Bergeman, 2004; Ready, Carvalho, & Weinberger, 2008), we also wondered
whether biases would increase for smaller samples. For a sample size of N=50 (with 35
observations per person), SE bias increased to -8.0%; for a sample size of N=25 (with 35
observations per person), SE bias increased to -11.8%. Thus, a small sample size at level 2 (i.e., a
small number of subjects) yielded biased estimates, but a small number of observations (i.e., 3)
combined with a large sample size yields unbiased parameters and standard errors.
Statistical power: The statistical power to detect age differences in the relationship between
happiness and sadness based on the MLM is also shown in the table below, and graphically
illustrated in the figure below. As can be seen, for a given number of overall observations
(sample size * observations per person), increasing the participant sample size (lower part of the
table) has a much greater impact on power than increasing the number of observations per person
(upper part of the table). Keeping the number of observations per person constant at 3, a sample
size of 1500 is sufficient to detect an effect of the size found in the PATS sample with a power
exceeding 90%. On the other hand, keeping the sample size at 180 individuals, the power to
detect this same effect does not reach 80% even with a very large number (i.e., 700) of
observations per person.
Table: Parameter bias, standard error bias, and power for parameter γ11
Total
number of
observations
540
2700
4500
6300
8100
9900
12060
126000
Sample size = 180, varying number of observations per person
Sample
Observations
% parameter
% SE bias
size
per person
bias
Power
180
180
180
180
180
180
180
180
0.23
0.55
0.62
0.63
0.66
0.67
0.70
0.75
3
15
25
35
45
55
67
700
-0.97
+0.97
+0.65
-2.90
-2.26
-1.29
+0.65
+1.0
-0.75
-1.95
+0.00
-2.96
-0.78
+0.00
+2.48
-3.07
Total
number of
observations
540
2700
4500
6300
8100
9900
12000
Observations per person = 3, varying sample size
Sample
Observations
% parameter
% SE bias
size
per person
bias
Power
180
900
1500
2100
2700
3300
4000
0.23
0.72
0.91
0.98
0.99
0.997
0.999
3
3
3
3
3
3
3
-0.97
+0.65
+1.29
+0.65
+0.32
+0.32
+0.65
-0.75
-0.81
+1.06
+0.00
+1.43
+1.59
+1.75
Power for varying ns (subjects or observations)
1
2100*3
2700*3 3300*3
4000*3
Varying the number of
subjects (keeping number of
observations at 3)
180*67
Varying number of
observations per subject
(keeping number of subjects
at 180)
1500*3
0.8
900*3
0.6
Power
180*25 180*35
180*45 180*55
180*15
0.4
0.2
180*3
0
0
2000
4000
6000
8000
10000
Total sample size (subjects * observations)
Mplus syntax for the Monte Carlo simulation:
MONTECARLO:
NAMES ARE happy sad age;
nobservations = 6300;
ncsizes =1;
csizes = 180 (35);
seed = 5859;
nreps = 1000;
within = sad;
between = age;
ANALYSIS:
12000
type = twolevel random;
MODEL POPULATION:
%WITHIN%
[sad@0]; sad@2;
slope | happy on sad;
happy*1.366;
%BETWEEN%
slope on age*0.031;
happy on age*0.044;
[happy*4.107];
[slope*-0.332];
happy*1.351;
slope*0.08;
happy with slope*0.032;
[age@0]; age@3.2;
MODEL:
%WITHIN%
slope | happy on sad;
happy*1.366;
%BETWEEN%
slope on age*0.031;
happy on age*0.044;
[happy*4.107];
[slope*-0.332];
happy*1.351;
slope*0.08;
happy with slope*0.032;
References
Carstensen, L. L., Pasupathi, M., Mayr, U., & Nesselroade, J. R. (2000). Emotional experience in
everyday life across the adult life span. Journal of Personality and Social Psychology,
79(4), 644-655.
Ong, A. D., & Bergeman, C. S. (2004). The complexity of emotions in later life. Journals of
Gerontology Series B-Psychological Sciences and Social Sciences, 59(3), P117-P122.
Ready, R. E., Carvalho, J. O., & Weinberger, M. I. (2008). Emotional Complexity in Younger,
Midlife, and Older Adults. Psychology and Aging, 23(4), 928-933.
Wang, J. (2012). Wiley Series in Probability and Statistics : Structural Equation Modeling with
MPlus : Methods and Applications (3rd Edition). Somerset, NJ, USA: John Wiley &
Sons.