Cluster Level Moderation
Jessaca Spybrook
Western Michigan University
Cluster Level Moderation
Outline for Session
 Brief conceptual overview
 Examples and models
 Power/Minimum detectable effect size
difference (MDESD)
2
Conceptual Overview
Move beyond main effect of treatment
Consider under what conditions a program
works
Critical questions for program developers,
funders, policymakers, school personnel,
and many others
3
Conceptual Overview
Examples
 Village level intervention
Access to health care
Local politics
Size of village
Adult literacy rates in village
4
Conceptual Overview
Example
 The Gambia Data, School-based intervention
Condition of buildings
Number of classrooms in the school
Number of teachers in the school
One or two shifts of school per day
Presence of library
Number of working toilets
5
Two Types of Moderator Effects
Treatment with Binary
 Example
Treatment by Presence of Library
Treatment with Continuous
 Example
Treatment by Number classes per school
6
Example
Scenario




The Gambia data (2009)
Students nested in schools
2,657 -> 1,573 students (pupils data)
271 -> 173 schools (head teacher data)
7
Example
Variables
 DV:
Numbers of words read correctly in 60 seconds
(reading fluency) [S2Q3_PP]
 IVs at L2:
Library [Q216]
WSD indicator [WSD]
Library x WSD [TRMTBYLI]
Product of centered Lib x WSD [CENTWSDL]
School mean reading fluency 2008 [MEAN08]
8
Guiding Question
Guiding Question
 Research suggests that the presence of a
library in a school is an important condition for
academic success. To test this theory, we are
interested in examining whether the effect of
WSD is different in schools with a library
compared to schools without a library.
9
The Model
Level 1 (students):
Yij   0 j  rij
Yij is reading fluency for student i in school j
 0 j is the mean reading fluency for school j
rij is the random error associated with student i in school j, var( rij )   2
Level 2 (schools):
 0 j   00   01Lib   02Trmt   03 Lib xTrmt  u0 j
 00 is the average school mean reading fluency for control schools with no library
 01 is the average difference between control schools with a library and without a library
 02 is the average difference between treatment and control for schools with no library
 03 is the increase or decrease in the treatment effect for schools with a library compared
to schools without a library
u0j is the random error associated with school mean, conditional on the X’s, var( u0 j )   00| X
10
Interpretation
Combined Model:
Yij   00   01Lib   02Trmt   03 LibxTrmt  u0 j  rij
See output
11
Results/Interpretations
Note that these are not statistically
significant 
 Average reading fluency for students in
control schools with no library is 33.70
 Average increase in reading fluency for
students in control schools with library
compared to those without a library is 0.76
 Average treatment effect for schools with no
library is -4.68
 Treatment effect for schools with a library is
higher than for schools without a library by an
average of 4.16
12
Centering
 Two options
 Uncentered (previous example)
 Center treatment, moderators, and then compute
product of the centered variables for the
moderator effect
13
Centering
 In the context of moderator effect, centering
will:
 Reduces multicollinearity
 Change the interpretation, estimates, and
standard errors of intercept and main effects
 NOT change the interpretation, estimate, or
standard error of the moderator effect
 See output
 Choice depends on research questions
14
Statistical Power for Cluster Level
Moderators
Suppose planning a CRT of intervention
similar to WSD in a similar context
 Use this study to plan future CRT
15
Statistical Power
Model:
Yij   00   01Lib   02Trmt   03 LibxTrmt  u0 j  rij
To increase precision, include pre-test.
New Model:
Yij   00   01Lib   02Trmt   03 LibxTrmt   04 Pr e  u0 j  rij
Statistical Power
Estimated treatment effect:

 
L
NL
L
NL
ˆ
 03  YT  YT  YC  YC
Variance of treatment effect:




Varˆ03   16 1  R|W2 's  00   2 / n / J
Statistical Power
Hypothesis Test:
H 0 :  03  0
H1 :  03  0
F statistic:
F
MS trmt x mod
MS clusters
Under the alternative hypothesis, F statistic
follows non-central F distribution with noncentrality parameter:
2
 03
 J 5 
2
2
16 1  R|W 's  00   / n / J


18
Statistical Power
Standardized Noncentrality Parameter:


2

16 1  R|W 's   1    / n/ J
2
03
where
 00

 00   2

 03
 00   2
19
Estimating Design Parameters
ICC
 Unconditional model (see output)
105

 0.19
105  443
R2
 Lib, WSD, Interaction, School-level mean
reading fluency in 2008 as “pre-test” (see
output)
2 105  23
R
 0.78
105
20
MDESD
Minimum detectable effect size difference
Total Number
of Clusters
n=20
n=100
n=500
40
0.52
0.41
0.38
60
0.42
0.33
0.31
80
0.36
0.28
0.26
100
0.33
0.25
0.24
120
0.30
0.23
0.22
0.22
0.17
0.16
…
…
220
Note: These values assume half the clusters assigned to treatment and cluster moderator, alpha =
2
0.05, power = 0.80, and   0.19, R  0.78
21
MDESD
Magnitude
 Context specific
Outcome (proximal vs. distal)
Intervention (level of intensity)
Strength of moderator
22
MDESD
Magnitude
 How to estimate?
Literature
Similar programs
Pilot study
In our example
• Moderator effect:4.00
• Standardized moderator effect:4.00/(10.23+21.05) = 0.13
Based on our parameters, assuming 100 kids per
school, need about 380 schools
23
Implications
Cluster level moderators
 Challenging given cost constraints!
 Need lots of clusters!
 If priority, need to consider in design stage
 Consider 3 cases
Balanced
Unbalanced
Unbalanced
Implications
Case 1: Balanced
60
30 L
15 T
30 NL
15C 15 T
30 T
30 C
15 T-L, 15 T-NL
15 C-L, 15 C-NL
15C
15 per group * 4 = 60 total
clusters
Implications
Case 2: Unbalanced
60
40L
20T
20NL
20C 10T
30 T
30 C
20 T-L, 10 T-NL
20X C-L, 10 C-NL
10C
Harmonic mean (20, 10)
Harmonic mean = 13.3
Effective sample size =
13.3*4=53 total clusters
Implications
Case 3: Unbalanced
60
40L
10T
20NL
30C 20T
30 T
Unbalanced case
10T-L, 20T-NL
20T-L, 0 C-NL
30 C
0C
No C-NL group!
Next Steps
Practice session in lab
Questions/comments via video session
28