Statistical Analysis
Professor Lynne Stokes
Department of Statistical Science
Lecture 18
Random Effects
Fixed vs. Random Factors
Fixed Factors
Levels are preselected, inferences limited
to these specific levels
Factors
Shaft
Sleeve
Lubricant
Levels
Steel, Aluminum
Porous, Nonporous
Lub 1, Lub 2, Lub 3, Lub 4
Manufacturer
Speed
A, B
High, Low
Fixed Factors (Effects)
Fixed Factors
Levels are preselected, inferences limited
to these specific levels
One-Factor Model
yij = m + ai + eij
Fixed Levels
Main Effects
mi - m = ai
Changes in the mean
mi
Parameters
Random Factors (Effects)
Random Factor Levels are a random sample
from a large population of possible levels.
Inferences are desired on the population of levels.
Factors
Lawnmower
Levels
1, 2, 3, 4, 5, 6
One-Factor Model
yij = m + ai + eij
Random Levels
Random Factors (Effects)
One-Factor Model
yij = m + ai + eij
Estimate Variance
Components
a2 , 2
Main Effects
Random ai
2

Variability = a
Skin Swelling Measurements
Factors
Laboratory animals (Random)
Location of the measurement: Back, Ear (Fixed)
Repeat measurements (2 / location)
Source
Animals
Locations
Error
Total
df
5
1
17
23
SS
0.4436
1.0626
0.1273
1.6355
MS
0.0887
1.0626
0.0075
F
11.85
141.88
p-Value
0.003
0.001
Automatic Cutoff Times
Factors
Manufacturers: A, B (Fixed)
Lawnmowers: 3 for each manufacturer (Random)
Speeds: High, Low (Fixed)
Source
Manufacturers
Lawnmowers
Speed
MxS
Error
Total
df
1
4
1
1
16
23
SS
2,521
2,853
20,886
11
2,117
23,388
MS
2,521
713
20,886
11
132
}
F
3.54
5.39
157.87
0.08
p-Value
0.133
0.006
0.000
0.780
MGH Table 13.6
Random Factor Effects
Assumption
Factor levels are a random sample from a large
population of possible levels
Subjects (people) in a medical study
 Laboratory animals
 Batches of raw materials
 Fields or farms in an agricultural
study
Inferences are desired on the
population
of levels, NOT just on the levels
 Blocks in a block design

included in the design
Random Effects Model Assumptions
(All Factors Random)


Levels of each factor are a random sample of all
possible levels of the factor
Random factor effects and model error terms are
distributed as mutually independent zero-mean normal
variates; e.g., ei~NID(0,e2) , ai~NID(0,a2), mutually
independent
Analysis of variance model contains random
variables for each random factor and interaction
Interactions of random factors
are assumed random
Skin Color Measurements
Factors
Participants -- representative of those from one ethnic group,
in a well-defined geographic region of the U.S.
Weeks -- No skin treatment, studying week-to-week variation
(No Repeats -- must be able to assume no interaction)
Source
Subjects
Weeks
Error
Total
df
6
2
12
20
SS
1904.58
2.75
20.46
1927.78
MS
317.43
1.37
1.71
F
186.17
0.81
p-Value
0.000
0.468
MGH Table 10.3
Two-Factor Random Effects Model:
Main Effects Only
Two-Factor Main Effects Model
yijk = m + ai + bj + eijk
i = 1, ..., a
j = 1, ..., b
bi ~ NID( 0,  2b )
a i ~ NID( 0,  2a )
 = 0  No Effect
Mutually Independent

0
e ijk ~ NID( 0,  e2 )
Two-Factor Random Effects Model
Two-Factor Model
yijk = m + ai + bj + (ab)ij + eijk
i = 1, ..., a
j = 1, ..., b
k = 1, ..., r
bi ~ NID( 0,  2b )
a i ~ NID( 0,  2a )
Mutually Independent
(ab) ij ~ NID( 0,  2ab )
 = 0  No Effect

0
e ijk ~ NID( 0,  e2 )
Two-Factor Model Differences
Fixed Effects
Mean
Random Effects
mij = m + ai + bj + (ab)ij
mij = m
Change the Mean
Variance
 2y =  e2
2
 2y =  a2   2b   ab
  e2
Variance Components
Expected Mean Squares


Functions of model parameters
Identify testable hypotheses


Components set to zero under H0
Identify appropriate F statistic ratios

Under H0, two E(MS) are identical
Properties of Quadratic Forms in Normally
Distributed Random Variables
y ~ N(m, )
E( yAy) = tr(A)  mAm
yAy ~  2 ( , )  E( yAy) =  mAm =   2
Expected Mean Squares
One Factor, Fixed Effects
yij = m + ai + eij
i = 1, ... , a ; j = 1, ... , r
eij ~ NID(0,e2)
 y i ~ NID(m  a i , r 1 e2 )
Expected Mean Squares
One Factor, Fixed Effects
 y i ~ NID(m  a i , r 1 e2 )
Sum of Squares
y = (y1 , ... , y a  )
SS A = ry(I  a 1J ) y

E{SS A / e2 } = (a  1) 

E{MS A } = e2

 rQ a
r
1

a
(
I

a
J )a
2
e
1
, Qa =
a(I  a 1J )a
a 1
a
=  a i  a 2 a  1
i =1
Expected Mean Squares
One Factor, Fixed Effects
 y i ~ NID(m  a i , r 1 e2 )
Sum of Squares
y = (y 1 , ... , y a ) 
SS A = ry (I  a 1J ) y
 E{SS A /
 e2 } =
 E{MS A } =
 e2
( a  1) 
 rQ a
r
1
a
(
I

a
J )a

2
e
1
, Qa =
a (I  a 1J )a
a1
E{MSA)=e2  a1 = a2 = ... = aa
Expected Mean Squares
Three-Factor Fixed Effects Model
Source
A
AB
ABC
Error
Mean Square
MSA
MSAB
MSABC
MSE
Expected Mean Square
e2 + bcr Qa
e2 + cr Qab
e + r Qabg
 e2
Typical Main Effects
and Interactions
1 a
2
Qa =
(
a

a
)
, etc.

i
•All effects tested against error
a  1 i =1
Expected Mean Squares
One Factor, Random Effects
yij = m + ai + eij
i = 1, ... , a ; j = 1, ... , r
ai ~ NID(0,a2) , eij ~ NID(0,e2)
 y i ~ NID(m ,  2a  r 1 e2 )
Independent
Expected Mean Squares
One Factor, Random Effects
 y i ~ NID(m ,  2a  r 1 e2 )
Sum of Squares
y = (y1 , ... , ya  )
SS A = ry(I  a 1J ) y
 E{SS A /(a2  r 1e2 )} = (a  1)r
 E{MS A } = e2  ra2
Expected Mean Squares
One Factor, Random Effects
 y i ~ NID(m ,  2a  r 1 e2 )
Sum of Squares
y = (y1 , ... , ya  )
SS A = ry(I  a 1J ) y
 E{SS A /(a2  r 1e2 )} = (a  1)r
 E{MS A } = e2  ra2
E{MSA)=e2  a2 = 0
Skin Color Measurements
Factors
Participants -- representative of those from one ethnic group,
in a well-defined geographic region of the U.S.
Weeks -- No skin treatment, studying week-to-week variation
(No Repeats -- Must be Able to Assume No Interaction)
Source
Subjects
Weeks
Error
Total
df
6
2
12
20
SS
1904.58
2.75
20.46
1927.78
MS
317.43
1.37
1.71
E(MS)
2  3a2
2  72w
2
Expected Mean Squares
Three-Factor Random Effects Model
Source
Mean Square
A
MSA
AB
ABC
Error
MSAB
MSABC
MSE
Expected Mean Square
e2 + rabc2 + crab2
+ brac2 + bcra2
e2 + rabc2 + crab2
e + rabc2
 e2
•Effects not necessarily tested against error
•Test main effects even if interactions are significant
•May not be an exact test (three or more factors, random
or mixed effects models; e.g. main effect for A)
Expected Mean Squares
Balanced Random Effects Models




Each E(MS) includes the error variance component
Each E(MS) includes the variance component for the
corresponding main effect or interaction
Each E(MS) includes all higher-order interaction
variance components that include the effect
The multipliers on the variance components equal the
number of data values in factor-level combination
defined by the subscript(s) of the effect
e.g., E(MSAB) = e2 + rabc2 +crab2
Expected Mean Squares
Balanced Experimental Designs
1. Specify the ANOVA Model
yijk = m + ai + bj + (ab)ij + eijk
Two Factors, Fixed Effects
MGH Appendix to Chapter 10
Expected Mean Squares
Balanced Experimental Designs
2. Label a Two-Way Table
a. One column for each model subscript
b. Row for each effect in the model
-- Ignore the constant term
-- Express the error term as a nested effect
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
i
ai
bj
(abij
ek(ij)
j
k
Expected Mean Squares
Balanced Experimental Designs
3. Column Subscript Corresponds to a Fixed Effect.
a. If the column subscript appears in the row effect & no
other subscripts in the row effect are nested within
the column subscript
-- Enter 0 if the column effect is in a fixed row effect
b. If the column subscript appears in the row effect & one or
more other subscripts in the row effect are nested
within the column subscript
-- Enter 1
c. If the column subscript does not appear in the row effect
-- Enter the number of levels of the factor
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 3a
i
ai
bj
(abij
ek(ij)
j
0
0
0
0
k
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 3b
i
ai
bj
(abij
ek(ij)
j
0
0
1
0
0
1
k
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 3c
ai
bj
(abij
ek(ij)
i
j
0
a
0
1
b
0
0
1
k
Expected Mean Squares
Balanced Experimental Designs
4. Column Subscript Corresponds to a Random Effect
a. If the column subscript appears in the row effect
-- Enter 1
b. If the column subscript does not appear in the row
effect
-- Enter the number of levels of the factor
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 4a
ai
bj
(abij
ek(ij)
i
j
k
0
a
0
1
b
0
0
1
1
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 4b
ai
bj
(abij
ek(ij)
i
j
k
0
a
0
1
b
0
0
1
r
r
r
1
Expected Mean Squares
Balanced Experimental Designs
5. Notation
a. f = Qfactor(s) for fixed main effects and
interactions
b. f = factor(s)2 for random main effects and
interactions
List each f parameter in a column on the same
line as its corresponding model term.
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 5
ai
bj
(abij
ek(ij)
i
j
k
f
0
a
0
1
b
0
b
1
r
r
r
1
Qa
Qb
Qab
 e2
Expected Mean Squares
Balanced Experimental Designs
6. MS = Mean Square, C = Set of All Subscripts for the Corresponding Model
Term
a. Identify the f parameters whose model terms contain all the subscripts in
C (Note: can have more than those in C)
b. Multipliers for each f :
-- Eliminate all columns having the subscripts in C
-- Eliminate all rows not in 6a.
-- Multiply remaining constants across rows for each f
c. E(MS) is the linear combination of the coefficients from
6b and the corresponding f parameters; E(MSE) = e2.
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 6a

B
AB
Error
E(MS)
f
e2 Qab , Qa
e2 Qab , Qb
e2 Qab
e2
Qa
Qb
Qab
 e2
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 6b: MSAB
ai
bj
(abij
ek(ij)
i
j
k
f
0
a
0
1
b
0
b
1
r
r
r
1
Qa
Qb
Qab
 e2
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 6c
E(MS)

B
AB
Error
e2 Qab  Qa
e2 Qab  Qb
e2  rQab
 e2
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 6b: MSB
ai
bj
(abij
ek(ij)
i
j
k
f
0
a
0
1
b
0
b
1
r
r
r
1
Qa
Qb
Qab
 e2
Two Factors, Fixed Effects
yijk = m + ai + bj + (ab)ij + eijk
Step 6c
E(MS)

B
AB
Error
e2 Qab  Qa
e2  ar Qb
e2  rQab
 e2
Two Factors, Fixed Effects
Source
df
SS
E(MS)
a-1
b-1
(a-1)(b-1)
SSA
SSB
SSAB
e2  brQA
e2  arQA
e2  rQAB
Error
ab(r-1)
SSE
 e2
Total
abr-1
TSS
A
B
AB
Under appropriate null hypotheses,
E(MS) for A, B, and AB same as E(MSE)
F = MS / MSE