Factorial ANOVA
Cal State Northridge
320
Andrew Ainsworth PhD
Topics in Factorial Designs
What is Factorial?
Assumptions
Analysis
Multiple Comparisons
– Main Effects
– Simple Effects
– Simple Comparisons
Effect Size estimates
Higher Order Analyses
Psy 320 - Cal State Northridge
2
Factorial?
Factorial – means that:
1. You have at least 2 IVs
2. And all levels of one variable occur in
combination with all levels of the other
variable(s).
Assumptions
– Same as one-way ANOVA but they
are tested within each cell
– i.e. Normality, Homogeneity and
Independence
Psy 320 - Cal State Northridge
3
Simplest Form: 2 x 2 ANOVA
B
b1
A
b2
a1
a2
Video Game
GTA
NBA 2K7
Gender
Men
Women
Psy 320 - Cal State Northridge
4
Analysis
Performing a factorial analysis does the job of
three analyses in one
– Two one-way ANOVAs, one for each IV (called a
main effect)
– And a test of the interaction between the IVs
– Interaction? – the effect of one IV depends on the
level of another IV
• The variability that is left over after you assess each IV
• The 2 IVs together work to affect scores over and above
either of them independently
Psy 320 - Cal State Northridge
5
Analysis
The between groups sums of squares
from 1-way ANOVA is further broken
down:
– Before SSbg = SSeffect
– Now SSbg = SSA + SSB + SSAB
– In a two IV factorial design A, B and AxB
all differentiate between groups,
therefore they all add to the SSbg
Psy 320 - Cal State Northridge
6
Analysis
Total variability = (variability of A around
GM) + (variability of B around GM) +
(variability of each group mean {AB} around
GM) + (variability of each person’s score
around their group mean)
SSTotal = SSA + SSB + SSAB + SSerror
2
2
2
(
Y

Y
)

n
(
Y

Y
)

n
(
Y

Y
)
 i GM  a a GM  b b GM
   nab (Yab  YGM ) 2   na (Ya  YGM ) 2   nb (Yb  YGM ) 2 
  (Yi  Yab )
2
Psy 320 - Cal State Northridge
7
Analysis
Degrees of Freedom
– dfA = #groupsA – 1
– dfB = #groupsB – 1
– dfAB = (a – 1)(b – 1)
– dferror = ab(n – 1) = abn – ab = N – ab
– dftotal = N – 1 = a – 1 + b – 1 + (a – 1)(b – 1)
+ N – ab
Psy 320 - Cal State Northridge
8
Analysis
Breakdown of
degrees of freedom
Breakdown of
sums of squares
SStotal
SSbg
SSA
SSB
N-1
SSwg
SSAB
ab-1
a-1
Psy 320 - Cal State Northridge
b-1
N-ab
(a-1)(b-1)
9
Analysis
Mean square
– The mean squares are calculated the same
– SS/df = MS
– You just have more of them, MSA, MSB,
MSAB, and MSWG
– This expands when you have more IVs
• One for each main effect, one for each
interaction (two-way, three-way, etc.)
Psy 320 - Cal State Northridge
10
Analysis
F-test
– Each effect and interaction is a separate
F-test
– Calculated the same way: MSeffect/MSWG
since MSWG is our error variance
estimate
– You look up a separate Fcrit for each test
using the dfeffect, dfWG and tabled values
Psy 320 - Cal State Northridge
11
Example
A: Profession
a1: Administrators
a2: Belly Dancers
a3: Politicians
2
2
2
Y

0

1


B: Vacation Length
b1: 1 week
b2: 2 weeks
b3: 3 weeks
0
4
5
1
7
8
0
6
6
5
5
9
7
6
8
6
7
8
5
9
3
6
9
3
8
9
2
 22  1046
Psy 320 - Cal State Northridge
12
Analysis
Sample data reconfigured into cell and
marginal means (with variances)
A:Profession
a1: Administrators
B: Vacation Length
b1: 1 week b2: 2 weeks b3: 3 weeks Marginal A means
Ya1b1 = 0.333 Ya1b2 = 5.667 Ya1b3 = 6.333
2
a1b1
s
Ya2b1 = 6
a2: Belly Dancers
a3: Politicians
Marginal B Means
Y
2
 1046
= 0.333
2
a2b1 =
s
1
Ya3b1 = 6.333
2
a3b1
s
= 2.333
2
a1b2
s
= 2.333
Ya2b2 = 6
2
a2b2
s
=1
Ya3b2 = 9
2
a3b2
s
=0
2
a1b3
s
= 2.333
Ya2b3 = 8.333
2
a2b3
s
= 0.333
Ya3b3 = 2.667
2
a3b3
s
= 0.333
Yb1 = 4.222 Yb2 = 6.889 Yb3 = 5.778
Psy 320 - Cal State Northridge
Ya1 = 4.111
Ya2 = 6.778
Ya3 = 6
Y... = 5.630
13
Example – Sums of Squares
SStotal   (Yi  YGM ) 2 
 (____  ____) 2  (____  ____) 2  (____  ____) 2 
(5  5.630)  (7  5.630)  (6  5.630) 
2
2
2
(5  5.630)  (6  5.630)  (8  5.630) 
2
2
2
(4  5.630)  (7  5.630)  (6  5.630) 
2

2
2
 (3  5.630) 2  (2  5.630) 2  190.296
Psy 320 - Cal State Northridge
14
Example – Sums of Squares
SS A   na (Ya  YGM )2 
 [___*(____  ____) 2 ]  [___*(____  ____) 2 ] 
[___*(6  5.630) ]  33.852
2
Psy 320 - Cal State Northridge
15
Example – Sums of Squares
SS B   nb (Yb  YGM ) 
2
 [___*(____  ____) ]  [___*(____  ____) ] 
2
2
[___*(5.778  5.630) ]  32.296
2
Psy 320 - Cal State Northridge
16
Example – Sums of Squares
SS AB   nab (Yab  YGM ) 2   na (Ya  YGM ) 2   nb (Yb  YGM ) 2 
 [___*(____  ____) 2 ]  [___*(____  ____) 2 ] 
[___*(____  ____) 2 ]  [___*(____  ____) 2 ] 
[___*(____  ____) 2 ]  [___*(9  5.630) 2 ] 
[___*(6.333  5.630) 2 ]  [___*(8.333  5.630) 2 ] 
[___*(2.667  5.630) 2 ]  170.296
 170.296  33.825  32.296  104.148
Psy 320 - Cal State Northridge
17
Example – Sums of Squares
SS Error   (Yi  Yab ) 2 
 (____  ____) 2  (____  ____) 2  (____  ____) 2 
(____  ____)  (____  ____)  (____  ____) 
2
2
2
(5  6.333)  (6  6.333)  (8  6.333) 
2
2
2
(4  5.667)  (7  5.667)  (6  5.667) 
2

2
2
 (3  2.667)  (2  2.667)  20
2
2
Psy 320 - Cal State Northridge
18
Analysis – Computational
Marginal Totals – we look in the
margins of a data set when
computing main effects
Cell totals – we look at the cell totals
when computing interactions
In order to use the computational
formulas we need to compute both
marginal and cell totals
Psy 320 - Cal State Northridge
19
Analysis – Computational
Sample data reconfigured into cell
and marginal totals
A: Profession
a1: Administrators
a2: Belly Dancers
a3: Politicians
Marginal Sums for B
B: Vacation Length
b1: 1 week
b2: 2 weeks
b3: 3 weeks
1
17
19
18
18
25
19
27
8
b1 = 38
b2 = 62
b3 = 52
Psy 320 - Cal State Northridge
Marginal Sums for A
a1 = 37
a2 = 61
a3 = 54
T = 152
20
Analysis – Computational
Formulas for SS
a

T


SS 

2
A
SS B
2
bn
abn
b




an
2
T2

abn
ab     a     b 






2
SS AB
2
n
SSerror   Y
2
bn
ab 




T2
SST   Y 
abn
an
2
T2

abn
2
n
2
21
Analysis – Computational
Example
a




2
T2
SS A

bn
abn
___ 2  ___ 2  542
___ 2
SS A 

 ____  ____  33.85
3(3)
3(3)(3)
b




2
T2
SS B


an
abn
382  ___ 2  ___ 2
___ 2
SS B 

 888  855.7  32.30
3(3)
3(3)(3)
Psy 320 - Cal State Northridge
22
Analysis – Computational
Example
ab     a     b 






2
2
2
T2
SS AB

n
bn
an
abn
___ 2  ___ 2  ___ 2  182  182  252  19 2  27 2  82
SS AB 
3
37 2  612  542 382  622  522 1522



3(3)
3(3)
3(3)(3)
 ____  889.55  888  855.7  104.15
Psy 320 - Cal State Northridge
23
Analysis – Computational
Example
SSerror   Y
2
ab 




2
n
12  17 2  192  182  182  252  19 2  27 2  82
SSerror  _____ 
3
 ____  1026  20
2
T
SST   Y 2 
abn
1522
SST  1046 
 1046  855.7  190.30
3(3)(3)
Psy 320 - Cal State Northridge
24
Analysis – Computational
Example
df A  a  1  3  1  2
df B  b  1  3  1  2
df AB  (a  1)(b  1)  (3  1)(3  1)  2(2)  4
df Error  abn  ab  27  9  18
dftotal  abn  1  27  1  26
Psy 320 - Cal State Northridge
25
Analysis
Example
Tests of Between-Subjects Effects
Dependent Variable: ENJOY
Source
PROFESSION
LENGTH OF STAY
PROFESSION * LENGTH
WITHIN GROUPS
TOTAL
Type III Sum
of Squares
33.852
32.296
104.148
20.000
190.296
df
2
2
4
18
26
Mean Square
16.926
16.148
26.037
1.111
F
15.233
14.533
23.433
Sig.
.000
.000
.000
The MSWG is also the pooled (average)
variance across the cells, since all n are equal:
(.333+2.333+2.333+1+1+.333+2.333+0+.333)/9 = 1.111
Psy 320 - Cal State Northridge
26
Analysis
Fcrit(2,18)=3.55
Fcrit(4,18)=2.93
Since 15.25 > 3.55, the effect for
profession is significant
Since 14.55 > 3.55, the effect for
length is significant
Since 23.46 > 2.93, the effect for
profession * length is significant
Psy 320 - Cal State Northridge
27
Effect Size Revisited
Eta Squared is calculated for each
effect

2
effect

SSeffect
SStotal
Omega Squared also for each effect

2
Effect

SS Effect  (k Effect  1) MSWG
SST  MSWG
Psy 320 - Cal State Northridge
28
Effect Size Example
Effect Size for Profession

2
Profession
SS Profession 33.852


 .178
SS total
190.296

2
Profession
SSProfession  (kProfession  1) MSWG


SST  MSWG

2
Profession
33.853  [(3  1) *1.111]

 .165
190.296  1.111
Psy 320 - Cal State Northridge
29
Multiple Comparisons
If a main effect is significant and has more
than 2 levels, than you need to do marginal
comparisons
If the interaction is significant
– You should break the interaction down by
performing a simple effect analysis of A at
each level of B (The effect of A at B1, A at B2, A
at B3, etc.) and vice versa
– If any of them are significant and if A has more
than 2 levels, follow up with simple comparisons
Psy 320 - Cal State Northridge
30
Multiple Comparisons
Simple
Effects
for B
b1
b1
b2
b2
b3
b3
a1
a2
a3
a1
Simple
Effects
for A
a1
a2
a3
a3
Psy 320 - Cal State Northridge
Simple
Comparison
for A
31
Specific Comparisons
If the comparisons were planned than
analyze them without any adjustment
to the critical value
If they were post-hoc than the values
needs to be adjusted (e.g. Tukey,
Bonferroni, etc.)
– This is the same as previously covered
Psy 320 - Cal State Northridge
32
Multiple Comparisons Example
Main Effect: Profession
Multiple Compari sons
Dependent Variable: ENJOY
LS D
(I) PROFES S
1 Adminis trators
2 Belly Dancers
3 Politicians
Bonferroni
1 Adminis trators
2 Belly Dancers
3 Politicians
(J) PROFE SS
2 Belly Dancers
3 Politicians
1 Adminis trators
3 Politicians
1 Adminis trators
2 Belly Dancers
2 Belly Dancers
3 Politicians
1 Adminis trators
3 Politicians
1 Adminis trators
2 Belly Dancers
Mean
Difference
(I-J)
St d.
-2. 67*
-1. 89*
2.67*
.78
1.89*
-.78
-2. 67*
-1. 89*
2.67*
.78
1.89*
-.78
E rror
.497
.497
.497
.497
.497
.497
.497
.497
.497
.497
.497
.497
Sig.
.000
.001
.000
.135
.001
.135
.000
.004
.000
.405
.004
.405
95% Confidenc e Int erval
Lower Bound Upper Bound
-3. 71
-1. 62
-2. 93
-.84
1.62
3.71
-.27
1.82
.84
2.93
-1. 82
.27
-3. 98
-1. 36
-3. 20
-.58
1.36
3.98
-.53
2.09
.58
3.20
-2. 09
.53
Based on observed means.
*. The mean differenc e is significant at the .05 level.
Psy 320 - Cal State Northridge
33
Multiple Comparisons Example
Main Effect: Length of Stay
Multiple Comparisons
Dependent Variable: ENJOY
LSD
(I) LENGTH
1 1 week
2 2 weeks
3 3 weeks
Bonferroni
1 1 week
2 2 weeks
3 3 weeks
(J) LENGTH
2 2 weeks
3 3 weeks
1 1 week
3 3 weeks
1 1 week
2 2 weeks
2 2 weeks
3 3 weeks
1 1 week
3 3 weeks
1 1 week
2 2 weeks
Mean
Difference
(I-J)
-2.67*
-1.56*
2.67*
1.11*
1.56*
-1.11*
-2.67*
-1.56*
2.67*
1.11
1.56*
-1.11
Std. Error
.497
.497
.497
.497
.497
.497
.497
.497
.497
.497
.497
.497
Sig.
.000
.006
.000
.038
.006
.038
.000
.017
.000
.115
.017
.115
95% Confidence Interval
Lower Bound Upper Bound
-3.71
-1.62
-2.60
-.51
1.62
3.71
.07
2.16
.51
2.60
-2.16
-.07
-3.98
-1.36
-2.87
-.24
1.36
3.98
-.20
2.42
.24
2.87
-2.42
.20
Based on observed means.
*. The mean difference is s ignificant at the .05 level.
Psy 320 - Cal State Northridge
34
Simple Effect and Simple Comp.
Profession at 1 week
ANOV A
ENJOY
Between Groups
W ithin Groups
Total
Sum of
Squares
68.222
7.333
75.556
df
2
6
8
Mean S quare
34.111
1.222
F
27.909
Sig.
.001
Multiple Compari sons
Dependent Variable: ENJOY
LS D
(I) PROFES S
1 Adminis trators
2 Belly Dancers
3 Politicians
Bonferroni
1 Adminis trators
2 Belly Dancers
3 Politicians
(J) PROFE SS
2 Belly Dancers
3 Politicians
1 Adminis trators
3 Politicians
1 Adminis trators
2 Belly Dancers
2 Belly Dancers
3 Politicians
1 Adminis trators
3 Politicians
1 Adminis trators
2 Belly Dancers
Mean
Difference
(I-J)
St d.
-5. 67*
-6. 00*
5.67*
-.33
6.00*
.33
-5. 67*
-6. 00*
5.67*
-.33
6.00*
.33
*. The mean differenc e is significant at the .05 level.
E rror
.903
.903
.903
.903
.903
.903
.903
.903
.903
.903
.903
.903
Sig.
.001
.001
.001
.725
.001
.725
.002
.002
.002
1.000
.002
1.000
95% Confidenc e Int erval
Lower Bound Upper Bound
-7. 88
-3. 46
-8. 21
-3. 79
3.46
7.88
-2. 54
1.88
3.79
8.21
-1. 88
2.54
-8. 63
-2. 70
-8. 97
-3. 03
2.70
8.63
-3. 30
2.63
3.03
8.97
-2. 63
3.30
35
Higher-Order Designs
Higher-order – meaning more than 2 IVs
– With 3 IVs; each with 2 levels you have a 2 x
2 x 2 design
– If we have even 5 subjects per cell we are
talking about a minimum of 40 subjects
– We are also talking about:
• SST = SSA + SSB + SSC + SSAB + SSAC + SSBC +
SSABC + SSWG
Psy 320 - Cal State Northridge
36
Higher-Order Designs
Higher-order – meaning more than 2 IVs
– With 4 IVs; each with 2 levels you have a 2 x
2 x 2 x 2 design
– If we have even 5 subjects per cell we are
talking about a minimum of 80 subjects
– We are also talking about:
• SST = SSA + SSB + SSC + SSD + SSAB + SSAC +
SSAD + SSBC + SSBD + SSCD + SSABC + SSABD +
SSACD + SSBCD + SSABCD + SSWG
Psy 320 - Cal State Northridge
37