EDF 802
Dr. Jeffrey Oescher
Factorial ANOVA Designs
I.
Introduction
A.
Purpose of quantitative research design
1.
MAXMINCON (Kerlinger, 1986)
a.
b.
c.
2.
Maximize experimental variance
Minimize error variance
Control for extraneous variance
F-ratio of MSb/MSw
a.
b.
c.
Experimental variance is manifested in MSb
Error variance is manifested in MSw
Example of a research study to examine the effects of verbal
reinforcement (praise, criticism, silence) on solving problems
(1)
(2)
R2 = .40 for this single verbal reinforcement factor (i.e.,
explained and unexplained variation)
The effects of adding a second factor of problem type (simple,
complex) to the study increased R2 to .65
(a)
(b)
3.
B.
A procedure for examining the variance in some dependent measure in terms of
the portion of variance that can be attributed to certain factors (i.e., two or
more)
Application
1.
D.
In any ratio, increasing the numerator and decreasing the denominator results
in a larger quotient.
Description
1.
C.
Explained variance increased by the addition of a
second problem type factor (.20) and an interaction
effect between verbal reinforcement and problem type
(.05)
Error variance was reduced from .60 to .35
Standard procedure for designs with two or more independent variables.
Data
-1-
1.
2.
E.
The dependent variable is interval or ratio
The independent variables are categorical
Terminology
1.
2.
Main effects: the effect associated with each factor. It is considered
independently of all other factors or interactions
Interaction: the effect associated with combinations of certain levels of each
factor
a.
b.
Plots of interactions - see Figure 1
Ordinal
(1)
(2)
c.
Disordinal
(1)
(2)
3.
Lines are not parallel and do not cross
There is a greater difference at one level than another level
Lines are not parallel and cross
The effect of one factor reverses itself as the levels of the other
factor change
Numerical example – See Table 1 for the descriptive statistics for the verbal
reinforcement and problem type example
Table 1
Descriptive Statistics for the Factorial ANOVA Data
Verbal Reinforcement
Problem Type
Total
Praise
Criticism
Silence
Simple
7.60
7.20
4.40
6.40
Complex
7.00
2.00
3.20
4.07
Total
7.30
4.60
3.80
a.
Main effects
(1)
(2)
b.
F.
Verbal reinforcement – Praise (7.30) vs. Criticism (4.60) vs.
Silence (3.80)
Problem type – Simple (6.40) vs. Complex (4.07)
Interaction effect – see Figure 2
Levels of factors (i.e., effects) – specific values of the factors being examined
-2-
1.
Fixed effects
a.
b.
2.
All possible levels of the independent variables are represented in the
design and analysis
Results can be generalized only to these levels
Random effects
a.
3.
Levels of both the independent variables are sampled from larger sets
of possible values
b.
Results can be generalized to the population of levels from which the
levels of the independent variables were randomly selected
Mixed effects
a.
b.
c.
Levels of one independent variable are fixed; levels of the other
independent variable are random
Results relative to the levels of the fixed variable can be generalized
only to those levels
Results relative to the levels of the random variable can be generalized
to the population of levels from which the levels were selected
-3-
II.
Concepts underlying factorial ANOVA
A.
Notation – See Table 2
Table 2
Factorial ANOVA Notation
Levels of K
Levels of J
A1
A2
Aj
Column
Means
B.
1
2
k
X111
X112
X11k
X211
X212
X21k
Xi11
Xi12
Xi1k
X121
X122
X12k
X221
X222
X22k
Xi21
Xi22
Xi2k
X1j1
X1j2
X1jk
X2j1
X2j2
X2jk
Xij1
Xij2
Xijk
X̄·1
X̄·2
X̄·k
Row Means
X̄1·
X̄2·
X̄j·
X̄··
Algebraic approach
1.
Deviation score for the ith person in the jth row and the kth column can be
partitioned into two components: 1) within cell and 2) between cell
SSt = SSw + SSb
2.
The between cell component can be partitioned into three components:
between rows, between columns, and between cells after the first two
components have been accounted for
SSb = SSj + SSk + SSjk
C.
Linear model
Xijk = μ + αj + βk + (αβ)jk +eijk
D.
Partitioning of variance and expected means squares
-4-
1.
Variation within cells
a.
b.
Differences between subjects in the Jth row and Kth column
E ( MS w )   e2
2.
Variation among the J row means
a.
Differences across rows [H0: μ1. = μ2. = … = μj]
E ( MS j )   
2
e
3.
J 1
Variation among the K column means
a.
b.
Differences across columns [H0: μ.1 = μ.2 = … = μ.k]
E ( MS k )   
2
e
4.
N 2j
nJ j2
 1
Variation due to interaction
a.
Differences among levels of one factor across the levels of the other
factors [H0: all αβ effects = 0]
b.
E ( MS  )   e2 
E.
H0: μ1. = μ2. = … = μj.
H0: μ.1 = μ.2 = … = μ.k
H0: all αβ effects = 0
Assumptions
A.
B.
C.
IV.
( J  1)( K  1)
Hypotheses
1.
2.
3.
III.
n( ) 2jk
Independence of samples
Scores on the dependent variable are normally distributed
Homogeneity of variance
Analyzing a two-factor ANOVA
-5-
A.
Sequencing of the analysis
1.
2.
Compute the F-statistics for the main effects and interactions
If a significant interaction exists
a.
b.
3.
If a non-significant interaction exists
a.
B.
Analyze the data for simple effects
Interpret any significant main effects in the context of the simple effects
Analyze any statistically significant main effect using post-hoc
procedures
The data from the verbal reinforcement/problem complexity example presented earlier
is summarized in Table 3
Table 3
Summary Table of the Factorial ANOVA Example
Source
SS
df
MS
F
p
Reinforcement (A)
67.27
2
33.64
9.61
.000
Problem Type(B)
40.84
1
40.84
11.67
.000
A*B
31.26
2
15.63
4.47
.000
Error
17.50
5
3.50
1.
Significant interaction method
a.
Simple effects of one of the independent variables at each level of the
other independent variable
(1)
Simple effects of verbal reinforcement (A) while levels of
problem type (B) are held constant
(a) While both praise (7.60) and criticism (7.20) facilitate the
solving of problems in comparison with silence (4.40), only
praise (7.00) facilitates the solving of complex problems when
compared to silence (3.20) and criticism (2.00)
(2)
Simple effects of problem type (B) while levels of reinforcement
(a)
For the praise condition, children perform slightly better
with a simple reasoning problem (7.60) than with the
complex problems (7.00); this difference is considerably
-6-
greater for children in the criticism condition (7.20 vs.
2.00); the difference for those in the silence condition is
somewhere between (4.40 vs. 3.20)
(3)
2.
Calculation of simple effects is discussed later in these notes
Significant main effects
a.
b.
These effects are being discussed hypothetically due to the significant
interaction effect that precludes any importance attributed to the main
effects
Effects of each of the main factors independent of the other factor
(1)
Consider the three column marginal means reflecting the main
effect of Reinforcement (A)
(a)
(b)
(2)
Consider the two row marginal means reflecting the effect of
Problem Type (B)
(a)
(b)
(3)
V.
When the data is combined for simple and complex
problems, praise is higher than criticism and criticism is
slightly higher than silence
These differences are not representative of the
corresponding differences when considering the other
independent variable
When the data is combined across levels of
reinforcement, children performed better with simple
rather than complex problems
These differences are not representative of the
corresponding differences when considering the other
independent variable
Calculation of main effects is discussed later in these notes
Issues related to factorial ANOVA
A.
Balanced and unbalanced designs
1.
Three methods for computing the sums of squares (SS)
a.
b.
c.
2.
Regression – unique SS for each effect (Type III SS)
Experimental – adjust only the interaction for main effects
Hierarchal – adjust each effect for those preceding it in the model (Type
I SS)
Use the regression method unless obvious reasons to do otherwise (e.g., race
-7-
education race*education)
B.
Models for factorial ANOVA
1.
2.
3.
C.
Fixed effects – MSw is the appropriate error term for all main effects and
interactions
Mixed effects – MSjk is used to test the fixed main effect; MSw is used to test
the random main effect and the interaction effect
Random effects – MSjk is used to test both main effects; MSw is used to test the
interaction effect
Post-hoc comparisons
1.
Simple effects
a.
b.
2.
Main effects
a.
D.
Conceptually similar to computing a one-way ANOVA on each level of a
factor within levels of the other factor
Should a significant result be obtained, follow with appropriate tests of
simple comparisons (e.g., Tukey, Scheffe, etc.)
Use appropriate post-hoc comparison tests (e.g., Tukey, Scheffe, etc.)
Determining sample size
1.
2.
3.
4.
Treat each independent variable separately
Meet or exceed the minimum sample size for each effect (i.e., choose the
sample size on the basis of the larger of the two minimum sample sizes for the
independent variables)
Remember that when using Table C.12 you are identifying the sample size for
each row or column without consideration of the cells themselves
See Table 4
a.
b.
c.
Using Table C.12 for Factor 1 and the assumptions stated in Table 4, 35
subjects are needed for each row. Dividing 35 across all four levels of
Factor 2 results in 9 subjects per cell.
Using Table C.12 for Factor 2 and the assumptions stated in Table 4, 40
subjects are needed for each row. Dividing 40 across all three levels of
Factor 1 results in 14 subjects per cell.
Based on 14 subjects for each of the 12 cells, the total sample size is
estimated at 168.
-8-
Table 4
Sample Size for a Factorial ANOVA
Factor 2
Factor 1
A
B
C
D
1
143 / 94
14 / 9
14 / 9
14 / 9
351
2
14 / 9
14 / 9
14 / 9
14 / 9
35
3
14 / 9
14 / 9
14 / 9
14 / 9
35
402
40
40
40
1
The actual value from Table C.12 using alpha=.05, power=.80, and an effect size of .75
The actual value from Table C.12 using alpha =.05, power=.80, and an effect size of .75
3
The number of subjects per cell based on dividing 40 across all three cells and rounding up
4
The number of subjects per cell based on dividing 35 across all four cells and rounding up
2
VI.
Numerical Examples
A.
B.
Verbal reinforcement and problem complexity example
Flexibility example
-9-
Figure 1
Interaction Effects for Factorial ANOVA
- 10 -
Figure 2
Interaction Effect for Verbal Reinforcement - Problem Type ANOVA
8
7
6
5
Simple
4
Complex
3
2
1
0
Simple
Complex
P
7.6
7
C
7.2
2
S
4.4
3.2
- 11 -