Two-Way (Independent) ANOVA
Two-Way ANOVA
• “Two-Way” means groups are defined by 2 independent
variables.
• These IVs are typically called factors.
• An experiment in which any combination of values for
the 2 factors can occur is called a completely crossed
factorial design.
• If all cells have the same n, the design is said to be
balanced.
• Still have only 1 dependent variable
PSYC 6130A, PROF. J. ELDER
2
Example: Visual Grating Detection in Noise
500 ms
500 ms
200 ms
Until Response
PSYC 6130A, PROF. J. ELDER
3
2 x 3 Design
0.5
Grating
Frequency
(c/deg)
1.7
4.3%
PSYC 6130A, PROF. J. ELDER
14.8%
4
Noise
Contrast
50.0%
Balanced Design
Factor B
Noise Contras t (Michels on units )
.043
.148
.500
Count
Count
Count
Spatial
Frequency
(cpd)
.500
1.700
Signal to Noise
at Thres hold
Signal to Noise
at Thres hold
Factor A
PSYC 6130A, PROF. J. ELDER
5
10
10
10
10
10
10
Descriptive Statistics
Noise Contras t (Michels on units )
.043
.148
.500
Mean
Mean
Mean
Spatial Frequency
(cpd)
.500
Signal to Noise
at Thres hold
1.700
Group Total
1
s
NT  1
Spatial
Frequency
(cpd)
Group
Total
Mean
.078
.064
.065
.069
.095
.087
.089
.076
.098
.082
.094
.082
Noise Contras t (Michelson units )
X ij  X   0.014571
.043
.148
.500
Std Deviation Std Deviation Std Deviation
.500
Signal to Noise
.006
.003
.003
at Thres hold
1.700 Signal to Noise
.004
.005
.008
at Thres hold

PSYC 6130A, PROF. J. ELDER

2
6
Interactions
• If there is no interaction between the factors (spatial
frequency, noise contrast), the dependent variable (SNR)
for each condition (cell) can be predicted from the
independent effects of factors A and B:
– Cell mean = Grand mean + Row effect + Column effect
Noise Contras t (Michels on units )
.043
.148
.500
Mean
Mean
Mean
Spatial Frequency
(cpd)
.500
Signal to Noise
at Thres hold
1.700
Group Total
PSYC 6130A, PROF. J. ELDER
7
Group
Total
Mean
.078
.064
.065
.069
.095
.087
.089
.076
.098
.082
.094
.082
Interactions
• If there are no interactions, curves should be parallel (effect
of noise contrast is independent of spatial frequency).
PSYC 6130A, PROF. J. ELDER
8
Types of Effects
PSYC 6130A, PROF. J. ELDER
9
Interactions
• In the general case,
Cell mean = Grand mean + Row effect + Column effect + Interaction effect
• Score deviations from cell means are considered error (unpredictable).
• Thus:
Score = Grand mean + Row effect + Column effect + Interaction effect + Error
•
OR
Score - Grand mean = Row effect + Column effect + Interaction effect + Error
PSYC 6130A, PROF. J. ELDER
10
Sum of Squares Analysis
SStotal  SSA  SSB  SSAB  SSerror
where SSerror  SSW
SSA  SSB  SSAB  SSbet
where SSbet is the between-groups sum-of-squares that would
be calculated by lumping all groups into one factor in a 1-Way ANOVA.
Thus SSAB  SSbet  SSA  SSB
This provides a means for calculating SSAB.
PSYC 6130A, PROF. J. ELDER
11
Multiple Subscript and Summation Notation
Single Subscript Notation
X
1
2
12
X3
3
14
PSYC 6130A, PROF. J. ELDER
13
Double Subscript Notation
X ij
PSYC 6130A, PROF. J. ELDER
14
Double Subscript Notation
• The first subscript refers to the row that the particular
value is in, the second subscript refers to the column.
X 11
X 21
X 31
PSYC 6130A, PROF. J. ELDER
X 12
X 22
X 32
X 13
X 23
X 33
15
Double Subscript Notation
• Test your understanding by identifying X 32 in the table
below.
1 43 13
23 42 33
12 11 23
PSYC 6130A, PROF. J. ELDER
16
Double Subscript Notation
• We will follow the notation of Howell:
n  number of scores in each cell in a balanced design.
a  number of levels for Factor A.
b  number of levels for Factor B.
PSYC 6130A, PROF. J. ELDER
17
Multi-Subscript Notation
• In two-way ANOVA, 3 indices are needed:
X ijk
Index i identifies the level of Factor A (the row)
Index j identifies the level of Factor B (the column)
Together, (i , j ) identify the cell of the data table.
Index k identifies the individual score within cell (i , j ).
PSYC 6130A, PROF. J. ELDER
18
Multi-Subscript Notation
• Statistics are calculated by summing over scores within
cells, and thus the third subscript (k) is dropped:
1 n
X ij   X ijk
n k 1
PSYC 6130A, PROF. J. ELDER
19
Multi-Subscript Notation
Thus X 23 refers to the sample mean for the cell in Row 2, Column 3 of the data table
(2nd level of Factor A and 3rd level of Factor B )
X 41 refers to the sample mean for the cell in Row 4, Column 1 of the data table
(4th level of Factor A and 1st level of Factor B )
PSYC 6130A, PROF. J. ELDER
20
Pooled Statistics
• Multi-factor ANOVA requires the
calculation of statistics that pool, or
‘collapse’ data over one or more
factors.
• We indicate the factors over which the
data are being pooled by substituting a
‘bullet’ • for the corresponding index.
PSYC 6130A, PROF. J. ELDER
21
Pooled Statistics
1 a n
1 a
X j 
X ij is a 'column mean'
 X ijk  a 
an i 1 k 1
i 1
obtained by averaging over all scores in column j
(all levels of Factor A)
1 b n
1 b
Xi 
X ij is a 'row mean'
 X ijk  b 
bn j 1 k 1
j 1
obtained by averaging over all scores in row i
(all levels of Factor B)
1 a b n
1 a b
1 a
1 b
X  
X ij   X i  =  X j
 X ijk  ab 
abn i 1 j 1 k 1
a i=1
b j=1
i 1 j 1
is a 'grand mean' obtained by averaging
over all scores in the table.
PSYC 6130A, PROF. J. ELDER
22
Six Step Procedure
Example
Noise Contrast (Michelson units)
.043
.148
.500
Mean
Mean
Mean
Spatial Frequency
(cpd)
.500
Signal to Noise
at Threshold (%)
1.700
Group Total
1
s
NT  1
Spatial
Frequency
(cpd)
 X
.500
1.700
ij
 X 

2
 1.5
Mean
7.8
6.4
6.5
6.9
9.5
8.7
8.9
7.6
9.8
8.2
9.4
8.2
Noise Contrast (Michelson units)
.043
.148
.500
Std Deviation Std Deviation Std Deviation
Signal to Noise
at Threshold
Signal to Noise
at Threshold
PSYC 6130A, PROF. J. ELDER
Group
Total
24
0.62
0.29
0.31
0.36
0.54
0.80
Step 1. State the Hypothesis
• Null hypothesis has 3 parts, e.g.,
– Mean SNR at threshold same for both spatial frequencies
– Mean SNR at threshold same for all noise levels
– No interactions
PSYC 6130A, PROF. J. ELDER
25
Step 2. Select Statistical Test and
Significance Level
• Normally use same a-level for testing all 3 F ratios.
PSYC 6130A, PROF. J. ELDER
26
Step 3. Select Samples and Collect Data
• Strive for a balanced design
• Ideally, randomly sample
• More probably, random assignment
PSYC 6130A, PROF. J. ELDER
27
Step 4. Find Regions of Rejection
• Generally have 3 different critical values for each F test
dfW  NT  ab for all tests.
Denominator
dfA  a  1
dfB  b  1
Numerator
dfAB  dfAdfB
dfT  NT  1
Note that
dfT  dfA  dfB  dfAB  dfW
PSYC 6130A, PROF. J. ELDER
28
Degrees of Freedom Tree
dfT  NT  1
dfbet  ab  1
dfW  NT  ab
dfB  b  1
dfA  a  1
dfAB  dfAdfB
PSYC 6130A, PROF. J. ELDER
29
Step 5. Calculate the Test Statistics

SSW   SSij   X ijk  X ij






SSbet  n  X ij  X 
SSA  bn  X i   X 
SSB  an  X  j  X 

2
 (n  1) sij2
2
2
2
SSAB  SSbet  SSA  SSB

Sanity Check: SSW  SSA  SSB  SSAB  SST   X ij  X 
PSYC 6130A, PROF. J. ELDER
30

2
 (NT  1)s 2
Step 5. Calculate the Test Statistics
SSA
MSA 
dfA
MSA
FA 
MSW
SSB
MSB 
dfB
MSB
FB 
MSW
MSAB
SSAB

dfAB
FAB 
MSAB
MSW
SSW
MSW 
dfW
PSYC 6130A, PROF. J. ELDER
31
Step 6. Make the Statistical Decisions
• Note that 3 independent statistical decisions are being
made.
• Thus the probability of one or more Type I errors is
greater than the α value used for each test.
• It is not common to correct for this.
• You should be aware of this issue as both a producer
and consumer of scientific results!
PSYC 6130A, PROF. J. ELDER
32
SPSS Output
Main effects
Tests of Between-Subjects Effects
Dependent Variable: Signal to Nois e at Threshold
Source
Corrected Model
Intercept
SpatialFreq
NoiseContrast
SpatialFreq *
NoiseContrast
Error
Total
Corrected Total
Type III Sum
of Squares
.011 a
.399
.009
.001
df
5
1
1
2
Mean Square
F
.002
81.107
.399 14644.217
.009
344.657
.001
18.802
.001
2
.000
.001
.412
.013
54
60
59
2.73E-005
a. R Squared = .882 (Adjus ted R Squared = .872)
Interaction
PSYC 6130A, PROF. J. ELDER
33
11.637
Sig.
.000
.000
.000
.000
Partial Eta
Squared
.882
.996
.865
.411
.000
.301
SPSS Output
Tests of Between-Subjects Effects
Dependent Variable: Signal to Nois e at Threshold
SSbet
nX
2
SSA
SSB
SSAB
SSW
X
SST
2
i
Type III Sum
nX 2of Squares
Source
Corrected Model
.011 a
Intercept
.399
SpatialFreq
.009
NoiseContrast
.001
SpatialFreq *
.001
NoiseContrast
Error
.001
Total
.412
Corrected Total
.013
df
5
1
1
2
nX
Mean Square
F
.002
81.107
.399 14644.217
.009
344.657
.001
18.802
2
.000
54
60
59
2.73E-005
2
a. R Squared =nX
.882 (Adjus ted R Squared = .872)
PSYC 6130A, PROF. J. ELDER
34
11.637
2
Sig.
.000
.000
.000
.000
Partial Eta
Squared
.882
.996
.865
.411
.000
.301
Assumptions of Two-Way Independent ANOVA
• Same as for One-Way
• If balanced, don’t have to worry about homogeneity of
variance.
PSYC 6130A, PROF. J. ELDER
35
Advantages of 2-Way ANOVA with 2
Experimental Factors
• One factor may not be of interest (e.g., gender), but may
affect the dependent variable.
• Explicitly partitioning the data according to this ‘nuisance’
variable can increase the power of tests on the
independent variable of interest.
PSYC 6130A, PROF. J. ELDER
36
Simple Effects
• When significant main
effects are
discovered, it is
common to also test
for simple effects.
PSYC 6130A, PROF. J. ELDER
37
Simple Effects
• A main effect is an effect
of one factor measured by
collapsing (pooling) over
all other factors.
• A simple effect is an effect
of one factor measured by
fixing all other factors.
• Although we found
significant main effects,
given the significant
interaction, these main
effects do not necessarily
imply similarly significant
simple effects.
PSYC 6130A, PROF. J. ELDER
38
Simple Effects
• Thus, particularly when a
significant interaction is
observed, a factorial
ANOVA is often followed
up by a series of one-way
ANOVAS to test simple
effects.
• For our example, there
are a total of 5 possible
simple effects to test.
PSYC 6130A, PROF. J. ELDER
39
Simple Effects
• To conduct follow-up oneway ANOVA tests of
simple effects in SPSS:
– Select Split File … from
the Data menu
– Click on Organize Output
by Groups
– Transfer the factor to be
held constant to the
space labeled “Groups
Based On.”
– Now proceed with oneway ANOVAS as usual.
PSYC 6130A, PROF. J. ELDER
40
Simple Effects
ANOVAa
Signal to Noise at Threshold
Sum of
Squares
df
Between Groups
.001
2
Within Groups
.001
27
Total
.002
29
a. Spatial Frequency (cpd) = .500
Mean Square
.001
.000
Test of Homogeneity of Variances
F
32.990
Sig.
.000
a
Signal to Noise at Threshold
Levene
Statistic
5.120
df1
df2
2
Sig.
27
.013
a. Spatial Frequency (cpd) = .500
b
Robust Tests of Equality of Means
Signal to Noise at Threshold
a
Statistic
df1
Welch
21.413
2
Brown-Forsythe
32.990
2
a. Asymptotically F distributed.
df2
16.975
17.382
Sig.
.000
.000
b. Spatial Frequency (cpd) = .500
PSYC 6130A, PROF. J. ELDER
41
Simple Effects
ANOVAa
Signal to Noise at Threshold
Sum of
Squares
df
Between Groups
.000
2
Within Groups
.001
27
Total
.001
29
a. Spatial Frequency (cpd) = 1.700
Mean Square
.000
.000
F
5.899
Sig.
.007
a
Test of Homogeneity of Variances
Signal to Noise at Threshold
Levene
Statistic
df1
df2
2.037
2
27
Sig.
.150
a. Spatial Frequency (cpd) = 1.700
b
Robust Tests of Equality of Means
Signal to Noise at Threshold
a
Statistic
df1
Welch
5.527
2
Brown-Forsythe
5.899
2
a. Asymptotically F distributed.
df2
16.511
19.883
Sig.
.015
.010
b. Spatial Frequency (cpd) = 1.700
PSYC 6130A, PROF. J. ELDER
42
Simple Effects
• Again note that multiple independent statistical decisions
are being made.
• Conditioning the test for simple effects on a significant
main effect provides protection if only 2 simple effects
are being tested.
• Otherwise, the probability of one or more Type I errors is
greater than the α value used for each test.
• It is not common to correct for this.
• You should be aware of this issue as both a producer
and consumer of scientific results!
PSYC 6130A, PROF. J. ELDER
43
End of Lecture
April 8, 2009
Planned or Posthoc Pairwise Comparisons
• If significant main (and possibly
simple) effects are found, it is
common to follow up with one or
more pairwise tests.
• It is most common to test
differences between marginal
means within a factor (i.e., pooling
over the other factor).
• In this example, there are only 3
meaningful posthoc tests on
marginal means. Why?
PSYC 6130A, PROF. J. ELDER
45
Pairwise Comparisons on Marginal Means
•
Since there are 3 levels of noise, we can consider
using Fisher’s LSD.
•
However, since variances do not appear
homogeneous, we should not use an LSD based
on pooling the variance over all 3 conditions.
Test of Homogeneity of Variances
Signal to Noise at Threshold
Levene
Statistic
12.229
df1
df2
2
57
Sig.
.000
Multiple Comparisons
Dependent Variable: Signal to Noise at Threshold
LSD
(I) Noise Contrast
(Michelson units)
.043
.148
.500
(J) Noise Contrast
(Michelson units)
.148
.500
.043
.500
.043
.148
Mean
Difference
(I-J)
.010120*
.004800
-.010120*
-.005320
-.004800
.005320
Std. Error
.004492
.004492
.004492
.004492
.004492
.004492
Sig.
.028
.290
.028
.241
.290
.241
95% Confidence Interval
Lower Bound Upper Bound
.00112
.01912
-.00420
.01380
-.01912
-.00112
-.01432
.00368
-.01380
.00420
-.00368
.01432
*. The mean difference is significant at the .05 level.
PSYC 6130A, PROF. J. ELDER
46
Pairwise Comparisons on Marginal Means
•
Alternative when variances
appear heterogeneous:
–
Compute Fisher’s LSD by hand,
calculating standard error
separately for each test (not
difficult)
–
One of the unequal variance
post-hoc tests offered by SPSS
Multiple Comparisons
Dependent Variable: Signal to Noise at Threshold
Games-Howell
(I) Noise Contrast
(Michelson units)
.043
.148
.500
Mean
Difference
(I-J)
.010120*
Std. Error
.003787
Sig.
.030
Lower Bound
.00085
.500
.004800
.004573
.552
-.00648
.01608
.043
-.010120*
.003787
.030
-.01939
-.00085
.500
-.005320
.005028
.546
-.01762
.00698
.043
-.004800
.004573
.552
-.01608
.00648
.148
.005320
.005028
.546
-.00698
.01762
(J) Noise Contrast
(Michelson units)
.148
95% Confidence Interval
*. The mean difference is significant at the .05 level.
PSYC 6130A, PROF. J. ELDER
47
Upper Bound
.01939
Planned or Posthoc Pairwise Comparisons
•
It is also possible to test differences between cell
means. Note that in this design, there are 15
possible pairwise cell comparisons.
•
It doesn’t make that much sense to compare 2
cells that are not in the same row or column (i.e.
that differ in both factors).
•
It is more likely that you would follow a
significant simple effect test with a set of
pairwise comparisons within a factor while
holding the other factor constant. There are 9
such comparisons possible here.
•
For example, within a spatial frequency
condition, what noise conditions differ
significantly?
•
This defines a total of 6 pairwise comparisons (2
families of 3 comparisons each).
PSYC 6130A, PROF. J. ELDER
48
Planned or Posthoc Pairwise Comparisons
•
Alternative when variances appear
heterogeneous:
–
Compute Fisher’s LSD by hand,
calculating standard error separately
for each test (not difficult)
–
One of the unequal variance post-hoc
tests offered by SPSS (assumes allpairs)
Multiple Comparisonsa
Dependent Variable: Signal to Noise at Threshold
Games-Howell
(I) Noise Contrast
(Michelson units)
.043
.148
.500
Mean
Difference
(I-J)
.005890*
Std. Error
.002067
Sig.
.030
Lower Bound
.00055
Upper Bound
.01123
.500
-.003160
.002790
.513
-.01056
.00424
.043
-.005890*
.002067
.030
-.01123
-.00055
.500
-.009050*
.003066
.024
-.01697
-.00113
.043
.003160
.002790
.513
-.00424
.01056
.148
.009050*
.003066
.024
.00113
.01697
(J) Noise Contrast
(Michelson units)
.148
95% Confidence Interval
*. The mean difference is significant at the .05 level.
a. Spatial Frequency (cpd) = 1.700
PSYC 6130A, PROF. J. ELDER
49
Interaction Comparisons
• If significant interactions are found in a design that is 2x3
or larger, it may be of interest to test the significance of
smaller (e.g., 2x2) interactions.
• These can be tested by ignoring specific subsets of the
data for each test (e.g., by using the SPSS Select Cases
function).
PSYC 6130A, PROF. J. ELDER
50
Unbalanced Designs for Two-Way ANOVA
• Dealing with unbalanced designs is easy for One-Way
ANOVA.
• Dealing with unbalanced designs is trickier for Two-Way.
PSYC 6130A, PROF. J. ELDER
51
Simple Solution
• Let n = harmonic mean of sample sizes.
• Calculate marginal means as an unweighted mean of
cell means (not the pooled mean).
PSYC 6130A, PROF. J. ELDER
52
Better Solution
• Regression approach to ANOVA (will not cover)
PSYC 6130A, PROF. J. ELDER
53