PSYC 6130
One-Way Independent ANOVA
Generalizing t-Tests
• t-Tests allow us to test hypotheses about differences
between two groups or conditions (e.g., treatment and
control).
• What do we do if we wish to compare multiple groups or
conditions simultaneously?
• Examples:
– Effects of 3 different therapies for autism
– Effects of 4 different SSRIs on seratonin re-uptake
– Effects of 5 different body orientations on judgement of induced
self-motion.
PSYC 6130, PROF. J. ELDER
2
Reinterpreting the 2-Sample t-Statistic
t2 
X

n
1
 X2

2
2
sp2
The denominator sp2 is an estimate of the variance  2 of the population,
derived by averaging the variances within the two samples:
sp2 
1 2
(s1  s22 )
2
PSYC 6130, PROF. J. ELDER
3
Reinterpreting the 2-Sample t-Statistic
t2 
X
n
1  X2

2
2
s
2
p
The numerator is also an estimate of the variance  2 of the population,
derived from the variance between the sample means.
To see this, recall that s X 
s
n
 s 2  ns X2
For 2 groups, s X2  ( X1  XG )2  ( X 2  XG )2 , where XG 
2
1
1

 

Thus, s   X1  ( X1  X 2 )    X 2  ( X1  X 2 ) 
2
2

 

2
2
X
2
1
 1

  ( X1  X 2 )    ( X 2  X1 ) 
2
 2


2
1
( X1  X 2 )2
2
PSYC 6130, PROF. J. ELDER
4
1
( X1  X 2 )
2
Example
X1
X2
-10.2 4.8
-1.8 6.7
15.2 -0.8
-0.4 8.9
12.3 23.1
-7.0 5.2
0.1 -0.1
-7.8 9.1
5.9 0.6
-2.5 -11.1
Mean
Std Dev
PSYC 6130, PROF. J. ELDER
0.4
8.4
5
4.6 2.5 X G
8.8
The F Distribution
t2 
n

X1  X 2

2
2
s
2
p
Thus, under the null hypothesis, the numerator and denominator are
independent estimates of the same population variance  2.
The ratio of 2 independent, unbiased estimates of the same variance
follows an F distribution.
0.5
F distribution for 2 groups
of size n=13
0.4
0.3
0.2
0.1
0
PSYC 6130, PROF. J. ELDER
0
2
4
6
6
8
10
Within and Between Variances
• Recall that the variance is, by definition, the mean squared deviation
of scores from their mean.
• Since the numerator of the t2 statistic estimates the variance from
the deviations of group means, it is called the mean-square-between
MSbet.
• Since the denominator of the t2 statistic estimates the variance from
the deviations within groups, it is called the mean-square-within
MSW.
• These definitions allow us to generalize to an arbitrary number of
groups.
Thus F 
MSbet
MSW
PSYC 6130, PROF. J. ELDER
7
Generalizing to > 2 Groups
F
MSbet
MSW
MSbet
MSW
n (X


i
i
 XG )2
dfbet
(n


i
, where XG 
 1)si2
dfw
PSYC 6130, PROF. J. ELDER
8
1
NT

all scores
Xi 
1
NT
n X
i
i
Degrees of Freedom
• Recall that the sample variance follows a scaled chisquare distribution, parameterized by the degrees of
freedom.
• Thus the F distribution is a ratio of two chi-square
distributions, each with different degrees of freedom.
dfbet  k  1, where k  number of groups.
dfW  NT  k, where NT = total number of subjects over all groups.
   ni  1
dftot  dfbet  dfW  NT  1
PSYC 6130, PROF. J. ELDER
9
Testing Hypotheses
F
Large values of F suggest that differences between the groups
MSbet
MSW
are inflating the MSbet estimate of  2  reject H0 .
1
F distribution for 3 groups of size n=13
p(F)
0.8
0.6
0.4
0.2
0
0
Fcrit
PSYC 6130, PROF. J. ELDER
2
4
6
3.32 for   .05 (Appendix F)
12
8
10
PSYC 6130, PROF. J. ELDER
13
When k=2
• ANOVA will give exactly the same result as two-tailed ttest.
• One-tailed tests must be done using t-tests.
PSYC 6130, PROF. J. ELDER
14
Example
From the Canadian Generalized Social Survey, Cycle 6 (1992)
PSYC 6130, PROF. J. ELDER
15
Example
Descriptives
During 12 months-Number of contacts: Psychologist
N
MARRIED
WIDOWED
SEPARATED OR DIVORCED
SINGLE
Total
PSYC 6130, PROF. J. ELDER
6601
1630
1012
2568
11811
16
Mean Std. Deviation
0.185
0.082
0.900
0.620
0.326
2.034
1.023
4.688
4.012
2.811
Reporting Results
• A one-way ANOVA demonstrates that frequency of
contact with clinical psychologists depends on marital
status. Widowed individuals had the least contact
(M=0.082). Married individuals (M=0.185) had
somewhat more contact. Single (M=0.620) and
separated or divorced (M=0.900) had substantially more
contact. F(3,11807)=33.3, MSE = 7.8, p<.001.
PSYC 6130, PROF. J. ELDER
17
Summary Table (SPSS)
ANOVA
During 12 months-Number of contacts: Psychologist
Between Groups
Within Groups
Total
Sum of
Squares
783.673
92531.091
93314.764
PSYC 6130, PROF. J. ELDER
df
3
11807
11810
Mean Square
261.224
7.837
18
F
33.332
Sig .
.000
Interpreting the F Ratio
F
estimate of treatment effect + between-group estimate of error variance
within-group estimate of error variance
PSYC 6130, PROF. J. ELDER
19
Effect Size and Proportion of Variance Accounted For
SSbet
Proportion of variance accounted for (sample):  
SStot
2
PSYC 6130, PROF. J. ELDER
20
(Approxiately) Unbiased Effect Size
Proportion of variance accounted for (population):  2 
PSYC 6130, PROF. J. ELDER
21
SSbet  (k  1)MSW
SStot  MSW
Reporting Results
• A one-way ANOVA demonstrates that frequency of
contact with clinical psychologists depends on marital
status. Widowed individuals had the least contact
(M=0.082). Married individuals (M=0.185) had
somewhat more contact. Single (M=0.620) and
separated or divorced (M=0.900) had substantially more
contact. F(3,11807)=33.3, p<.001. However, the size of
the effect was relatively small:  2
PSYC 6130, PROF. J. ELDER
22
0.008.
Planning a Study: ANOVA and Power
X
Estimating power for ANOVA:   n

 can be used to plan experiments, relating n, k and  (Appendix ncF)
  .05 :
PSYC 6130, PROF. J. ELDER
23
Example
• You are interested in whether there is a link between
PSYC 6130 final grades and the professor teaching the
section.
• Grades typically have a standard deviation of about 15%
• There are typically 3 sections, each with around 12
students.
• What is the probability you would pick up an effect if the
standard deviation of the mean grade is around 5%?
PSYC 6130, PROF. J. ELDER
24
Advantages of ANOVA
• Avoid inflation in error rate due to multiple comparisons
• Can detect an effect of the treatment even when no 2
groups are significantly different.
PSYC 6130, PROF. J. ELDER
25
6-Step Process for ANOVA
1. State the hypotheses
H0 : 1  2  ...  n
H A : i , j  [1,..., n] : i   j
2. Select the statistical test and significance level
3. Select the samples and collect the data
4. Find the region of rejection
5. Calculate the test statistic
6. Make the statistical decision
PSYC 6130, PROF. J. ELDER
26
Sums of Squares Approach
F
MSbet
MSW
MSbet 
MSW 
SSbet
, where SSbet   ni ( X i  XG )2
dfbet
SSW
, where SSW   (ni  1)si2
dfw
NB :
SStotal  SSbet  SSW
MStotal  MSbet  MSW
PSYC 6130, PROF. J. ELDER
27
ANOVA Assumptions
• Independent random sampling
• Normal distributions
• Homogeneity of variance
PSYC 6130, PROF. J. ELDER
28
More on Homogeneity of Variance
s12
k  2: F (df1, df2 )  2
s2
Where s1  larger of the 2 std devs
k  2:
Hartley's Fmax
2
smax
 2
smin
Problem: sensitive to deviations from normality.
Levene's test:
Test of Homogeneity of Variances
More robust
During 12 months-Number of contacts: Psychologist
Levene
Statistic
115.537
Used by SPSS
PSYC 6130, PROF. J. ELDER
29
df1
3
df2
11807
Sig .
.000
Levene’s Test: Basic Idea
1. Replace each score X1i , X2i ,... with its absolute deviation from the sample mean:
d1i | X1i  X1 |
d2i | X 2i  X 2 |
2. Now run an analysis of variance on d1i ,d2i ,... :
F
MSbet
MSW
SSbet
, where SSbet   ni (di  dG )2
dfbet
SSW
MSW 
, where SSW   (ni  1)sdi2
dfw
MSbet 
SPSS reports an F-statistic for Levene’s test
• Allows the homogeneity of variance for two or more variables to be tested.
PSYC 6130, PROF. J. ELDER
30
What to do if Homogeneity of Variance Assumption
is Rejected
• Some adjustment procedures are available in SPSS
(e.g., Welch 1951).
• We will not cover the theory behind these adjustments.
PSYC 6130, PROF. J. ELDER
31
Fixed vs Random Effects
• Fixed Effects: interested only in the specified levels of
the independent variable
(e.g., single/married/divorced/widowed)
• Random Effects: interested in a large number of
possible levels of the independent variable – randomly
sampling only a few of these.
e.g.,
– Does the order of questions on a questionnaire effect the
results?
– Does the order of stimuli in a psychophysical experiment effect
the results?
PSYC 6130, PROF. J. ELDER
32
Fixed vs Random Effects
• One-Way Independent ANOVA calculation is the same
for fixed and random effect designs.
• Power and effect size calculations differ.
• More complex ANOVA designs differ.
• We restrict our attention in this course to fixed effect
designs.
PSYC 6130, PROF. J. ELDER
33
Qualitative vs Quantitative Independent Variables
• In principle, ANOVA can be applied to either qualitative
or quantitative variables.
• If IV is quantitative and effect is roughly linear, usually
have more power using regression (only using up 2
degrees of freedom, instead of k).
• If effect is complex (e.g., non-monotonic):
– Use a higher-order regression model (e.g., quadratic)
– Use ANOVA (makes no smoothness assumptions)
PSYC 6130, PROF. J. ELDER
34