Psy B07
ANALYSIS OF VARIANCE
Chapter 1
Slide 1
Psy B07
t-test refresher
 In chapter 7 we talked about analyses that
could be conducted to test whether pairs of
means were significantly different.
 For example, consider an experiment in which
we are testing whether using caffeine
improves final marks on an exam. We might
have two groups, one group (say 12 subjects)
who is given normal coffee while they study,
another group (say also 12 subjects) who is
given the same amount of decaffeinated
coffee.
Chapter 1
Slide 2
Psy B07
t-test refresher
Sub
We could now
look at the
exam marks for
those students
and compare
the means of
the two groups
using a
“betweensubjects” (or
independent
samples) t-test:
Chapter 1
Caf (X)
Decaf(Y)
X2
Y2
4624
5476
3481
3721
4225
5184
6400
3364
4225
3600
6084
5625
1
2
3
4
5
6
7
8
9
10
11
12
72
65
68
83
79
92
69
74
78
83
88
71
68
74
59
61
65
72
80
58
65
60
78
75
5184
4225
4624
6889
6241
8464
4761
5476
6084
6889
7744
5041
=
922
815
71622 56009
Slide 3
Psy B07
t-test refresher
t
x1  x2
2
1
2
2
s
s

N1 N 2
 3.30
Chapter 1
Slide 4
Psy B07
t-test refresher
The critical point of the previous example is the
following:
The basic logic for testing whether or not two
means are different is to compare the size of
the differences between the groups (which we
assume is due to caffeine), relative to the
differences within the groups (which we
assume is due to random variation .. or error).
Chapter 1
Slide 5
Psy B07
t-test refresher
measure of effect (or treatment)
assessed by examining variance
(or difference) between the groups
measure of random variation (or error)
assessed by examining variance
within the groups
This exact logic underlies virtually all statistical tests,
including analysis of variance, an analysis that allows us
to compare multiple means simultaneously.
Chapter 1
Slide 6
Psy B07
Analysis of Variance (ANOVA) –
the why?
The purpose of analysis of variance is to let us ask whether
means are different when we have more than just two
means (or, said another way, when our variable has
more than two levels).
 In the caffeine study for example, we were
interested in only one variable (caffeine)
and we examined two levels of that
variable, no caffeine versus some caffeine.
 Alternately, we might want to test different
dosages of caffeine where each dosage
would now be considered a “level” of
caffeine
Chapter 1
Slide 7
Psy B07
Analysis of Variance (ANOVA) –
the why?
 As you’ll see in PsyC08, as you learn about
more complicated ANOVAs (and the
experimental designs associated with them)
we may even be interested in multiple
variables, each of which may have more than
two levels.
 For example, we might want to
simultaneously consider the effect of
caffeine (perhaps several different dose
levels) and gender (generally just two
levels) on test performance.
Chapter 1
Slide 8
Psy B07
Analysis of Variance (ANOVA) –
the what?
The critical question is, is the variance
between the groups significantly bigger
than the variance within the groups to
allow us to conclude that the between
group differences are more than just
random variation?
Chapter 1
Slide 9
Psy B07
Analysis of Variance (ANOVA) –
the what?
Score
on
Exam
No Caffeine
Chapter 1
Moderate Dose
Heavy Dose
Slide 10
Psy B07
Analysis of Variance (ANOVA) –
the what?
Score
on
Exam
No Caffeine
Chapter 1
Moderate Dose
Heavy Dose
Slide 11
Psy B07
Analysis of Variance (ANOVA) –
the what?
Score
on
Exam
No Caffeine
Chapter 1
Moderate Dose
Heavy Dose
Slide 12
Psy B07
Analysis of Variance (ANOVA) –
the how?
The textbook presents the logic in a more
verbal/statistical manner, and it can’t
hurt to think of this in as manner
different ways as possible, so, in that
style:
Let’s say we were interested in testing
three doses of caffeine; none, moderate
and high.
Chapter 1
Slide 13
Psy B07
Analysis of Variance (ANOVA) –
the how?
First of all, use of analysis of variance assumes
that these groups have (1) data that is
approximately normally distributed, (2)
approximately equal variances, and (3) that
the observations that make up each group are
independent.
Given the first two assumptions, only the means
can be different across the groups - thus, if the
variable we are interested in is having an
affect on performance, we assume it will do so
by affecting the mean performance level.
Chapter 1
Slide 14
Psy B07
Analysis of Variance (ANOVA) –
the how?
Chapter 1
Sub
High
Moderate
None
1
2
3
4
5
6
7
8
9
10
11
12
72
65
68
83
79
92
69
74
78
83
88
71
68
80
64
65
69
79
80
63
69
70
83
75
68
74
59
61
65
72
80
58
65
60
78
75
=
922
865
815
Slide 15
Psy B07
Analysis of Variance (ANOVA) –
the how?
Mean =
s2 =
s=
Chapter 1
78.83
71.06
8.43
72.08
48.99
7.00
67.92
59.72
7.73
Slide 16
Psy B07
Analysis of Variance (ANOVA) –
the how?
From this data, we can generate two estimates
of the population variance 2.
“Error” estimate (σ2e ): One estimate we can
generate makes no assumptions about the
veracity (trueness or falseness) of the null
hypothesis.
 Specifically, the variance within each
group provides an estimate of σ2e.
Chapter 1
Slide 17
Psy B07
Analysis of Variance (ANOVA) –
the how?
Given the assumption of equal variance (all of
which provide estimates of 2), our best
estimate of 2 would be the mean of the group
variances.
  s k
2
e
2
j
This estimate of the population variance is
sometimes called the mean squared error
(MSe) or the mean squared within (MSwithin).
Chapter 1
Slide 18
Psy B07
Analysis of Variance (ANOVA) –
the how?
Treatment estimate (σ2t ): Alternatively, if we assume
the null hypothesis is true (i.e., that there is no
difference between the groups), then another way
to estimate the population variance is to use the
variance of the means across the groups.
By the central limit theorem, the variance of our
sample means equals the population variance
divided by n, where n equals the number of
subjects in each group.
2
2

x

Chapter 1
n
s
Slide 19
Psy B07
Analysis of Variance (ANOVA) –
the how?
Therefore, employing some algebra:
   ns
2
2
x
This is also called the mean squared treatment
(MStreat) or mean squared between (MSbetween).
Chapter 1
Slide 20
Psy B07
Analysis of Variance (ANOVA) –
the how?
OK, so if the null hypothesis really is true and
there is no difference between the groups,
then these two estimates will be the same:
2
2
=
 e
However, if the treatment is having an effect,
this will inflate σ2τ as it will not only reflect
variance due to random variation, but also
variance due to the treatment (or variable).
Chapter 1
Slide 21
Psy B07
Analysis of Variance (ANOVA) –
the how?
The treatment will not affect σ2e , therefore, by
comparing these two estimates of the
population variance, we can assess whether
the treatment is having an effect:
 2 MStreat MSbetween
=
=
 2e MSerror MSwithin
Measure of Chance Variance + Treatment Effect
Measure of Chance Variance Only
Chapter 1
Slide 22
Psy B07
Analysis of Variance (ANOVA) –
the how?
1) Calculate a SSerror, SStreat, and SStotal.
2) Calculate a dferror, dftreat and dftotal
3) By dividing each SS by its relevant df, we
then arrive at MSerror and MStreat (and
MStotal).
4) Then we divide MStreat by MSerror to get our
F-ratio, which we then use for hypothesis
testing.
Chapter 1
Slide 23
Psy B07
Sums of Squares
The sum of squares is simply a measure of the
sum of the squared deviations of observations
from some mean:
(x  x)
2
OK, so rather than directly calculating the MSerror
and MStreat (which are actually estimates of the
variance within and between groups), we can
calculate SSerror and SStreat.
Chapter 1
Slide 24
Psy B07
ANOVA
Sub
High
1
72
2
65
3
68
4
83
5
79
6
92
7
69
8
74
9
78
10
83
11
88
12
71
X =
922
2
X = 71622
Mean = 76.83
Chapter 1
Moderate
68
80
64
65
69
79
80
63
69
70
83
75
865
62891
72.08
None
68
74
59
61
65
72
80
58
65
60
78
75
815
56009
67.92
Slide 25
Psy B07
SSerror
 To calculate SSerror, we subtract the
mean of each condition from each score,
square the differences, and add them
up, and then add up all the sums of
squares
SSerror  ( x ij  x j )
Chapter 1
2
Slide 26
Psy B07
SSerror
There is a different way of doing this. First,
calculate ΣX2 for each group
For example, for Group 1, the X2 would equal
(722+652+….+882+712) = 71622.
Once we have them, we then calculate the sum
of squares for each group using the
computational formula:
(X)
SSj  X 
n
2
2
Chapter 1
Slide 27
Psy B07
SSerror
For example, for Group 1, the math would be:
(922 )
SS1  71622 
12
 781 .67
2
To get SSerror we then sum all the SSerrors.
SSerror = SS1+SS2+SS3 = 781.67+538.92+656.92
= 1977.50
Chapter 1
Slide 28
Psy B07
SStreat
 To calculate SStreat we subtract the
grand mean from each group mean,
square the differences, sum them up,
and multiply by n.
SStreat  n( x j  x..)
2
Chapter 1
Slide 29
Psy B07
SStreat
 Again, there is a different way of doing
this. Basically, all we need are our three
means and the squares of those means.
We then calculate the sum of the means,
and the sum of the squared means:
x  76.83  72.08  67.92  216 .83
x 2  5903 .36  5196 .01  4612 .67
 15712 .04
Chapter 1
Slide 30
Psy B07
SStreat
Now we can calculate the SS using a formula similar to the
2
one before:
(x )
SStreat  x 
k
(216 .83) 2
 15712 .04 
3
 39.81
2
Once again, because we are dealing with means and not
observations, we need to multiply this number by the n
that went into each mean to get the real SStreat
SStreat = 12(39.81) = 477.72
Chapter 1
Slide 31
Psy B07
SStotal
 The sum of squares total is simply the
sum of squares of all of the data points,
ignoring the fact that there are separate
groups at all.
 To calculate it, subtract the grand mean
from every score, square the
differences, and add them up
SStotal  ( x ij  x..)
2
Chapter 1
Slide 32
Psy B07
SStotal
 Surprise, surprise – there is another way of
calculating this as well
 Here you will need the sum of all the data
points, and the sum of all the data points
squared.
An easy way to get this is to just add up the X
and the X2 for the groups:
X = X1+X2+X3 = 922+865+815 = 2602
X2 = X21+X22+X23 = 71622+62891+56009 = 190522
Chapter 1
Slide 33
Psy B07
SStotal
Then, again using a version of the old SS formula:
2
(

x
)
SStotal  x 2 
N
(2602 ) 2
 190522 
36
 2455 .22
If all is right in the world, then SStotal should equal
SSwithin+SStreat. For us, it does.
Chapter 1
Slide 34
Psy B07
df
OK, so now we have our three sum of squares,
step two is to figure the appropriate degrees
of freedom for each.
Here’s the formulae:
dferror=k(n-1)
dftreat=k-1
dftotal=N-1
where k = the number of groups, n = the number
of subjects within each group, and N = the
total number of subjects.
Chapter 1
Slide 35
Psy B07
From SS to MS to F
MS estimates for treatment and within
are calculated by dividing the
appropriate sum of squares by its
associated degrees of freedom.
We then compute an F-ratio by dividing
the MStreat by the MSerror.
Finally, we place all these values in a
Source Table that clearly shows all the
steps leading up to the final F value.
Chapter 1
Slide 36
Psy B07
ANOVA source table
The source table for our data would look
like this:
Source
Treatment
Error
Total
SS
477.72
1977.50
2455.22
df
2
33
35
MS
238.86
59.92
F
3.99
OK, now what?
Chapter 1
Slide 37
Psy B07
Hypothesis Testing
Now we are finally ready to get back to the
notion of hypothesis testing. . .that is, we are
not ready to answer the following question:
If there is really no effect of caffeine on
performance, what is the probability of
observing an F-ratio as large as 3.99.
More specifically, is that probability less that our
chosen level of alpha (e.g., .05).
Chapter 1
Slide 38
Psy B07
Sampling distribution of F
How do we arrive at the probability of observing
some specific F value?
Recall our example when we created 3 groups by
randomly sampling individuals from the same
population and asking them for some piece of
data (e.g. age).
In this case, the null hypothesis should be true …
the means of the three groups should only vary
as a result of chance (or error) variation
Chapter 1
Slide 39
Psy B07
Sampling distribution of F
If we perform an analysis of variance on this
data, the F value should be about 1. However,
it will not be exactly 1; rather, there will be a
distribution with a mean of 1 and some
variance around that mean.
This distribution is termed the F distribution, and
its exact shape varies as a function of dftreat
and dferror.
The important point here is that for any given
degrees of freedom, the function can be
mathematically specified, allowing one to
perform calculus and, therefore, to find the
probabilities of certain values.
Chapter 1
Slide 40
Psy B07
Hypothesis Testing
All we really want to know is whether the F we
have obtained in our analysis is significantly
larger than we would expect by chance.
That is, we want to know whether it falls within
the extreme “high” 5% of the chance
distribution.
Thus, all we really need to know is the critical F
value that “cuts off” the extreme 5% of the
distribution.
If our obtained F is larger than the critical F, we
know it is in the “rejection region” and,
therefore, that the probability of obtaining an
F that large is less than 5%.
Chapter 1
Slide 41
Psy B07
Finishing the example
From the table, Fcrit(2,33) = 3.32
Since Fobt (3.99) > Fcrit (3.32) we
reject the null hypothesis
Mean
=1
Chapter 1
Fcrit
= 3.32
Slide 42
Psy B07
Finishing the example
 One thing to keep in mind – all an
ANOVA (significant) tells you is that
there is a difference between the
means. You can’t tell where exactly this
difference lies just yet. That’s in
chapter 12 – and PsyC08
Chapter 1
Slide 43
Psy B07
Violation of Assumptions
The textbook discusses this issue in detail and offers a
couple of solutions (including some really nasty
formulae) for what to do when the variances of the
groups are not homogeneous.
What I want you to know is the following:
1) If the biggest variance is more than 4 times
larger than the smallest variance, you may
have a problem.
2) There are things that you can do to
calculate an F if the variances are
heterogeneous.
Chapter 1
Slide 44
Psy B07
The Structural Model
 Let’s assume that the average height of all
people is 5’7”. Let’s also assume that males
tend to be 2” taller than females, on average.
 Given this, I can describe anyone’s height
using three components: 1) the mean height of
all people, 2) the component due to sex, and
3) individual contributions
 My height is about 6’0”. I can break this down
into: 5’7”+2”+3”
Chapter 1
Slide 45
Psy B07
The Structural Model
 In more general terms, we can write the
model out like this:
x 
Chapter 1
Slide 46
Psy B07
X ij   ..  t . j   ij
 GM  ( X. j  GM)  (X ij  X. j )
 GM  GM  X. j  X. j  X ij
 X ij
( X ij  GM )  ( X . j  GM )  ( X ij X ij )
 X . j  X . j  X ij  GM
 X ijGM
( X ij  GM )  ( X jGM )  ( X ij  X j )
( X ij  GM ) 2  ( X j  GM ) 2  2( X j  GM )( X ij  X ij )  ( X ij  X j ) 2
SS total   ( X ij  GM ) 2   ( X j  GM ) 2   ( X ij  X j ) 2
j
i
i
j
j
i
  ( X j  GM ) 2   ( X ij  X j ) 2
j
i
j
i
 n ( X j  GM ) 2   ( X ij  X j ) 2
j
SS total  SS treatment
Chapter 1
j
i
 SS error
Slide 47