Analysis of Variance (ANOVA): The F-Test
t- tests are limited to only being able to compare 2 groups at a
time. Using multiple t-tests inflates the chance of a Type one error.
F-tests can compare 2, 3 or more groups in a complex experiment
while keeping Type I error probability at a specified level (e.g. .05)
The F-Ratio : ANOVA divides the variability (V) in an
experiment into 2 parts
Total Variability
Between Groups Variance
(BGV)
1. Treatment Effect
2. Individual Differences
3. Experimental Error
Within Group Variance
(WGV)
1. No Treatment effect
2. Individual Differences
3. Experimental Error
The value of F comes from the F-ratio, which divides the BGV by
the WGV:
F
BGV
WGV
OR
F
MeanSquareBG
MeanSquareWG
Note: The treatment effect (if any) will show in the numerator (the
BGV or Mean SquareBG)
 Therefore, if there is no treatment effect it should be the case
that F is approximately 1.0.
 If there is a significant effect, F should be > 1.
1
 Null Hypothesis for ANOVA:
Ho: 1=2=3…k
k refers to the number
of groups or
experimental conditions
 If Ho is rejected (treatment effect present) F should be > 1.0
 Alternate or working hypothesis for ANOVA:
At least one population mean () is different from the
others
The F – Distribution
 The F – distribution is positively skewed, therefore the F –
test is always a one tailed test
2
Solving for F – Ratio (ANOVA)
New Symbols:
k: the number of levels or groups in the experiment
N: total number of subjects in the experiment
n: number of subjects in each group, e.g. n1=5, n2=5, n3=5
T: X for each group, e.g. T1=47, T2=42, T3=58
G: X for the entire experiment (T  G)
Formula Summary Table for ANOVA
Source
Between
Groups
df
k-1
Within
Groups
Total
N-k
SS

MS
T 2 G2

n
N
 SS inside
each group
N-1
SS BG
df BG
SSW G
dfW G
F
p
MS BG
MS W G
From
table
G2
X  N
2
3
Example Problem:
Does medication reduce the incidence of manic episodes among
bipolar clients? To examine this question, 2 medications were
compared to a control group over a 5 - year period. Numbers
below are the number of manic episodes for patients in each group.
Prozac
11
10
9
9
8
Lithium
13
12
12
10
9
Placebo Control
15
12
17
10
12
 This is a one Factor or one-way ANOVA – Factor refers to
the number of Independent Variables
 This experiment has one factor (Drug) with 3 levels
 Had the experiment looked at the effects of these same drugs
on manic episodes for Males and Females separately, there
would be a second IV or factor namely gender. This would
then be a two-factor or two-way ANOVA
4
Conducting an ANOVA can be divided into 2 main parts:
 Conducting the ANOVA itself
 Conducting Post Hoc tests if the ANOVA is significant
ANOVA
Step 1: State the hypothesis and select the alpha level:
 Ho: 1=2=3 Medications have no effect on number of
manic episodes
 H1: At least one population mean () is different from the
others
Step 2: Identify the critical region
From the F Table:
F.05 (2,12) = 3.88
F.01 (2,12) = 6.93
5
Step 3: Solve for the test statistic, in this case F
3 a) compute the following values
Prozac
Lithium
T
X2
SS
n
M
47
447
5.20*
5
9.40
56
638
10.80*
5
11.20
G = 169
N = 15
Placebo
Control
66
902
30.80*
5
13.20
X2Total = 1987 k = 3
2
SS

X


*
( X ) 2
n
for each group
6
3b) Construct a blank summary table and complete the
calculations
Source
Between
Groups
Within
Groups
Total
df
SS
MS
F
p
Calculations for the summary table
3b-1)
df
dfBG = k-1 = 3-1 = 2
dfWG = N-k = 15-3 = 12
dfTotal = N-1 = 15-1 = 14
3b-2) SS
SSBG =

T 2 G2

n
N
T12 T22 T32 G 2




n1 n2 n3
N
47 2 56 2 66 2 169 2



=
5
5
5
15
7
=441.8 + 627.2 + 871.2 – 1904.07
 36.13
SSWG =
 SS inside each group
 5.2  10.8  30.8
 46.8
G2
SSTotal =  X 
N
2
169 2
 1987 
15
 1987  1904.07
 82.93
** Check SS BG  SSW G  SSTotal
36.13  46.8  82.93
8
SS BG
3b-3) MS BG  df
BG

36.13
2
 18.07
MSW G 

SSW G
df W G
46.8
12
 3.90
F
3b4)

MS BG
MS W G
18.07
3.90
 4.63
9
The completed summary table:
Source
Between
Groups
Within
Groups
Total
df
2
SS
36.13
12
46.8
14
82.93
MS
18.07
F
p
4.63
<.05
3.90
Step 4: Make a decision
Reject Ho: At least one of the groups has a significantly different
number of manic episodes
Step 5: If the ANOVA was not significant, make a final
statement in the conventional manner
e.g. The results of the study show that type of medication has
no significant effect on the number of manic episodes
However, this was not our finding. We found a significant
effect of Drug on number of Manic episodes. Therefore, we
need to conduct post hoc comparisons (also called paired or
pair wise comparisons) to determine which group(s) is/are
different from one another.
10
Post hoc comparisons between group means:
There are several types of post hoc comparisons – we will examine
a relatively conservative type known as the Scheffe F – test
This test guards against Type I errors by:
 Maintaining the same df as the overall F
 Using the same critical values to evaluate F – Scheffe
as the overall F
Post Hoc Comparisons: Step 1
The first step is to rank order the difference (largest to smallest)
between all possible group means
Greatest difference:
Mcontrol Vs MProzac
(13.2 – 9.4 = 3.8)
2nd greatest difference : Mcontrol Vs MLithium
(13.3 – 11.2 = 2.0)
3rd greatest difference MLithium Vs MProzac
(11.2 – 9.4 = 1.8)
11
Note:
 df remain the same as in the overall ANOVA (e.g. 2,12)
 MSWG remains the same as in the overall ANOVA (e.g. 3.90)
 A new MSBG is computed for each pairwise comparison
Post Hoc Comparisons: Step 2: Conduct the pair wise
comparisons in the order indicated by the ranking:
First pair wise comparison (Control Vs Prozac):
SSBG for Control Vs Prozac
SSBG =

T 2 G2

n
N
47 2 662 1132



5
5
10
Note: only 2 groups:
new G and new N
= 871.2 + 441.8 – 1276.9
= 36.1
Calculate a new MSBG for control Vs Prozac
SS BG
MSBG = df
BG
Retain same df as
overall ANOVA
12
36.1
 18.05
2
FScheffe =

MS BG
MS W G
New MSBG divided by
MSWG from overall
ANOVA
18.05
3.90
 4.63
Comparing 4.63 to the critical value: F.05(2,12) = 3.88
FScheffe (2,12) = 4.63, p<.05, reject Ho
Patients in the control group did not significantly differ in
manic episodes from patients in the Prozac group
13
Second pair wise comparison (Control Vs Lithium):
Because the first pair wise comparison was significant, we go on to
the second: Control Vs Lithium
SSBG for Control Vs Lithium
SSBG =

T 2 G2

n
N
662 562 1222



5
5
10
Note: again, only 2
groups: new G and new
N
= 871.2 + 627.2 – 1488.4
= 10.0
MSBG for Control Vs lithium
SS BG
MSBG = df
BG

Retain same df as
overall ANOVA
10.0
2
 5 .0
FScheffe =
MS BG
MSW G
New MSBG divided by
MSWG from overall
ANOVA
14

5.0
3.90
 1.28
Comparing 1.28 to the critical value: F.05(2,12) = 3.88
FScheffe (2,12) = 1.28, p>.05, fail to reject Ho
Patients in the control group did not significantly differ in
manic episodes from patients in the Lithium group
Note:
 Once the first non significant post hoc comparison is
revealed, the comparisons stop (otherwise we could start
inflating the possibility of Type 1 errors)
 We know that the comparison between Lithium and Prozac
will not be significant, because the difference between these
means is even smaller than with the 2nd comparison.
Step Final statement:
Overall, the results of the ANOVA indicated a significant effect of
medication on number of manic episodes F. (2,12) = 4.63, p<.05.
Scheffe post hoc comparisons revealed that Prozac reduced manic
episodes compared to the control FScheffe (2,12) = 4.63, p<.05. No
other comparisons were significant.
15