COURSE: JUST 3900
INTRODUCTORY STATISTICS
FOR CRIMINAL JUSTICE
Chapter 12:
Introduction to Analysis of Variance
Instructor:
Dr. John J. Kerbs, Associate Professor
Joint Ph.D. in Social Work and Sociology
The Logic and the Process of
Analysis of Variance


Chapter 12 presents the general logic and basic
formulas for the hypothesis testing procedure known
as analysis of variance (ANOVA).
The purpose of ANOVA is much the same as the t
tests presented in the preceding chapters: the goal is
to determine whether the mean differences that are
obtained for sample data are sufficiently large to
justify a conclusion that there are mean differences
between the populations from which the samples
were obtained.
The Logic & Process of ANOVA
The difference between ANOVA and the t tests
is that ANOVA can be used in situations where
there are two or more means being compared,
whereas the t tests are limited to situations
where only two means are involved.
 ANOVA is necessary to protect researchers
from excessive risk of a Type I error in
situations where a study is comparing more
than two population means.

The Logic & Process of ANOVA


These situations would require a series of several t
tests to evaluate all of the mean differences.
(Remember, a t test can compare only two means at
a time.)
Although each t test can be done with a specific αlevel (risk of Type I error), the α-levels accumulate
over a series of tests so that the final experimentwise α-level can be quite large.
 Note: While experiment-wise Type I error does
accumulate, it is not a simple additive process.
The Logic & Process of ANOVA
ANOVA allows researcher to evaluate all of the
mean differences in a single hypothesis test
using a single α-level and, thereby, keeps the
risk of a Type I error under control no matter
how many different means are being
compared.
 Although ANOVA can be used in a variety of
different research situations, this chapter
presents only independent-measures designs
involving only one independent variable.

Typical Research Design for
Analysis of Variance
ANOVA TERMS
In ANOVA, the variable (independent or quasiindependent) that designates the groups being
compared is called a factor.
 In ANOVA, the individual groups or treatment
conditions that are used to make up a factor
are called levels of the factor.


Example: A study that looks at three different
telephone conditions would have three levels of the
factor.
Hypotheses for ANOVA

There are multiple means involved and so the
hypotheses can read as follows:

H0: µ1 = µ2 = µ3
H1: There is at least one mean difference among
the populations - - or
 H1: µ1 ≠ µ2 ≠ µ3 (All three means are different) - - or
 H1: µ1 = µ3 , but µ2 is different - - or
 H1: µ1 = µ2 , but µ3 is different - - or
 H1: µ2 = µ3 , but µ1 is different .

The Logic & Process of ANOVA:
Understanding the F-Ratio
The test statistic for ANOVA is an F-ratio,
which is a ratio of two sample variances. In the
context of ANOVA, the sample variances are
called mean squares, or MS values.
 The top of the F-ratio, MSbetween, measures the
size of mean differences between samples.
The bottom of the ratio, MSwithin, measures the
magnitude of differences that would be
expected without any treatment effects.

The Logic & Process of ANOVA:
Understanding the F-Ratio

Thus, the F-ratio has the same basic structure
as the independent-measures t statistic
presented in Chapter 10.

F=
𝑂𝑏𝑡𝑎𝑖𝑛𝑒𝑑 𝑚𝑒𝑎𝑛 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 (𝑖𝑛𝑐𝑙𝑢𝑑𝑒 𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑒𝑓𝑓𝑒𝑐𝑡𝑠)
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒𝑠 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑏𝑦 𝑐ℎ𝑎𝑛𝑐𝑒 (𝑤𝑖𝑡ℎ𝑜𝑢𝑡 𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑒𝑓𝑓𝑒𝑐𝑡𝑠)

F=
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐵𝑒𝑡𝑤𝑒𝑒𝑛 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑠
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑊𝑖𝑡ℎ𝑖𝑛 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑠

F=
𝑆𝑦𝑠𝑡𝑒𝑚𝑎𝑡𝑖𝑐 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠+𝑅𝑎𝑛𝑑𝑜𝑚,𝑈𝑛𝑠𝑦𝑠𝑡𝑒𝑚𝑎𝑡𝑖𝑐 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠
𝑅𝑎𝑛𝑑𝑜𝑚,𝑈𝑛𝑠𝑦𝑠𝑡𝑒𝑚𝑎𝑡𝑖𝑐 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠

F=

=
=
𝑀𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒𝑠 𝐼𝑛𝑐𝑙𝑢𝑑𝑖𝑛𝑔 𝐴𝑛𝑦 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠
𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒𝑠 𝑤𝑖𝑡ℎ 𝑁𝑜 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠
0 +𝑅𝑎𝑛𝑑𝑜𝑚,𝑈𝑛𝑠𝑦𝑠𝑡𝑒𝑚𝑎𝑡𝑖𝑐 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠
𝑅𝑎𝑛𝑑𝑜𝑚,𝑈𝑛𝑠𝑦𝑠𝑡𝑒𝑚𝑎𝑡𝑖𝑐 𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝐸𝑓𝑓𝑒𝑐𝑡𝑠
= 1.00
NOTE: The denominator
for an F-Ratio is called
an Error Term (Variance
Caused By Random
Differences)
NOTE: Zero Treatment Effects Gives an F-Ratio of 1.00 in This Case.
The Logic & Process of ANOVA:
Understanding Total Variability
The Logic & Process of ANOVA:
Understanding F-Ratio Values
A large value for the F-ratio indicates that the
obtained sample mean differences are greater
than would be expected if the treatments had
no effect.
 Each of the sample variances, MS values, in
the F-ratio is computed using the basic formula
for sample variance:
SS
2
sample variance = S = ──
df

The Logic & Process of ANOVA:
Within & Between Treatment
Variability

To obtain the SS and df values, you must go
through an analysis that separates the total
variability for the entire set of data into two
basic components:
within-treatment variability (which will be the
denominator) and
 between-treatment variability (which will become
the numerator of the F-ratio).

The Logic & Process of ANOVA:
Within Treatment Variability

The two components of the F-ratio. The first component is
Within-Treatment Variability:
 Within-Treatments Variability: MSwithin measures the size
of the differences that exist inside each of the samples.
 Because all the individuals in a sample receive exactly the
same treatment, any differences (or variance) within a
sample cannot be caused by different treatments.
 Thus, these differences are caused by only one source:
 Chance or Error: The unpredictable differences that exist
between individual scores are not caused by any
systematic factors and are simply considered to be
random chance or error.
The Logic & Process of ANOVA:
Between Treatment Variability

The two components of the F-ratio. The second
component is Between-Treatment Variability:
 Between-Treatments Variability: MSbetween
measures the size of the differences between the
sample means. For example, suppose that three
treatments, each with a sample of n = 5 subjects,
have means of M1 = 1, M2 = 2, and M3 = 3.
 Notice that the three means are different; that is,
they are variable.
The Logic & Process of ANOVA
By computing the variance for the three means
we can measure the size of the differences.
 Although it is possible to compute a variance
for the set of sample means, it usually is easier
to use the total, T, for each sample instead of
the mean, and compute variance for the set of
T values.

The Logic & Process of ANOVA:
Two Sources of Variance

Logically, the differences (or variance) between
means can be caused by two sources:

Treatment Effects: If the treatments have different effects,
this could cause the mean for one treatment to be higher
(or lower) than the mean for another treatment.

Chance or Sampling Error: If there is no treatment effect
at all, you would still expect some differences between
samples. Mean differences from one sample to another
are an example of random, unsystematic sampling error.
The Logic & Process of ANOVA:
Back to the F-Ratio

Considering these sources of variability,
the structure of the F-ratio becomes:
treatment effect + random differences
F = ──────────────────────
random differences
The Logic & Process of ANOVA:
F-Ratio Values
When the null hypothesis is true and there are
no differences between treatments, the F-ratio
is balanced.
 That is, when the "treatment effect" is zero, the
top and bottom of the F-ratio are measuring the
same variance.
 In this case, you should expect an F-ratio near
1.00. When the sample data produce an Fratio near 1.00, we will conclude that there is
no significant treatment effect.

The Logic & Process of ANOVA:
F-Ratio Values


On the other hand, a large treatment effect will
produce a large value for the F-ratio. Thus, when the
sample data produce a large F-ratio we will reject the
null hypothesis and conclude that there are significant
differences between treatments.
To determine whether an F-ratio is large enough to be
significant, you must select an α-level, find the df
values for the numerator and denominator of the Fratio, and consult the F-distribution table to find the
critical value.
The Logic & Process of ANOVA:
Structure & Sequence of ANOVA
Calculations
Analysis of Variance & Post Tests
The null hypothesis for ANOVA states that for
the general population there are no mean
differences among the treatments being
compared; H0: μ1 = μ2 = μ3 = . . .
 When the null hypothesis is rejected, the
conclusion is that there are significant mean
differences.
 However, the ANOVA simply establishes that
differences exist, it does not indicate exactly
which treatments are different.

ANOVA Calculations & Notations:
Learn Your Terms!!!



k is used to identify the number of treatment conditions
n is used to identify the number of scores in each treatment
condition
N is used to identify the total number scores in the entire study



T stands for treatment total and is calculated by ∑X, which
equals the sum of the scores for each treatment condition
G stands for the sum of all scores in a study (Grand Total)


N = kn, when samples are the same size
Calculate by adding up all N scores or by adding treatment total (G=∑T)
You will also need SS and M for each sample, and ∑X2 for the
entire set of all scores.
ANOVA Calculations:
Step 1 (Analysis of Sum of Squares)
𝐺2
𝑁

SStotal = ∑X 2 -

SSwithin treatments = ∑SS inside each treatment = SS1+SS2+SS3

SSbetween treatments = SStotal - SSwithin treatments

ALTERNATIVE FORMULA FOR SSbetween

SSbetween treatments =

𝑇2
∑
𝑛
-
𝐺2
𝑁
Use formula to check math
Note that we use treatment totals instead of treatment means here
ANOVA Calculations:
Step 2 (Analysis of DF)

Calculate Total Degrees of Freedom (df total)


Calculate Within-Treatment Degrees of Freedom
(dfwithin)



df within = ∑(n-1) = ∑df in each treatment or
df within = N – k
Calculate Between-Treatments Degrees of Freedom
(dfbetween)


df total = N – 1
df between = k – 1
Check to see if df total = df within + df between
ANOVA Calculations:
Step 3 (Calculation of Variances)


Again, in ANOVA, the term Mean Square or MS is
used in the place of the term “variance.”
Calculate MS between Treatments


MS between =
MS within =
s2
s2
between
within
=
=
𝑆𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑑𝑓𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑆𝑆𝑤𝑖𝑡ℎ𝑖𝑛
𝑑𝑓𝑤𝑖𝑡ℎ𝑖𝑛
ANOVA Calculations:
Step 4 (Calculate F-Ratio)
s2between

F = ------------- =
𝑀𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛
s2within

To determine if this is a significant F-ratio, determine if it is
larger than the critical value, which can be identified in Table
B4 (F-Distribution) on p. 705 of your textbook.



Look up critical value for critical region based upon degrees of freedom
in the numerator (df between) and the degrees of freedom in the
denominator (df within)
If F ≤ Fcrit, then fail to reject H0
If F > Fcrit, then reject H0
Measuring Effect Size for an
Analysis of Variance



As with other hypothesis tests, an ANOVA evaluates
the significance of the sample mean differences; that
is, are the differences bigger than would be
reasonable to expect just by chance.
With large samples, however, it is possible for
relatively small mean differences to be statistically
significant.
Thus, the hypothesis test does not necessarily
provide information about the actual size of the mean
differences.
Measuring Effect Size for an
Analysis of Variance (cont'd.)
To supplement the hypothesis test, it is
recommended that you calculate a measure of
effect size.
 For an analysis of variance the common
technique for measuring effect size is to
compute the percentage of variance that is
accounted for by the treatment effects.

Measuring Effect Size for an
Analysis of Variance (cont'd.)
For the t statistics, this percentage was
identified as r2, but in the context of ANOVA
the percentage is identified as η2 (the Greek
letter eta, squared).
 The formula for computing effect size is:

SSbetween treatments
η2 = ───────────
SStotal
Post Hoc Tests

With more than two treatments, this creates a
problem. Specifically, you must follow the
ANOVA with additional tests, called post hoc
tests, to determine exactly which treatments
are different and which are not.
The Tukey’s HSD and Scheffé test are examples
of post hoc tests.
 These tests are done after an ANOVA where H0 is
rejected with more than two treatment conditions.
The tests compare the treatments, two at a time, to
test the significance of the mean differences.

Post Hoc Tests

Tukey’s HSD (Honestly Significant Difference)
Compares any two treatment conditions to
determine if there is a significance difference
 Formula:

 HSD
 For
=q
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛
𝑛
values of q, please see Table B 5 on page 708 of
your textbook.
 To locate the critical q, you need to know the number of
treatments in the overall experiment (k), the df for
MSwithin, and the alpha level (typical to use the same as
the one selected for your ANOVA)
Post Hoc Tests

Scheffe Test
Uses extremely cautious approach to reducing
Type I Error
 One of the safest possible post-hoc tests because
it has one of the smallest risks of Type I Error
 Uses an F-Ratio for two treatments

 Numerator
= MS between using two treatments
 Denominator = MS within for overall ANOVA
Post Hoc Tests

Scheffe Test

Formulas
𝑇2
∑
𝑛
𝐺2
𝑁

SSbetween treatments =

dfbetween = k - 1 (k defined by overall ANOVA study)


MS between =
FA versus B =
 Note:
-
(compute for 2 groups)
𝑆𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑑𝑓𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑀𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛
𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛
Error term here (MSwithin) is from overall ANOVA
 Note: Critical F values found in Table B4 (p.705) with
dfbetween and dfwithin for overall ANOVA
Post Hoc Tests

Assumptions - Independent Measures ANOVA



1. The observations within each sample must be independent (see
page 254)
2. The populations from which the samples are selected must be
normal
3. The populations from which the samples are selected must have
equal variances (homogeneity of variances)
 Do not be too concerned about assumptions concerning normality
when you study large sample, but do be concerned if there is
reason to believe that the assumption has been violated.
 If you suspect that the assumption of homogeneity of variances is
violated, please use Hartley’s F-max test as discussed in Ch. 10.
Post Hoc Tests

Assumptions - Independent Measures ANOVA

If you suspect that one or more of the 3 key
assumptions for ANOVA have been violated or you
have large sample variance that prevents you from
finding significant results:
 Transform
original scores to ranks, which leaves you
with ordinal data
 Use the Kruskal-Wallis Test (see Appendix E for more
details)