Chapter 11: Multiple
Comparisons & Analysis
of Variance
One population, Two
population, ...
• Chapter 9: Inference (confidence intervals, hypothesis
testing) for mean for one group/one population
• Chapter 9: Inference (confidence intervals, hypothesis
testing) to compare the means of two groups/two
populations
• To review... briefly look at a few of those one and twomean inference procedures/situations
Ho:
μ=1
Ha:
μ>1
where μ = mean heat conductivity transmitted per square
meter of surface per degree Celsius difference on the
two sides of the glass
Is there evidence that the conductivity of this type of
glass is greater than 1? Carry out an appropriate test.
Does logging significantly change the mean number of species in a
plot after 8 years? Give appropriate statistical evidence to
support your conclusion. Assume both populations are
Normally distributed.
We want to test
Ho: μU = μL
OR
μU – μL = 0
H a : μU ≠ μL
OR
μU – μL ≠ 0
where μU & μL are the mean number of species in unlogged and
logged plots, respectfully
• Is there good evidence that red wine drinkers’ mean polyphenol
levels were different from white wine drinkers’ mean polyphenol
levels? Assume both populations are approximately Normal.
• We want to test:
Ho:
μR = μW
or
μR – μW = 0
Ha:
μR ≠ μW
or
μR – μW ≠ 0
where μR & μW are the mean percent change in polyphenols for men
who drink red and white wine, respectfully.
Nothing magical about
the numbers one or two...
• Sometimes there is a need to compare three, four, five,
or more groups with each other.
• ANOVA (Analysis of Variance) is a method for doing
that; tests whether there is an association between a
categorical variable that identifies different groups, and
a numerical variable.
• The phrase “Analysis of Variance” can be misleading;
the procedure really looks at means/compares means.
ANOVA example...
• We may want to know which diet (Weight Watchers,
Jenny Craig, Atkins, Slim-Fast, etc.), the categorical
variable, is best for losing particular amounts of weight
(3 pounds, 10 pounds, 25 pounds), the numerical
variable.
• ANOVA accurately performs multiple comparisons
• If we chose to conduct several individual comparisons
(several two-sample/two-population) procedures (twosample t procedure, confidence interval, hypothesis
test), life gets very messy (more on this in a moment)
How does anova
work?
• Basic idea: Accurately calculates variation within a
given group as well as between several groups
• This leads to a test statistic (like a t score/value) that
accurately compares several different groups without
the problem of multiple comparisons.
• Like all procedures we have discussed, ANOVA works
best if certain conditions are met; more on this later...
...popcorn & kernels popped &
problems with multiple comparison ...
In chapter 9, we would have...
Ho: mean no oil = mean medium oil
Ha: mean no oil ≠ mean medium oil
Ho: mean no oil = mean maximum oil
Ha: mean no oil ≠ mean maximum oil
Ho: mean medium oil = mean maximum oil
Ha: mean medium oil ≠ mean maximum oil
Three different hypothesis tests... This is called multiple comparison... Comparing
multiple pairs of means
Remember α...
• Rejection zone (when conducting an hypothesis test);
significance level; usually 5% (0.05)
• α is also the probability of committing a type I error
(rejecting the null hypothesis when it really is true)
• Basic problem with multiple comparisons is that even
though the probability of something going wrong
(making an incorrect decision; committing an error) on
one occasion is small (5%), if we keep repeating the
experiment, eventually something will go wrong.
Big chances to make
big mistakes...
• Essentially, by doing multiple tests, we are creating
more opportunities to mistakenly reject the null
hypothesis.
• The more tests we do, the greater the probability that
we will mistakenly reject the null hypothesis at least
one.
• For our three hypothesis tests, each with α= 0.05, the
overall significance level (or probability that we will
conclude that at least one amount of oil is more
effective than another, when the truth is that all
amounts are equally effective is 14%.
Big chances to make
big mistakes...
• 14% doesn’t seem too high... But we were shooting for
5%... And this is just with 3 procedures.
• DETOUR...by the way, how did I calculate that 14%?
• Remember binomial distributions? Let’s take a
moment and review how I calculated that 14%. Pull
up Minitab...
So, anova to the
rescue...
• ANOVA tests whether a categorical variable is
associated with a numerical variable. This is the same
as testing whether the mean value of a numerical
variable is different in different groups
• ANOVA looks at the variation within each group and
between all groups; then creates a ratio comparing
these numbers called the F-statistic
Variation.Between.Groups
• F = Variation.Within.Groups
ANOVA looks at variation
within & between
• Look at variation within each group
• Look at variation between all groups group
ANOVA looks at variation
within & between
• Look at variation within each group
• Look at variation between all groups group
ANOVA looks at variation
within & between
Look at variation within each group
Look at variation between all groups group
Like all other
procedures...
• We have conditions that must be checked and met
• Random Sample & Independent Measurements
• Independent Groups
• Same Variance
Let’s practice a few...
• Does the amount of oil used affect the number of kernels popped
when one is making popcorn? Researchers randomly assigned
bags with 50 unpopped kernels to be popped with no oil, a
medium amount of oil (1/2 tsp), or the maximum amount of oil
(1 tsp). Thirty-six bags were assigned to each group. After 75
seconds, the popped kernels were counted.
• Using significance level of 5%, fan ANOVA test was carried out.
• Hypothesize:
• Ho: μnone = μmedium = μmax
• Ha: The mean number of kernels popped differs by amount of oil
used.
Ho: μnone = μmedium = μmax
Ha: The mean number of kernels popped differs by amount of oil used.
• Does the amount of oil used affect the number of kernels
popped when one is making popcorn? Using significance
level of 5%, carry out an ANOVA test.
• Check Conditions:
• Random Sample & Independent Measurements
• Independent Groups
• Same Variance
Ho: μnone = μmedium = μmax
Ha: The mean number of kernels popped differs by amount of oil used.
• Does the amount of oil used affect the number of kernels
popped when one is making popcorn? Using significance
level of 5%, carry out an ANOVA test.
Ho: μnone = μmedium = μmax
Ha: The mean number of kernels popped differs by amount of oil used.
• Reject null hypothesis. With anαof 5% and a p-value of
about 2%, we reject the null hypothesis that the mean number
of kernels popped are all equal.
Now let’s use
minitab...
Test weather the mean number of tv’s differs
among these three schools. Use a 5% significance
level. Assume conditions have been checked and
met.
Ho: All populations are =
ha: the population means are ≠
• Minitab; enter data into three columns with labels for
each (OC, VA, MC)
• Go to ANOVA, one-way ANOVA, choose ‘responses
are in separate columns for each factor level’
• In responses box, insert your three columns of data
(OC, VA, MC); then OK
• F-Value = 0.48; p-value = 0.6205
• Interpretation: Fail to reject. With an α level of 5%
and a p-value over 62%, we do not have enough
evidence to show that the population means are ≠
ANOVA HW & Test
questions...
• No need to do the HW in this section
• Just be familiar with the general PP concepts
• Test questions on ANOVA? Minimal. Will be on
final, but not a main focus (probably just 1-2 simple
questions about ANOVA)