Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Lesson 11
Chapter 10: Comparing k(> 2) Populations
Michael Akritas
Department of Statistics
The Pennsylvania State University
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
1
Comparing k> 2 Means: The ANOVA F Test
2
Comparing k> 2 Propostions: The χ2 Test
3
The Kruskal-Wallis Test – NOT COVERED
4
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Statistical Model and Hypothesis
Xi1 , Xi2 , . . . , Xini , i = 1, . . . , k, are independent samples.
Xij = µi + ij , Var(ij ) = σi2 .
P
Write µi = µ + αi , where µ = k −1 ni=1 µi , and αi = µi − µ
is the effect of population i (of factor level i).
Of interest is the hypothesis of no effects (all αi = 0), or
H0 : µ1 = µ2 = · · · = µk vs Ha : H0 is false.
The sample mean and sample variance from population i
are: X i , Si2 , i = 1, . . . , k.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The basic idea
As in regression, variability is represented by the so-called
sums of squares, or SS, and the ANOVA approach
decomposes the total variability into components.
For the comparison of k means, one of the components is
called between groups variability and the other is called
within groups variability.
If the between groups variability is large compared to the
within groups variability, the hypothesis of equal means is
rejected.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Assume homoscedasticity, i.e. Var(ij ) = σ 2 , for all
i = 1, . . . , k and define:
SST =
ni
k X
X
(Xij − X ·· )2 Total Sum of Squares
i=1 j=1
SSE =
ni
k X
X
(Xij − X i· )2 Error Sum of Squares
i=1 j=1
SSTr =
k
X
ni (X i· − X ·· )2 Treatment Sum of Squares
i=1
It can be shown that SST = SSTr + SSE.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Each SS has its own DF (here N = n1 + · · · + nk ):
DFSST = N − 1, DFSSE = N − k , DFSSTr = k − 1
Dividing each SS by its DF we obtain the mean squares:
MSE =
SSE
SSTr
, MSTr =
.
N −k
k −1
MSE equals the k-sample version of the pooled variance:
MSE = Sp2 =
(n1 − 1)S12 + · · · + (nk − 1)Sk2
n1 + · · · + nk − k
Thus, MSE is an unbiased estimator of the common
intrinsic error variance σ 2 .
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The ANOVA table summarizes these calculations:
Source
df
SS
MS
Treatment
k −1
SSTr
MSTr=
SSTr
k −1
Error
N −k
SSE
MSE=
SSE
N −k
F
FH0 =
MSTr
MSE
Total
N − 1 SST
Under H0 , and the assumption of normality
FH0 ∼ Fk−1,N−k
Thus, H0 is rejected at level α if F > Fα,k−1,N−k .
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Example
The following data resulted from comparing the degree of
soiling in fabric treated with three different mixtures of
methacrylic acid.
Mix 1:
0.56
1.12
0.90
1.07
0.94
Mix 2:
0.72
0.69
0.87
0.78
0.91
Mix 3:
0.62
1.08
1.07
0.99
0.93
Test H0 : µ1 = µ2 = µ3 vs Ha : H0 is false, at α = 0.1.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Example (Continued)
Solution. Because of the small sample sizes, we need to
assume that the three populations are normal and
homoscedastic. The ANOVA table (obtained from R) is
Source
Treatment
Error
Total
df
k −1=2
N − k = 12
N − 1 = 14
SS
.0608
.3701
.4309
MS
.0304
.0308
F
.99
The rejection rule specifies that H0 be rejected if F > F0.1,2,12 .
Since F0.1,2,12 = 2.81, H0 is not rejected.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
One-way ANOVA in R
Read the data http://sites.stat.psu.edu/˜mga/
401/Data/FabricSoilingData.txt into the data
frame fs
anova(aov(Value ∼ Sample, data=fs))
anova(aov(fs$Value ∼ fs$Sample)) # also works the same
Can also use lm: anova(lm(Value ∼ Sample, data=fs))
Try also: plot(aov(Value ∼ Sample, data=fs))
NOTE: Must use only non-numeric sample indicators.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
F and χ2 Tests
·
If Y ∼ Fν1 ,ν2 , with ν2 ”large”, then ν1 Y ∼ χ2ν1 .
·
Thus, Q = (k − 1)FH0 ∼ χ2k−1 .
Another look at Q:
Pk
Q=
i=1 ni (X i·
Sp2
− X )2
This is the form used for H0 : p1 = · · · = pk :
Pk
Q=
b −p
b )2
b(1 − p
b)
p
i=1 ni (pi
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Contingency Table Form
The TS for comparing k (> 2) proportions is the statistic Q
given above.
However, an alternative form, called the contingency table
form, is more common:
k X
2
X
(O`i − E`i )2
Q=
, where
E`i
i=1 `=1
bi , O2i = ni 1 − p
bi , E1i = ni p
b, E2i = ni 1 − p
b .
O1i = ni p
·
If H0 : p1 = · · · = pk is true, Q ∼ χ2k−1 . Thus, H0 is rejected
at level α if
Q > χ2k−1 (α).
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Example (Pilot response time for different panel designs)
The sample sizes, ni , and number of times, O1i , that the
response times were below 3 seconds for the four designs are
as follows: n1 = 45, O11 = 29; n2 = 50, O12 = 42; n3 = 55,
O13 = 28; n4 = 50, O14 = 24. Perform the test at α = 0.05.
b=
Solution. Here, p
Q =
O11 + O12 + O13 + O14
= 0.615. Thus,
n1 + n2 + n3 + n4
45(0.6444 − 0.615)2 50(0.84 − 0.615)2
+
0.2368
0.2368
+
55(0.5091 − 0.615)2 50(0.48 − 0.615)2
+
= 17.307.
0.2368
0.2368
Since 17.307 > χ23 (0.05) = 7.815, H0 is rejected.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Contingency Table χ2 Test in R
The above example can be done in R as follows:
table =matrix(c(29, 16, 42, 8, 28, 27, 24, 26), nrow=2,
dimnames=list(Less=c(”Yes”,”No”), Design=c(”1”,”2”, ”3”, ”4”)))
table # just to see what was entered
chisq.test(table)
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Notation
The rank-based procedures test
H0F : F1 = · · · = Fk .
(4.1)
If H0F holds, then so does H0 : µ1 = · · · = µk .
Combine the observations, Xi1 , . . . , Xini , i = 1, . . . , k , from
the k samples into an overall set of N = n1 + · · · + nk
observations, and arrange them from smallest to largest.
Let Rij denote the (mid-)rank of observation Xij , and set
n
Ri =
ni−1
X
Rij ,
2
SR,i
j
i 2
1 X
=
Rij − R i ,
ni − 1
j=1
for the sample average and variance of ranks in group i.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Kruskal-Wallis TS is
KWk =
k
1 X
2
SKW
i=1
ni
N +1 2
Ri −
,
2
where
2
SKW
ni k
N +1 2
1 XX
Rij −
=
,
N −1
2
i=1 j=1
is the sample variance of the collection of all ranks.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
If there are no ties,
k
X
12
KWk =
ni
N(N + 1)
i=1
N +1 2
.
Ri −
2
With continuous distributions, the exact null distribution of
the K-W TS is known even with very small ni . (However, it
depends on the ni .)
·
Under H0 and if ni > 8, KWk ∼ χ2k−1 .
The RR is KWk > χ2k−1 (α).
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Variations of Kruskal-Wallis test
0 −1
b
QRk = (Ck R)0 (Ck V(R)C
k ) (Ck R), where
0
b
R = R1 , . . . , Rk , and V(R)
= diag(SR,1 /n1 , . . . , SR,k /nk ).
The version that uses the pooled variance,
2
SR,p
=
2 + · · · + (n − 1)S 2
(n1 − 1)SR,1
k
R,k
N −k
,
2 can also be used if the rank variances
instead of each SR,i
are not too dissimilar.
QRk applies to data with or without ties.
·
Under H0 and if ni > 8, QRk ∼ χ2k−1 .
The RR is QRk > χ2k −1 (α).
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
Example (Rank procedures for the flammability data)
The ranks, rank averages and rank variances are
Material 1
Material 2
Material 3
1
4
2
18
3
17
Ranks
8
15.5
7
5
15.5
14
10
6
13
12
9
11
Ri
10.75
5.67
12.08
2
SR,i
35.98
4.67
28.64
1
For the K-W test, we break the tie of observation 2.07
(groups 1 and 3) by setting 2.07 in group 1 to 2.0701.
Then, KW3 = 4.83, which yields a p-value of 0.089.
2
With the variances pooled, QR3 (ties) = 5.9549, which
corresponds to a p-value of 0.051.
3
Without pooling the variances, QR3 (ties) = 10.04, which
corresponds to a p-value of 0.0066.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
1
Comparing k> 2 Means: The ANOVA F Test
2
Comparing k> 2 Propostions: The χ2 Test
3
The Kruskal-Wallis Test – NOT COVERED
4
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
When H0 : µ1 = µ2 = · · · = µk is rejected, it is not clear
which µi ’s are significantly different.
It would seem that this question can be addressed quite
simply by making all pairwise comparisons.
For example, if k = 5 and H0 is rejected we can make CIs
for all 10 pairwise differences,
µ1 − µ2 , . . . , µ1 − µ5 , µ2 − µ3 , . . . , µ2 − µ5 , . . . , µ4 − µ5 ,
With some fine tuning this idea leads to a correct multiple
comparisons method.
The fine-tuning is needed to assure that the overall, or
experiment-wise error rate does not exceed α.
The experiment-wise error rate is the probability of at least
one pair of means being declared different when all means
are equal.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
To appreciate the need for fine-tuning, suppose the above
10 CIs are independent. (They are not really!)
Then the null probability that all 10 confidence intervals
contain zero is (1 − α)10 , and thus the experiment-wise
error rate is 1 − (1 − α)10 . If α = 0.05, this is
1 − (1 − 0.05)10 = 0.401.
In spite of the unrealistic independence assumption, the
above calculation gives an fairly close approximation to the
true experiment-wise error rate.
Confidence intervals which control the experiment-wise
error rate at a desired level α will be called (1 − α)100%
simultaneous confidence intervals.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
We will see two methods of fine-tuning the naive procedure
of using traditional confidence intervals in order to bring
the experiment-wise error rate to a desired level α.
One method is based on Bonferroni’s inequality, which
gives an upper bound on the experiment-wise error rate.
The other is Tukey’s procedure, which gives the exact
experiment-wise error rate in the case of sampling normal
homoscedastic populations, but can also be used as a
good approximation with large samples from any
distribution.
Tukey’s method can also be applied on the ranks, with
smaller sample sizes, when sampling from skewed
homoscedastic distributions.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
1
Comparing k> 2 Means: The ANOVA F Test
2
Comparing k> 2 Propostions: The χ2 Test
3
The Kruskal-Wallis Test – NOT COVERED
4
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Bonferroni’s CIs achieve the desired experiment-wise error
rate by adjusting the level of the traditional CIs.
Exact adjustment is not possible due to the dependence of
the CIs.
However, Bonferroni’s inequality asserts that, when each
of m CIs are performed at level α, then the
experiment-wise error rate, is no greater than mα.
For example, if each of the 10 CIs is constructed at level
0.05/10, and if the CIs were constructed independently,
the experiment-wise error rate would be
1 − (1 −
0.05 10
) = 1 − (1 − 0.005)10 = 0.0489,
10
which, indeed, is no larger than 0.05.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
• The above discussion leads to the following procedure for
constructing (1 − α)100% Bonferroni simultaneous CIs and
multiple comparisons:
For each of the m contrast construct a (1 − α/m)100% CI.
This set of m CIs are the (1 − α)100% Bonferroni
simultaneous CIs for the m contrasts.
Bonferroni multiple comparisons at level α: If any of the
m (1 − α)100% Bonferroni simultaneous CIs does not
contain zero, the corresponding contrast is declared
significantly different from zero at experiment-wise level α.
Bonferroni multiple comparisons can also be conducted
with pairwise testing, without CIs. Each of the m pairwise
tests is conducted at level α/m. Those rejected are
declared significantly different at experiment-wise level α.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Example
Test, at α = 0.05, the null hypothesis that the four panel
designs have no effect on whether or not the pilot reaction time
is below 3 seconds (H0 : p1 = p2 = p3 = p4 vs Ha : H0 is false)
using Bonferroni multiple comparisons.
Solution: We will construct 95% Bonferroni simultaneous CIs
for the contrasts
p1 − p2 , p1 − p3 , p1 − p4 , p2 − p3 , p2 − p4 , p3 − p4 .
Because there are m = 6 contrasts, we construct
(1 − 0.05/6)100% = 99.17% CIs for each of the above
contrasts. The data are: n1 = 45, O11 = 29; n2 = 50, O12 = 42;
n3 = 55, O13 = 28; n4 = 50, O14 = 24.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Example (Continued)
The resulting CIs are:
Contrast
p1 − p2
p1 − p3
p1 − p4
p2 − p3
p2 − p4
p3 − p4
99.17% CI
(-0.428, 0.0373)
(-0.124, 0.394)
(-0.101, 0.429)
(0.106, 0.555)
(0.129, 0.591)
(-0.229, 0.287)
Contains zero?
Yes
Yes
Yes
No
No
Yes
Thus, p2 is significantly different, at experiment-wise level
α = 0.05, from p3 and p4 . All other contrasts are not
significantly different from zero.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Example
Consider the data in Exercise 10.5.1-2 on the exams scores for
students exposed to three different teaching methods. Use the
Bonferroni multiple comparisons procedure, based on the
rank-sum test, to identify the methods achieving significantly, at
α = 0.05, better score results.
Solution: In this example, we are interested only in multiple
comparisons, not in simultaneous CIs. Because the desired
experiment-wise error rate is 0.05, we will conduct each of the
m = 3 pair-wise comparisons (A vs B, A vs C, and B vs C) at
level 0.05/3 = 0.0167. If the p-value of one of these
comparisons is smaller than 0.0167, the corresponding
methods are declared different at level α = 0.05. The results
from the tree rank-sum tests are summarized in the following
table:
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Example (Continued)
Comparison
A vs B
A vs C
B vs C
p-value
0.104
0.0136
0.1415
Less than 0.0167?
No
Yes
No
Thus, methods A and C are significantly different at α = 0.05,
but methods A and B, as well as methods B and C are not
significantly different.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
1
Comparing k> 2 Means: The ANOVA F Test
2
Comparing k> 2 Propostions: The χ2 Test
3
The Kruskal-Wallis Test – NOT COVERED
4
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
Assume normality (or all ni ≥ 30) and homoscedasticity.
They are based on the studentized range distribution
which is characterized by two degrees of freedom.
The numerator degrees of freedom equals k (the number of
populations).
The denominator degrees of freedom equals the degrees of
freedom for the SSE, i.e., N − k , with N = n1 + · · · + nk .
Tables for the studentized range distribution are available
but we will only use R output.
The R commands for constructing them are given next.
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations
Outline
Comparing k> 2 Means: The ANOVA F Test
Comparing k> 2 Propostions: The χ2 Test
The Kruskal-Wallis Test – NOT COVERED
Multiple Comparisons and Simultaneous CIs
The Experiment-Wise Error Rate
Bonferroni Multiple Comparisons and Simultaneous CIs
Tukey’s Multiple Comparisons and Simultaneous CIs
R Commands for Tukey’s Simultaneous CIs
Use the iron concentration data from http://sites.
stat.psu.edu/˜mga/401/Data/FeData.txt. Then
do:
out=aov(ore$conc ∼ ore$ind)
TukeyHSD(out) # lm instead of aov will NOT work here
TukeyHSD(out,conf.level=0.99) # for 99% simultaneous CIs
plot(TukeyHSD(out))
Michael Akritas
Lesson 11 Chapter 10: Comparing k(> 2) Populations