Comparing k > 2 Groups - Numeric
Responses
• Extension of Methods used to Compare 2 Groups
• Parallel Groups and Crossover Designs
• Normal and non-normal data structures
Data Normal
Design
Nonnormal
Parallel
Groups
(CRD)
Crossover
(RBD)
KruskalWallis Test
F-Test
1-Way
ANOVA
F-Test
2-Way
ANOVA
Friedman’s
Test
Parallel Groups - Completely
Randomized Design (CRD)
• Controlled Experiments - Subjects assigned at
random to one of the k treatments to be compared
• Observational Studies - Subjects are sampled from
k existing groups
• Statistical model Yij is a subject from group i:
Yij     i   ij  i   ij
where  is the overall mean, i is the effect
of treatment i , ij is a random error, and i is
the population mean for group i
1-Way ANOVA for Normal Data (CRD)
• For each group obtain the mean, standard deviation, and
sample size:
yi 
 yij
j
ni
si 
2
(
y

y
)
 ij i
j
ni  1
• Obtain the overall mean and sample size
n  n1    nk
n1 y1    nk y k i  j yij
y

n
n
Analysis of Variance - Sums of Squares
• Total Variation
TotalSS  i 1  j 1 ( yij  y) 2
k
ni
dfTotal  n  1
• Between Group Variation
SST  i 1  j 1 ( y i  y)  i 1 ni ( y i  y) 2
k
ni
2
k
dfT  k  1
• Within Group Variation
SSE  i 1  j 1 ( yij  y i )  i 1 (ni  1) si2
k
ni
TotalSS  SST  SSE
2
k
dfTotal  dfT  df E
df E  n  k
Analysis of Variance Table and F-Test
Source of
Variation
Treatments
Error
Total
Sum of Squares
SST
SSE
Total SS
Degrres of
Freedom
k-1
n-k
n-1
Mean Square
MST=SST/(k-1)
MSE=SSE/(n-k)
F
F=MST/MSE
• H0: No differences among Group Means (1k=0)
• HA: Group means are not all equal (Not all i are 0)
MST
T .S . : Fobs 
MSE
R.R. : Fobs  F ,k 1,n  k
P  val : P( F  Fobs )
(Table A.4)
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• Three Conditions (Treatments):
– Music and Self-sedation (i = 1)
– Self-Sedation Only (i = 2)
– Music alone (i = 3)
• Outcomes
– Patient satisfaction score (all 3 conditions)
– Amount of self-controlled dose (conditions 1 and 2)
Source: Lee, et al (2002)
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• Summary Statistics and Sums of Squares Calculations:
Trt (i)
1
2
3
Total
ni
55
55
55
165
Mean
7.8
6.8
7.4
overall mean=7.33
Std. Dev.
2.1
2.3
2.3
---
SST  55(7.8  7.33) 2  55(6.8  7.33) 2  55(7.4  7.33) 2  31.29 dfT  3  1  2
SSE  (55  1)( 2.1) 2  (55  1)( 2.3) 2  (55  1)( 2.3) 2  809.46
df E  165  3  162
TotalSS  31.29  809.46  840.75
dfTotal  2  162  164
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• Analysis of Variance and F-Test for Treatment effects
Source of
Variation
Treatments
Error
Total
Sum of Squares
31.29
809.46
840.75
Degrres of
Freedom
2
162
164
Mean Square
15.65
5.00
F
3.13
•H0: No differences among Group Means (13=0)
• HA: Group means are not all equal (Not all i are 0)
15.65
T .S . : Fobs 
 3.13
5.00
R.R. : Fobs  F.05, k 2 ,162  3.055
P  val : P ( F  3.13)  0.05
(Table A.4)
Post-hoc Comparisons of Treatments
• If differences in group means are determined from
the F-test, researchers want to compare pairs of
groups. Three popular methods include:
– Dunnett’s Method - Compare active treatments with a
control group. Consists of k-1 comparisons, and utilizes
a special table.
– Bonferroni’s Method - Adjusts individual comparison
error rates so that all conclusions will be correct at
desired confidence/significance level. Any number of
comparisons can be made.
– Tukey’s Method - Specifically compares all k(k-1)/2
pairs of groups. Utilizes a special table.
Bonferroni’s Method (Most General)
• Wish to make C comparisons of pairs of groups with
simultaneous confidence intervals or 2-sided tests
• Want the overall confidence level for all intervals to be
“correct” to be 95% or the overall type I error rate for all tests
to be 0.05
• For confidence intervals, construct (1-(0.05/C))100% CIs
for the difference in each pair of group means (wider than
95% CIs)
• Conduct each test at =0.05/C significance level (rejection
region cut-offs more extreme than when =0.05)
Bonferroni’s Method (Most General)
• Simultaneous CI’s for pairs of group means:
y  y   t
i
j
 / 2c ,nk
1 1
MSE   
n n 
j 
 i
• If entire interval is positive, conclude i > j
• If entire interval is negative, conclude i < j
• If interval contains 0, cannot conclude i  j
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• C=3 comparisons: 1 vs 2, 1 vs 3, 2 vs 3. Want all
intervals to contain true difference with 95% confidence
• Will construct (1-(0.05/3))100% = 98.33% CIs for
differences among pairs of group means
t.05 / 2 ( 3),162  z.0083  2.40
MSE  5.00
n1  n2  n3  55
1
1 
1 
 1

t.05 / 2 ( 3),162 MSE

 2.40 5.00

  1.02
n n 
 55 55 
j 
 i
1vs2 : (7.8  6.8)  1.02  (0.02,2.02)
1vs3 : (7.8  7.4)  1.02  (0.62,1.42)
2vs3 : (6.8  7.4)  1.02  (1.62,0.42)
Note all intervals contain 0, but first is very close to 0 at lower end
CRD with Non-Normal Data
Kruskal-Wallis Test
• Extension of Wilcoxon Rank-Sum Test to k>2
Groups
• Procedure:
– Rank the observations across groups from smallest (1)
to largest (n = n1+...+nk), adjusting for ties
– Compute the rank sums for each group: T1,...,Tk . Note
that T1+...+Tk = n(n+1)/2
Kruskal-Wallis Test
• H0: The k population distributions are identical (1=...=k)
• HA: Not all k distributions are identical (Not all i are equal)
2
12
k Ti
T .S . : H 
 3(n  1)

i 1
n(n  1)
ni
R.R. : H   ,k 1
2
P  val : P (   H )
2
Post-hoc comparisons of pairs of groups can be made by pairwise
application of rank-sum test with Bonferroni adjustment
Example - Thalidomide for Weight Gain
in HIV-1+ Patients with and without TB
• k=4 Groups, n1=n2=n3=n4=8 patients per group (n=32)
• Group 1: TB+ patients assigned Thalidomide
• Group 2: TB- patients assigned Thalidomide
• Group 3: TB+ patients assigned Placebo
• Group 4: TB- patients assigned Placebo
• Response - 21 day weight gains (kg) -- Negative values
are weight losses
Source: Klausner, et al (1996)
Example - Thalidomide for Weight Gain
in HIV-1+ Patients with and without TB
TB+/Thal
9.0 (32)
6.0 (31)
4.5 (30)
2.0 (20.5)
2.5 (23)
3.0 (25)
1.0 (15.5)
1.5 (18.5)
T1=195.5
TB-/Thal
2.5 (23)
3.5 (26.5)
4.0 (28.5)
1.0 (15.5)
0.5 (12)
4.0 (28.5)
1.5 (18.5)
2.0 (20.5)
T2=173.0
TB+/Plac
0.0 (9)
1.0 (15.5)
-1.0 (6)
-2.0 (4)
-3.0 (1.5)
-3.0 (1.5)
0.5 (12)
-2.5 (3)
T3=52.5
TB-/Plac
-0.5 (7)
0.0 (9)
2.5 (23)
0.5 (12)
-1.5 (5)
0.0 (9)
1.0 (15.5)
3.5 (26.5)
T4=107.0
12  (195.5) 2 (173.0) 2 (52.5) 2 (107.0) 2 

  3(33)  17.98
T .S . : H 



32(33)  8
8
8
8 
R.R. : H  .205, 41  7.815
Weight Gain Example - SPSS Output
F-Test and Post-Hoc Comparisons
O
W
m
d
F
S
i
a
g
f
B
8
3
3
6
0
W
3
8
3
T
0
1
4
3
2
Mean of WTGAIN
1
0
-1
-2
TB+/Thalidom ide
GROUP
TB-/Thalidom ide
TB+/Placebo
TB-/Placebo
Weight Gain Example - SPSS Output
F-Test and Post-Hoc Comparisons
C
o
m
D
ep
M
ean
f
i de
n
er
en
per
d.
(
er
S
I
(
(
I
J
i
J
E
)
g
B
)
)
B
G
T
T
T
uk
B
B
89
1
6
18
7
3
4
1
6
T
B
52
3
6
00
5
8*
4
4
1
T
B
58
0
6
18
16
0*
4
4
T
T
B
B
27
1
6
18
9
3
4
6
1
T
B
20
2
6
03
4
5*
4
1
9
T
B
27
8
6
02
96
8
4
1
T
T
B
B
35
3
6
00
2
8
4
1
4
*
T
B
04
2
6
03
0
5
4
9
1
*
T
B
646
3
6
95
2
8
4
1
T
T
B
B
416
0
6
18
8
0
4
4
*
T
B
896
8
6
02
7
8
4
1
T
B
52
3
6
95
46
8
4
1
B
T
T
B
on
B
99
1
6
0
7
3
4
0
4
9
T
B
62
3
6
00
5
8*
4
1
4
T
B
68
0
6
22
13
0*
4
7
T
T
B
B
37
1
6
0
9
3
4
0
9
4
T
B
31
2
6
04
38
5*
4
2
T
B
37
8
6
12
99
8
4
4
T
T
B
B
25
3
6
00
2
8
4
4
1
*
T
B
938
2
6
04
1
5
4
2
*
T
B
749
3
6
01
2
8
4
4
T
T
B
B
313
0
6
22
8
0
4
7
*
T
B
999
8
6
12
7
8
4
4
T
B
62
3
6
01
49
8
4
4
*
.
T
he
Weight Gain Example - SPSS Output
Kruskal-Wallis H-Test
n
n
N
G
W
T
8
4
T
8
3
T
8
6
T
8
8
T
2
t
a
a
G
C
d
A
a
K
b
G
Crossover Designs: Randomized Block
Design (RBD)
• k > 2 Treatments (groups) to be compared
• b individuals receive each treatment (preferably in
random order). Subjects are called Blocks.
• Outcome when Treatment i is assigned to Subject j
is labeled Yij
• Effect of Trt i is labeled i
• Effect of Subject j is labeled bj
• Random error term is labeled ij
Crossover Designs - RBD
• Model:
Yij     i  b j   ij  i  b j   ij
• Test for differences among treatment effects:
• H0: 1  ...  k  0
(1  ...  k )
• HA: Not all i = 0 (Not all i are equal)
RBD - ANOVA F-Test (Normal Data)
• Data Structure: (k Treatments, b Subjects)
• Mean for Treatment i:
y i.
• Mean for Subject (Block) j:
• Overall Mean:
y. j
y
• Overall sample size: n = bk
• ANOVA:Treatment, Block, and Error Sums of Squares

SST  b y  y 
SSB  k  y  y 
TotalSS  i 1  j 1 yij  y
k
b
2
k
i 1
i.
2
b
j 1

.j
SSE  TotalSS  SST  SSB
2
df Total  bk  1
df T  k  1
df B  b  1
df E  (b  1)( k  1)
RBD - ANOVA F-Test (Normal Data)
• ANOVA Table:
Source
Treatments
Blocks
Error
Total
SS
SST
SSB
SSE
TotalSS
df
k-1
b-1
(b-1)(k-1)
bk-1
MS
MST = SST/(k-1)
MSB = SSB/(b-1)
MSE = SSE/[(b-1)(k-1)]
•H0: 1  ...  k  0 (1  ...  k )
• HA: Not all i = 0
T .S . : Fobs
R.R. : Fobs
(Not all i are equal)
MST

MSE
 F , k 1,( b 1)( k 1)
P  val : P ( F  Fobs )
F
F = MST/MSE
Example - Theophylline Interaction
• Goal: Determine whether Cimetidine or Famotidine interact
with Theophylline
• 3 Treatments: Theo/Cim, Theo/Fam, Theo/Placebo
• 14 Blocks: Each subject received each treatment
• Response: Theophylline clearance (liters/hour)
Subject
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Source: Bachmann, et al (1995)
TRT Mean
T/C
3.69
3.61
1.15
4.02
1.00
1.75
1.45
2.59
1.57
2.34
1.31
2.43
2.33
2.34
2.26
T/F
5.13
7.04
1.46
4.44
1.15
2.11
2.12
3.25
2.11
5.20
1.98
2.38
3.53
2.33
3.16
T/P
5.88
5.89
1.46
4.05
1.09
2.59
1.69
3.16
2.06
4.59
2.08
2.61
3.42
2.54
3.08
BLK Mean
4.90
5.51
1.36
4.17
1.08
2.15
1.75
3.00
1.91
4.04
1.79
2.47
3.09
2.40
2.83
Example - Theophylline Interaction
n
-
D
I
I
I
S
q
d
S
F
S
u
i
f
g
a
C
7
5
4
8
0
I
n
3
1
3
9
0
T
5
2
3
1
0
S
1
3
4
3
0
E
9
6
1
T
9
2
C
5
1
a
R
• The test for differences in mean theophylline clearance
is given in the third line of the table
•T.S.: Fobs=10.59
• R.R.: Fobs  F.05,2,26 = 3.37 (From F-table)
• P-value: .000 (Sig. Level)
C
Example - Theophylline Interaction
Post-hoc Comparisons


Bonferroni : y i  y j  t / 2 c ,( b 1)( k 1)


Tukey : y i  y j  q.05,k ,( b 1)( k 1)
2
MSE  
b
1
MSE  
b
o
t  2.57
q  3.514
m
D
e
M
e a
d
e
e
r
e
e
e
S
.
I
(
r
r
(
i
I
E
J
J
g
)
B
B
)
T
T
T
u
h
h
3
0
3
3
6
6
1
7
5
*
T
h
3
0
3
3
6
6
2
7
5
*
T
T
h
h
3
0
3
3
6
6
1
5
7
*
T
h
3
2
0
0
0
6
8
1
1
T
T
h
h
3
0
3
3
6
6
2
5
7
*
T
h
3
2
0
0
0
6
8
1
1
B
T
T
o
h
h
3
0
9
7
6
6
1
8
4
*
T
h
3
0
9
7
6
6
2
8
4
*
T
T
h
h
3
0
7
9
6
6
1
4
8
*
T
h
3
0
6
6
0
6
0
2
2
T
T
h
h
3
0
7
9
6
6
2
4
8
*
T
h
3
0
6
6
0
6
0
2
2
B
a
*
.
T
h
Example - Theophylline Interaction
Plot of Data (Marginal means are raw data)
Estimated Marginal Means of CLRNCE
7
6
5
TRT
4
Theophylline/Cimetid
3
ine
2
Theophylline/Famotid
1
ine
0
Theophylline/Placebo
1
3
2
5
4
SUBJECT
7
6
9
8
11
10
13
12
14
RBD -- Non-Normal Data
Friedman’s Test
• When data are non-normal, test is based on ranks
• Procedure to obtain test statistic:
– Rank the k treatments within each block (1=smallest,
k=largest) adjusting for ties
– Compute rank sums for treatments (Ti) across blocks
– H0: The k populations are identical (1=...=k)
– HA: Differences exist among the k group means
12
k
2
T .S . : Fr 
T
 3b(k  1)

i 1 i
bk (k  1)
R.R. : Fr  2 ,k 1
P  val : P(  2  Fr )
Example - tmax for 3 formulation/fasting states
• k=3 Treatments of Valproate: Capsule/Fasting (i=1),
Capsule/nonfasting (i=2), Enteric-Coated/fasting (i=3)
• b=11 subjects
• Response - Time to maximum concentration (tmax)
Subject
Source: Carrigan, et al (1990)
1
2
3
4
5
6
7
8
9
10
11
Rank sum
C/F
3.5 (2)
4.0 (2)
3.5 (2)
3.0 (1.5)
3.5 (1.5)
3.0 (1)
4.0 (2.5)
3.5 (2)
3.5 (1.5)
3.0 (1)
4.5 (2)
T1=19.0
C/NF
4.5 (3)
4.5 (3)
4.5 (3)
4.5 (3)
5.0 (3)
5.5 (3)
4.0 (2.5)
4.5 (3)
5.0 (3)
4.5 (3)
6.0 (3)
T2=32.5
EC/F
2.5 (1)
3.0 (1)
3.0 (1)
3.0 (1.5)
3.5 (1.5)
3.5 (2)
2.5 (1)
3.0 (1)
3.5 (1.5)
3.5 (2)
3.0 (1)
T3=14.5
Example - tmax for 3 formulation/fasting states
H0: The k populations are identical (1=...=k)
HA: Differences exist among the k group means


12
T .S . : Fr 
19.0 2  32.52  14.52  3(11)(3  1)  15.95
11(3)(3  1)
R.R. : Fr  .205,31  5.99