Comparing I > 2 Groups - Numeric Responses
• Extension of Methods used to Compare 2 Groups
• Independent and Dependent Samples
• Normal and non-normal data structures
Data
Design
Independent
Samples
(CRD)
Normal
Non-normal
F-Test
1-Way
ANOVA
Kruskal-Wallis
Test
Dependent
Samples
(RBD)
F-Test
2-Way
ANOVA
Friedman’s
Test
Independent Samples - Completely
Randomized Design (CRD)
• Controlled Experiments - Subjects assigned at
random to one of the I treatments to be compared
• Observational Studies - Subjects are sampled from
I existing groups
• Statistical model xij is a subject from group i:
xij   i   ij
where i is the population mean of
group/treatment i , ij is a random error
1-Way ANOVA for Normal Data (CRD)
• For each group obtain the mean, standard deviation, and
sample size:
xi 
 xij
j
ni
si 
2
(
x

x
)
i
 ij
j
ni  1
• Obtain the overall mean and sample size
N  n1    nI
n1 x1    nI x I i  j xij
x

N
N
Analysis of Variance - Sums of
Squares/Degrees of Freedom
• Total Variation
SST  i 1  j 1 ( xij  x) 2
I
ni
DFT  N  1
• Among Group Variation
SSG  i 1  j 1 ( x i  x)  i 1 ni ( x i  x) 2
I
ni
I
2
DFG  I  1
• Within Group Variation
SSE  i 1  ji1 ( xij  x i ) 2  i 1 (ni  1) si2
I
n
SST  SSG  SSE
I
DFT  DFG  DFE
DFE  N  I
Analysis of Variance Table and F-Test
Source of
Variation
Treatments
Error
Total
Sum of Squares
SSG
SSE
SST
Degrres of
Freedom
I-1
N-I
N-1
Mean Square
MSG=SSG/(I-1)
MSE=SSE/(N-I)
F
F=MSG/MSE
• H0: No differences among Group Means (1I)
• HA: Group means are not all equal (Not all i are equal)
MSG
T .S . : Fobs 
MSE
R.R. : Fobs  F , I 1, N  I
P  value : P( F  Fobs )
(Table E )
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• Three Conditions (Treatments):
– Music and Self-sedation (i = 1)
– Self-Sedation Only (i = 2)
– Music alone (i = 3)
• Outcomes
– Patient satisfaction score (all 3 conditions)
– Amount of self-controlled dose (conditions 1 and 2)
Source: Lee, et al (2002)
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• Summary Statistics and Sums of Squares Calculations:
Trt (i)
1
2
3
Total
ni
55
55
55
165
Mean
7.8
6.8
7.4
overall mean=7.33
Std. Dev.
2.1
2.3
2.3
---
SSG  55(7.8  7.33) 2  55(6.8  7.33) 2  55(7.4  7.33) 2  31.29 DFG  3  1  2
SSE  (55  1)( 2.1) 2  (55  1)( 2.3) 2  (55  1)( 2.3) 2  809.46
SST  31.29  809.46  840.75
DFE  165  3  162
DFT  2  162  164
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• Analysis of Variance and F-Test for Treatment effects
Source of
Variation
Treatments
Error
Total
Sum of Squares
31.29
809.46
840.75
Degrres of
Freedom
2
162
164
Mean Square
15.65
5.00
•H0: No differences among Group Means (123)
• HA: Group means are not all equal (Not all i are equal)
15.65
T .S . : Fobs 
 3.13
5.00
R.R. : Fobs  F.05, 2 ,162  3.055
(Table E )
P  value : P ( F  3.13)  0.05
F
3.13
Post-hoc Comparisons of Treatments
• If differences in group means are determined from
the F-test, researchers want to compare pairs of
groups. Three popular methods include:
– Dunnett’s Method - Compare active treatments with a
control group. Consists of I-1 comparisons, and utilizes
a special table.
– Bonferroni’s Method - Adjusts individual comparison
error rates so that all conclusions will be correct at
desired confidence/significance level. Any number of
comparisons can be made.
– Tukey’s Method - Specifically compares all I(I-1)/2
pairs of groups. Utilizes a special table.
Bonferroni’s Method (Most General)
• Wish to make C comparisons of pairs of groups with
simultaneous confidence intervals or 2-sided tests
• Want the overall confidence level for all intervals to be
“correct” to be 95% or the overall type I error rate for all tests
to be 0.05
• For confidence intervals, construct (1-(0.05/C))100% CIs
for the difference in each pair of group means (wider than
95% CIs)
• Conduct each test at =0.05/C significance level (rejection
region cut-offs more extreme than when =0.05)
Bonferroni’s Method (Most General)
• Simultaneous CI’s for pairs of group means:
x  x  t
i
j
 / 2c, N  I
1 1
MSE   
n n 
j 
 i
• If entire interval is positive, conclude i > j
• If entire interval is negative, conclude i < j
• If interval contains 0, cannot conclude i  j
Example - Relaxation Music in PatientControlled Sedation in Colonoscopy
• C=3 comparisons: 1 vs 2, 1 vs 3, 2 vs 3. Want all
intervals to contain true difference with 95% confidence
• Will construct (1-(0.05/3))100% = 98.33% CIs for
differences among pairs of group means
t.05 / 2 ( 3),162  z.0083  2.40
MSE  5.00
n1  n2  n3  55
1
1 
1 
 1

t.05 / 2 ( 3),162 MSE

 2.40 5.00

  1.02
n n 
 55 55 
j 
 i
1vs2 : (7.8  6.8)  1.02  (0.02,2.02)
1vs3 : (7.8  7.4)  1.02  (0.62,1.42)
2vs3 : (6.8  7.4)  1.02  (1.62,0.42)
Note all intervals contain 0, but first is very close to 0 at lower end
CRD with Non-Normal Data
Kruskal-Wallis Test
• Extension of Wilcoxon Rank-Sum Test to I>2
Groups
• Procedure:
– Rank the observations across groups from smallest (1)
to largest (N = n1+...+nI), adjusting for ties
– Compute the rank sums for each group: R1,...,RI . Note
that R1+...+RI = N(N+1)/2
Kruskal-Wallis Test
• H0: The I population distributions have same distribution
• HA: Not all I distributions are identical
2

12
I Ri 
 i 1
  3( N  1)
T .S . : H 
N ( N  1) 
ni 
2
R.R. : H   , I 1
P  value : P(   H )
2
Post-hoc comparisons of pairs of groups can be made by pairwise
application of rank-sum test with Bonferroni adjustment
Example - Thalidomide for Weight Gain
in HIV-1+ Patients with and without TB
• I=4 Groups, n1=n2=n3=n4=8 patients per group (N=32)
• Group 1: TB+ patients assigned Thalidomide
• Group 2: TB- patients assigned Thalidomide
• Group 3: TB+ patients assigned Placebo
• Group 4: TB- patients assigned Placebo
• Response - 21 day weight gains (kg) -- Negative values
are weight losses
Source: Klausner, et al (1996)
Example - Thalidomide for Weight Gain
in HIV-1+ Patients with and without TB
TB+/Thal
9.0 (32)
6.0 (31)
4.5 (30)
2.0 (20.5)
2.5 (23)
3.0 (25)
1.0 (15.5)
1.5 (18.5)
R1=195.5
TB-/Thal
2.5 (23)
3.5 (26.5)
4.0 (28.5)
1.0 (15.5)
0.5 (12)
4.0 (28.5)
1.5 (18.5)
2.0 (20.5)
R2=173.0
TB+/Plac
0.0 (9)
1.0 (15.5)
-1.0 (6)
-2.0 (4)
-3.0 (1.5)
-3.0 (1.5)
0.5 (12)
-2.5 (3)
R3=52.5
TB-/Plac
-0.5 (7)
0.0 (9)
2.5 (23)
0.5 (12)
-1.5 (5)
0.0 (9)
1.0 (15.5)
3.5 (26.5)
R4=107.0
12  (195.5) 2 (173.0) 2 (52.5) 2 (107.0) 2 

  3(33)  17.98
T .S . : H 



32(33)  8
8
8
8 
R.R. : H  .205, 41  7.815
Weight Gain Example - SPSS Output
F-Test and Post-Hoc Comparisons
O
W
m
d
F
S
i
a
g
f
B
8
3
3
6
0
W
3
8
3
T
0
1
4
3
2
Mean of WTGAIN
1
0
-1
-2
TB+/Thalidom ide
GROUP
TB-/Thalidom ide
TB+/Placebo
TB-/Placebo
Weight Gain Example - SPSS Output
F-Test and Post-Hoc Comparisons
C
o
m
D
ep
M
ean
f
i de
n
er
en
per
d.
(
er
S
I
(
(
I
J
i
J
E
)
g
B
)
)
B
G
T
T
T
uk
B
B
89
1
6
18
7
3
4
1
6
T
B
52
3
6
00
5
8*
4
4
1
T
B
58
0
6
18
16
0*
4
4
T
T
B
B
27
1
6
18
9
3
4
6
1
T
B
20
2
6
03
4
5*
4
1
9
T
B
27
8
6
02
96
8
4
1
T
T
B
B
35
3
6
00
2
8
4
1
4
*
T
B
04
2
6
03
0
5
4
9
1
*
T
B
646
3
6
95
2
8
4
1
T
T
B
B
416
0
6
18
8
0
4
4
*
T
B
896
8
6
02
7
8
4
1
T
B
52
3
6
95
46
8
4
1
B
T
T
B
on
B
99
1
6
0
7
3
4
0
4
9
T
B
62
3
6
00
5
8*
4
1
4
T
B
68
0
6
22
13
0*
4
7
T
T
B
B
37
1
6
0
9
3
4
0
9
4
T
B
31
2
6
04
38
5*
4
2
T
B
37
8
6
12
99
8
4
4
T
T
B
B
25
3
6
00
2
8
4
4
1
*
T
B
938
2
6
04
1
5
4
2
*
T
B
749
3
6
01
2
8
4
4
T
T
B
B
313
0
6
22
8
0
4
7
*
T
B
999
8
6
12
7
8
4
4
T
B
62
3
6
01
49
8
4
4
*
.
T
he
Weight Gain Example - SPSS Output
Kruskal-Wallis H-Test
n
n
N
G
W
T
8
4
T
8
3
T
8
6
T
8
8
T
2
t
a
a
G
C
d
A
a
K
b
G
Dependent Samples: Randomized Block
Design (RBD)
• I > 2 Treatments (groups) to be compared
• J individuals receive each treatment (preferably in
random order). Subjects are called Blocks.
• Outcome when Treatment i is assigned to Subject j
is labeled xij
• Effect of Trt i is labeled i
• Effect of Subject j is labeled bj
• Random error term is labeled ij
Dependent Samples - RBD
• Model:
xij     i  b j   ij  i  b j   ij
• Test for differences among treatment effects:
• H0: 1  ...  I  0
(1  ...  I)
• HA: Not all i = 0 (Not all i are equal)
RBD - ANOVA F-Test (Normal Data)
• Data Structure: (I Treatments, J Subjects or Blocks)
• Mean for Treatment i:
x i.
• Mean for Subject (Block) j:
• Overall Mean:
x. j
x
• Overall sample size: N = IJ
• ANOVA:Treatment, Block, and Error Sums of Squares

SST  i 1  j 1 xij  x
I
J

SSB  I  x

 x
SSG  J i 1 x i .  x
I
J
j 1
.j

2
DFT  IJ  1
2
DFG  I  1
2
DFB  J  1
SSE  SST  SSG  SSB
DFE  ( I  1)( J  1)
RBD - ANOVA F-Test (Normal Data)
• ANOVA Table:
Source
Treatments
Blocks
Error
Total
SS
SSG
SSB
SSE
SST
df
I-1
J-1
(I-1)(J-1)
IJ-1
MS
MSG = SSG/(I-1)
MSB = SSB/(J-1)
MSE = SSE/[(I-1)(J-1)]
•H0: 1  ...  I  0 (1  ...  I)
• HA: Not all i = 0
T .S . : Fobs
R.R. : Fobs
(Not all i are equal)
MSG

MSE
 F , I 1,( I 1)( J 1)
P  value : P ( F  Fobs )
F
F = MSG/MSE
Example - Theophylline Interaction
• Goal: Determine whether Cimetidine or Famotidine interact
with Theophylline
• 3 Treatments: Theo/Cim, Theo/Fam, Theo/Placebo
• 14 Blocks: Each subject received each treatment
• Response: Theophylline clearance (liters/hour)
Subject
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Source: Bachmann, et al (1995)
TRT Mean
T/C
3.69
3.61
1.15
4.02
1.00
1.75
1.45
2.59
1.57
2.34
1.31
2.43
2.33
2.34
2.26
T/F
5.13
7.04
1.46
4.44
1.15
2.11
2.12
3.25
2.11
5.20
1.98
2.38
3.53
2.33
3.16
T/P
5.88
5.89
1.46
4.05
1.09
2.59
1.69
3.16
2.06
4.59
2.08
2.61
3.42
2.54
3.08
BLK Mean
4.90
5.51
1.36
4.17
1.08
2.15
1.75
3.00
1.91
4.04
1.79
2.47
3.09
2.40
2.83
Example - Theophylline Interaction
n
-
D
I
I
I
S
q
d
S
F
S
u
i
f
g
a
C
7
5
4
8
0
I
n
3
1
3
9
0
T
5
2
3
1
0
S
1
3
4
3
0
E
9
6
1
T
9
2
C
5
1
a
R
• The test for differences in mean theophylline clearance
is given in the third line of the table
•T.S.: Fobs=10.59
• R.R.: Fobs  F.05,2,26 = 3.37 (From F-table)
• P-value: .000 (Sig. Level)
C
Example - Theophylline Interaction
Post-hoc Comparisons


Bonferroni : x i.  x j .  t / 2 c ,( I 1)( J 1)


Tukey : x i .  x j .  q.05, I ,( I 1)( J 1)
2
MSE  
J
1
MSE  
J
o
t  2.57
q  3.514
m
D
e
M
e a
d
e
e
r
e
e
e
S
.
I
(
r
r
(
i
I
E
J
J
g
)
B
B
)
T
T
T
u
h
h
3
0
3
3
6
6
1
7
5
*
T
h
3
0
3
3
6
6
2
7
5
*
T
T
h
h
3
0
3
3
6
6
1
5
7
*
T
h
3
2
0
0
0
6
8
1
1
T
T
h
h
3
0
3
3
6
6
2
5
7
*
T
h
3
2
0
0
0
6
8
1
1
B
T
T
o
h
h
3
0
9
7
6
6
1
8
4
*
T
h
3
0
9
7
6
6
2
8
4
*
T
T
h
h
3
0
7
9
6
6
1
4
8
*
T
h
3
0
6
6
0
6
0
2
2
T
T
h
h
3
0
7
9
6
6
2
4
8
*
T
h
3
0
6
6
0
6
0
2
2
B
a
*
.
T
h
Example - Theophylline Interaction
Plot of Data (Marginal means are raw data)
Estimated Marginal Means of CLRNCE
7
6
5
TRT
4
Theophylline/Cimetid
3
ine
2
Theophylline/Famotid
1
ine
0
Theophylline/Placebo
1
3
2
5
4
SUBJECT
7
6
9
8
11
10
13
12
14
RBD -- Non-Normal Data
Friedman’s Test
• When data are non-normal, test is based on ranks
• Procedure to obtain test statistic:
– Rank the I treatments within each block (1=smallest,
I=largest) adjusting for ties
– Compute rank sums for treatments (Ri) across blocks
– H0: The I populations are identical (1=...=I)
– HA: Differences exist among the I group means
12
I
2
T .S . : Fr 
R
 3J ( I  1)

i 1 i
IJ ( I  1)
R.R. : Fr  2 , I 1
P  value : P(  2  Fr )
Example - tmax for 3 formulation/fasting states
• I=3 Treatments of Valproate: Capsule/Fasting (i=1),
Capsule/nonfasting (i=2), Enteric-Coated/fasting (i=3)
• J=11 subjects
• Response - Time to maximum concentration (tmax)
Subject
Source: Carrigan, et al (1990)
1
2
3
4
5
6
7
8
9
10
11
Rank sum
C/F
3.5 (2)
4.0 (2)
3.5 (2)
3.0 (1.5)
3.5 (1.5)
3.0 (1)
4.0 (2.5)
3.5 (2)
3.5 (1.5)
3.0 (1)
4.5 (2)
T1=19.0
C/NF
4.5 (3)
4.5 (3)
4.5 (3)
4.5 (3)
5.0 (3)
5.5 (3)
4.0 (2.5)
4.5 (3)
5.0 (3)
4.5 (3)
6.0 (3)
T2=32.5
EC/F
2.5 (1)
3.0 (1)
3.0 (1)
3.0 (1.5)
3.5 (1.5)
3.5 (2)
2.5 (1)
3.0 (1)
3.5 (1.5)
3.5 (2)
3.0 (1)
T3=14.5
Example - tmax for 3 formulation/fasting states
H0: The I populations are identical (1=...=I)
HA: Differences exist among the I group means


12
T .S . : Fr 
19.0 2  32.52  14.52  3(11)(3  1)  15.95
11(3)(3  1)
R.R. : Fr  .205,31  5.99
Data Sources
• Lee,D.W., K.W. Chan, C.M. Poon, et al (2002). “Relaxation
Music Decreases the Dose of Patient-Controlled Sedation During
Colonoscopy: A Prospective Randomized Controlled Trial,”
Gastrointestinal Endoscopy, 55:33-36.
• Klausner,J.D., S. Makonkawkeyoon, P. Akarasewi, et al (1996).
“The Effect of Thalidomide on the Pathogenesis of HIV-1 and M.
tuberculosis Infection,” Journal of Acquired Immune Deficiency
Syndromes and Human Retrovirology, 11:247-257
• Bachmann, K., T.J. Sullivan, J.H. Reese, et al (1995).
“Controlled Study of the Putative Interaction Between
Famotidine and Theophylline in Patients with Chronic
Obstructive Pulmonary Disorder,” Journal of Clinical
Pharmacology, 35:529-535.