Experimental Design and the
Analysis of Variance
Comparing t > 2 Groups - Numeric Responses
• Extension of Methods used to Compare 2 Groups
• Independent Samples and Paired Data Designs
• Normal and non-normal data distributions
Data
Design
Normal
Nonnormal
Independent
Samples
(CRD)
Paired Data
(RBD)
F-Test
1-Way
ANOVA
F-Test
2-Way
ANOVA
KruskalWallis Test
Friedman’s
Test
Completely Randomized Design (CRD)
• Controlled Experiments - Subjects assigned at random
to one of the t treatments to be compared
• Observational Studies - Subjects are sampled from t
existing groups
• Statistical model yij is measurement from the jth subject
from group i:
yij     i   ij  i   ij
where  is the overall mean, i is the effect
of treatment i , ij is a random error, and i is
the population mean for group i
1-Way ANOVA for Normal Data (CRD)
• For each group obtain the mean, standard deviation, and
sample size:
y i. 
 yij
j
ni
si 
2
(
y

y
)
 ij i.
j
ni  1
• Obtain the overall mean and sample size
N  n1  ...  nt
n1 y1.  ...  nt y t . i  j yij
y .. 

N
N
Analysis of Variance - Sums of Squares
• Total Variation
TSS  i 1  j 1 ( yij  y.. ) 2
k
ni
dfTotal  N  1
• Between Group (Sample) Variation
SST  i 1  ji1 ( y i.  y.. ) 2  i 1 ni ( y i.  y.. ) 2
t
n
t
dfT  t  1
• Within Group (Sample) Variation
SSE  i 1  j 1 ( yij  y i. )  i 1 (ni  1) si2
t
ni
TSS  SST  SSE
2
t
dfTotal  dfT  df E
df E  N  t
Analysis of Variance Table and F-Test
Source of
Variation
Treatments
Error
Total
Sum of Squares
SST
SSE
TSS
Degrres of
Freedom
t-1
N-t
N-1
Mean Square
MST=SST/(t-1)
MSE=SSE/(N-t)
F
F=MST/MSE
• Assumption: All distributions normal with common variance
•H0: No differences among Group Means (1    t =0)
• HA: Group means are not all equal (Not all i are 0)
MST
T .S . : Fobs 
MSE
R.R. : Fobs  F ,t 1, N t
P  val : P( F  Fobs )
(Table 9)
Expected Mean Squares
• Model: yij =  +i + ij with ij ~ N(0,s2), Si = 0:
E ( MSE )  s 2
t
E ( MST )  s 2 
2
n

 i i
i 1
t 1
t
s 
2

E ( MST )

E ( MSE )
2
n

 i i
t
i 1
t 1
s2
 1
2
n

 i i
i 1
2
s (t  1)
E ( MST )
When H 0 : 1     t  0 is true,
1
E ( MSE )
E ( MST )
otherwise ( H a is true),
1
E ( MSE )
Expected Mean Squares
• 3 Factors effect magnitude of F-statistic (for fixed t)
– True group effects (1,…,t)
– Group sample sizes (n1,…,nt)
– Within group variance (s2)
• Fobs = MST/MSE
• When H0 is true (1=…=t=0), E(MST)/E(MSE)=1
• Marginal Effects of each factor (all other factors fixed)
– As spread in (1,…,t)  E(MST)/E(MSE) 
– As (n1,…,nt)  E(MST)/E(MSE)  (when H0 false)
– As s2  E(MST)/E(MSE)  (when H0 false)
0.09
0.09
0.08
0.08
0.07
0.07
0.06
0.06
0.05
0.05
0.04
0.04
0.03
E ( MST )
E ( MSE )
0.02
0.01
0
0
20
40
60
80
100
120
140
160
180
200
A) =100, t1=-20, t2=0, t3=20, s = 20
n
4
8
12
20
0.09
0.08
0.07
A
9
17
25
41
B
129
257
385
641
C
1.5
2
2.5
3.5
0.03
0.02
0.01
0
0
20
40
60
80
100
120
140
160
180
200
B) =100, t1=-20, t2=0, t3=20, s = 5
D
9
17
25
41
0.09
0.08
0.06
0.07
0.05
0.06
0.04
0.05
0.03
0.04
0.02
0.03
0.01
0.02
0
0.01
0
20
40
60
80
100
120
140
160
180
0
0
C) =100, t1=-5, t2=0, t3=5, s = 20
20
40
60
80
100
120
140
160
180
D) =100, t1=-5, t2=0, t3=5, s = 5
200
Example - Seasonal Diet Patterns in Ravens
• “Treatments” - t = 4 seasons of year (3 “replicates” each)
–
–
–
–
Winter: November, December, January
Spring: February, March, April
Summer: May, June, July
Fall: August, September, October
• Response (Y) - Vegetation (percent of total pellet weight)
• Transformation (For approximate normality):
 Y 

Y '  arcsin 

 100 
Source: K.A. Engel and L.S. Young (1989). “Spatial and Temporal Patterns in the Diet of Common
Ravens in Southwestern Idaho,” The Condor, 91:372-378
Seasonal Diet Patterns in Ravens - Data/Means
Y
j=1
j=2
j=3
Winter(i=1)
94.3
90.3
83.0
Fall(i=2)
80.7
90.5
91.8
Summer(i=3)
80.5
74.3
32.4
Fall (i=4)
67.8
91.8
89.3
Y'
j=1
j=2
j=3
Winter(i=1)
1.329721
1.254080
1.145808
Fall(i=2)
1.115957
1.257474
1.280374
Summer(i=3)
1.113428
1.039152
0.605545
Fall (i=4)
0.967390
1.280374
1.237554
1.329721  1.254080  1.145808
y 1. 
 1.24203
3
1.115957  1.257474  1.280374
y 2. 
 1.217935
3
1.113428  1.039152  0.605545
y 3. 
 0.919375
3
0.967390  1.280374  1.237554
y 4. 
 1.16773
3
1.329721  ...  1.237554
y .. 
 1.135572
12
Seasonal Diet Patterns in Ravens - Data/Means
Plot of Transformed Data by Season
1.500000
1.400000
1.300000
Transformed % Vegetation
1.200000
1.100000
1.000000
0.900000
0.800000
0.700000
0.600000
0.500000
0
1
2
3
Season
4
5
Seasonal Diet Patterns in Ravens - ANOVA
Total Variation : (dfTotal  12 - 1  11)
TSS  (1.329721  1.135572) 2  ...  (1.27554  1.135572) 2  0.438425
Between Group Variation : (dfT  4 - 1  3)


SST  3 (1.24203  1.135572) 2  ...  (1.161773  1.135572) 2  0.197387
Within Group Variation : (dfE  12 - 4  8)
SSE  (1.329721  1.243203) 2  ...  (1.237554  1.161773) 2  0.241038
ANOVA
Source of Variation
Between Groups
Within Groups
SS
0.197387
0.241038
df
3
8
Total
0.438425
11
MS
F
P-value
0.065796 2.183752 0.167768
0.03013
F crit
4.06618
Do not conclude that seasons differ with respect to vegetation intake
Seasonal Diet Patterns in Ravens - Spreadsheet
Month
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
Season
1
1
1
2
2
2
3
3
3
4
4
4
Y'
1.329721
1.254080
1.145808
1.115957
1.257474
1.280374
1.113428
1.039152
0.605545
0.967390
1.280374
1.237554
Season MeanOverall Mean
1.243203
1.135572
1.243203
1.135572
1.243203
1.135572
1.217935
1.135572
1.217935
1.135572
1.217935
1.135572
0.919375
1.135572
0.919375
1.135572
0.919375
1.135572
1.161773
1.135572
1.161773
1.135572
1.161773
1.135572
Sum
TSS
0.037694
0.014044
0.000105
0.000385
0.014860
0.020968
0.000490
0.009297
0.280928
0.028285
0.020968
0.010400
0.438425
Total SS
Between Season SS
(Y’-Overall Mean)2
(Group Mean-Overall Mean)2
SST
0.011584
0.011584
0.011584
0.006784
0.006784
0.006784
0.046741
0.046741
0.046741
0.000687
0.000687
0.000687
0.197387
SSE
0.007485
0.000118
0.009486
0.010400
0.001563
0.003899
0.037657
0.014346
0.098489
0.037785
0.014066
0.005743
0.241038
Within Season SS
(Y’-Group Mean)2
CRD with Non-Normal Data
Kruskal-Wallis Test
• Extension of Wilcoxon Rank-Sum Test to k > 2 Groups
• Procedure:
– Rank the observations across groups from smallest (1) to largest
( N = n1+...+nk ), adjusting for ties
– Compute the rank sums for each group: T1,...,Tk . Note that
T1+...+Tk = N(N+1)/2
Kruskal-Wallis Test
• H0: The k population distributions are identical (1=...=k)
• HA: Not all k distributions are identical (Not all i are equal)
2
12
k Ti
T .S . : H 

3
(
N

1
)

N ( N  1) i 1 ni
R.R. : H   ,k 1
2
P  val : P(   H )
2
An adjustment to H is suggested when there are many ties in the
data. Formula is given on page 344 of O&L.
Example - Seasonal Diet Patterns in Ravens
Month
NOV
DEC
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
Season
1
1
1
2
2
2
3
3
3
4
4
4
Y'
1.329721
1.254080
1.145808
1.115957
1.257474
1.280374
1.113428
1.039152
0.605545
0.967390
1.280374
1.237554
H 0 : No seasonal difference
Rank
12
8
6
5
9
10.5
4
3
1
2
10.5
7
• T1 = 12+8+6 = 26
• T2 = 5+9+10.5 = 24.5
• T3 = 4+3+1 = 8
• T4 = 2+10.5+7 = 19.5
H a : Seasonal Difference s
 (26) 2 (24.5) 2 (8) 2 (19.5) 2 
12
T .S . : H 




  3(12  1)  44.12  39  5.12
12(12  1)  3
3
3
3 
R.R.(  0.05) : H   .205, 41  7.815
P  value : P(  2  H  5.12)  .1632
Post-hoc Comparisons of Treatments
• If differences in group means are determined from the Ftest, researchers want to compare pairs of groups. Three
popular methods include:
– Fisher’s LSD - Upon rejecting the null hypothesis of no
differences in group means, LSD method is equivalent to doing
pairwise comparisons among all pairs of groups as in Chapter 6.
– Tukey’s Method - Specifically compares all t(t-1)/2 pairs of
groups. Utilizes a special table (Table 11, p. 701).
– Bonferroni’s Method - Adjusts individual comparison error rates
so that all conclusions will be correct at desired
confidence/significance level. Any number of comparisons can be
made. Very general approach can be applied to any inferential
problem
Fisher’s Least Significant Difference Procedure
• Protected Version is to only apply method after
significant result in overall F-test
• For each pair of groups, compute the least significant
difference (LSD) that the sample means need to differ
by to conclude the population means are not equal
LSDij  t / 2
1 1
MSE   
n n 
j 
 i
with df  N  t
Conclude  i   j if y i.  y j .  LSDij


Fisher' s Confidence Interval : y i.  y j .  LSDij
Tukey’s W Procedure
• More conservative than Fisher’s LSD (minimum
significant difference and confidence interval width are
higher).
• Derived so that the probability that at least one false
difference is detected is  (experimentwise error rate)
MSE
Wij  q (t , )
n
q given in Table 11, p. 701 with   N - t
Conclude i   j if y i.  y j .  Wij
Tukey's Confidence Interval:
y
i.

 y j .  Wij
1 1
q (t , )
When the sample sizes are unequal, use Wij 
MSE   
n n 
2
j 
 i
Bonferroni’s Method (Most General)
• Wish to make C comparisons of pairs of groups with simultaneous
confidence intervals or 2-sided tests
•When all pair of treatments are to be compared, C = t(t-1)/2
• Want the overall confidence level for all intervals to be “correct” to
be 95% or the overall type I error rate for all tests to be 0.05
• For confidence intervals, construct (1-(0.05/C))100% CIs for the
difference in each pair of group means (wider than 95% CIs)
• Conduct each test at =0.05/C significance level (rejection region
cut-offs more extreme than when =0.05)
• Critical t-values are given in table on class website, we will use
notation: t/2,C, where C=#Comparisons,  = df
Bonferroni’s Method (Most General)
1 1
Bij  t / 2,C ,v MSE   
n n 
j 
 i
(t given on class website with v  N-t )
Conclude i   j if y i.  y j .  Bij


Bonferroni ' s Confidence Interval : y i.  y j .  Bij
Example - Seasonal Diet Patterns in Ravens
Note: No differences were found, these calculations are only
for demonstration purposes
MSE  0.03013 ni  3 t.025,8  2.306 q.05,t  4,df E 8  4.53 t.025,C 6,df E 8  3.479
1 1
LSDij  2.306 (0.03013)    0.3268
3 3
1
Wij  4.53 (0.03013)   0.4540
3
1 1
Bij  3.479 (0.03013)    0.4930
3 3
Comparison(i vs j) Group i Mean Group j MeanDifference
1 vs 2
1.243203
1.217935
0.025267
1 vs 3
1.243203
0.919375
0.323828
1 vs 4
1.243203
1.161773
0.081430
2 vs 3
1.217935
0.919375
0.298560
2 vs 4
1.217935
1.161773
0.056162
3 vs 4
0.919375
1.161773
-0.242398
Randomized Block Design (RBD)
• t > 2 Treatments (groups) to be compared
• b Blocks of homogeneous units are sampled. Blocks can
be individual subjects. Blocks are made up of t subunits
• Subunits within a block receive one treatment. When
subjects are blocks, receive treatments in random order.
• Outcome when Treatment i is assigned to Block j is
labeled Yij
• Effect of Trt i is labeled i
• Effect of Block j is labeled bj
• Random error term is labeled ij
• Efficiency gain from removing block-to-block
variability from experimental error
Randomized Complete Block Designs
Yij     i  b j   ij  i  b j   ij
t

i 1
i
 0 E ( ij )  0
V ( ij )  s 2
Note:
1
1
Y

...

Y

   1  b1  11   ...     1  b b  1b   
 11
1b 
b
b
   1  b    1
Y 1 
Y 2     2  b    2

 


 Y 1  Y 2    1  b    1     2  b    2  1   2    1   2
• Test for differences among treatment effects:
• H0: 1  ...  t  0
• HA: Not all i = 0
(1  ...  t )
(Not all i are equal)

RBD - ANOVA F-Test (Normal Data)
• Data Structure: (t Treatments, b Blocks)
• Mean for Treatment i:
y i.
• Mean for Subject (Block) j:
• Overall Mean:
y. j
y ..
• Overall sample size: N = bt
• ANOVA:Treatment, Block, and Error Sums of Squares

TSS   i 1  j 1 yij  y ..
t
b

SSB  t   y

y 

2
dfTotal  bt  1
SST  b  i 1 y i .  y ..
2
dfT  t  1
b
2
df B  b  1
t
j 1
t
b
i 1
j 1

.j
..
SSE    yij  y i .  y . j  y ..

2
 TSS  SST  SSB
b i 1 i2
t
E MSE  s 2
E MST   s 2 
t 1
df E  (b  1)(t  1)
RBD - ANOVA F-Test (Normal Data)
• ANOVA Table:
Source
Treatments
Blocks
Error
Total
SS
SST
SSB
SSE
TSS
df
t-1
b-1
(b-1)(t-1)
bt-1
MS
MST = SST/(t-1)
MSB = SSB/(b-1)
MSE = SSE/[(b-1)(t-1)]
•H0: 1  ...  t  0 (1  ...  t )
• HA: Not all i = 0
T .S . : Fobs
R.R. : Fobs
(Not all i are equal)
MST

MSE
 F ,t 1,( b 1)( t 1)
P  val : P ( F  Fobs )
F
F = MST/MSE
Pairwise Comparison of Treatment Means
• Tukey’s Method- q in Studentized Range Table with  = (b-1)(t-1)
MSE
Wij  q (t , v)
b
Conclude  i   j if y i.  y j .  Wij


Tukey' s Confidence Interval : y i.  y j .  Wij
• Bonferroni’s Method - t-values from table on class
website with  = (b-1)(t-1) and C=t(t-1)/2
Bij  t / 2,C ,v
2MSE
b
Conclude  i   j if y i.  y j .  Bij


Bonferroni ' s Confidence Interval : y i.  y j .  Bij
Expected Mean Squares / Relative Efficiency
• Expected Mean Squares: As with CRD, the Expected Mean
Squares for Treatment and Error are functions of the sample
sizes (b, the number of blocks), the true treatment effects
(1,…,t) and the variance of the random error terms (s2)
• By assigning all treatments to units within blocks, error
variance is (much) smaller for RBD than CRD (which
combines block variation&random error into error term)
• Relative Efficiency of RBD to CRD (how many times as
many replicates would be needed for CRD to have as
precise of estimates of treatment means as RBD does):
MSECR (b  1) MSB  b(t  1) MSE
RE ( RCB , CR) 

MSE RCB
(bt  1) MSE
Example - Caffeine and Endurance
•
•
•
•
Treatments: t=4 Doses of Caffeine: 0, 5, 9, 13 mg
Blocks: b=9 Well-conditioned cyclists
Response: yij=Minutes to exhaustion for cyclist j @ dose i
Data:
Dose \ Subject
0
5
9
13
1
36.05
42.47
51.50
37.55
2
52.47
85.15
65.00
59.30
3
56.55
63.20
73.10
79.12
4
45.20
52.10
64.40
58.33
5
35.25
66.20
57.45
70.54
6
66.38
73.25
76.49
69.47
7
40.57
44.50
40.55
46.48
8
57.15
57.17
66.47
66.35
9
28.34
35.05
33.17
36.20
Plot of Y versus Subject by Dose
90.00
80.00
70.00
Time to Exhaustion
60.00
50.00
0 mg
5 mg
9mg
40.00
13 mg
30.00
20.00
10.00
0.00
0
1
2
3
4
5
Cyclist
6
7
8
9
10
Example - Caffeine and Endurance
Subject\Dose
1
2
3
4
5
6
7
8
9
Dose Mean
Dose Dev
Squared Dev
TSS
0mg
36.05
52.47
56.55
45.20
35.25
66.38
40.57
57.15
28.34
46.44
-8.80
77.38
5mg
42.47
85.15
63.20
52.10
66.20
73.25
44.50
57.17
35.05
57.68
2.44
5.95
9mg
51.50
65.00
73.10
64.40
57.45
76.49
40.55
66.47
33.17
58.68
3.44
11.86
13mg
37.55
59.30
79.12
58.33
70.54
69.47
46.48
66.35
36.20
58.15
2.91
8.48
Subj MeanSubj Dev Sqr Dev
41.89
-13.34
178.07
65.48
10.24
104.93
67.99
12.76
162.71
55.01
-0.23
0.05
57.36
2.12
4.51
71.40
16.16
261.17
43.03
-12.21
149.12
61.79
6.55
42.88
33.19
-22.05
486.06
55.24
1389.50
103.68
7752.773
TSS  (36.05  55.24) 2    (36.20  55.24) 2  7752.773 dfTotal  4(9)  1  35

SSB  4(41.89  55.24)

   (33.19  55.24)   4(1389.50)  5558.00
SST  9 (46.44  55.24) 2    (58.15  55.24) 2  9(103.68)  933.12 dfT  4  1  3
2
2
df B  9  1  8
SSE  (36.05  41.89  46.44  55.24) 2    (36.20  33.19  58.15  55.24) 2 
 TSS  SST  SSB  7752.773  933.12  5558  1261.653 df E  (4  1)(9  1)  24
Example - Caffeine and Endurance
Source
Dose
Cyclist
Error
Total
df
3
8
24
35
SS
933.12
5558.00
1261.65
7752.77
MS
311.04
694.75
52.57
H 0 : No Caffeine Dose Effect (1     4  0)
H A : Difference s Exist Among Doses
MST 311.04
T .S . : Fobs 

 5.92
MSE 52.57
R.R.(  0.05) : Fobs  F.05,3, 24  3.01
P  value : P( F  5.92)  .0036 (From EXCEL)
Conclude that true means are not all equal
F
5.92
Example - Caffeine and Endurance
Tukey' s W : q.05, 4, 24
1
 3.90 W  3.90 52.57   9.43
9
Bonferroni ' s B : t.05 / 2,6, 24
Doses
5mg vs 0mg
9mg vs 0mg
13mg vs 0mg
9mg vs 5mg
13mg vs 5mg
13mg vs 9mg
2
 2.875 B  2.875 52.57   9.83
9
High Mean
57.6767
58.6811
58.1489
58.6811
58.1489
58.1489
Low Mean Difference Conclude
46.4400
11.2367
50
46.4400
12.2411
90
46.4400
11.7089
130
57.6767
1.0044
NSD
57.6767
0.4722
NSD
58.6811
-0.5322
NSD
Example - Caffeine and Endurance
Relative Efficiency of Randomized Block to Completely Randomized Design :
t  4 b  9 MSB  694.75 MSE  52.57
RE ( RCB , CR) 
(b  1) MSB  b(t  1) MSE 8(694.75)  9(3)(52.57) 6977.39


 3.79
(bt  1) MSE
(9(4)  1)(52.57)
1839.95
Would have needed 3.79 times as many cyclists per dose to have the
same precision on the estimates of mean endurance time.
• 9(3.79)  35 cyclists per dose
• 4(35) = 140 total cyclists
RBD -- Non-Normal Data
Friedman’s Test
• When data are non-normal, test is based on ranks
• Procedure to obtain test statistic:
– Rank the k treatments within each block (1=smallest,
k=largest) adjusting for ties
– Compute rank sums for treatments (Ti) across blocks
– H0: The k populations are identical (1=...=k)
– HA: Differences exist among the k group means
12
k
2
T .S . : Fr 
T
 3b(k  1)

i 1 i
bk (k  1)
R.R. : Fr  2 ,k 1
P  val : P(  2  Fr )
Example - Caffeine and Endurance
Subject\Dose
1
2
3
4
5
6
7
8
9
0mg
36.05
52.47
56.55
45.2
35.25
66.38
40.57
57.15
28.34
5mg
42.47
85.15
63.2
52.1
66.2
73.25
44.5
57.17
35.05
9mg
51.5
65
73.1
64.4
57.45
76.49
40.55
66.47
33.17
13mg
37.55
59.3
79.12
58.33
70.54
69.47
46.48
66.35
36.2
Ranks
Total
0mg
1
1
1
1
1
1
2
1
1
10
5mg
3
4
2
2
3
3
3
2
3
25
9mg
4
3
3
4
2
4
1
4
2
27
13mg
2
2
4
3
4
2
4
3
4
28
H 0 : No Dose Difference s
H a : Dose Difference s Exist


12
26856
2
2
T .S . : Fr 
(10)    (28)  3(9)( 4  1) 
 135  14.2
9(4)( 4  1)
180
R.R.(  0.05) : Fr   .205, 41  7.815
P - value : P(  2  14.2)  .0026 (From EXCEL)
Conclude Means (Medians) are not all equal
Latin Square Design
• Design used to compare t treatments when there are
two sources of extraneous variation (types of blocks),
each observed at t levels
• Best suited for analyses when t  10
• Classic Example: Car Tire Comparison
– Treatments: 4 Brands of tires (A,B,C,D)
– Extraneous Source 1: Car (1,2,3,4)
– Extraneous Source 2: Position (Driver Front, Passenger
Front, Driver Rear, Passenger Rear)
Car\Position
1
2
3
4
DF
A
B
C
D
PF
B
C
D
A
DR
C
D
A
B
PR
D
A
B
C
Latin Square Design - Model
• Model (t treatments, rows, columns, N=t2) :
yijk     k  b i   k   ijk
  Overall Mean
^
  y ...
^
 k  Effect of Treatment k  k  y .. k  y ...
bi
j
 Effect due to row i
^
b i  y i..  y ...
^
 Effect due to Column j  j  y . j .  y ...
 ijk  Random Error Term
Latin Square Design - ANOVA & F-Test
t

t
Total Sum of Squares : TSS   yijk  y ...

2
df  t 2  1
i 1 j 1
t

Treatment Sum of Squares SST  t  y .. k  y ...
k 1
t

Row Sum of Squares SSR  t  y i..  y ...


Column Sum of Squares SSC  t  y . j .  y ...
j 1
dfT  t  1
2
i 1
t

2
df R  t  1

2
df C  t  1
Error Sum of Squares SSE  TSS  SST  SSR  SSC
• H0: 1 = … = t = 0
df E  (t 2  1)  3(t  1)  (t  1)(t  2)
Ha: Not all k = 0
• TS: Fobs = MST/MSE = (SST/(t-1))/(SSE/((t-1)(t-2)))
• RR: Fobs  F, t-1, (t-1)(t-2)
Pairwise Comparison of Treatment Means
• Tukey’s Method- q in Studentized Range Table with  = (t-1)(t-2)
MSE
Wij  q (t , v)
t
Conclude  i   j if y i.  y j .  Wij


Tukey' s Confidence Interval : y i.  y j .  Wij
• Bonferroni’s Method - t-values from table on class
website with  = (t-1)(t-2) and C=t(t-1)/2
Bij  t / 2,C ,v
2MSE
t
Conclude  i   j if y i.  y j .  Bij


Bonferroni ' s Confidence Interval : y i.  y j .  Bij
Expected Mean Squares / Relative Efficiency
• Expected Mean Squares: As with CRD, the Expected Mean
Squares for Treatment and Error are functions of the sample
sizes (t, the number of blocks), the true treatment effects
(1,…,t) and the variance of the random error terms (s2)
• By assigning all treatments to units within blocks, error
variance is (much) smaller for LS than CRD (which
combines block variation&random error into error term)
• Relative Efficiency of LS to CRD (how many times as
many replicates would be needed for CRD to have as
precise of estimates of treatment means as LS does):
MSECR MSR  MSC  (t  1) MSE
RE ( LS , CR) 

MSE LS
(t  1) MSE
Power Approach to Sample Size Choice – R Code
When the means are not all equal, the F -statistic is non-central F :
t
F * ~ F  t  1, N  r ,   where  
 n 
i 1
i
i
  
where  
s2
t
When all sample sizes are equal:  
t
2
n   i   
i 1
n 
i 1
N
where  

i 1
t
The power of the test, when conducted at the significance level of  :

i
t
2
s2
i

Pr F *  F 1   ; t  1, N  t  | F * ~ F  t  1, N  t ,  
In R:
F 1   ; t  1, N  t   qf (1   , t  1, N  t )
Power = 1  b  1  pf  qf (1   , t  1, N  t ), t  1, N  t ,  
i
2-Way ANOVA
• 2 nominal or ordinal factors are believed to
be related to a quantitative response
• Additive Effects: The effects of the levels of
each factor do not depend on the levels of
the other factor.
• Interaction: The effects of levels of each
factor depend on the levels of the other
factor
• Notation: ij is the mean response when
factor A is at level i and Factor B at j
2-Way ANOVA - Model
yijk     i  b j  bij   ijk
i  1,..., a
j  1,..., b k  1,..., r
yijk  Measuremen t on k th unit receiving Factors A at level i, B at level j
  Overall Mean
 i  Effect of i th level of factor A
b j  Effect of j th level of factor B
bij  Interactio n effect whe n i th level of A and j th level of B are combined
 ijk  Random Error Terms
•Model depends on whether all levels of interest for a factor are
included in experiment:
• Fixed Effects: All levels of factors A and B included
• Random Effects: Subset of levels included for factors A and B
• Mixed Effects: One factor has all levels, other factor a subset
Fixed Effects Model
• Factor A: Effects are fixed constants and sum to 0
• Factor B: Effects are fixed constants and sum to 0
• Interaction: Effects are fixed constants and sum to 0
over all levels of factor B, for each level of factor A,
and vice versa
• Error Terms: Random Variables that are assumed to be
independent and normally distributed with mean 0,
variance s2
a
i  0,
i 1
b
bj  0
j 1
a
bij  0 j
i 1
2

b

0

i

~
N
0
,
s
 ij
ijk
 
b
j 1
Example - Thalidomide for AIDS
•
•
•
•
Response: 28-day weight gain in AIDS patients
Factor A: Drug: Thalidomide/Placebo
Factor B: TB Status of Patient: TB+/TBSubjects: 32 patients (16 TB+ and 16 TB-).
Random assignment of 8 from each group to
each drug). Data:
–
–
–
–
Thalidomide/TB+: 9,6,4.5,2,2.5,3,1,1.5
Thalidomide/TB-: 2.5,3.5,4,1,0.5,4,1.5,2
Placebo/TB+: 0,1,-1,-2,-3,-3,0.5,-2.5
Placebo/TB-: -0.5,0,2.5,0.5,-1.5,0,1,3.5
ANOVA Approach
• Total Variation (TSS) is partitioned into 4
components:
– Factor A: Variation in means among levels of A
– Factor B: Variation in means among levels of B
– Interaction: Variation in means among
combinations of levels of A and B that are not due
to A or B alone
– Error: Variation among subjects within the same
combinations of levels of A and B (Within SS)
Analysis of Variance
a
b
r

Total Variation : TSS   yijk  y ...

2
dfTotal  abr  1
i 1 j 1 k 1
a




Factor A Sum of Squares : SSA  br  y i..  y ...
2
df A  a  1
i 1
b
Factor B Sum of Squares : SSB  ar  y . j .  y ...
2
j 1
a
b

df B  b  1
Interactio n Sum of Squares : SSAB  r  y ij.  y i..  y . j .  y ...

2
i 1 j 1
a
b
r

Error Sum of Squares : SSE   yijk  y ij.
i 1 j 1 k 1
• TSS = SSA + SSB + SSAB + SSE
• dfTotal = dfA + dfB + dfAB + dfE

2
df E  ab(r  1)
df AB  (a  1)(b  1)
ANOVA Approach - Fixed Effects
Source
Factor A
Factor B
Interaction
Error
Total
df
a-1
b-1
(a-1)(b-1)
ab(r-1)
abr-1
SS
SSA
SSB
SSAB
SSE
TSS
MS
MSA=SSA/(a-1)
MSB=SSB/(b-1)
MSAB=SSAB/[(a-1)(b-1)]
MSE=SSE/[ab(r-1)]
F
FA=MSA/MSE
FB=MSB/MSE
FAB=MSAB/MSE
• Procedure:
• First test for interaction effects
• If interaction test not significant, test for Factor A and B effects
Test for Interactio n :
H 0 : b11  ...  bab  0
Test for Factor A
H 0 : 1  ...   a  0
Test for Factor B
H 0 : b1  ...  b b  0
H a : Not all bij  0
H a : Not all  i  0
H a : Not all b j  0
MSAB
TS : FAB 
MSE
RR : FAB  F ,( a 1)( b 1),ab( r 1)
MSA
MSB
TS : FA 
TS : FB 
MSE
MSE
RR : FA  F ,( a 1),ab( r 1) RR : FB  F (b 1),ab( r 1)
Example - Thalidomide for AIDS
Individual Patients
Group Means


tb
7.5

Negative

Positive
3.000

wtgain
5.0










2.5
0.0
-2.5
meanwg











2.000
1.000

0.000
-1.000
Placebo
Placebo
Thalidomide
Thalidomide
drug
drug
Report
WTGAIN
GROUP
TB+/Thalidomide
TB-/Thalidomide
TB+/Placebo
TB-/Placebo
Total

Mean
3.688
2.375
-1.250
.688
1.375
N
8
8
8
8
32
Std. Deviation
2.6984
1.3562
1.6036
1.6243
2.6027
Example - Thalidomide for AIDS
Tests of Between-Subjects Effects
Dependent Variable: WTGAIN
Source
Corrected Model
Intercept
DRUG
TB
DRUG * TB
Error
Total
Corrected Total
Type III Sum
of Squares
109.688 a
60.500
87.781
.781
21.125
100.313
270.500
210.000
df
3
1
1
1
1
28
32
31
Mean Square
36.563
60.500
87.781
.781
21.125
3.583
F
10.206
16.887
24.502
.218
5.897
Sig.
.000
.000
.000
.644
.022
a. R Squared = .522 (Adjus ted R Squared = .471)
• There is a significant Drug*TB interaction (FDT=5.897, P=.022)
• The Drug effect depends on TB status (and vice versa)
Comparing Main Effects (No Interaction)
• Tukey’s Method- q in Studentized Range Table with  = ab(r-1)
MSE
Wij  q (a, v)
br
A
MSE
Wij  q (b, v)
ar
B
Conclude :  i   j if y i..  y j ..  WijA


b i  b j if y .i.  y . j .  W B

Tukey' s CI : ( i   j ) : y i..  y j ..  WijA

ij
( b i  b j ) : y .i.  y . j .  WijB
• Bonferroni’s Method - t-values in Bonferroni table with  =ab (r-1)
Bij  t / 2,a ( a 1) / 2,v
A
2MSE
br
Bij  t / 2,b (b 1) / 2,v
B
Conclude :  i   j if y i..  y j ..  BijA


2MSE
ar
b i  b j if y .i.  y . j .  B B
Bonferroni ' s CI : (αi -α j ) : y i..  y j ..  BijA


ij
( b i -b j ) : y .i.  y . j .  BijB
Comparing Main Effects (Interaction)
• Tukey’s Method- q in Studentized Range Table with  = ab(r-1)
MSE
Wij  q (a, v)
r
A
Within k th level of Factor B, Conclude :  i   j if y ik .  y jk .  WijA


Tukey' s CI : ( i   j ) : y ik .  y jk .  WijA Similar for Factor B in A
• Bonferroni’s Method - t-values in Bonferroni table with  =ab (r-1)
Bij  t / 2,a ( a 1) / 2,v
A
2MSE
r
Within k th level of B, Conclude :  i   j if y ik .  y jk .  BijA


Bonferroni ' s CI : (αi -α j ) : y ik .  y jk .  BijA
Miscellaneous Topics
• 2-Factor ANOVA can be conducted in a Randomized
Block Design, where each block is made up of ab
experimental units. Analysis is direct extension of
RBD with 1-factor ANOVA
• Factorial Experiments can be conducted with any
number of factors. Higher order interactions can be
formed (for instance, the AB interaction effects may
differ for various levels of factor C).
• When experiments are not balanced, calculations are
immensely messier and you must use statistical
software packages for calculations
Unequal Sample Sizes
• When sample sizes are unequal, calculations and
parameter interpretations (especially marginal
ones) become messier
• Observational studies often have unequal sample
sizes due to availability of sampling units for
certain combinations of factor levels (villagers of
certain types in a rural study for instance)
• Experimental studies, even when planned with
equal sample sizes can end up unbalanced through
technical problems or “drop outs”
• Some conditions may be cheaper to measure than
others, and will have larger sample sizes
• Some situations have particular contrasts of
higher importance
Regression Approach - I
Sample Sizes: # of Cases when Factor A is at level i, B @ j: nij
b
ni   nij
j 1
a
n j   nij
i 1
a
nij
b
nT   nij
Yij    Yijk
i 1 j 1
k 1
a
b
a
b
i 1
j 1
i 1
j 1
Restrictions on Effects:   i   b j   b ij   b ij  0
  a  1   2  ...   a 1


nij
 ijk ~ N  0, s 2  (independent)
Model: Yijk     i  b j  b ij   ijk
 bb   b1  b 2  ...  b b 1
Y ij  
Yij 
b ib   b i1  b i 2  ...  b i ,b1
b aj   b 1 j  b 2 j  ...  b a 1 j
Regression Approach - II
Regression Model:
Yijk    1 X ijk 1  ...   a 1 X ijk ,a 1  b1 X ijka  ...  bb 1 X ijk ,a b  2 
b 11 X ijk1 X ijka  ...  b a 1,b1 X ijk ,a 1 X ijk ,a b2   ijk
 1 if case from level 1 of factor A

where: X ijk1   1 if case from level a of factor A
0
otherwise

1 if case from level a-1 of factor A

X ijk ,a 1   1 if case from level a of factor A
0
otherwise

where: X ijka
X ijk ,a b  2
 1 if case from level 1 of factor B

  1 if case from level b of factor B
0
otherwise

1 if case from level b-1 of factor B

  1 if case from level b of factor B
0
otherwise

Regression Approach – Example I
Writer Type (B)
Style (Factor A)
Poets (B1)
Conceptualists (A1)
Eliot
(Finders)
Cummings
Plath
Pound
Wilbur
n & Mean
n11=5
Experimentalists (A2) Bishop
(Seekers)
Moore
Williams
Lowell
Stevens
Frost
n & Mean
n21=6
Year at Peak
23
26
30
30
34
28.60
29
32
40
41
42
48
38.67
Novelists (B2)
Fitzgerald
Hemingway
Melville
Lawrence
Joyce
n12=5
James
Faulkner
Dickens
Woolf
Conrad
Twain
Hardy
n22=7
Year at Peak
29
30
32
35
40
33.20
38
39
41
45
47
50
51
44.43
Yijk    1 X ijk 1  b1 X ijk 2  b 11 X ijk 1 X ijk 2   ijk
 2  1
b 2   b1
b 12  b 21   b 11
b 22  b 11
X_ijk1
1
1
1
1
1
1
1
1
1
1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
X_ijk2
1
1
1
1
1
-1
-1
-1
-1
-1
1
1
1
1
1
1
-1
-1
-1
-1
-1
-1
-1
X1X2
1
1
1
1
1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
-1
1
1
1
1
1
1
1
Testing Strategies – Models Fit
Model 1: all Factor A, Factor B, and Interaction AB Effects
Model 2:all Factor A, Factor B Effects (Remove Interaction)
Model 3: all Factor B,Interaction AB Effects (Remove A)
Model 4:all Factor A,Interaction AB Effects (Remove B)
To test for Interaction Effects, Model 1 is Full Model, Model 2 is
Reduced dfNumerator=(a-1)(b-1) dfden=nT-ab
Testing for Factor A Effects, Full=Model 1, Reduced=Model 3
dfNumerator=(a-1) dfden=nT-ab
Testing for Factor B Effects, Full=Model 1, Reduced=Model 4
dfNumerator=(b-1) dfden=nT-ab
Regression Approach – Example - Continued
Yijk    1 X ijk 1  b1 X ijk 2  b 11 X ijk1 X ijk 2   ijk
Model 1: E Yijk     1 X ijk 1  b1 X ijk 2  b 11 X ijk1 X ijk 2
^
Y  36.22  5.32 X 1  2.59 X 2  0.29 X 1 X 2
Model 2: E Yijk     1 X ijk 1  b1 X ijk 2 Y  36.23  5.33 X 1  2.63 X 2
^
Model 3: E Yijk     b1 X ijk 2  b 11 X ijk1 X ijk 2 Y  36.90  2.77 X 2  0.47 X 1 X 2
^
Model 4: E Yijk     1 X ijk 1  b 11 X ijk 1 X ijk 2 Y  36.31  5.41X 1  0.62 X 1 X 2
^
ANOVA
Regression
Residual
Total
Model1
df
3
19
22
SS
827.91
557.05
1384.96
Model2
df
2
20
22
SS
826.01
558.95
1384.96
Model3
df
2
20
22
SS
188.77
1196.19
1384.96
Model4
df
2
20
22
SS
676.58
708.37
1384.96
Regression Approach – Example - Continued
H 0 : b 11  b 12  b 21  b 22  0
SSE  R   558.95 df E  R   20
H A : Interaction Exists
SSE  F   557.05 df E  F   19
 SSE  R   SSE  F    df E  R   df E  F   
TS : F 

 SSE  F  df E  F  
 558.95  557.05   20  19  
*
 0.065 RR : FAB
 F .95,1,19   4.381
557.05 19
*
AB
ANOVA
Regression
Residual
Total
Model1
df
3
19
22
SS
827.91
557.05
1384.96
Model2
df
2
20
22
SS
826.01
558.95
1384.96
Model3
df
2
20
22
SS
188.77
1196.19
1384.96
Model4
df
2
20
22
SS
676.58
708.37
1384.96
Regression Approach – Example - Continued
H 0 : 1   2  0 H A : Factor A Effects Exist:
SSE  R   1196.19 df E  R   20
1196.19  557.05   20  19  
F 
 21.80 RR : FA*  F .95,1,19   4.381
557.05 19
*
A
H 0 : b1  b 2  0 H A : Factor B Effects Exist:
SSE  R   708.37 df E  R   20
 708.37  557.05   20  19  
F 
 5.16 RR : FB*  F .95,1,19   4.381
557.05 19
*
B
ANOVA
Regression
Residual
Total
Model1
df
3
19
22
SS
827.91
557.05
1384.96
Model2
df
2
20
22
SS
826.01
558.95
1384.96
Model3
df
2
20
22
SS
188.77
1196.19
1384.96
Model4
df
2
20
22
SS
676.58
708.37
1384.96
Estimating Treatment and Factor Level Means/Contrasts
Treatment Means:
nij
Parameter: ij
^
Estimator:  ij 
Y
 
ijk
k 1
nij
MSE
nij
^
Estimated Standard Error: s  ij 
= Y ij 
Factor A Means:
b
Parameter: i =

j 1
b
ij
^
Estimator:  i 
b
Y
 
ij 
^
j 1
Estimated Standard Error: s  i 
b
MSE b 1

b 2 j 1 nij
Factor B Means:
a
Parameter:  j =
 ij
i 1
a
a
^
Estimator:   j 
Y
 
ij 
^
Estimated Standard Error: s   j 
i 1
a
Contrast or Linear Function of Factor A Means:
a
Parameter: LA   ci i
i 1
a
^
 
^
^
Estimator: L A   ci  i Estimated Standard Error: s L A 
i 1
Contrast or Linear Function of Factor B Means:
b
Parameter: LB   c j  j
j 1
b
^
MSE a 1

a 2 i 1 nij
MSE a 2 b 1
 ci 
b 2 i 1 j 1 nij
 
^
MSE b 2 a 1
cj 
a 2 j 1 i 1 nij
^
Estimator: L B   c j   j Estimated Standard Error: s L B 
j 1
Contrast or Linear Function of Treatment Means:
a
b
Parameter: LAB   cij ij
i 1 j 1
^
a
b
 
^
a
b
Estimator: L AB   cij Y ij  Estimated Standard Error: s L AB  MSE 
i 1 j 1
i 1 j 1
cij2
nij
Standard Error Multipliers
Single Comparisons:
t 1   / 2  ; nT  ab 
General Multiple Comparisons of Treatment (Cell) Means :
Scheffe: S 
Bonferroni:
 ab  1 F 1   ; ab  1, nT  ab 
B  t 1    2 g   , nT  ab 
Tukey (all pairs of treatment means): T 
1
q 1   ; ab, nT  ab 
2
General Multiple Comparisons of Factor Level Means :
 a  1 F 1   ; a  1, nT  ab  Factor B:
Factor A or Factor B : B  t 1    2 g   , nT  ab 
Scheffe: Factor A: S A 
Bonferroni:
Tukey: Factor A: TA 
1
q 1   ; a, nT  ab 
2
Factor B: TB 
SB 
 b  1 F 1   ; b  1, nT  ab 
1
q 1   ; b, nT  ab 
2
Creative Life Cycles – Comparing Treatment Means
Comparing all 4 Treatment Means(athough no interaction was present):
 
MSE
29.32

 2.42
n11
5
Y 12  33.20 n12  5 s Y 12 
 
MSE
29.32

 2.21
n21
6
Y 22  44.43 n22  7 s Y 22 
Y 11  28.60 n11  5 s Y 11 
Y 21  38.67 n21  6 s Y 21 
T
 
 
MSE
29.32

 2.42
n12
5
MSE
29.32

 2.05
n22
7
 1
1
1
1 
q  0.95, 4, 23  4  19  
 3.977   2.812 s Y ij  Y i ' j '  MSE   
2
2
 nij ni ' j ' 




1 1
Y 11  Y 12  28.60  33.20  4.60 s Y 11  Y 12  29.32     3.43
5 5
HSD  2.812(3.43)  9.65


HSD  2.812(3.28)  9.22


HSD  2.812(3.17)  8.92


1 1
Y 11  Y 21  28.60  38.67  10.07 s Y 11  Y 21  29.32     3.28
5 6
1 1
Y 11  Y 22  28.60  44.43  15.83 s Y 11  Y 22  29.32     3.17
5 7
1 1
Y 12  Y 21  33.20  38.67  5.47 s Y 12  Y 21  29.32     3.28
5 6




1 1
Y 12  Y 22  33.20  44.43  11.23 s Y 12  Y 22  29.32     3.17
5 7
HSD  2.812(3.28)  9.22
HSD  2.812(3.17)  8.92
1 1
Y 21  Y 22  38.67  44.43  5.76 s Y 21  Y 22  29.32     3.01
HSD  2.812(3.01)  8.47
6 7
Conceptualists/Poets Conceptualists/Novelists Experimentalists/Poets Experimentalists/Novelists
Creative Life Cycles – Comparing Factor Level Means
Factor A (Style):
b
^
 1 
Y
1 j
j 1
b

Y 11  Y 12 28.60  33.20

 30.90
2
2

Y 21  Y 22 38.67  44.43

 41.55
2
2
b
^
 2 
^
Y
2 j
j 1
b
^
 1   2  30.9  41.55  10.65

^

^
29.32 2
1 1 1 1
1  (1) 2        10.402  3.23

2
2
5 5 6 7
s  1   2 
t  0.975, 23  4  19   2.093
95% CI for 1  2 (Conceptualists - Experimentalists): -10.7  (2.093)(3.23)  10.65  6.75 
Factor B (Writer Type):
a
^
 1 
Y
i1
i 1
a

Y 11  Y 21 28.60  38.67

 33.635
2
2

Y 12  Y 22 33.20  44.43

 38.815
2
2
b
^
 2 
^
Y
i 2
j 1
b
^
 1   2  33.635  38.815  5.18

^
^

s  1   2 
29.32 2
1 1 1 1
1  (1) 2        10.402  3.23

2
2
5 5 6 7
t  0.975, 23  4  19   2.093
95% CI for 1  2 (Poets - Novelists): - 5.18  (2.093)(3.23)  5.18  6.75 
 11.93,1.57 
 17.40, 3.90 
1-Way Random Effects Model
Treatment Levels in Experiment are Sample from a Population of Levels
Effects are Random Variables (Not Fixed Constants)
Yij     i   ij i  1,..., t ; j  1,..., n
 i ~ NID  0, s 2   ij ~ NID  0, s 2 
 i  ,  ij  independent
ANOVA obtained as in Fixed Effects Model:
t

SST  n Y i  Y 
i 1
t
n


SSE   Yij  Y i
i 1 j 1
2

2
 SST 
2
2
dfT  t  1 E MST   E 
  s   ns 
 t 1 
 SSE 
2
df E  t  n  1 E MSE  E 
  s
 t  n  1 
^
Estimating Population Mean:   Y 
^
SE    
 
MST
tn
^
1   100% CI:   t /2;t 1
MST  MSE
n
MST
Testing for Trt Effects: H 0 : s 2  0 H A : s 2  0 TS : F 
RR : F  F ;t l ,t  n 1
MSE
^ 2
^ 2
Point Estimates of Variance Components: s   MSE s  
MST
tn
Randomized Complete Block Designs – Subjects as Blocks
Yij     i  b j   ij  i  b j   ij
t

i 1
i
 0 E ( ij )  0
i  1,..., t ; j  1,..., b
V ( ij )  s 2
Random Blocks: b j ~ N  0, s b2 
Note:
1
1
Y

...

Y

   1  b1  11   ...     1  b b  1b   
 11
1b 
b
b
   1  b    1
Y 1 
Y 2     2  b    2

 


 Y 1  Y 2    1  b    1     2  b    2  1   2    1   2
• Test for differences among treatment effects:
• H0: 1  ...  t  0
HA: Not all i = 0
• TS: F = MST/MSE
RR: F ≥ F;t-1,(t-1)(b-1)

Mixed Effects Models
• Assume:
– Factor A Fixed (All levels of interest in study)
1  2  …   0
– Factor B Random (Sample of levels used in study)
bj ~ N(0,sb2)
(Independent)
• AB Interaction terms Random
b)ij ~ N(0,sab2 (Independent)
• Analysis of Variance is computed exactly as in
Fixed Effects case (Sums of Squares, df’s, MS’s)
• Error terms for tests change (See next slide).
Expected Mean Squares for 2-Way ANOVA
Mean Square
df
Fixed Model
Random Model
Mixed Model
(A Fixed, B Fixed) (A Random, B Random) (A Fixed, B Random)
a
MSA
a -1
s 2+
bn 
i 1
a
2
i
2
s 2  ns b
 bns 2
a 1
2
s 2  ns b

bn  i2
i 1
a 1
b
MSB
b -1
s 2+
an b j2
j 1
b 1
a
MSAB
MSE
 a  1 b  1
ab  n  1
s 2+
2
s 2  ns b
 ans b2
b
n b ij
i 1 j 1
2
 a  1 b  1
s2
2
s 2  ns b
 ans b2
2
s 2  ns b
s2
2
s 2  ns b
s2
ANOVA Approach – Mixed Effects
Source
Factor A
Factor B
Interaction
Error
Total
df
a-1
b-1
(a-1)(b-1)
ab(r-1)
abr-1
SS
SSA
SSB
SSAB
SSE
TSS
MS
MSA=SSA/(a-1)
MSB=SSB/(b-1)
MSAB=SSAB/[(a-1)(b-1)]
MSE=SSE/[ab(n-1)]
F
FA=MSA/MSAB
FB=MSB/MSAB
FAB=MSAB/MSE
• Procedure:
• First test for interaction effects
• If interaction test not significant, test for Factor A and B effects
Test for Interactio n :
Test for Factor A
Test for Factor B
2
H 0 : s ab
0
H 0 : 1  ...   a  0
H 0 : s b2  0
2
H a : s ab
0
H a : Not all  i  0
H a : s b2  0
MSAB
TS : FAB 
MSE
RR : FAB  F ,( a 1)( b 1),ab( r 1)
MSA
MSB
TS : FA 
TS : FB 
MSAB
MSAB
RR : FA  F ,( a 1),( a 1)( b 1) RR : FB  F ,(b 1),( a 1)( b 1)
Comparing Main Effects for A (No Interaction)
• Tukey’s Method- q in Studentized Range Table with  = (a-1)(b-1)
MSAB
Wij  q (a, v)
br
A
Conclude :  i   j if y i..  y j ..  WijA


Tukey' s CI : ( i   j ) : y i..  y j ..  WijA
• Bonferroni’s Method - t-values in Bonferroni table with  = (a-1)(b-1)
Bij  t / 2,a ( a 1) / 2,v
A
2MSAB
br
Conclude :  i   j if y i..  y j ..  BijA


Bonferroni ' s CI : (αi -α j ) : y i..  y j ..  B ijA
Random Effects Models
• Assume:
– Factor A Random (Sample of levels used in study)
i ~ N(0,sa2)
(Independent)
– Factor B Random (Sample of levels used in study)
bj ~ N(0,sb2)
(Independent)
• AB Interaction terms Random
b)ij ~ N(0,sab2 (Independent)
• Analysis of Variance is computed exactly as in
Fixed Effects case (Sums of Squares, df’s, MS’s)
• Error terms for tests change (See next slide).
ANOVA Approach – Mixed Effects
Source
Factor A
Factor B
Interaction
Error
Total
df
a-1
b-1
(a-1)(b-1)
ab(n-1)
abn-1
SS
SSA
SSB
SSAB
SSE
TSS
MS
MSA=SSA/(a-1)
MSB=SSB/(b-1)
MSAB=SSAB/[(a-1)(b-1)]
MSE=SSE/[ab(n-1)]
F
FA=MSA/MSAB
FB=MSB/MSAB
FAB=MSAB/MSE
• Procedure:
• First test for interaction effects
• If interaction test not significant, test for Factor A and B effects
Test for Interactio n :
2
H 0 : s ab
0
2
H a : s ab
0
MSAB
TS : FAB 
MSE
RR : FAB  F ,( a 1)( b 1),ab( r 1)
Test for Factor A
Test for Factor B
H 0 : s a2  0
H 0 : s b2  0
H a : s a2  0
H a : s b2  0
MSA
MSB
TS : FA 
TS : FB 
MSAB
MSAB
RR : FA  F ,( a 1),( a 1)( b 1) RR : FB  F (b 1),( a 1)( b 1)
Nested Designs
• Designs where levels of one factor are nested (as
opposed to crossed) wrt other factor
• Examples Include:
–
–
–
–
–
Classrooms nested within schools
Litters nested within Feed Varieties
Hair swatches nested within shampoo types
Swamps of varying sizes (e.g. large, medium, small)
Restaurants nested within national chains
Nested Design - Model
Yijk     i  b j (i )   ijk
i  1,...a j  1,..., bi k  1,..., r
where :
Yijk  Response for k th rep of Factor A at i th level, B at jth level within A
  Overall Mean
 i  Effect of i th level of A (Fixed or Random)
b j (i )  Effect of jth level of B within i th level of A (Fixed or Random)
 ijk  Random error term for k th rep when A is at i, B is at j(i)
Nested Design - ANOVA
Total Variation :
bi
a

r
TSS   Yijk  Y ...

a
dfTotal  r  bi  1
2
i 1 j 1 k 1
i 1
Factor A :

a
SSA  r  bi Y i..  Y ...

2
df A  a  1
i 1
Factor B Nested Within A
bi
a

SSB ( A)  r  Y ij.  Y i..
i 1 j 1

2
a
df B ( A)   bi  a
i 1
Error :
a
bi
r

SSE   Yijk  Y ij.
i 1 j 1 k 1

2
a
df E  (r  1) bi
i 1
Factors A and B Fixed
a
i  0
i 1
2


b

0
i

1
,...,
a

~
N
0
,
s
 j (i )
ijk
bi
j 1
Tests for Difference s Among Factor A Effects
H 0 : 1  ...   a  0 H A : Not all  i  0
MSA
Test Statistic : FA 
P - value : PF  FA 
MSE
Rejection Region : FA  F ,a 1,( r 1) b
i
Tests for Difference s Among Factor B Effects
H 0 : b j (i )  0 i, j H A : Not all b j (i )  0
MSB ( A)
P - value : P F  FB ( A) 
MSE
Rejection Region : FB ( A)  F , b  a ,( r 1) b
i
i
Test Statistic : FB ( A) 
Comparing Main Effects for A
• Tukey’s Method- q in Studentized Range Table with  = (r-1)Sbi
MSE  1
1 
Wij  q (a, v)


2  rbi rb j 
A
Conclude :  i   j if y i..  y j ..  WijA


Tukey' s CI : ( i   j ) : y i..  y j ..  WijA
• Bonferroni’s Method - t-values in Bonferroni table with  = (r-1)Sbi
B ij  t / 2,a ( a 1) / 2,v
A
 1

1

MSE 

 rb rb 
j 
 i
Conclude :  i   j if y i..  y j ..  B ijA


Bonferroni ' s CI : (αi -α j ) : y i..  y j ..  B ijA
Comparing Effects for Factor B Within A
• Tukey’s Method- q in Studentized Range Table with  = (r-1)Sbi
B
Wij ( k )
MSE
 q (bk , v)
r
Conclude : b i ( k )  b j ( k ) if y ki.  y kj .  WijB( k )


Tukey' s CI : ( b i ( k )  b j ( k ) ) : y ki.  y kj .  WijB( k )
• Bonferroni’s Method - t-values in Bonferroni table with  = (r-1)Sbi
Bij ( k )  t / 2,bk (bk 1) / 2,v
B
2
MSE  
r
Conclude : b i ( k )  b j ( k ) if y ki.  y kj .  BijB( k )


Bonferroni ' s CI : ( b i ( k ) -b j ( k ) ) : y ki.  y kj .  BijB( k )
Factor A Fixed and B Random
a

i 1
i

 0 b j (i ) ~ N 0, s b2

 ijk ~ N 0, s 2 
Tests for Difference s Among Factor A Effects
H 0 : 1  ...   a  0 H A : Not all  i  0
MSA
Test Statistic : FA 
P - value : PF  FA 
MSB ( A)
Rejection Region : FA  F ,a 1, b  a
i
Tests for Difference s Among Factor B Effects
H 0 : s b2  0 i, j
H A : s b2  0
MSB ( A)
P - value : P F  FB ( A) 
MSE
Rejection Region : FB ( A)  F , b  a ,( r 1) b
i
i
Test Statistic : FB ( A) 
Comparing Main Effects for A (B Random)
• Tukey’s Method- q in Studentized Range Table with  = Sbi-a
MSB ( A)  1
1 
Wij  q (a, v)



2
rb
rb
i
j


A
Conclude :  i   j if y i..  y j ..  WijA


Tukey' s CI : ( i   j ) : y i..  y j ..  WijA
• Bonferroni’s Method - t-values in Bonferroni table with  = Sbi-a
B ij  t / 2,a ( a 1) / 2,v
A
 1

1

MSB ( A)

 rb rb 
j 
 i
Conclude :  i   j if y i..  y j ..  B ijA


Bonferroni ' s CI : (αi -α j ) : y i..  y j ..  B ijA
Factors A and B Random
 i ~ N 0, s a2  b j (i ) ~ N 0, s b2   ijk ~ N 0, s 2 
Tests for Difference s Among Factor A Effects
H 0 : s a2  0 H A : s a2  0
MSA
Test Statistic : FA 
P - value : PF  FA 
MSB ( A)
Rejection Region : FA  F ,a 1, b  a
i
Tests for Difference s Among Factor B Effects
H 0 : s b2  0 i, j
H A : s b2  0
MSB ( A)
P - value : P F  FB ( A) 
MSE
Rejection Region : FB ( A)  F , b  a ,( r 1) b
i
i
Test Statistic : FB ( A) 
Elements of Split-Plot Designs
• Split-Plot Experiment: Factorial design with at least 2
factors, where experimental units wrt factors differ in
“size” or “observational points”.
• Whole plot: Largest experimental unit
• Whole Plot Factor: Factor that has levels assigned to
whole plots. Can be extended to 2 or more factors
• Subplot: Experimental units that the whole plot is split
into (where observations are made)
• Subplot Factor: Factor that has levels assigned to
subplots
• Blocks: Aggregates of whole plots that receive all levels
of whole plot factor
Split Plot Design
Block 1
A=1, B=1
A=1, B=2
A=1, B=3
A=1, B=4
A=2, B=1
A=2, B=2
A=2, B=3
A=2, B=4
A=3, B=1
A=3, B=2
A=3, B=3
A=3, B=4
Block 2
A=1, B=1
A=1, B=2
A=1, B=3
A=1, B=4
A=2, B=1
A=2, B=2
A=2, B=3
A=2, B=4
A=3, B=1
A=3, B=2
A=3, B=3
A=3, B=4
Block 3
A=1, B=1
A=1, B=2
A=1, B=3
A=1, B=4
A=2, B=1
A=2, B=2
A=2, B=3
A=2, B=4
A=3, B=1
A=3, B=2
A=3, B=3
A=3, B=4
Block 4
A=1, B=1
A=1, B=2
A=1, B=3
A=1, B=4
A=2, B=1
A=2, B=2
A=2, B=3
A=2, B=4
A=3, B=1
A=3, B=2
A=3, B=3
A=3, B=4
Note: Within each block we would assign at random the 3
levels of A to the whole plots and the 4 levels of B to the
subplots within whole plots
Examples
• Agriculture: Varieties of a crop or gas may need to be
grown in large areas, while varieties of fertilizer or
varying growth periods may be observed in subsets of
the area.
• Engineering: May need long heating periods for a
process and may be able to compare several
formulations of a by-product within each level of the
heating factor.
• Behavioral Sciences: Many studies involve repeated
measurements on the same subjects and are analyzed as
a split-plot (See Repeated Measures lecture)
Design Structure
• Blocks: b groups of experimental units to be exposed
to all combinations of whole plot and subplot factors
• Whole plots: a experimental units to which the whole
plot factor levels will be assigned to at random within
blocks
• Subplots: c subunits within whole plots to which the
subplot factor levels will be assigned to at random.
• Fully balanced experiment will have n=abc
observations
Data Elements (Fixed Factors, Random Blocks)
• Yijk: Observation from wpt i, block j, and spt k
•  : Overall mean level
•  i : Effect of ith level of whole plot factor (Fixed)
• bj: Effect of jth block (Random)
• (ab )ij : Random error corresponding to whole plot
elements in block j where wpt i is applied
•  k: Effect of kth level of subplot factor (Fixed)
•  )ik: Interaction btwn wpt i and spt k
• (bc )jk: Interaction btwn block j and spt k (often set to 0)
•  ijk: Random Error= (bc )jk+ (abc )ijk
• Note that if block/spt interaction is assumed to be 0, 
represents the block/spt within wpt interaction
Model and Common Assumptions
• Yijk =  +  i + b j + (ab )ij +  k + ( )ik +  ijk
a

i 1
i
0
b j ~ NID (0, s b2 )
2
( ab) ij ~ NID (0, s ab
)
c

k 1
k
0
a
 ( )
i 1
c
ik
 ( ) ik  0
k 1
 ijk ~ NID (0.s 2 )
COV (b j , ( ab) ij )  COV (b j ,  ijk )  COV (( ab) ij ,  ijk )  0
Tests for Fixed Effects
Whole Plot Trt Effects : H 0 : 1     a  0
Test Statistic : FW P 
MSW P
MS BLOCK *W P
PW P  P ( F  FW P | F ~ Fa 1.( a 1)( b 1) )
Subplot Trt Effects : H 0 :  1     c  0
Test Statistic : FSP 
MS SP
MS ERROR
PSP  P ( F  FSP | F ~ Fc 1.a ( b 1)( c 1) )
WP  SP Interactio n : H 0 : ( ) ik  0 i, k
Test Statistic : FW P SP
MSW P SP

MS ERROR
PW P SP  P ( F  FW P SP | F ~ F( a 1)( c 1).a ( b 1)( c 1) )
Comparing Factor Levels
Whole Plot Factor Levels :
95% CI for  i   i '  :
Y
i ..

 Y i '..  t
2 MS BLOCK W P
bc
Sub Plot Factor Levels :
Y

2 MS ERROR
ab
Sub Plot Effects Within same whole plot (Interacti on) :
95% CI for ( k   k ' ) :
.. k
 Y .. k '  t
95% CI for  k   k '   ( )ik  ( ) ik '  :
Y
i .k

 Y i .k '  t
2 MS ERROR
b
Whole Plot Effects within same sub plot (Interacti on) :
Y
i .k

 Y i '.k  t
2MS BLOCK W P  (c  1) MS ERROR 
bc
2

(c  1) MS ERROR  MS BLOCK W P 

 (c  1) MS ERROR 2 MS BLOCK W P 2 

^

 a (b  1)(c  1)

(a  1)(b  1) 
(df given below)
Repeated Measures Designs
• a Treatments/Conditions to compare
• N subjects to be included in study (each subject
will receive only one treatment)
– n subjects receive trt i: an = N
• t time periods of data will be obtained
• Effects of trt, time and trtxtime interaction of
primary interest.
– Between Subject Factor: Treatment
– Within Subject Factors: Time, TrtxTime
Model
Yijk     i  b j (i )  t k  (t ) ik   ijk
  overall mean
 i  effect of trt i
a

i 1
i
0

b j (i )  effect of j th subject in trt i b j (i ) ~ NID 0, s b2
t k  effect of k time period
th
t
t
k 1
k

0
(t ) ik  interactio n between tr t i and time k
 ijk  random error term  ijk ~ NID 0, s 2 
a
 (t )
i 1
t
ik
  (t ) ik  0
k 1
Note the random error term is actually the interaction between
subjects (within treatments) and time
Tests for Fixed Effects
Treatment Effects : H 0 : 1     a  0
Test Statistic : FTRTS 
MS TRTS
MS SUBJECTS (TRTS )
PTRTS  P ( F  FTRTS | F ~ Fa 1, a ( n 1) )
Time Effects : H 0 : t 1    t t  0
Test Statistic : FTIME 
MS TIME
MS ERROR
PTIME  P ( F  FTIME | F ~ Ft 1, a ( n 1)( t 1) )
Treatment/ Time Interactio n : H 0 : (t ) ik  0 i, k
Test Statistic : FTRT TIME 
MS TRT TIME
MS ERROR
PTRT TIME  P ( F  FTRT TIME | F ~ F( a 1)( t 1), a ( n 1)( t 1) )
Comparing Factor Levels
Comparing Treatment Levels :
95% CI for  i   i ' :
Y
i ..

 Y i '..  t
2 MS SUBJECTS (TRTS )
nt
Comparing Time Levels :
Y
 t
2 MS ERROR
95% CI for t k  t k ' :
.. k  Y .. k '
an
Comparing Treatment Levels Within Time Levels :
Y
i .k

 Y i '.k  t
2MS SUBJECTS (TRTS )  (t  1) MS ERROR 
nt
with approximat e df :
^

(t  1)MS
ERROR  MS SUBJECT (TRT )


2
 (t  1) MS ERROR 2 MS SUBJECT (TRT )


a (n  1)
 a(n  1)(t  1)

2

