Notes on Expected Mean Squares
• There are two approaches or models for random effects: The “restricted” model and
the “unrestricted” model. We’ll focus on the “unrestricted” model, which is the
default in Minitab, and which makes the most sense. The restricted model basically
incorporates the assumption that a random effect-by-fixed effect interaction sums to 0
over the levels of the fixed effect. I cannot imagine why, in real life, one would assume
this to be so.
• The general idea is that each Mean Square arising from the model (appearing in the
ANOVA table) has an expected value of the form “part 1 + part 2” where “part 2”
is 0 if the associated null hypothesis is true.
One looks to find another term in the model whose expected mean square is “part 1”.
One then uses this other term as the denominator of the F-statistic.
Then it follows that
F ≈
part 1 + part 2
part 1
F ≈
part 1 + 0
=1
part 1
and if H0 is true then
and if H0 is false then F tends to be greater than 1.
• There is a set of not-too-complicated rules for determining the expected mean squares
(for both the unrestricted and restricted models) as long as the design is balanced.
• These are not-too-complicated but a bit awkward to state. Minitab knows how to
work out the expected mean squares and will do so if asked.
• Most of the time you won’t actually need to use these rules, since Minitab does the
work for you. However my firm belief is that you are less likely to make mistakes
over-all if you are capable of handling these rules yourself.
In order to state the rules, we first need to establish an elephant-ful of notation:
- Let U denote any “effect” — main effect or interaction. Think of U as a sequence
of letters such as ABC where A, B, and C represent main effects. (Of course U
could consist of a single letter such as A.)
- If a main effect C is nested within an effect U , then we represent C as U C . E.g
if C is nested within AB (i.e. within each combination of levels of A and B, then
we represent C as ABC; note that there will be no AC or BC (and of course no
actual C) terms in the model. A bit of useful jargon: In such cases, i.e. where
ABC appears in the model to represent the effect C which is nested within AB,
we say that A and B appear in ABC “by convention”.
- Let the number of levels of each main effect A, B, C, . . . , be denoted by the
corresponding lower case letter, a, b, c, . . . . Let the number of replicates be r .
- With each effect U in the model there is associated a non-negative quantity
denoted by σU2 . If U is a random effect then σU2 is indeed its variance. If U
is a fixed effect then σU2 is just a “quadratic form” (sum of squares) involving
the parameters associated with that effect. In any case the effect U is actually
present if and only if σU2 6= 0 .
- The variance of the error term in the model is denoted by σ 2 .
• THE RULES:
1. The expected mean square for U always contains a σ 2 and a σU2 .
2. It also contains a σ 2 term for any interaction in which all all the letters of U
appear and at least one letter represents a random effect. (For the restricted
model the proviso is that all other letters must represent random effects. Note
however that this allows sequences of letters used to represent nested random
effects, even if some of the individual letters on their own represent fixed effects.)
3. The coefficient of σ2 is always 1.
4. The coefficient of the σ 2 for any other term is r times the product of all the lower
case letters a, b, c, . . . whose corresponding upper case letters DO NOT appear
in that term.
• E.g. # 1: A fixed, B random, crossed with each other, so the model is
Y = mean + A + B + AB + Error
Source
A
B
AB
Error
Expected Mean Square
2 + rσ 2
σ2 + rbσA
AB
2 + rσ 2
σ2 + raσB
AB
2
σ2 + rσAB
σ2
From this we see that the proper denominator to test for the A and B effects is the
mean square for AB, and the proper denominator to test for the AB effect is the
mean square for Error.
• E.g. # 2: A fixed, B random, crossed with each other, C random and nested within
AB, The model is
Y = mean + A + B + AB + ABC + Error
(so A and B appear in ABC by convention).
Source
A
B
AB
ABC
Error
Expected Mean Square
2 + rcσ 2 + rσ 2
σ2 + rbcσA
AB
ABC
2 + rcσ 2 + rσ 2
σ2 + racσB
AB
ABC
2 + rσ 2
σ2 + rcσAB
ABC
2
σ2 + rσABC
2
σ
2
The appropriate denominator to test for an A effect is M SAB; for a B effect it is
M SABC; for an AB effect it is also M SABC, and for an ABC effect it is M SError.
• E.g. # 3: A fixed, B and C random, all crossed, so the model is
Y = mean + A + B + AB + C + AC + BC + ABC + Error
Source
A
B
AB
C
AC
BC
ABC
Error
Expected Mean Square
2 + rcσ 2 + rbσ 2 + rσ 2
σ2 + rbcσA
AB
AC
ABC
2
2 + rcσ 2 + raσ 2 + rσ 2
σ + racσB
AB
BC
ABC
2 + rσ 2
σ2 + rcσAB
ABC
2 + raσ 2 + rσ 2
σ2 + rabσC
BC
ABC
2 + rσ 2
σ2 + rbσAC
ABC
2 + rσ 2
σ2 + raσBC
ABC
2
σ2 + rσABC
σ2
This seems to be a perfectly sensible design, and yet there is no appropriate denominator to test for any of the main effects. In particular you can’t test for the fixed
effect A. This is weird. Minitab will (these days) produce a “synthetic F-test” in such
situations; but such synthetic tests are approximations and should be interpreted with
even more than the usual amount of caution.
• E.g. # 4: (a weird one; I’m not completely sure that I’ve got this right or that it even
makes sense . . . . . . )
A fixed, B random, C random and nested within A, A crossed with B and B crossed
with C. The model is
Y = mean + A + B + AB + AC + ABC + Error
Note that since C is nested within A we represent C as AC, and then B crossed with
C is represented as B × AC which becomes ABC .
(E.g. A is different measuring devices; B is technicians; C is specimens to be measured;
each technician measures each specimen r times and each technician uses all devices;
however the specimens have to be prepared in a particular way for each device and
so a specimen to be measured by device number 1 cannot be measured by device
number 2, etc.) (O.K., so it’s a bit far-fetched, but I’ll bet you that weirder designs
have happened in real life!)
Source
A
B
AB
AC
ABC
Error
Expected Mean Square
2 + rcσ 2 + rbσ 2 + rσ 2
σ2 + rbcσA
AB
AC
ABC
2
2 + rcσ 2 + rσ 2
σ + racσB
AB
ABC
2 + rσ 2
σ2 + rcσAB
ABC
2
2 + rσ 2
σ + rbσAC
ABC
2
σ2 + rσABC
2
σ
3
There is no appropriate denominator to test for an A effect, which is probably what
you’re really interested in — but that’s not too surprizing since it’s such a weird
design.
• Just for the sake of completeness, let’s run through the foregoing examples under the
restricted model.
E.g. # 1 (restricted model):
Source
A
B
AB
Error
Expected Mean Square
2 + rσ 2
σ2 + rbσA
AB
2
σ2 + raσB
2
σ2 + rσAB
σ2
Under the restricted model we see that the right denominator to test for the A effect
is the mean square for AB, and the right denominator to test for the B and AB effects
is the mean square for Error.
E.g. # 2 (restricted model):
Source
A
B
AB
ABC
Error
Expected Mean Square
2 + rcσ 2 + rσ 2
σ2 + rbcσA
AB
ABC
2 + rσ 2
2
σ + racσB
ABC
2 + rσ 2
σ2 + rcσAB
ABC
2
2
σ + rσABC
σ2
Under the restricted model the appropriate denominator to test for an A effect is
M SAB; for a B effect it is M SABC; for an AB effect it is also M SABC, and for an
ABC effect it is M SError.
E.g. # 3 (restricted model):
Source
A
B
AB
C
AC
BC
ABC
Error
Expected Mean Square
2 + rcσ 2 + rbσ 2 + rσ 2
σ2 + rbcσA
AB
AC
ABC
2
2 + raσ 2
σ + racσB
BC
2 + rσ 2
σ2 + rcσAB
ABC
2
2 + raσ 2
σ + rabσC
BC
2 + rσ 2
σ2 + rbσAC
ABC
2
σ2 + raσBC
2
σ2 + rσABC
2
σ
Under the restricted model there is no appropriate denominator to test for an A effect
(but B and C are O.K.)
4
E.g. # 4 (restricted model):
Source
A
B
AB
AC
ABC
Error
Expected Mean Square
2 + rcσ 2 + rbσ 2 + rσ 2
σ2 + rbcσA
AB
AC
ABC
2
2 + rσ 2
σ + racσB
ABC
2 + rσ 2
σ2 + rcσAB
ABC
2
2 + rσ 2
σ + rbσAC
ABC
2
σ2 + rσABC
2
σ
Again under the restricted model there is no appropriate denominator to test for an
A effect.
• We will now display Minitab’s version of all this. Minitab doesn’t work in the abstract
so it had to be given some data. The data used to generate these analyses were white
noise, so don’t expect to see anything sensible in the actual tests. (Things should
always turn out to be “non-significant”.) Specific numbers had to be chosen for the
numbers of levels of each factor, and for the number of replicates in each cell. The
numbers used were a = 3, b = 5, c = 4, and r = 3.
Note 1: Minitab writes the EMS terms corresponding to fixed effects as “Q[...]”
and those corresponding to random effects (variances) simply as “(...)” where the
... bits are the numbers corresponding to terms in the model. Also the coefficients
of the “Q[...]” terms are omitted under the unrestricted model (although they are
put in for the restricted model). Minitab (misleadingly, I think) indicates a sort of
dependence on interactions between fixed effects under the unrestricted model. E.g.
Minitab’s MS for A in the model
Y = mean + A + B + AB + C + AC + BC + ABC + Error
where A and B are both fixed effects and C is random, a, b, c and r as above, is
(8) + 3(7) + 15(5) + Q[1,4]
where [1 refers to A, 4] refers to AB, ((5) refers to AC, (7) refers to ABC, and (8)
refers to Error. The rules tell us that the MS for A is
2
2
2
σ2 + rbcσA
+ rbσAC
+ rσABC
with no allusion to AB. (There would be an AB term if B were random.) There’s
always got to be something weird going on to confuse you.
Note 2: The following Minitab output was produced with an “older” version of
Minitab. These days, where it says “ * No exact F-test can be calculated.”,
Minitab would actually calculate the approximated or “synthetic” F-test.
. . . . . . (continued over page)
5
First the unrestricted model:
MTB > # E.g. 1 (unrestricted model):
MTB > ANOVA ’Y’ = A B A*B;
SUBC>
Random B;
SUBC>
EMS.
Source
A
B
A*B
Error
Total
DF
2
4
8
165
179
Source
Variance Error Expected Mean Square
component term (using unrestricted model)
3
(4) + 12(3) + Q[1]
-0.00394
3
(4) + 12(3) + 36(2)
-0.00944
4
(4) + 12(3)
0.92079
(4)
1
2
3
4
A
B
A*B
Error
SS
1.9874
2.6626
6.4598
151.9308
163.0406
MS
0.9937
0.6656
0.8075
0.9208
F
1.23
0.82
0.88
P
0.342
0.545
0.537
MTB > # E.g. 2 (unrestricted model):
MTB > ANOVA ’Y’ = A B A*B C(A B);
SUBC>
Random B C;
SUBC>
EMS.
Source
A
B
A*B
C(A B)
Error
Total
DF
2
4
8
45
120
179
Source
Variance Error Expected Mean Square
component term (using unrestricted model)
3
(5) + 3(4) + 12(3) + Q[1]
-0.00394
3
(5) + 3(4) + 12(3) + 36(2)
-0.01734
4
(5) + 3(4) + 12(3)
0.04341
5
(5) + 3(4)
0.88528
(5)
1
2
3
4
5
A
B
A*B
C(A B)
Error
SS
1.9874
2.6626
6.4598
45.6975
106.2333
163.0406
MS
0.9937
0.6656
0.8075
1.0155
0.8853
F
1.23
0.82
0.80
1.15
P
0.342
0.545
0.610
0.275
. . . . . . (continued over page)
6
MTB > # E.g. 3 (unrestricted model):
MTB > ANOVA ’Y’ = A B A*B C A*C B*C A*B*C;
SUBC>
Random B C;
SUBC>
EMS.
Source
A
B
A*B
C
A*C
B*C
A*B*C
Error
Total
DF
2
4
8
3
6
12
24
120
179
Source
Variance Error Expected Mean Square
component term (using unrestricted model)
*
(8) + 3(7) + 15(5) + 12(3) + Q[1]
-0.02670
*
(8) + 3(7) + 9(6) + 12(3) + 36(2)
0.00729
7
(8) + 3(7) + 12(3)
-0.01580
*
(8) + 3(7) + 9(6) + 15(5) + 45(4)
0.02329
7
(8) + 3(7) + 15(5)
0.09102
7
(8) + 3(7) + 9(6)
-0.05510
8
(8) + 3(7)
0.88528
(8)
1
2
3
4
5
6
7
8
A
B
A*B
C
A*C
B*C
A*B*C
Error
SS
1.9874
2.6626
6.4598
3.5322
6.4157
18.4703
17.2793
106.2333
163.0406
MS
0.9937
0.6656
0.8075
1.1774
1.0693
1.5392
0.7200
0.8853
F
*
*
1.12
*
1.49
2.14
0.81
P
0.384
0.225
0.055
0.714
* No exact F-test can be calculated.
MTB > # E.g. 4 (unrestricted model):
MTB > ANOVA ’Y’ = A B A*B C(A) B*C(A) ;
SUBC>
Random B C;
SUBC>
EMS.
Source
A
B
A*B
C(A)
B*C(A)
Error
Total
DF
2
4
8
9
36
120
179
Source
Variance Error Expected Mean Square
component term (using unrestricted model)
*
(6) + 3(5) + 15(4) + 12(3) + Q[1]
-0.00394
3
(6) + 3(5) + 12(3) + 36(2)
-0.01546
5
(6) + 3(5) + 12(3)
0.00748
5
(6) + 3(5) + 15(4)
0.03592
6
(6) + 3(5)
0.88528
(6)
1
2
3
4
5
6
A
B
A*B
C(A)
B*C(A)
Error
SS
1.9874
2.6626
6.4598
9.9478
35.7497
106.2333
163.0406
MS
0.9937
0.6656
0.8075
1.1053
0.9930
0.8853
* No exact F-test can be calculated.
7
F
*
0.82
0.81
1.11
1.12
P
0.545
0.596
0.379
0.316
And now the restricted model:
MTB > # E.g. 1 (restricted model):
MTB > ANOVA ’Y’ = A B A*B;
SUBC>
Random B;
SUBC>
Restrict;
SUBC>
EMS.
Source
A
B
A*B
Error
Total
DF
2
4
8
165
179
Source
Variance Error Expected Mean Square
component term (using restricted model)
3
(4) + 12(3) + 60Q[1]
-0.00709
4
(4) + 36(2)
-0.00944
4
(4) + 12(3)
0.92079
(4)
1
2
3
4
A
B
A*B
Error
SS
1.9874
2.6626
6.4598
151.9308
163.0406
MS
0.9937
0.6656
0.8075
0.9208
F
1.23
0.72
0.88
P
0.342
0.577
0.537
MTB > # E.g. 2 (restricted model):
MTB > ANOVA ’Y’ = A B A*B C(A B);
SUBC>
Random B C;
SUBC>
Restrict;
SUBC>
EMS.
Source
A
B
A*B
C(A B)
Error
Total
DF
2
4
8
45
120
179
Source
Variance Error Expected Mean Square
component term (using restricted model)
3
(5) + 3(4) + 12(3) + 60Q[1]
-0.00972
4
(5) + 3(4) + 36(2)
-0.01734
4
(5) + 3(4) + 12(3)
0.04341
5
(5) + 3(4)
0.88528
(5)
1
2
3
4
5
A
B
A*B
C(A B)
Error
SS
1.9874
2.6626
6.4598
45.6975
106.2333
163.0406
MS
0.9937
0.6656
0.8075
1.0155
0.8853
F
1.23
0.66
0.80
1.15
P
0.342
0.626
0.610
0.275
. . . . . . (continued over page)
8
MTB > # E.g 3 (restricted model):
MTB > ANOVA ’Y’ = A B A*B C A*C B*C A*B*C;
SUBC>
Random B C;
SUBC>
Restrict;
SUBC>
EMS.
Source
A
B
A*B
C
A*C
B*C
A*B*C
Error
Total
DF
2
4
8
3
6
12
24
120
179
Source
Variance Error Expected Mean Square
component term (using restricted model)
*
(8) + 3(7) + 15(5) + 12(3) + 60Q[1]
-0.02427
6
(8) + 9(6) + 36(2)
0.00729
7
(8) + 3(7) + 12(3)
-0.00804
6
(8) + 9(6) + 45(4)
0.02329
7
(8) + 3(7) + 15(5)
0.07266
8
(8) + 9(6)
-0.05510
8
(8) + 3(7)
0.88528
(8)
1
2
3
4
5
6
7
8
A
B
A*B
C
A*C
B*C
A*B*C
Error
SS
1.9874
2.6626
6.4598
3.5322
6.4157
18.4703
17.2793
106.2333
163.0406
MS
0.9937
0.6656
0.8075
1.1774
1.0693
1.5392
0.7200
0.8853
F
*
0.43
1.12
0.76
1.49
1.74
0.81
P
0.783
0.384
0.535
0.225
0.067
0.714
* No exact F-test can be calculated.
MTB > # E.g. 4 (restricted model):
MTB > ANOVA ’Y’ = A B A*B C(A) B*C(A);
SUBC>
Random B C;
SUBC>
Restrict;
SUBC>
EMS.
Source
A
B
A*B
C(A)
B*C(A)
Error
Total
DF
2
4
8
9
36
120
179
SS
1.9874
2.6626
6.4598
9.9478
35.7497
106.2333
163.0406
MS
0.9937
0.6656
0.8075
1.1053
0.9930
0.8853
F
*
0.67
0.81
1.11
1.12
P
0.617
0.596
0.379
0.316
. . . . . . (continued over page)
9
Source
1
2
3
4
5
6
A
B
A*B
C(A)
B*C(A)
Error
Variance Error Expected Mean Square
component term (using restricted model)
*
(6) + 3(5) + 15(4) + 12(3) + 60Q[1]
-0.00909
5
(6) + 3(5) + 36(2)
-0.01546
5
(6) + 3(5) + 12(3)
0.00748
5
(6) + 3(5) + 15(4)
0.03592
6
(6) + 3(5)
0.88528
(6)
* No exact F-test can be calculated.
• It is actually possible to convert from the restricted to the unrestricted model by
making appropriate changes of variable, and to show that the two are really the same
thing!!! So what’s going on when the restricted model tells you to form your F-statistic
one way, and the unrestricted model tells you to form it another way? The long and
the short of it, it seems to me, is: You are simply testing different hypotheses.
For instance, in E.g. 1, the two F statistics for testing for a “B” effect both look like
2 = 0. However, if you go through the change of variable
they are testing H0 : racσB
2 under the restricted model is precisely equal to
jazz, you see that the value of racσB
2
2
that of racσB + rcσAB under the unrestricted model.
Thus testing for a “B” effect using the restricted model (i.e. using M SE as the
2 + rcσ 2
denominator of the F statistic) really amounts to testing H 0 : racσB
AB = 0
2
2
which can only be true if both σB and σAB are 0. I.e. you are really testing that
both the “B” effect and its interaction with A (as they would be interpreted under
the unrestricted model) are null. This does not seem to me to be what one would
want to do. I strongly recommend sticking with the unrestricted model.
10