The ANOVA Approach to the Analysis
of Linear Mixed-Effects Models
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
1 / 58
We begin with a relatively simple special case. Suppose
yijk = µ + τi + uij + eijk , (i = 1, . . . , t; j = 1, . . . , n; k = 1, . . . , m)
β = (µ, τ1 , . . . , τt )0 , u = (u11 , u12 , . . . , utn )0 , e = (e111 , e112 , . . . , etnm )0 ,
β ∈ Rt+1 , an unknown parameter vector,
u
e
∼N
0
0
2
σu I 0
,
, where
0 σe2 I
σu2 , σe2 ∈ R+ are unknown variance components.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
2 / 58
This is the standard model for a CRD with t treatments, n
experimental units per treatment, and m observations per
experimental unit.
We can write the model as y = Xβ + Zu + e, where
X = [1tnm×1 , It×t ⊗ 1nm×1 ]
c
Copyright 2016
Dan Nettleton (Iowa State University)
and
Z = [Itn×tn ⊗ 1m×1 ].
Statistics 510
3 / 58
Special Case of t = n = m = 2






X=




1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
1
1
1
1










µ






 , β = τ1 , Z = 


τ2






c
Copyright 2016
Dan Nettleton (Iowa State University)
1
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
1
1








, u = 






u11
u12 

u21 
u22
Statistics 510
4 / 58
Connection to ANOVA
Let X1 = 1tnm×1 , X2 = [1tnm×1 , It×t ⊗ 1nm×1 ], X3 = [Itn×tn ⊗ 1m×1 ].
Note that C(X1 ) ⊂ C(X2 ) ⊂ C(X3 ), X = X2 , and Z = X3 .
As usual, let Pj = PXj = Xj (X0j Xj )− X0j for j = 1, 2, 3.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
5 / 58
An ANOVA Table
Sum of Squares Degrees of Freedom
y0 (P2 − P1 )y
rank(X2 ) − rank(X1 ) = t − 1
y0 (P3 − P2 )y
rank(X3 ) − rank(X2 ) = tn − t
y0 (I − P3 )y
rank(I) − rank(X3 ) = tnm − tn
y0 (I − P1 )y
rank(I) − rank(X1 ) = tnm − 1
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
6 / 58
Shortcut for Obtaining DF from Source
Source
DF
treatments
t−1
exp.units(treatments)
(n − 1)t
obs.units(exp.units, treatments)
(m − 1)nt
c.total
tnm − 1
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
7 / 58
Shortcut for Obtaining SS from DF
Source
DF
Sum of Squares
trt
t−1
Pt
Pn Pm
− y··· )2
xu(trt)
t(n − 1)
Pt
Pn Pm
− yi·· )2
Pt
Pn Pm
− yij· )2
Pt
Pn Pm
− y··· )2
ou(xu, trt) tn(m − 1)
c.total
tnm − 1
c
Copyright 2016
Dan Nettleton (Iowa State University)
i=1
i=1
i=1
i=1
j=1
j=1
j=1
j=1
k=1 (yi··
k=1 (yij·
k=1 (yijk
k=1 (yijk
Statistics 510
8 / 58
Source
DF
Sum of Squares
trt
t−1
nm
xu(trt)
tn − t
m
Pt
i=1 (yi··
Pt
i=1
Mean Square
− y··· )2
Pn
j=1 (yij·
− yi·· )2
ou(xu, trt) tnm − tn
Pt
Pn Pm
− yij· )2
tnm − 1
Pt
Pn Pm
− y··· )2
c.total
i=1
i=1
c
Copyright 2016
Dan Nettleton (Iowa State University)
j=1
j=1
k=1 (yijk
k=1 (yijk
SS/DF
Statistics 510
9 / 58
Expected Mean Squares
Based on our linear mixed-effects model (y = Xβ + Zu + e
and associated assumptions), we can find the expected
value of each mean square in the ANOVA table.
Examining these expected values helps us see ways to
1)
test hypotheses of interest by computing
ratios of mean squares, and
2)
estimate variance components by computing
linear combinations of mean squares.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
10 / 58
Expected Mean Squares
For balanced designs, there are shortcuts (not presented
here) for writing down expected mean squares.
Rather than memorizing shortcuts, I think it is better to know
how to derive expected mean squares.
Before going through one example derivation, we will prove
a useful result that you may already be familiar with.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
11 / 58
Expectation of Sample Variance Numerator
ind
Suppose w1 , . . . , wk ∼ (µw , σw2 ). Then
kσw2
=
k
X
σw2
=
i=1
k
X
2
E(wi − µw ) =
i=1
k
X
E(wi − w̄· + w̄· − µw )2
i=1
( k
)
X
2
= E
(wi − w̄· + w̄· − µw )
i=1
= E
( k
X
(wi − w̄· )2 + (w̄· − µw )2 + 2(w̄· − µw )(wi − w̄· )
)
i=1
( k
)
k
k
X
X
X
2
2
= E
(wi − w̄· ) +
(w̄· − µw ) + 2(w̄· − µw )
(wi − w̄· )
i=1
c
Copyright 2016
Dan Nettleton (Iowa State University)
i=1
i=1
Statistics 510
12 / 58
Expectation of Sample Variance Numerator (ctd.)
kσw2 = E
( k
X
(wi − w̄· )2 +
i=1
= E
( k
X
k
X
)
(w̄· − µw )2
i=1
)
2
2
(wi − w̄· ) + k(w̄· − µw )
i=1
= E
( k
X
)
(wi − w̄· )2
+ kE(w̄· − µw )2
i=1
= E
= E
( k
X
i=1
( k
X
)
(wi − w̄· )2
+ kVar(w̄· )
)
(wi − w̄· )
i=1
c
Copyright 2016
Dan Nettleton (Iowa State University)
2
+
kσw2 /k
( k
)
X
2
=E
(wi − w̄· ) + σw2
i=1
Statistics 510
13 / 58
Expectation of Sample Variance Numerator (ctd.)
We have shown
kσw2 = E
( k
X
)
(wi − w̄· )2
+ σw2 .
i=1
Therefore,
( k
)
X
E
(wi − w̄· )2 = (k − 1)σw2 .
i=1
This is just a special case of the Gauss-Markov model result
E(σ̂ 2 ) = σ 2 . (y = [w1 , . . . , wk ]0 , X = 1, β = [µw ], σ 2 = σw2 )
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
14 / 58
Expected Value of MStrt
E(MStrt ) =
nm
t−1
Pt
E(yi.. − y··· )2
=
nm
t−1
Pt
E(µ + τi + ui· + ei·· − µ − τ · − u·· − e··· )2
=
nm
t−1
Pt
E(τi − τ · + ui· − u·· + ei·· − e··· )2
=
nm
t−1
Pt
− τ · )2 + E(ui· − u·· )2 + E(ei·· − e··· )2 ]
=
nm
[
t−1
Pt
P
− τ · )2 + E{ ti=1 (ui· − u·· )2 }
i=1
i=1
i=1
i=1 [(τi
i=1 (τi
c
Copyright 2016
Dan Nettleton (Iowa State University)
+E{
Pt
i=1 (ei··
− e··· )2 }]
Statistics 510
15 / 58
So, to simplify E(MStrt ) further, note that
σu2
i.i.d.
u1· , . . . , ut· ∼ N 0,
.
n
Thus,
)
( t
X
σ2
E
(ui· − u·· )2 = (t − 1) u .
n
i=1
Similarly,
e1·· , . . . , et··
so that
E
( t
X
σ2
∼ N 0, e
nm
i.i.d.
)
(ei·· − e··· )2
i=1
c
Copyright 2016
Dan Nettleton (Iowa State University)
= (t − 1)
σe2
.
nm
Statistics 510
16 / 58
It follows that
)
" t
( t
X
nm X
E(MStrt ) =
(τi − τ · )2 + E
(ui· − u·· )2
t − 1 i=1
i=1
( t
)#
X
+E
(ei·· − e··· )2
i=1
" t
nm X
σ2
=
(τi − τ · )2 + (t − 1) u
t − 1 i=1
n
σ2
+(t − 1) e
nm
=
t
nm X
(τi − τ .)2 + mσu2 + σe2 .
t − 1 i=1
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
17 / 58
Similar calculations allow us to add an Expected Mean Squares
(EMS) column to our ANOVA table.
Source
EMS
trt
σe2 + mσu2 +
xu(trt)
σe2 + mσu2
ou(xu, trt)
σe2
c
Copyright 2016
Dan Nettleton (Iowa State University)
nm
t−1
Pt
i=1 (τi
− τ .)2
Statistics 510
18 / 58
Expected Mean Squares (EMS) could be computed using
E(y0 Ay) = tr(AΣ) + E(y)0 AE(y),
where
Σ = Var(y) = ZGZ0 + R = σu2 Itn× tn ⊗ 110 m× m + σe2 Itnm× tnm
and




E(y) = 


c
Copyright 2016
Dan Nettleton (Iowa State University)
µ + τ1
µ + τ2
.
.
.
µ + τt




 ⊗ 1nm× 1 .


Statistics 510
19 / 58
Furthermore, with some nontrivial work, it can be shown that
!
t
X
nm
y0 (P2 − P1 )y
∼ χ2t−1
(τi − τ .)2 ,
σe2 + mσu2
2(σe2 + mσu2 ) i=1
y0 (P3 − P2 )y
∼ χ2tn−t ,
2
2
σe + mσu
y0 (I − P3 )y
∼ χ2tnm−tn ,
σe2
and that these three χ2 random variables are independent.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
20 / 58
It follows that
MStrt
y0 (P2 − P1 )y/(t − 1)
= 0
MSxu(trt)
y (P3 − P2 )y/(tn − t)
h0
i
y (P2 −P1 )y
/(t − 1)
σe2 +mσu2
h
i
=
y0 (P3 −P2 )y
/(tn − t)
σ 2 +mσ 2
F1 =
e
u
∼ Ft−1, tn−t
!
t
X
nm
(τi − τ · )2 .
2(σe2 + mσu2 ) i=1
Thus, we can use F1 to test H0 : τ1 = · · · = τt .
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
21 / 58
Also,
MSxu(trt)
y0 (P3 − P2 )y/(tn − t)
= 0
MSou(xu,trt)
y (I − P3 )y/(tnm − tn)
h
i
y0 (P3 −P2 )y /(tn − t)
2
2
σ 2 +mσu2
σe + mσu
h0 e
i
=
2
y (P3 −P2 )y
σe
/(tnm − tn)
σe2
2
σe + mσu2
∼
Ftn−t, tnm−tn .
σe2
F2 =
Thus, we can use F2 to test H0 : σu2 = 0.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
22 / 58
Estimating σu2
Note that
E
MSxu(trt) − MSou(xu,trt)
m
=
(σe2 + mσu2 ) − σe2
= σu2 .
m
Thus,
MSxu(trt) − MSou(xu,trt)
m
2
is an unbiased estimator of σu .
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
23 / 58
Although
MSxu(trt) − MSou(xu,trt)
m
2
is an unbiased estimator of σu , this estimator can take
negative values.
This is undesirable because σu2 , the variance of the u
random effects, cannot be negative.
Later in the course, we will discuss likelihood based
methods for estimating variance components that honor the
parameter space.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
24 / 58
Estimation of Estimable Cβ
As we have seen previously,
Σ ≡ Var(y) = σu2 Itn× tn ⊗ 110 m× m + σe2 Itnm× tnm .
It turns out that
β̂ Σ = (X0 Σ−1 X)− X0 Σ−1 y = (X0 X)− X0 y = β̂.
Thus, the GLS estimator of any estimable Cβ is equal to the
OLS estimator in this special case.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
25 / 58
An Analysis Based on the Average for Each
Experimental Unit
Recall that our model is
yijk = µ + τi + uij + eijk , (i = 1, ..., t; j = 1, ..., n; k = 1, ..., m)
The average of observations for experimental unit ij is
yij· = µ + τi + uij + eij·
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
26 / 58
If we define
ij = uij + eij· ∀ i, j
and
σ 2 = σu2 +
σe2
,
m
we have
yij· = µ + τi + ij ,
where the ij terms are iid N(0, σ 2 ). Thus, averaging the same
number (m) of multiple observations per experimental unit
results in a Gauss-Markov linear model with normal errors for
the averages
yij· : i = 1, ..., t; j = 1, ..., n .
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
27 / 58
Inferences about estimable functions of β obtained by
analyzing these averages are identical to the results
obtained using the ANOVA approach as long as the number
of multiple observations per experimental unit is the same
for all experimental units.
When using the averages as data, our estimate of σ 2 is an
2
estimate of σu2 + σme .
We can’t separately estimate σu2 and σe2 , but this doesn’t
matter if our focus is on inference for estimable functions of
β.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
28 / 58
Because




E(y) = 


µ + τ1
µ + τ2
.
.
.
µ + τt




 ⊗ 1nm× 1 ,


the only estimable quantities are linear combinations of the
treatment means µ + τ1 , µ + τ2 , ..., µ + τt , whose Best Linear
Unbiased Estimators are y1·· , y2·· , . . . , yt·· , respectively.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
29 / 58
Thus, any estimable Cβ can always be written as


µ + τ1
 µ + τ2 
 for some matrix A.
A
..


.
µ + τt
It follows that the BLUE of Cβ can be written as


y1··
 y2·· 

A
 ...  .
c
Copyright 2016
Dan Nettleton (Iowa State University)
yt··
Statistics 510
30 / 58
Now note that
Var(yi ..) = Var(µ + τi + ui . + ei ..)
= Var(ui . + ei ..)
= Var(ui .) + Var(ei ..)
σ2
σ2
= u+ e
n nm 1
σ2
=
σu2 + e
n
m
2
σ
.
=
n
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
31 / 58
Thus




Var 


y1..
y2..
.
.
.
yt..


 σ 2

 = It×t

n

which implies that the variance of the BLUE of Cβ is
 

y1··
  . 
2
 

σ
σ2




Var A  .  = A
It×t A0 = AA0 .
n
n
  . 
yt··
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
32 / 58
Thus, we don’t need separate estimates of σu2 and σe2 to
carry out inference for estimable Cβ.
We do need to estimate σ 2 = σu2 +
σe2
.
m
This can equivalently be estimated by
MSxu(trt)
m
or by the MSE in an analysis of the experimental unit means
yij· : i = 1, ..., t; j = 1, ..., n. .
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
33 / 58
For example, suppose we want to estimate τ1 − τ2 . The BLUE is
y1·· − y2·· whose variance is
Var(y1·· − y2·· ) = Var(y1·· ) + Var(y2·· )
2
σ2
σu
σe2
+
= 2 =2
n
n
nm
2 2
(σe + mσu2 )
=
nm
2
=
E(MSxu(trt) )
nm
Thus,
c 1·· − y2·· ) = 2MSxu(trt) .
Var(y
nm
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
34 / 58
A 100(1 − α)% confidence interval for τ1 − τ2 is
r
2MSxu(trt)
.
y1·· − y2·· ± tt(n−1),1−α/2
nm
A test of H0 : τ1 = τ2 can be based on

τ1 − τ2
y − y2··
t = q1··
∼ tt(n−1)  q
2MSxu(trt)
nm
c
Copyright 2016
Dan Nettleton (Iowa State University)
2(σe2 +mσu2 )
nm

.
Statistics 510
35 / 58
What if the number of observations per experimental unit is
not the same for all experimental units?
Let us look at two miniature examples to understand how
this type of unbalancedness affects estimation and
inference.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
36 / 58
First Example


y111
 y121 

y=
 y211  ,
y212

1
 1
X=
 0
0
c
Copyright 2016
Dan Nettleton (Iowa State University)

0
0 
,
1 
1

1
 0
Z=
 0
0
0
1
0
0

0
0 

1 
1
Statistics 510
37 / 58
First Example


y111
 y121 

y=
 y211  ,
y212

1
 1
X=
 0
0
X1 = 1,

ȳ···
 ȳ··· 

P1 y = 
 ȳ···  ,
ȳ···


0
0 
,
1 
1
X2 = X,

ȳ1·1
 ȳ1·1 

P2 y = 
 ȳ21·  ,
ȳ21·
c
Copyright 2016
Dan Nettleton (Iowa State University)


1
 0
Z=
 0
0
0
1
0
0

0
0 

1 
1
X3 = Z

y111
 y121 

P3 y = 
 ȳ21· 
ȳ21·

Statistics 510
38 / 58
Note that
ȳ1·1 − ȳ··· = ȳ1·1 − (ȳ1·1 + ȳ21· )/2 = (ȳ1·1 − ȳ21· )/2
and
ȳ21· − ȳ··· = ȳ21· − (ȳ1·1 + ȳ21· )/2 = −(ȳ1·1 − ȳ21· )/2.
Thus,
MStrt = y0 (P2 − P1 )y = 2(y1·1 − y··· )2 + 2(y21· − y··· )2 = (y1·1 − y21· )2
1
MSxu(trt) = y0 (P3 −P2 )y = (y111 −y1·1 )2 +(y121 −y1·1 )2 = (y111 −y121 )2
2
1
MSou(xu,trt) = y0 (I−P3 )y = (y211 −y21· )2 +(y212 −y21· )2 = (y211 −y212 )2 .
2
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
39 / 58
E(MStrt ) = E(y1·1 − y21· )2
= E(τ1 − τ2 + u1· − u21 + e1·1 − e21· )2
= (τ1 − τ2 )2 + Var(u1· ) + Var(u21 ) + Var(e1·1 ) + Var(e21· )
= (τ1 − τ2 )2 +
σu2
2
+ σu2 +
σe2
2
+
σe2
2
= (τ1 − τ2 )2 + 1.5σu2 + σe2
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
40 / 58
E(MSxu(trt) )
=
1
E(y111
2
=
1
E(u11
2
=
1
(2σu2
2
− y121 )2
− u12 + e111 − e121 )2
+ 2σe2 )
= σu2 + σe2
E(MSou(xu,trt) ) =
1
E(y211
2
− y212 )2
=
1
E(e211
2
− e212 )2
= σe2
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
41 / 58
SOURCE EMS
trt
(τ1 − τ2 )2 + 1.5σu2 + σe2
xu(trt)
σu2 + σe2
ou(xu, trt)
σe2
With some nontrivial work, it can be shown that
MSxu(trt)
(τ1 − τ2 )2
MStrt
F=
/
∼ F1,1
.
1.5σu2 + σe2
σu2 + σe2
3σu2 + 2σe2
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
42 / 58
The test statistic that we used to test
H0 : τ1 = · · · = τt
in the balanced case is not F distributed in this unbalanced case.
1.5σu2 + σe2
(τ1 − τ2 )2
MStrt
∼
F1,1
MSxu(trt)
σu2 + σe2
3σu2 + 2σe2
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
43 / 58
A Statistic with an Approximate F Distribution
We’d like our denominator to be an unbiased estimator of
1.5σu2 + σe2 in this case.
Consider 1.5MSxu(trt) − 0.5MSou(xu,trt)
The expectation is
1.5(σu2 + σe2 ) − 0.5σe2 = 1.5σu2 + σe2 .
The ratio
MStrt
1.5MSxu(trt) − 0.5MSou(xu,trt)
can be used as an approximate F statistic with 1 numerator
DF and a denominator DF obtained using the
Cochran-Satterthwaite method.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
44 / 58
The Cochran-Satterthwaite method will be explained in the
next set of notes.
We should not expect this approximate F-test to be reliable
in this case because of our pitifully small dataset.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
45 / 58
Best Linear Unbiased Estimates in this First Example
What do the BLUEs of the treatment means look like in this
case? Recall




1 0 0
1 0


 1 0 
µ1
, Z =  0 1 0 .
β=
, X=
 0 1 
 0 0 1 
µ2
0 0 1
0 1
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
46 / 58
Σ = Var(y) = ZGZ0 + R = σu2 ZZ0 + σe2 I




1 0 0 0
1 0 0 0
 0 1 0 0 


 + σe2  0 1 0 0 
= σu2 
 0 0 1 1 
 0 0 1 0 
0 0 1 1
0 0 0 1
 2
  2
σu 0 0 0
σe 0 0 0
 0 σu2 0 0   0 σe2 0 0
 
= 
 0 0 σu2 σu2  +  0 0 σe2 0
0 0 σu2 σu2
0 0 0 σe2
 2

σu + σe2
0
0
0


0
σu2 + σe2
0
0

= 
2
2
2


0
0
σu + σe
σu
2
2
2
0
0
σu
σu + σe
c
Copyright 2016
Dan Nettleton (Iowa State University)




Statistics 510
47 / 58
It follows that
β̂ Σ = (X0 Σ−1 X)− X0 Σ−1 y
1 1
0 0
y1·1
2
2
y=
=
y21·
0 0 12 21
Fortunately, this is a linear estimator that does not depend on
unknown variance components.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
48 / 58
Second Example


y111
 y112 

y=
 y121  ,
y211

1
 1
X=
 1
0
c
Copyright 2016
Dan Nettleton (Iowa State University)

0
0 
,
0 
1

1
 1
Z=
 0
0
0
0
1
0

0
0 

0 
1
Statistics 510
49 / 58
In this case, it can be shown that
β̂ Σ = (X0 Σ−1 X)− X0 Σ−1 y

"
=
"
=
σe2 +σu2
3σe2 +4σu2
σe2 +σu2
3σe2 +4σu2
σe2 +2σu2
3σe2 +4σu2
0
0
0
2σe2 +2σu2
3σe2 +4σu2
c
Copyright 2016
Dan Nettleton (Iowa State University)
y11·
+
y211
σe2 +2σu2
3σe2 +4σu2
0
1
#
y121

y111
 y112 


 y121 
y211
#
.
Statistics 510
50 / 58
It is straightforward to show that the weights on y11· and y121 are
1
Var(y11· )
1
Var(y11· )
+
1
Var(y121 )
and
1
Var(y121 )
1
Var(y11· )
+
1
Var(y121 )
, respectively.
This is a special case of a more general phenomenon: the BLUE
is a weighted average of independent linear unbiased estimators
with weights for the linear unbiased estimators proportional to
the inverse variances of the linear unbiased estimators.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
51 / 58
Of course, in this case and in many others,
" 2 2
#
σe2 +2σu2
2σe +2σu
y
+
y
3σe2 +4σu2 121
β̂ Σ = 3σe2 +4σu2 11·
y211
is not an estimator because it is a function of unknown
parameters.
Thus, we use β̂ Σb as our estimator (i.e., we replace σe2 and σu2 by
estimates in the expression above).
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
52 / 58
β̂ Σb is an approximation to the BLUE.
β̂ Σb is not even a linear estimator in this case.
Its exact distribution is unknown.
When sample sizes are large, it is reasonable to assume
that the distribution of β̂ Σb is approximately the same as the
distribution of β̂ Σ .
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
53 / 58
Var(β̂ Σ ) = Var[(X0 Σ−1 X)−1 X0 Σ−1 y]
= (X0 Σ−1 X)−1 X0 Σ−1 Var(y)[(X0 Σ−1 X)−1 X0 Σ−1 ]0
= (X0 Σ−1 X)−1 X0 Σ−1 ΣΣ−1 X(X0 Σ−1 X)−1
= (X0 Σ−1 X)−1 X0 Σ−1 X(X0 Σ−1 X)−1
= (X0 Σ−1 X)−1
−1
−1
−1
b X)−1 X0 Σ
b y] =???? ≈ (X0 Σ
b X)−1
Var(β̂ Σb ) = Var[(X0 Σ
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
54 / 58
Summary of Main Points
Many of the concepts we have seen by examining special
cases hold in greater generality.
For many of the linear mixed models commonly used in
practice, balanced data are nice because...
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
55 / 58
1
It is relatively easy to determine degrees of freedom, sums of
squares, and expected mean squares in an ANOVA table.
2
Ratios of appropriate mean squares can be used to obtain exact
F-tests.
3
For estimable Cβ, Cβ̂ Σ
b = Cβ̂. (OLS = GLS).
4
5
When Var(c0 β̂) = constant × E(MS), exact inferences about c0 β
can be obtained by constructing t-tests or confidence intervals
based on
c0 β̂ − c0 β
∼ tDF(MS) .
t= p
constant × (MS)
Simple analysis based on experimental unit averages gives the
same results as those obtained by linear mixed model analysis of
the full data set.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
56 / 58
When data are unbalanced, the analysis of linear mixed may be
considerably more complicated.
1
Approximate F-tests can be obtained by forming linear
combinations of Mean Squares to obtain denominators for
test statistics.
2
The estimator Cβ̂ Σb may be a nonlinear estimator of Cβ
whose exact distribution is unknown.
3
Approximate inference for Cβ is often obtained by using the
distribution of Cβ̂ Σb , with unknowns in that distribution
replaced by estimates.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
57 / 58
Whether data are balanced or unbalanced, unbiased estimators
of variance components can be obtained using linear
combinations of mean squares from the ANOVA table.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
58 / 58