Inference for fixed effects and
variance components
1/16
Linear mixed models
In general, a linear mixed model may be represented as
Y = X β + Zu + ε,
where
I
Y is an n × 1 vector of response;
I
X is an n × p design matrix;
I
β is a p × 1 vector of “fixed” unknown parameter values;
I
Z is an n × q model matrix of known constants;
I
u is a q × 1 random vector;
I
ε is an n × 1 random error.
2/16
Linear mixed models
We typically assume that
E(ε) = 0, Var(ε) = R, E(u) = 0, Var(u) = G.
As a result,
Var(Y ) = Σ(γ) = ZGZ T + R.
3/16
Generalized LSE
I
For any estimable function C T β, the unique BLUE is
T β = C T (X T Σ−1 X )− X T Σ−1 Y .
Cd
I
It can be shown that
T β ∼ N(C T β, C T (X T Σ−1 X )− C).
Cd
I
The inference for C T β can be built based on the
asymptotic normality.
4/16
Example: random blocks (penicillin production)
I
Comparison of four processes for producing penicillin.
I
Four processes A, B, C and D, levels of a “fixed” effect
treatment.
I
Random sample of five batches of raw material, corn steep
liquor.
I
Split each batch into four parts. Run each process on one
part, and randomize the order in which the processes are
run with each batch.
5/16
Example: Penicillin production
Recall that the model is
Yij = µ + αi + γj + eij i = 1, · · · , a j = 1, · · · , b,
where Yij is the yield for the i-th process applied to the j-th
batch, γj ’s are independent and identically distributed (IID)
random batch effects, eij ’s are IID random errors. Moreover, γj ’s
are independent of eij ’s.
6/16
Example continued
I
Assume that we would like to estimate and construct
confidence intervals for µ + αi .
I
An unbiased estimate for µ + αi is Ȳi· = b−1
Pb
j=1 Yij
(is
this BLUE?).
I
It is easy to know that Var(Ȳi· ) = (σγ2 + σ 2 )/b. Based on the
ANOVA estimates of the variance components, we have
o
1n1
\
(MSB − MSE) + MSE
Var(
Ȳi· ) =
b a
1
a−1
=
MSB +
MSE.
ab
ab
7/16
Example continued
I
A 1 − α approximate confidence interval for µ + αi is
\
Ȳi· ± zα/2 {Var(
Ȳi· )}1/2 .
I
In small sample case, one might wish to use t-quantile to
\
replace standard normal quantile. Since Var(
Ȳ ) is a linear
i·
combination of mean squares, it is not immediately clear
what degree of freedom should be used.
8/16
Cochran-Satterthwaite (CS) approximation
Suppose MS1 , MS2 , · · · , MSk are independent random
variables such that
dfi MSi
∼ χ2dfi for i = 1, · · · , k .
E(MSi )
Consider a linear combination of MSi , which is
S 2 = a1 MS1 + a2 MS2 + · · · + ak MSk .
We would like to approximate the distribution of S 2 by a skewed
chi-square distribution bχ2ν .
9/16
CS approximation
The idea of CS approximation is to match the first two moments
of bχ2ν with the first two moments of S 2 . Then solving these two
equations to estimate the unknown parameters b and ν in the
skewed chi-square distribution.
10/16
Moments of S 2
It is easy to see the following
E(S 2 ) =
k
X
ai E(MSi );
i=1
Var(S 2 ) =
=
k
X
i=1
k
X
Var(MSi )ai2
E 2 (MSi ) 2
a × (2dfi )
(dfi )2 i
i=1
k
X
=2
i=1
ai2
E 2 (MSi )
.
dfi
11/16
Moments of skewed chi-square bχ2ν
The first two moments of bχ2ν are
E(bχ2ν ) = bν;
Var(bχ2ν ) = 2b2 ν.
Matching with the moments of S 2 , we have the following two
estimating equations:
bν =
k
X
ai E(MSi );
i=1
2
2b ν = 2
k
X
i=1
ai2
E 2 (MSi )
.
dfi
12/16
CS approximation
The solution of the above estimating equations are
b=
k
X
ai2
i=1
2
2
E 2 (MSi )
/E(S 2 )
dfi
ν = E (S )/
k
X
ai2
i=1
E 2 (MSi )
.
dfi
Therefore, we can estimate b and ν by
b̂ =
k
X
i=1
ai2
MSi2 2
/S
dfi
ν̂ = (S 2 )2 /
k
X
i=1
ai2
MSi2
.
dfi
13/16
Example: penicillin production
A t-type confidence interval for µ + αi is
\
Ȳi· ± tν̂ {Var(
Ȳi· )}1/2 .
Here ν is approximate by the CS approximation. Namely,
n 1 MSB 2
o
a − 1 2
MSE 2
2
\
ν̂ = {Var(
Ȳi· )}2 /
+
.
ab b − 1
ab
(a − 1)(b − 1)
14/16
Inference for variance components
Suppose that a variance component parameter γ can be
estimated by a linear combination of mean squares, say
S 2 = a1 MS1 + a2 MS2 + · · · + ak MSk .
Then an approximate 1 − α confidence interval for γ is
!
ν̂S 2
ν̂S 2
,
,
χ2ν̂,α/2 χ2ν̂,1−α/2
where ν̂ is the approximated degree of freedom using the CS
method.
15/16
Example: penicillin production
Suppose we want to construct a 1 − α confidence interval for
σγ2 . Recall that
σ̂γ2 =
1
(MSB − MSE).
a
Then the estimated degree of freedom through the CS
approximation is
ν̂ = {σ̂γ2 }2 /
n 1 MSB 2
o
1
MSE 2
+
.
a2 b − 1
a2 (a − 1)(b − 1)
Thus a 1 − α confidence interval for σγ2 is
ν̂ σ̂γ2
ν̂ σ̂γ2
,
χ2ν̂,α/2 χ2ν̂,1−α/2
!
.
16/16