• Consider the following model:
Yi|bi ∼ N (Xiβ + Zibi, R)
Random effects models
bi ∼ N (0, D)
• Easily accomodate heterogeneous
inspection schedules
• Remarks:
– bi are called random effects
• Model heterogeneity among subjects
due to unmeasured factors
– Zi and Xi are non-random model
matrices
– R represents the variability within
subjects, often R = σ 2 I
– Random subject effects
– Random regression coefficients
– D represents the variability between
subjects
• Explicitly partitition the variability
into within and between subject
components
– Marginal distribution:
Yi ∼ N (Xβ, ZiT DZi + R)
– Correlation induced through random
effects
520
521
Random Block Effects
Comparison of four processes for
producing penicillin
P rocess
P rocess
P rocess
P rocess
A
B
C
D
⎫
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎬
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎪
⎭
Here, batch effects are considered as
random block effects:
Fixed treatment factor
Blocks correspond to different batches of
an important raw material, corn steep
liquor
• Batches are sampled from a population
of many possible batches
• To repeat this experiment you would
need to use a different set of batches
of raw material
Data Source:
• Random sample of five batches
• Split each batch into four parts:
– run each process on one part
Box, Hunter & Hunter (1978),
Statistics for Experimenters,
Wiley & Sons, New York.
Data file:
penclln.dat
SAS code:
penclln.sas
R code:
penclln.R
– randomize the order in which the
processes are run within each batch
522
523
Model
Here
Yij =
↑
Yield
for the
i-th process
applied
to the
j-th batch
μ + αi
↑
mean
yield
for the
i-th process,
averaging
across the
entire
population
of
possible
batches
+bj
+eij
↑
↑
random random
batch
error
effect
μi = E(Yij ) = E(μ + αi + bj + eij )
= μ + αi + E(bj ) + E(eij )
= μ + αi
i = 1, 2, 3, 4
represents the mean yield for the i-th process, averaging across all possible batches.
PROC GLM and PROC MIXED in SAS
would fit a restricted model with α4 = 0.
Then
where
bj ∼ N ID(0, σb2 )
• μ = μ4 is the mean yield for process D
eij ∼ N ID(0, σe2 )
• αi = μi − μ4
i = 1, 2, 3, 4.
and any eij is independent of any bj .
524
In R or S-PLUS you could use the
”treatment” constraints where α1 = 0.
Then
525
Variance-covariance structure:
V ar(Yij ) = V ar(μ + αi + bj + eij )
= V ar(bj + eij )
• μ = μ1 is the mean yield for process A
• αi = μi − μ1
= V ar(bj ) + V ar(eij )
i = 1, 2, 3, 4.
= σb2 + σe2
Alternatively, you could choose the
solution to the normal equations
corresponding to the ”sum” constraints
where
for all (i, j)
Different runs on the same batch:
Cov(Yij , Ykj )
= Cov(μ + αi + bj + eij , μ + αk + bj + ekj )
• α1 + α2 + α3 + α4 = 0
• μ = (μ1 + μ2 + μ3 + μ4)/4 is an overall
mean yield
• αi = μi − μ is the difference between
the mean yield for the i-th process and
the overall mean yield.
526
= Cov(bj + eij , bj + ekj )
= Cov(bj , bj ) + Cov(bj , ekj ) + Cov(eij , bj )
+Cov(eij , ekj )
= V ar(bj ) = σb2
for all i = k
527
Correlation among yields for runs on the
same batch:
ρ =
⎡
Cov(Yij , Ykj )
V ar(Yij )V ar(Ykj )
σb2
=
Results from the four runs on a single
batch:
V ar
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
Y1j
Y2j
Y3j
Y4j
⎤
⎡
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
=
σβ2 + σe2
σb2
σb2
σb2
σb2 + σe2
σb2
σb2 + σe2
σb2
σb2
σb2
σb2
σb2
Cov(Yij , Yk) = 0
⎤
⎡
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
=
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
+ ⎢⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
0
0
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
1
0
0
0
0
0
0
0
0
1
1
1
1
0
0
0
0
0
0
0
0
σb2
+ σe2
This special type of covariance structure is
called compound symmetry
for j = Write this model as Y = Xβ + Zb + e
Y11
Y21
Y31
Y41
Y12
Y22
Y32
Y42
Y13
Y23
Y33
Y43
Y14
Y24
Y34
Y44
Y15
Y25
Y35
Y45
σb2
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
= σb2J + σe2 I
↑
↑
matrix
identity
of
matrix
ones
528
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
σb2
⎤
for i = k
σb2 + σe2
Results for runs on different batches are
uncorrelated (independent):
⎡
σb2
Here
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥⎡
⎥
⎥
⎥⎢
⎥
⎥⎢
⎥⎢
⎥⎢
⎥⎢
⎥⎣
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
0
0
0
0
529
D = V ar(b) = σb2I5×5
μ
α1
α2
α3
α4
R = V ar(e) = σe2 In×n
⎤
⎥
⎥
⎥
⎥
⎥
⎦
and
V ar(Y) = V ar(Xβ + Zb + e)
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1
1
⎤
⎡
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥⎡
⎥
⎥
⎥⎢
⎥
⎥⎢
⎥⎢
⎥⎢
⎥⎢
⎥⎣
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
b1
b2
b3
b4
b5
⎤
⎥
⎥
⎥
⎥
⎥
⎦
+
e11
e21
e31
e41
e12
e22
e32
e42
e13
e23
e33
e43
e14
e24
e34
e44
e15
e25
e35
e45
= V ar(Zb) + V ar(e)
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
= ZDZ T + R
= σb2 ZZ T + σe2I
⎡
=
530
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
σb2J + σe2I
⎤
σb2J
+
σe2I
...
σb2J + σe2I
531
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
Current status: Moisture content varied
Hierarchical Random Effects Model
Analysis of sources of variation in a
process used to monitor the production
of a pigment paste.
• about a mean of approximately 25
(or 2.5%)
• with standard deviation of about 6
Current Monitoring Procedure:
Problem: Variation in moisture content
was regarded as excessive.
• Sample barrels of pigment paste
• Take one sample from each barrel
Sources of Variation
• Send the sample to a lab for determination of moisture content
Measured Response: (Y ) moisture content of the pigment paste (units of one
tenth of 1%).
532
533
Model:
Yijk = μ + bi + δij + eijk
where
Data Collection: Hierarchical (or nested)
Study Design
• Sampled b barrels of pigment paste
Yijk is the moisture content determination
for the k-th part of the j -th sample
from the i-th barrel
μ is the overall mean moisture content
• s samples are teaken from the content
of each barrel
• Each sample is mixed and divided into
r parts. Each part is sent to the lab.
There are n = (b)(s)(r) observations.
bi is a random barrel effect:
bi ∼ N ID(0, σb2)
δij is a random sample effect:
δij ∼ N ID(0, σδ2)
eijk corresponds to random measurement
(lab analysis) error:
eijk ∼ N ID(0, σe2)
534
535
Covariance structure:
V ar(Yijk ) = V ar(μ + bi + δij + eijk )
= V ar(bi) + V ar(δij ) + V ar(eijk )
= σb2 + σδ2 + σe2
Two parts of one sample:
Cov(Yijk , Yij)
Observations from different barrels:
= Cov(μ + bi + δij + eijk , μ + bi + δij + eij)
Cov(Yijk , Ycm) = 0,
i = c
= Cov(bi, bi) + Cov(δij , δij )
= σb2 + σδ2
for k = Observations from different samples taken
from the same barrel:
Cov(Yijk , Yim) = Cov(bi, bi)
= σb2
j = m
536
Write this model in the form:
Y = Xβ + Zb + e
⎡
In this study
b = 15 barrels were sampled
s = 2 samples were taken from each barrel
r = 2 sub-samples were analyzed from each
sample taken from each barrel
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
Data file: pigment.dat
SAS code: pigment.sas
R code: pigment.R
537
Y111
Y112
Y121
Y122
Y211
Y212
Y221
Y222
...
Y15,1,1
Y15,1,2
Y15,2,1
Y15,2,2
1
1
1
1
0
0
0
0
...
0
0
0
0
0
0
0
0
1
1
1
1
...
0
0
0
0
⎤
⎡
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
=
...
...
...
...
...
...
...
...
...
...
...
...
...
0
0
0
0
0
0
0
0
...
1
1
1
1
1
1
1
1
1
1
1
1
...
1
1
1
1
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
1
1
0
0
0
0
0
0
...
0
0
0
0
[ μ ] +
0
0
1
1
0
0
0
0
...
0
0
0
0
0
0
0
0
1
1
0
0
...
0
0
0
0
0
0
0
0
0
0
1
1
...
0
0
0
0
...
...
...
...
...
...
...
...
...
...
...
...
...
0
0
0
0
0
0
0
0
...
1
1
0
0
0
0
0
0
0
0
0
0
...
0
0
1
1
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
b1
b2
...
b15
δ1,1
δ1,2
δ2,1
δ2,2
...
δ15,1
δ15,2
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
+e
538
Random Coefficients Model
where
R = V ar(e) =
σe2I
⎡
D = V ar(u) =
⎢
⎢
⎣
σb2I
0
σδ2I
0
Yij = β0 + β1tj + b0i + b1itj + eij
⎤
⎥
⎥
⎦
• bi = (b0i b1i)T and D is 2 × 2 with
elements dkl, k = 1, 2, l = 1, 2
Then
E(Y) = Xβ = 1μ
• separate regression line for each
subject, but borrow strength through
the distribution of bi
V ar(Y) = Σ = ZDZ T + R
⎡
= Z
⎢
⎢
⎢
⎢
⎣
⎤
σb2Ib 0
σδ2Ibs
0
⎥
⎥
⎥
⎥
⎦
Z T + σe2Ibsr
= σb2(Ib ⊗ Jsr ) + σδ2(Ibs ⊗ Jr ) + σe2Ibsr
because Z = Ib ⊗ 1sr Ibs ⊗ 1r
(sr) × 1
vector
of ones
• random effects design vector for the
j − th observation on the i − th subject
is Zij = (1, tj )
• Variance changes over time:
V ar(Yij ) = d11 + t2j d22 + 2tj d12 + σ 2
r×1
vector
of ones
• Cov(Yij , Yik ) = d11 + tj tk d22 + (tj + tk )d12
539
Estimation in Random Effects
Models
estimation
R)−1 (Yi
data set2;
infile ’c:\st565\dhlz.example1_4.data’;
input diet cow week protein;
if(protein ge 9.99) then protein=’.’;
run;
proc sort data=set2; by diet week; run;
• Predictions of random effects
b̂i = ZiD(ZiDZi +
Fitting Random effects Models in
SAS
/* Fit random effects models to
the milk protein data */
• Similar to marginal models
• Reference for likelihood
(Harville, 1977, JASA)
540
− Xiβ̂)
• Compromise between estimates based
solely on subject i and an overall estimate across all subjects
541
/* Fit cubic trends across time with a compound
symmetry covariance structure */
proc mixed data=set2;
class diet cow;
model protein = diet diet*week diet*week*week
diet*week*week*week
/ s outpm=means dfm=kr;
random intercept / subject=cow g;
run;
542
/* plot the fitted curves */
axis1 label=(f=swiss h=1.8 a=90 r=0 "Protein (percent)")
order = 2.5 to 4.5 by .5
value=(f=swiss h=1.8) w=3.0
length= 4.0in;
axis2 label=(f=swiss h=2.0 "Time(weeks)")
order = 0 to 20 by 5
value=(f=swiss h=1.8) w= 3.0
length = 6.5 in;
SYMBOL1 V=CIRCLE H=1.7 w=3 l=1 i=join ;
SYMBOL2 V=DIAMOND H=1.7 w=3 l=3 i=join ;
SYMBOL3 V=square H=1.7 w=3 l=9 i=join ;
PROC GPLOT DATA=means;
PLOT pred*week=diet /
vaxis=axis1 haxis=axis2;
TITLE1 ls=0.01in H=2.0 F=swiss "Estimated Protein Content";
footnote ls=0.01in;
RUN;
543
544
Dimensions
Covariance Parameters
Columns in X
Columns in Z Per Subject
Subjects
Max Obs Per Subject
Observations Used
Observations Not Used
Total Observations
The Mixed Procedure
Model Information
Data Set
Dependent Variable
Covariance Structure
Subject Effect
Estimation Method
Residual Variance Method
Fixed Effects SE Method
Degrees of Freedom Method
WORK.SET2
protein
Variance Components
cow
REML
Profile
Prasad-Rao-JeskeKackar-Harville
Kenward-Roger
Iteration History
Class Level Information
Class
diet
cow
Levels
3
79
2
13
1
79
19
1337
0
1337
Iteration
Evaluations
-2 Res Log Like
Criterion
0
1
2
1
2
1
730.59188696
438.77899831
438.77813937
0.00000086
0.00000000
Values
1 2 3
1 2 3
14 15
24 25
34 35
44 45
54 55
64 65
74 75
Estimated G Matrix
4 5 6
16 17
26 27
36 37
46 47
56 57
66 67
76 77
7 8 9
18 19
28 29
38 39
48 49
58 59
68 69
78 79
10
20
30
40
50
60
70
11
21
31
41
51
61
71
545
12
22
32
42
52
62
72
13
23
33
43
53
63
73
Row
1
Effect
cow
Intercept
1
Col1
0.02807
Covariance Parameter Estimates
Cov Parm
Subject
Intercept
Residual
cow
Estimate
0.02807
0.06546
546
Fit Statistics
Solution for Fixed Effects
-2 Res Log Likelihood
AIC (smaller is better)
AICC (smaller is better)
BIC (smaller is better)
438.8
442.8
442.8
447.5
Effect
Intercept
diet
diet
diet
week*diet
diet
Estimate
Standard
Error
DF
t Value
1
2
3
1
3.7948
0.1316
0.0930
0
-0.1601
0.06549
0.09450
0.09254
.
0.02555
699
700
698
.
1249
57.94
1.39
1.01
.
-6.27
diet
Intercept
diet
diet
diet
week*diet
1
2
3
1
Standard
Error
DF
t Value
2
3
1
2
3
1
2
3
-0.1890
-0.1696
0.01552
0.01925
0.01593
-0.00043
-0.00056
-0.00045
0.02459
0.02472
0.002994
0.002886
0.002901
0.000101
0.000097
0.000098
1249
1250
1250
1249
1250
1251
1250
1251
-7.69
-6.86
5.18
6.67
5.49
-4.23
-5.77
-4.58
Solution for Fixed Effects
Solution for Fixed Effects
Effect
Estimate
week*diet
week*diet
week*week*diet
week*week*diet
week*week*diet
week*week*week*diet
week*week*week*diet
week*week*week*diet
Solution for Fixed Effects
Effect
diet
Pr > |t|
<.0001
0.1642
0.3151
.
<.0001
Effect
diet
week*diet
week*diet
week*week*diet
week*week*diet
week*week*diet
week*week*week*diet
week*week*week*diet
week*week*week*diet
2
3
1
2
3
1
2
3
547
Pr > |t|
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
<.0001
548
Solution for Random Effects
Effect
cow
Estimate
Std Err
Pred
Intercept
Intercept
Intercept
Intercept
Intercept
Intercept
Intercept
Intercept
Intercept
Intercept
.
.
.
1
2
3
4
5
6
7
8
9
10
0.3252
-0.2416
-0.2709
-0.3007
0.1232
0.2452
0.003223
0.2385
0.006427
0.08659
0.06413
0.06413
0.07072
0.06517
0.06413
0.07072
0.06517
0.06524
0.06413
0.06413
DF
t Value
409 5.07
409 -3.77
444 -3.83
417 -4.61
409 1.92
444 3.47
417 0.05
417 3.66
417 0.10
417 1.35
Pr>|t|
<.0001
0.0002
0.0001
<.0001
0.0555
0.0006
0.9606
0.0003
0.9202
0.1777
/* Fit a model with random coefficients */
proc mixed data=set2;
class diet cow;
model protein = diet diet*week diet*week*week
diet*week*week*week
/ s outpm=means outp=pred dfm=kr;
random int week week*week / subject=cow s
g type=un v vcorr;
run;
/* Plot predicted values for some cows */
data predt; set pred; if(cow<6); run;
Type 3 Tests of Fixed Effects
proc sort data=predt; by cow week; run;
Effect
diet
week*diet
week*week*diet
week*week*week*diet
Num
DF
Den
DF
F Value
Pr > F
2
3
3
3
699
1249
1250
1251
1.04
48.49
33.83
24.03
0.3547
<.0001
<.0001
<.0001
549
proc print data=predt; run;
550
axis1 label=(f=swiss h=1.8 a=90 r=0 "Protein (percent)")
order = 2.5 to 4.5 by .5
value=(f=swiss h=1.8) w=3.0
length= 4.0in;
axis2 label=(f=swiss h=2.0 "Time(weeks)")
order = 0 to 20 by 5
value=(f=swiss h=1.8) w= 3.0
length = 6.5 in;
SYMBOL1
SYMBOL2
SYMBOL3
SYMBOL4
SYMBOL5
SYMBOL6
V=none
V=none
V=none
V=none
V=none
V=none
H=1.7
H=1.7
H=1.7
H=1.7
H=1.7
H=1.7
w=3
w=3
w=3
w=3
w=3
w=3
l=1 i=join ;
l=3 i=join ;
l=9 i=join ;
l=17 i=join ;
l=22 i=join ;
l=12 i=join ;
PROC GPLOT DATA=predt;
PLOT pred*week=cow /
vaxis=axis1 haxis=axis2;
TITLE1 ls=0.01in H=2.0 F=swiss "Predicted Protein Content";
footnote ls=0.01in;
RUN;
551
552
The Mixed Procedure
Dimensions
Model Information
Data Set
Dependent Variable
Covariance Structure
Subject Effect
Estimation Method
Residual Variance Method
Fixed Effects SE Method
Degrees of Freedom Method
Covariance Parameters
Columns in X
Columns in Z Per Subject
Subjects
Max Obs Per Subject
Observations Used
Observations Not Used
Total Observations
WORK.SET2
protein
Unstructured
cow
REML
Profile
Prasad-Rao-JeskeKackar-Harville
Kenward-Roger
7
13
3
79
19
1337
0
1337
Iteration History
Class Level Information
Class
diet
cow
Levels
3
79
Values
1 2 3
1 2 3
14 15
24 25
34 35
44 45
54 55
64 65
74 75
4 5 6
16 17
26 27
36 37
46 47
56 57
66 67
76 77
7 8 9
18 19
28 29
38 39
48 49
58 59
68 69
78 79
10
20
30
40
50
60
70
11
21
31
41
51
61
71
553
12
22
32
42
52
62
72
13
23
33
43
53
63
73
Iteration
Evaluations
0
1
2
3
4
5
6
7
1
2
1
1
1
1
1
1
-2 Res Log Like
Criterion
730.59188696
166.76227401
140.77244395
128.34035064
124.10501018
123.33585993
123.29664620
123.29650139
0.01759401
0.00853476
0.00301705
0.00058812
0.00003264
0.00000012
0.00000000
Convergence criteria met.
554
Estimated G Matrix
Row
Effect
1
2
3
cow
Intercept
week
week*week
1
1
1
Col1
Col2
Col3
0.08742
-0.01551
0.000727
-0.01551
0.006617
-0.00042
0.000727
-0.00042
0.000030
Covariance Parameter Estimates
Estimated V Correlation Matrix for cow 1
Row
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Col1
1.0000
0.5697
0.5044
0.4321
0.3629
0.3018
0.2500
0.2070
0.1716
0.1423
0.1181
0.0981
0.0825
0.0712
0.0645
0.0616
0.0614
0.0631
0.0658
Col2
0.5697
1.0000
0.5341
0.4911
0.4435
0.3969
0.3533
0.3124
0.2727
0.2318
0.1872
0.1378
0.08621
0.03812
-0.00106
-0.02965
-0.04904
-0.06155
-0.06933
Col3
Col4
Col5
Col6
0.5044
0.5341
1.0000
0.5249
0.4995
0.4686
0.4348
0.3984
0.3574
0.3088
0.2487
0.1757
0.0944
0.0151
-0.05203
-0.1031
-0.1396
-0.1649
-0.1823
0.4321
0.4911
0.5249
1.0000
0.5305
0.5144
0.4909
0.4605
0.4211
0.3689
0.2994
0.2105
0.1080
0.00581
-0.08247
-0.1510
-0.2011
-0.2368
-0.2622
0.3629
0.4435
0.4995
0.5305
1.0000
0.5386
0.5248
0.5010
0.4654
0.4136
0.3403
0.2431
0.1281
0.01124
-0.09129
-0.1721
-0.2321
-0.2757
-0.3073
0.3018
0.3969
0.4686
0.5144
0.5386
1.0000
0.5415
0.5248
0.4946
0.4464
0.3744
0.2755
0.1555
0.03128
-0.07948
-0.1681
-0.2348
-0.2840
-0.3202
Cov Parm
Subject
Estimate
UN(1,1)
UN(2,1)
UN(2,2)
UN(3,1)
UN(3,2)
UN(3,3)
Residual
cow
cow
cow
cow
cow
cow
0.08742
-0.01551
0.006617
0.000727
-0.00042
0.000030
0.03983
Fit Statistics
-2 Res Log Likelihood
AIC (smaller is better)
AICC (smaller is better)
BIC (smaller is better)
123.3
137.3
137.4
153.9
Null Model Likelihood Ratio Test
DF
Chi-Square
Pr > ChiSq
6
607.30
<.0001
555
556
Solution for Random Effects
Effect
Solution for Fixed Effects
Effect
Intercept
diet
diet
diet
week*diet
week*diet
week*diet
week*week*diet
week*week*diet
week*week*diet
week*week*week*diet
week*week*week*diet
week*week*week*diet
diet
1
2
3
1
2
3
1
2
3
1
2
3
Estimate
Standard
Error
DF
t Value
3.8451
0.1299
0.1037
0
-0.1990
-0.2369
-0.2098
0.0226
0.0279
0.0234
-0.0007
-0.0009
-0.0008
0.0724
0.1046
0.1024
.
0.0261
0.0251
0.0252
0.00270
0.00261
0.00263
0.00008
0.00008
0.00008
116
117
116
.
269
269
272
807
806
811
1169
1171
1172
53.06
1.24
1.01
.
-7.61
-9.41
-8.30
8.36
10.69
8.92
-8.83
-11.60
-9.55
Intercept
week
week*week
Intercept
week
week*week
Intercept
week
week*week
Intercept
week
week*week
Intercept
week
week*week
.
.
cow
Estimate
Std Err
Pred
DF
1
1
1
2
2
2
3
3
3
4
4
4
5
5
5
0.03921
-0.00188
0.003093
-0.3790
0.02334
-0.00023
0.01524
-0.02565
-0.00145
-0.2091
-0.05969
0.004394
0.1802
0.003571
-0.00010
0.1466
0.03528
0.001922
0.1466
0.03528
0.001922
0.1644
0.04908
0.003210
0.1496
0.03731
0.002080
0.1466
0.03528
0.001922
315
300
258
315
300
258
263
248
268
308
304
292
315
300
258
Pr>|t|
0.27
-0.05
1.61
-2.59
0.66
-0.12
0.09
-0.52
-0.45
-1.40
-1.60
2.11
1.23
0.10
-0.05
0.7893
0.9575
0.1088
0.0102
0.5088
0.9028
0.9262
0.6017
0.6521
0.1631
0.1107
0.0355
0.2199
0.9194
0.9572
Type 3 Tests of Fixed Effects
Effect
diet
week*diet
week*week*diet
week*week*week*diet
557
t Value
Num
DF
Den
DF
F Value
Pr > F
2
3
3
3
116
270
808
1170
0.88
71.82
87.90
101.28
0.4181
<.0001
<.0001
<.0001
558
# Compute sample means
lme function in R or S-Plus
#
#
#
#
#
This file posted as
milkprotein3.R
means <- tapply(set1$protein,
list(set1$week,set1$diet),mean)
means
This code is applied to the milk protein
data from DHLZ, page 8. Missing data
are not included.
set1 <- read.table("c:/st565/dhlz.example1_4.data",
col.names=c("diet","cow","week","protein"))
#
#
Make a profile plot of the means
Unix users should insert the motif( )
# Create factors
#
command
set1$dietf <- as.factor(set1$diet)
set1$weekf <- as.factor(set1$week)
set1$cowf <- as.factor(set1$cow)
x.axis <- unique(set1$week)
#
par(fin=c(6.0,6.0),pch=18,mkh=.1,mex=1.5,
cex=1.2,lwd=3)
matplot(c(1,19), c(3.0,4.0), type="n",
xlab="Time(weeks)", ylab="Protein (percent)",
main= "Observed Protein Means")
matlines(x.axis,means,type=’l’,lty=c(1,3,7))
matpoints(x.axis,means, pch=c(16,17,15))
legend(1,2.45,legend=c("Barley diet",
’Barley+lupins’,’Lupin diet’),lty=c(1,3,7),bty=’n’)
Sort the data set by subject
i <- order(set1$cow,set1$week)
set1 <- set1[i,]
# Delete the list called i
rm(i)
559
560
Fixed effects: protein ~ dietf + week + dietf * week
+ week^2 + dietf * week^2
+ week^3 + dietf * week^3
#
#
Use the lme( ) function to fit a
model with random effects
options(contrasts=c("contr.treatment","contr.poly"))
set1.lme <- lme(protein ~ dietf+ week+
dietf*week+week^2+dietf*week^2 +
week^3 + dietf*week^3,
data=set1,
random = ~ 1 | cow,
method=c("REML"))
summary(set1.lme)
Linear mixed-effects model fit by REML
Data: set1
AIC
BIC
logLik
466.7781 539.4265 -219.3891
(Intercept)
dietf2
dietf3
week
I(week^2)
I(week^3)
dietf2week
dietf3week
dietf2I(week^2)
dietf3I(week^2)
dietf2I(week^3)
dietf3I(week^3)
Value
3.926357
-0.038575
-0.131599
-0.160102
0.015522
-0.000427
-0.028928
-0.009489
0.003723
0.000407
-0.000134
-0.000021
Std.Error
0.06811872
0.09441989
0.09449547
0.02554696
0.00299418
0.00010094
0.03545609
0.03554716
0.00415875
0.00416914
0.00014023
0.00014057
DF
1249
76
76
1249
1249
1249
1249
1249
1249
1249
1249
1249
t-value p-value
57.63991 <.0001
-0.40855 0.6840
-1.39264 0.1678
-6.26697 <.0001
5.18418 <.0001
-4.23121 <.0001
-0.81587 0.4147
-0.26694 0.7896
0.89519 0.3709
0.09754 0.9223
-0.95631 0.3393
-0.14699 0.8832
anova(set1.lme)
Number of Observations: 1337
Number of Groups: 79
numDF denDF F-value p-value
(Intercept)
1 1249 28937.95 <.0001
dietf
2
76
8.42 0.0005
week
1 149
20.06 <.0001
I(week^2)
1 1249
109.03 <.0001
I(week^3)
1 1249
71.03 <.0001
dietf:week
2 1249
3.55 0.0290
dietf:I(week^2)
2 1249
0.06 0.9459
dietf:I(week^3)
2 1249
0.54 0.5831
Random effects:
Formula: ~ 1 | cow
(Intercept) Residual
StdDev:
0.1675376 0.2558492
561
562
ranef(set1.lme)
(Intercept)
1 0.32519586024
2 -0.24155619694
3 -0.27091288079
4 -0.30072984598
5 0.12315273646
.
.
.
.
79 0.32079113086
Random Effects:
Level: cow
lower
est.
upper
sd((Intercept)) 0.1396647 0.1675376 0.2009732
Within-group standard error:
lower
est.
upper
0.2460085 0.2558492 0.2660834
intervals(set1.lme)
Approximate 95% confidence intervals
#
#
Use the lme( ) function to fit a
model with random coefficients
Fixed effects:
(Intercept)
dietf2
dietf3
week
I(week^2)
I(week^3)
dietf2week
dietf3week
dietf2I(week^2)
dietf3I(week^2)
dietf2I(week^3)
dietf3I(week^3)
lower
3.7927171859
-0.2266283337
-0.3198026229
-0.2102218035
0.0096481836
-0.0006251481
-0.0984877756
-0.0792276391
-0.0044360202
-0.0077726367
-0.0004092187
-0.0002964424
est.
upper
3.9263569251 4.0599966643
-0.0385748259 0.1494786818
-0.131598525
0.0566054378
-0.1601021051 -0.1099824066
0.0155223621 0.0213965405
-0.0004271114 -0.0002290746
-0.0289277043 0.0406323671
-0.0094889137 0.0602498117
0.0037228947 0.0118818097
0.0004066622 0.0085859612
-0.0001341042 0.0001410104
-0.0000206624 0.0002551176
set1.lme <- lme(protein ~ dietf+ week+
dietf*week+week^2+dietf*week^2 +
week^3 + dietf*week^3,
data=set1,
random = ~ 1+week+week^2 | cow,
method=c("REML"))
summary(set1.lme)
anova(set1.lme)
ranef(set1.lme)
intervals(set1.lme)
563
Linear mixed-effects model fit by REML
Data: set1
AIC
BIC
logLik
161.2965 259.8907 -61.64825
564
Correlation:
Random effects:
Formula: ~ 1 + week + week^2 | cow
Structure: General positive-definite
StdDev
Corr
(Intercept) 0.295664925 (Intr) week
week 0.081347490 -0.645
I(week^2) 0.005501347 0.447 -0.947
Residual 0.199562253
Fixed effects: protein ~ dietf + week + dietf * week
+ week^2 + dietf * week^2 + week^3
+ dietf * week^3
Value Std.Error
DF
t-value p-value
(Intercept) 3.974995 0.0753535 1249 52.75128 <.0001
dietf2 -0.026205 0.1044958
76 -0.25077 0.8027
dietf3 -0.129897 0.1045439
76 -1.24251 0.2179
week -0.199019 0.0261378 1249 -7.61421 <.0001
I(week^2) 0.022631 0.0027043 1249
8.36843 <.0001
I(week^3) -0.000776 0.0000876 1249 -8.85542 <.0001
dietf2week -0.037903 0.0362780 1249 -1.04479 0.2963
dietf3week -0.010783 0.0363472 1249 -0.29666 0.7668
dietf2I(week^2) 0.005314 0.0037583 1249
1.41381 0.1577
dietf3I(week^2) 0.000852 0.0037718 1249
0.22598 0.8213
dietf2I(week^3) -0.000213 0.0001220 1249 -1.74177 0.0818
dietf3I(week^3) -0.000048 0.001228 1249 -0.39441 0.6933
565
dietf2
dietf3
week
I(week^2)
I(week^3)
dietf2week
dietf3week
dietf2I(week^2)
dietf3I(week^2)
dietf2I(week^3)
dietf3I(week^3)
dietf3week
dietf2I(week^2)
dietf3I(week^2)
dietf2I(week^3)
dietf3I(week^3)
(Intr)
-0.721
-0.721
-0.754
0.596
-0.439
0.543
0.542
-0.429
-0.428
0.315
0.313
dietf2 dietf3
0.520
0.544
-0.430
0.317
-0.754
-0.391
0.596
0.308
-0.439
-0.226
0.544
-0.430
0.317
-0.392
-0.754
0.309
0.596
-0.227
-0.438
week I(wk^2 I(w^3)
-0.933
0.710 -0.896
-0.720 0.672 -0.512
-0.719 0.671 -0.511
0.671 -0.720 0.644
0.669 -0.717 0.642
-0.510 0.643 -0.718
-0.507 0.639 -0.714
dtf2wk dtf3wk d2I(^2) d3I(^2) d2I(^3)
0.518
-0.933 -0.483
-0.482 -0.933
0.516
0.710
0.367 -0.896
-0.461
0.365
0.710 -0.460
-0.896
0.512
Standardized Within-Group Residuals:
Min
Q1
Med
Q3
Max
-3.752192 -0.5654391 0.02836916 0.5621147 3.434687
566
Approximate 95% confidence intervals
Fixed effects:
Number of Observations: 1337
Number of Groups: 79
numDF denDF F-value p-value
(Intercept)
1 1249 29705.56 <.0001
dietf
2
76
7.72 0.0009
week
1 1249
38.47 <.0001
I(week^2)
1 1249
1.89 0.1694
I(week^3)
1 1249
302.27 <.0001
dietf:week
2 1249
0.53 0.5895
dietf:I(week^2)
2 1249
0.06 0.9374
dietf:I(week^3)
2 1249
1.69 0.1859
(Intercept)
dietf2
dietf3
week
I(week^2)
I(week^3)
dietf2week
dietf3week
dietf2I(week^2)
dietf3I(week^2)
dietf2I(week^3)
dietf3I(week^3)
(Intercept
week
I(week^2)
1 0.039206052 -0.001883403 0.00309339114
2 -0.378983496 0.023338731 -0.00023488864
3 0.015243884 -0.025647900 -0.00144883744
4 -0.209119127 -0.059693792 0.00439366422
5 0.180158852 0.003570968 -0.00010320278
.
.
79 0.419292873 0.028281414 -0.00396485404
lower
3.8271613446
-0.2343261474
-0.3381143060
-0.2502976761
0.0173252299
-0.0009477113
-0.109073817
-0.0820911147
-0.0020597720
-0.0065474101
-0.0004518548
-0.0002893208
est.
upper
3.97499479114 4.12282823766
-0.02620469306 0.18191676129
-0.12989706460 0.07832017678
-0.19901880219 -0.14773992829
0.02263068817 0.02793614644
-0.00077583067 -0.00060395004
-0.03790280224 0.03326977719
-0.01078290812 0.06052529845
0.00531358071 0.01268693342
0.00085236560 0.00825214129
-0.00021250127 0.00002685228
-0.00004842894 0.00019246293
Random Effects:
Level: cow
lower
est.
upper
sd((Intercept)) 0.240068896 0.295664925 0.36413608
sd(week) 0.065897066 0.081347490 0.10042047
sd(I(week^2)) 0.004486629 0.005501347 0.00674556
cor((Intercept),week) -0.778682934 -0.644780518 -0.45470071
cor((Intercept),I(week^2)) 0.204690609 0.447225855 0.63802215
cor(week,I(week^2)) -0.969644061 -0.947480303 -0.90987460
Within-group standard error:
lower
est.
upper
0.1913373 0.1995623 0.2081408
567
568