Repeated Measures Example
In an exercise therapy study, subjects were assigned to one of three
Measurements
of
strength
(Y )
were taken on days 2, 4, 6, 8, 10,
weightlifting programs
12 and 14 for each subject.
(i=1) The number of repetitions of
weightlifting was increased as
subjects became stronger (RI)
(i=2) The
amount
of
weight
was
Source:
Littel,
Freund,
and
Spector
(1991) SAS System for Linear
Models
increased as subjects became
R code: RepeatedMeasures.R
stronger (WI)
(i=3) Subjects did not participate in
weightlifting (XCont)
1
2
Mixed model
Yijk = μ + αi + Sij + τk + γik + eijk
Yijk strength measurement for
prog. i, subj. j , time point k
αi
Sij
τk
γik
eijk
fixed program effect
Average strength after 2k days on
the i-th program is
μik = E(Yijk )
= E(μ + αi + Sij + τk + γik + eijk )
= μ + αi + E(Sij ) + τk + γik + E(eijk )
random subject effect
= μ + αi + τk + γik
fixed time effect
for i = 1, 2, 3 and k = 1, 2, . . . , 7
fixed prog.×time interaction
The variance of any single
observation is
random error
where the random effects are all
independent and
V ar(Yijk ) = V ar(μ + αi + Sij + τk + αik + eijk )
= V ar(Sij + eijk )
= V ar(Sij ) + V ar(eijk )
2)
Sij ∼ N ID(0, σS
ijk ∼ N ID(0, σ2)
2 + σ2
= σS
e
3
4
Correlation between observations
taken on the same subject:
Cov(Yijk , Yij)
= Cov(μ + αi + Sij + τk + γik + eijk ,
μ + αi + Sij + τ + γi + eij)
The correlation between Yijk and
Yij is
2
σS
2 + σ2
σS
e
= Cov(Sij + eijk , Sij + eij)
= Cov(Sij , Sij ) + Cov(Sij , eij)
Observations
+Cov(eijk , Sij ) + Cov(eijk , eij)
= V ar(Sij )
≡ρ
taken
on
different
subjects are uncorrelated.
for k = 2.
= σS
5
6
For the set of observations taken
on a single subject, we have
⎛⎡
V ar
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎜⎢
⎝⎣
Yij1
Yij3
..
Yij7
Write this model in the form
⎤⎞
Y = Xβ + Zu + e
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎥⎟
⎦⎠
⎡
2
2
σe2 + σS
σS
2
2
σS
σe2 + σS
=
..
..
2
2
σS
σS
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
⎡
2
···
σS
2
···
σS
2
...
σS
2 σ2 + σ2
σS
e
S
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
2J
= σe2I + σS
↑
J is a matrix of ones
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
Y111
Y112
...
Y117
Y121
Y122
...
Y127
...
Y211
Y212
...
Y217
...
Y3,n31
Y3,n32
...
Y3,n37
⎤
⎡
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
=
1 1 0
1 1 0
...
1 1 0
1 1 0
1 1 0
...
1 1 0
...
1 0 1
1 0 1
...
1 0 1
...
1 0 0
1 0 0
...
...
1 0 0
0 1000000 |
0 0100000 |
...
|
0 0000001 |
0 1000000 |
0 0100000 |
...
|
0 0000001 |
|
0 1000000 |
0 0100000 |
...
|
0 0000001 |
|
1 1000000 |
1 0100000 |
...
|
1 0000001 |
⎤
21
columns
for
interaction
This covariance structure is called
compound symmetry.
7
8
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
μ
α1
α2
α3
τ1
τ2
τ3
τ4
τ5
τ6
τ7
γ11
γ12
...
γ37
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
In this case:
⎡
1
1
...
1
0
0
...
+ 0
...
...
...
0
0
...
0
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
0
0
...
0
1
1
...
1
...
...
...
0
0
...
0
··· 0
··· 0
...
··· 0
··· 0
··· 0
...
··· 0
...
...
...
··· 1
··· 1
...
··· 1
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
⎡
⎤
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
e111
e112
...
e117
e121
e122
⎤
...
S11 ⎥⎥
⎥
e127
S12 ⎥⎥⎥
...
... ⎥⎥⎥
... ⎥⎥⎥⎥ + e
211
... ⎥⎥⎥⎥
e
212
⎥
⎦
...
S3,n3
e217
...
e3,n31
e3,n32
...
e3,n37
R = V ar(e) = σe2I(7r)×(7r)
2I
G = V ar(u) = σS
r×r
where r is the number
of subjects
Σ = V ar(Y) = ZGZ T + R
is a block diagonal
matrix with one block
of the form
2J
(σe2I7×7 + σS
7×7)
for each subject
9
Mean strength at a
10
Program means
particular time
(averaging across time)
in a particular program
LSM EAN = μ̂ + α̂i + τ̂k + γ̂ik
= Ȳi.k =
V ar(Ȳi.k ) =
SȲ
i.k
=
⎛
⎜
⎜
⎜
⎝
1 ni
ni j=1
= μ̂ + α̂i +
Yijk
V ar(Ȳi...) =
2
σe2 + σS
ni
⎞
1
⎟ 1
M Serror + M SSubj ⎟⎟⎠
7
7
ni
6
LSM EAN = Ȳi..
↑
Cochran-Satterthwaite
degrees of freedom are 65.8
11
SȲ
i..
=
1 7
(τ̂k + γ̂ik )
7 k=1
2
σe2 + 7σS
7ni
M Ssubjects
2ni
There are n1 = 16, n2 = 21, n3 = 20
subjects in the three programs.
12
Mean strength at a
particular time point
Difference between strength
(averaging across programs)
means at two time points
LSM EAN = μ̂ + τ̂k +
=
1 3
(α̂i + γ̂ik )
3 i=1
1 3
(Ȳi.k )
3 i=1
=
(averaging across programs)
Ȳ..k
⎛
⎝
because n1 = 16, n2 = 21, n3 = 20.
V ar(LSM EAN ) =
SLSM EAN =
3
⎝
7
1
⎞⎛
M Serror + M Ssubj
7
3
⎟⎜ ⎟⎜
⎠⎝
⎛
⎞
⎞
⎞
1 ⎛⎜⎜ 1 ⎛⎜⎜ 3
3
⎟
⎟
= ⎝
Ȳi.k ⎠ − ⎝
Ȳi.⎟⎟⎠
3 i=1
3 i=1
1 3 1 n
i
=
(Yijk − Yij)
i=1
j=1
3
ni
2
2
1 3 σe + σS
9 i=1
ni
⎛
1
⎜ 6
⎜
⎞
τ̂k + 13 3i=1 γ̂ik ⎠ − ⎝τ̂ − 13 3i=1 γ̂i⎠
1
i=1 ni
↑
subject effects
cancel out
⎞
⎟
⎟
⎠
↑
Cochran-Satterthwaite
degrees of freedom are 65.8
13
14
Difference between strength
means for two programs
at a specific time point
Variance formula:
2σe2 3 1
9 Σi=1 ni
(μ̂ + α̂i + τ̂ik + γ̂ik ) − (μ̂ + α̂ + τ̂k + γ̂k )
= Ȳi.k − Ȳ.k
Standard error:
2M Serror Σ3 1 = 0.206
i=1 ni
9
2 )(
V ar(Ȳi.k − Ȳ.k ) = (σe2 + σS
↑
use degrees of freedom
for error = 324
SȲ −Ȳ =
i.k
.k
⎛
⎜
⎜
⎝
6
1
ni
+
1
n
⎞⎛
⎞
1
1 ⎟⎟
⎟⎜ 1
M Serror + M Ssubj ⎟⎠ ⎜⎝ +
⎠
7
7
ni
n
↑
Use Cochran-Satterthwaite
degrees of freedom = 65.8
15
)
16
Difference between strength
Difference in strength
means at two time points
means for two programs
within a particular program
(averaging across time
points)
(μ̂ + α̂i + τ̂k + γ̂ik ) − (μ̂ + α̂i + τ̂ + γ̂i)
Ȳi.. − Ȳ.. = (α̂i −
= Ȳi.k − Ȳi.
V ar(Ȳi.k − Ȳi.) =
⎛
⎞
n
i
V ar ⎜⎝ n1 Σj=1
(Yijk − Yij)⎟⎠ =
i
⎛
1 1 7
7
γ̂ik ) − (α̂ +
γ̂k )
7 k=1
7 k=1
2 )(
V ar(Ȳi.. − Ȳ..) = (σe2 + 7σS
2σe2
ni
⎞
SȲ −Ȳ = M Serror ⎜⎝ n2 ⎟⎠
i.k
i.
i
=
SȲ −Ȳ
i..
..
↑
use degrees of freedom
for error=324
1
7ni
M Ssubjects(
+
1
7ni
1
7n
+
)
1
7n
)
↑
54 d.f.
17
18
Specifying other covariance
This model was expressed in the
matrices
form
We began with the model
Y = Xβ + Zu + e
Yijk = μ + αi + Sij + τk + γik + eijk
where
where
⎡
⎤
S11 ⎥⎥⎥
.. ⎥⎥⎥
u=
⎥
⎥
S3,20 ⎦
contained the random subject effects
2)
Sij ∼ N ID(0, σS
eijk ∼ N ID(0, σe2)
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
and the {Sij } are distributed
independently of the {eijk }
19
20
If you are not interested in predict-
Here
ing subject effects (random subject
2I
G = V ar(u) = σS
effects are included only to introduce correlation among repeated
R = V ar(e) = σe2I
measures on the same subject), you
can work with an alternative expres-
Σ = V ar(Y) = ZGZ T + R
⎡
2
= σS
⎡
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
=
2J
σe2I + σS
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
J
sion of the same model
⎤
J
...
J
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
Y = Xβ + e∗
+ σe2I
where
⎤
...
2J
σe2I + σS
R = V ar(e∗)
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
⎡
=
where J is a matrix of ones.
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
2J
σe2I + σS
⎤
...
2J
σe2I + σS
21
22
Replace the mixed model
Y = Xβ + Zu + e
First Order Autoregressive:
AR(1)
⎡
with the model
Y = Xβ + e∗
W = σ2
where
⎡
⎤
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
W 0 ··· 0
0 W ··· 0
V ar(Y) = V ar(e∗) =
............
0 0 ··· W
23
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎢
⎣
ρ
ρ2 ρ3
ρ
1
ρ
ρ2
ρ2
ρ
1
ρ
ρ3 ρ2
ρ
1
1
⎤
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦
where σ 2 ∈ (0, ∞) and ρ ∈ (−1, 1) are
unknown parameters.
24
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎥
⎦