The Cochran-Satterthwaite Approximation
for Linear Combinations of Mean Squares
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
1 / 18
Suppose M1 , ..., Mk are independent mean squares and that
di Mi
∼ χ2di
E(Mi )
∀ i = 1, . . . , k.
It follows that
di Mi
E(Mi ) 2
di Mi
= di , Var
= 2di , and Mi ∼
χdi
E
E(Mi )
E(Mi )
di
for all i = 1, . . . , k.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
2 / 18
Consider the random variable
M = a1 M1 + a2 M2 + · · · + ak Mk ,
where a1 , a2 , . . . , ak are known constants in R.
Note that M is a linear combination of scaled χ2 random
variables.
The Cochran-Satterthwaite approximation works by assuming
that M is approximately distributed as a scaled χ2 , just like each
of the variables in the linear combination.
dM · 2
· E(M) 2
χd .
∼ χd ⇐⇒ M ∼
E(M)
d
What choice for d makes the approximation most reasonable?
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
3 / 18
If
·
M∼
E(M) 2
χd ,
d
then
E(M)
d
2
E(M)
d
2
Var(M) ≈
=
Var χ2d
(2d)
2 [E(M)]2
=
d
2
2M
≈
.
d
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
4 / 18
Now note that
Var(M) = a21 Var(M1 ) + · · · + a2k Var(MK )
=
a21
h
E(M1 )
d1
i2
2d1 + · · · +
= 2
Pk
a2i [E(Mi )]2
di
≈ 2
Pk
a2i Mi2 /di .
i=1
i=1
c
Copyright 2016
Dan Nettleton (Iowa State University)
a2k
h
E(Mk )
dk
i2
2dk
Statistics 510
5 / 18
Equating these two variance approximations yields
k
X
2M 2
=2
a2i Mi2 /di .
d
i=1
Solving for d yields
d = Pk
M
2
2 2
i=1 ai Mi /di
P
= Pk
k
i=1
ai Mi
2
2 2
i=1 ai Mi /di
.
This is the Cochran-Satterthwaite formula for the approximate
degrees of freedom associated with the linear combination of
mean squares defined by M.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
6 / 18
Recall the first example from the last slide set.


y111
 y121 

y=
 y211  ,
y212

1
 1
X=
 0
0
X1 = 1,

0
0 
,
1 
1
X2 = X,

1
 0
Z=
 0
0
0
1
0
0

0
0 

1 
1
X3 = Z
y0 (P2 − P1 )y + y0 (P3 − P2 )y + y0 (I − P3 )y = y0 (I − P1 )y
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
7 / 18
Expected Mean Squares
SOURCE EMS
trt
1.5σu2 + σe2 + (τ1 − τ2 )2
xu(trt)
σu2 + σe2
ou(xu, trt)
σe2
E(1.5MSxu(trt) − 0.5MSou(xu,trt) ) = 1.5(σu2 + σe2 ) − 0.5σe2
c
Copyright 2016
Dan Nettleton (Iowa State University)
= 1.5σu2 + σe2
Statistics 510
8 / 18
An Approximate F Test
The statistic
F=
MStrt
1.5MSxu(trt) − 0.5MSou(xu,trt)
is approximately F distributed with 1 numerator degree of
freedom and denominator degrees freedom approximated by the
Cochran-Satterthwaite Method:
d=
(1.5MSxu(trt) − 0.5MSou(xu,trt) )2
2
2 .
(1.5)2 MSxu(trt) + (−0.5)2 MSou(xu,trt)
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
9 / 18
SAS Code for Example
data d;
input trt xu y;
cards;
1 1 6.4
1 2 4.2
2 1 1.5
2 1 0.9
;
run;
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
10 / 18
SAS Code for Example
proc mixed method=type1;
class trt xu;
model y=trt / ddfm=satterthwaite;
random xu(trt);
run;
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
11 / 18
The Mixed Procedure
Model Information
Data Set
Dependent Variable
Covariance Structure
Estimation Method
Residual Variance Method
Fixed Effects SE Method
Degrees of Freedom Method
WORK.D
y
Variance Components
Type 1
Factor
Model-Based
Satterthwaite
Class Level Information
Class
trt
xu
Levels
2
2
Values
1 2
1 2
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
12 / 18
Dimensions
Covariance Parameters
Columns in X
Columns in Z
Subjects
Max Obs Per Subject
2
3
3
1
4
Number of Observations
Number of Observations Read
Number of Observations Used
Number of Observations Not Used
c
Copyright 2016
Dan Nettleton (Iowa State University)
4
4
0
Statistics 510
13 / 18
Type 1 Analysis of Variance
Source
trt
xu(trt)
Residual
DF
Sum of
Squares
Mean Square
1
1
1
16.810000
2.420000
0.180000
16.810000
2.420000
0.180000
Source
Expected Mean Square
Error Term
trt
Var(Residual)+1.5
Var(xu(trt))+Q(trt)
1.5 MS(xu(trt))
- 0.5 MS(Residual)
xu(trt)
Var(Residual)+Var(xu(trt))
MS(Residual)
Residual
Var(Residual)
.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
14 / 18
Degrees of Freedom for Satterthwaite Approximation
d =
=
(1.5MSxu(trt) − 0.5MSou(xu,trt) )2
2
2
(1.5)2 MSxu(trt) + (−0.5)2 MSou(xu,trt)
(1.5 × 2.42 − 0.5 × 0.18)2
(1.5)2 [2.42]2 + (−0.5)2 [0.18]2
= 0.9504437
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
15 / 18
Type 1 Analysis of Variance
Source
trt
xu(trt)
Residual
Error
DF
F Value
Pr > F
0.9504
1
.
4.75
13.44
.
0.2840
0.1695
.
Covariance Parameter
Estimates
Cov Parm
Estimate
xu(trt)
Residual
2.2400
0.1800
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
16 / 18
Concluding Remarks
This example was chosen to be small so that we could write out
all the data and see how each observation was involved in the
analysis.
It would be surprising if the approximate F test worked well for
this example because of the very low sample size.
It would be difficult to draw any meaningful conclusions with 4
observations of the response on 3 experimental units.
We will see more practically relevant examples where the
samples sizes are larger and the approximate F-based
inferences may be reasonable.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
17 / 18
Concluding Remarks
In more complicated examples, there may be more than one
linear combination of mean squares with the desired
expectation.
In such cases, linear combinations with non-negative
coefficients are recommended over those with a mix of positive
and negative coefficients.
c
Copyright 2016
Dan Nettleton (Iowa State University)
Statistics 510
18 / 18