General Linear Test of a General Linear
Hypothesis
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 35
Suppose the NTGMM holds so that
y = Xβ + ε,
where
c
Copyright 2012
Dan Nettleton (Iowa State University)
ε ∼ N(0, σ 2 I).
Statistics 611
2 / 35
Suppose C is a known q × p matrix and d is a known q × 1 vector.
The general linear hypothesis
H0 : Cβ = d
is testable if rank(C) = q and each component of Cβ is estimable.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
3 / 35
Supposep×m
A of rank s.
Can H0 : β ∈ C(A) be written as a testable general linear hypothesis?
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
4 / 35
Suppose
yijk = µ + αi + βj + γij + εijk ,

µ
i = 1, 2; j = 1, 2; k = 1, . . . , nij .

 
 α1 
 
 
 α2 
 
 
 β1 
 

Let β = 
 β2  .
 
γ11 
 
 
γ12 
 
γ 
 21 
γ22
Write a testable general linear hypothesis for “no interaction.”
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
6 / 35
Suppose
H0 : Cβ = d
is testable.
Find the distribution of the BLUE of Cβ.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
9 / 35
Find the distribution of
(Cβ̂ − d)0 (σ 2 C(X0 X)− C0 )−1 (Cβ̂ − d).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
11 / 35
Show that Cβ̂ and SSE are independent.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
13 / 35
Now note that independence of Cβ̂ and SSE =⇒
(Cβ̂ − d)0 (σ 2 C(X0 X)− C0 )−1 (Cβ̂ − d)
and
SSE
y0 (I − PX )y
σ̂ 2
=
=
σ 2 (n − r)
σ 2 (n − r)
σ2
are independent.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
18 / 35
We have previously shown that
SSE
∼ χ2n−r .
σ2
Thus,
(n − r)σ̂ 2
∼ χ2n−r
σ2
and
σ̂ 2
∼ χ2n−r /(n − r).
σ2
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
19 / 35
It follows that
(Cβ̂ − d)0 (σ 2 C(X0 X)− C0 )−1 (Cβ̂ − d)/q
σ̂ 2 /σ 2
=
(Cβ̂ − d)0 (C(X0 X)− C0 )−1 (Cβ̂ − d)
≡F
qσ̂ 2
∼ Fq,n−r (φ),
where
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
20 / 35
φ=
1
(Cβ − d)0 (C(X0 X)− C0 )−1 (Cβ − d)
2σ 2
as defined previously.
We can use F to test
H0 : φ = 0 ⇐⇒ H0 : Cβ − d = 0
(∵ (C(X0 X)− C0 )−1 is PD.)
⇐⇒ H0 : Cβ = d.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
21 / 35
To test
H0 : Cβ = d
at level α, we reject H0 iff
F ≥ Fq,n−r,α
where Fq,n−r,α is the upper α quantile of the Fq,n−r distribution.
By Result 5.13, the power of the test is a strictly increasing function of
φ.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
22 / 35
Now suppose that
H0 : c0 β = d
is testable.
By arguments analogous to the previous F case, it is straightforward to
show that
c0 β̂ − d
t≡ p
∼ tn−r
σ̂ 2 c0 (X0 X)− c
c
Copyright 2012
Dan Nettleton (Iowa State University)
c0 β − d
p
σ 2 c0 (X0 X)− c
!
.
Statistics 611
23 / 35
We can conduct tests of
H0 : c0 β = d
against
HA1 : c0 β < d
HA2 : c0 β > d,
or
HA : c0 β 6= d
by comparing the observed value of t to the tn−r distribution.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
24 / 35
Returning to the F-test of testable GLH
H0 : Cβ = d,
note that there are multiple ways to express the same null hypothesis.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
25 / 35
For example, suppose
yij = µi + εij
(i = 1, 2, 3; j = 1, . . . , ni ).
Find different matrices C1 , C2 and C3 3
 
µ1
 

Ck 
µ2  = 0 ⇐⇒ µ1 = µ2 = µ3
µ3
c
Copyright 2012
Dan Nettleton (Iowa State University)
∀ k = 1, 2, 3.
Statistics 611
26 / 35
C1 =
C2 =
"
C3 =
"
1 −1
1
0
"
1 −1
1
0
1
−1
#
0
C1 µ =
#
0
C2 µ =
"
#
,
C3 µ =
µ1 − µ2
#
µ1 − µ3
"
,
−1
1/2 1/2 −1
c
Copyright 2012
Dan Nettleton (Iowa State University)
,
−1
0
"
µ1 − µ2
#
µ2 − µ3
µ 1 − µ2
µ1 +µ2
2
− µ3
#
.
Statistics 611
27 / 35
Suppose
H01 : C1 β = d1
and H02 : C2 β = d2
are both testable and
S1 ≡ {β : C1 β = d1 }
= {β : C2 β = d2 } ≡ S2 .
Show that the F-test of H01 is the same as the F-test of H02 .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
28 / 35
Recall that
X0 = X0 PX = X0 X(X0 X)− X0 .
Thus, X(X0 X)− is a GI of X0
∀ X0 .
It follows that C0k (Ck C0k )− is a GI of Ck . Thus,
Sk = {C0k (Ck C0k )− dk + (I − C0k (Ck C0k )− Ck )z : z ∈ Rp }
for k = 1, 2.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
29 / 35
Because S1 = S2 , C1 times any element of S2 equal d1 ; i.e.,
β ∈ S2 =⇒ C1 β = d1 .
Thus
C1 [C02 (C2 C02 )− d2 + (I − C02 (C2 C02 )− C2 )z] = d1
∀ z ∈ Rp
=⇒ C1 C02 (C2 C02 )− d2 − d1 + (C1 − C1 C02 (C2 C02 )− C2 )z = 0
=⇒ C1 = C1 C02 (C2 C02 )− C2
C1 C02 (C2 C02 )− d2 = d1
c
Copyright 2012
Dan Nettleton (Iowa State University)
∀ z ∈ Rp
and
by Result A.8.
Statistics 611
30 / 35
Now
C1 = C1 C02 (C2 C02 )− C2
=⇒ C1 = C1 PC02
=⇒ C01 = PC02 C01
=⇒ C(C01 ) ⊆ C(PC02 ) = C(C02 ).
Repeating the entire argument with the roles of C1 and C2 reversed
gives
C(C02 ) ⊆ C(C01 )
so that
C(C02 ) = C(C01 ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
31 / 35
Because
C1 β = d 1
and C2 β = d2
are both testable, C01 and C02 are both full-column rank = q.
∴ ∃ a unique nonsingular matrixq×q
B3
C01 = C02 B.
We have previously shown that
C01 = PC02 C01 = C02 [(C2 C02 )− ]0 C2 C01 .
Thus,
B = [(C2 C02 )− ]0 C2 C01
c
Copyright 2012
Dan Nettleton (Iowa State University)
and
B0 = C1 C02 (C2 C02 )− .
Statistics 611
32 / 35
We have also shown previously that
C1 C02 (C2 C02 )− d2 = d1 =⇒ B0 d2 = d1 .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
33 / 35
Now consider the quadratic form
(C1 b − d1 )0 (C1 (X0 X)− C01 )−1 (C1 b − d1 )
= (B0 C2 b − B0 d2 )0 (B0 C2 (X0 X)− C02 B)−1 (B0 C2 b − B0 d2 )
= (C2 b − d2 )0 BB−1 (C2 (X0 X)− C02 )−1 (B0 )−1 B0 (C2 b − d2 )
= (C2 b − d2 )0 (C2 (X0 X)− C02 )−1 (C2 b − d2 ).
This is true for ∀ b ∈ Rp including β̂ and the true parameter vector β.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
34 / 35
∴ the F statistics for testing
H01 : C1 β = d1
and H02 : C2 β = d2
are identical, as are the noncentrality parameters associated with
those F statistics.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
35 / 35