Reparameterization in Testing
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 26
Example:
Suppose
yi = β1 x1i + β2 x2i + εi
(i = 1, . . . , n),
where
i.i.d.
ε1 , . . . , εn ∼ N(0, σ 2 ).
Consider testing
H0 : β1 = β2 .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
2 / 26
H0 : β1 = β2 ⇐⇒ H0 : Cβ = 0,
where
C = [1, −1],
c
Copyright 2012
Dan Nettleton (Iowa State University)
and β =
" #
β1
β2
.
Statistics 611
3 / 26
Let
 
xj1
.
.
xj = 
.
xjn
for j = 1, 2.
Then
X = [x1 , x2 ].
"
Suppose
1
#
−1
∈ C(X0 ) so that
H0 : β1 = β2
is testable.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
4 / 26
Provide the RNE for the restriction imposed by the null hypothesis.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
5 / 26
One way to find the BLUE of β subject to β1 = β2 is to solve these
equations.
Can you think of an easier way?
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
7 / 26
This is a special case of a general reparametrization strategy.
Suppose
H0 : Cβ = d
is testable.
The set of al β satisfying H0 is
Θ0 = {C− d + (I − C− C)γ : γ ∈ Rp }.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
12 / 26
Thus,
y = Xβ + ε,
where β is constrained to H0 : Cβ = d is equivalent to
y = X[C− d + (I − C− C)γ] + ε,
γ ∈ Rp
⇐⇒
y − XC− d = X(I − C− C)γ + ε,
c
Copyright 2012
Dan Nettleton (Iowa State University)
γ ∈ Rp .
Statistics 611
13 / 26
The model
y − XC− d = X(I − C− C)γ + ε,
γ ∈ Rp .
is unconstrained with response vector y − XC− d and design matrix
X(I − C− C).
Thus,
γ̂ = [(I − C− C)0 X0 X(I − C− C)]− (I − C− C)0 X0 (y − XC− d)
solves the unconstrained least squares problem
min ky − XC− d − X(I − C− C)γk2 .
γ∈Rp
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
14 / 26
Now
min ky − XC− d − X(I − C− C)γk2
γ∈Rp
⇐⇒ minp ky − X[C− d + (I − C− C)γ]k2
γ∈R
⇐⇒ min ky − Xβk2 .
β∈Θ0
Thus,
β̃ = C− d + (I − C− C)γ̂
solves
min ky − Xβk2 .
β∈Θ0
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
15 / 26
Show how this works for our simple example.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
16 / 26
In practice, when testing
H0 : Cβ = d,
we don’t often explicitly solve the RNE.
It is more common to reparameterize and carry out an unconstrained
maximization for the reparameterized model.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
21 / 26
We then use
(SSEReduced − SSEFull ) /(DFR − DFF )
SSEFull /DFF
as our test statistics, where SSEReduced is the SSE from the
reparameterized model with
DFR = n − rank(X(I − C− C)).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
22 / 26
We have shown that
ky − XC− d − X(I − C− C)γk2 = ky − Xβ̃k2 .
Thus, SSE for the reparameterized model is Q(β̃), the SSE for the
constrained model.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
23 / 26
Furthermore, it can be shown that
rank(X(I − C− C)) = rank(X) − rank(C)
= r − q.
Thus, DF for the SSE in the parameterized model is
n−r+q
so that
DFR − DFF = n − r + q − (n − r) = q.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
24 / 26
It follows that
[Q(β̃) − Q(β̂)]/q
(SSEReduced − SSEFull ) /(DFR − DFF )
=
.
SSEFull /DFF
Q(β̂)/(n − r)
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
25 / 26
Thus, the reparameterization strategy is yet another way to arrive at
the general linear test or, equivalently, the LRT of
c
Copyright 2012
Dan Nettleton (Iowa State University)
H0 : Cβ = d.
Statistics 611
26 / 26