Exercises Chapter 4
Statistical Hypothesis Testing
Advanced Econometrics - HEC Lausanne
Christophe Hurlin
University of Orléans
December 5, 2013
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
1 / 88
Exercise 1
Parametric tests and the Neyman Pearson lemma
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
2 / 88
Problem
We consider two continuous independent random variables U and W
normally distributed with N 0, σ2 . The transformed variable X de…ned
by:
p
X = U2 + W 2
has a Rayleigh distribution with a parameter σ2 :
X
Rayleigh σ2
with a pdf fX x; σ2 de…ned by:
fX x; σ2 =
Christophe Hurlin (University of Orléans)
x
exp
σ2
x2
2σ2
8x 2 [0, +∞[
Advanced Econometrics - HEC Lausanne
December 5, 2013
3 / 88
Problem (cont’d)
Question 1: we consider an i.i.d. sample fX1 , X2 , .., XN g . Derive the
MLE estimator of σ2
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
4 / 88
Solution
fX x; σ2 =
x2
2σ2
x
exp
σ2
8x 2 [0, +∞[
The log-likelihood of the i.i.d. sample fx1 , x2 , .., xN g is
`N σ 2 ; x =
N
∑ ln fX xi ; σ2 =
i =1
N
∑ ln (xi )
N ln σ2
i =1
1
2σ2
N
∑ xi2
i =1
b2 is de…ned as to be:
The ML estimator σ
b2 = arg max `N σ2 ; x
σ
σ 2 >0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
5 / 88
Solution (cont’d)
b2 = arg max
σ
σ 2 >0
N
∑ ln (xi )
1
2σ2
N ln σ2
i =1
N
∑ xi2
i =1
FOC (log-likelihood equations)
∂ `N σ 2 ; x
∂σ2
=
So, the ML estimator of σ2 is
b2
σ
b2 =
σ
Christophe Hurlin (University of Orléans)
N
1
+ 4
2
b
σ
2b
σ
1
2N
N
∑ xi2 = 0
i =1
N
∑ Xi2
i =1
Advanced Econometrics - HEC Lausanne
December 5, 2013
6 / 88
Solution (cont’d)
∂ `N σ 2 ; x
=
∂σ2
N
1
+ 4
2
σ
2σ
N
∑ xi2
i =1
SOC:
∂2 `N σ 2 ; x
∂σ4
=
b2
σ
=
=
N
b4
σ
1
b6
σ
N
∑ xi2
i =1
b2
N
2N σ
b4
b6
σ
σ
N
<0
b4
σ
2
b2 . So, we have a maximum.
since ∑N
i =1 xi = 2N σ
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
7 / 88
Problem (cont’d)
b2 ?
Question 2: what is the asymptotic distribution of the MLE estimator σ
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
8 / 88
Solution
The average Fisher information matrix associated to the sample is:
!
2`
2; X
∂
σ
N
IN σ2 = Eσ2
∂σ4
!
N
1 N 2
= Eσ 2
+ 6 ∑ Xi
σ4
σ i =1
1
N
+ 4
4
σ
σ
=
N
∑ Eσ 2
i =1
Xi2
σ2
Since (X /σ)2 = (U/σ)2 + (W /σ)2 where U/σ and W /σ are two
independent standard normal variables, then X 2 /σ2 χ2 (2) with
Eσ 2
Christophe Hurlin (University of Orléans)
Xi2
σ2
=2
Advanced Econometrics - HEC Lausanne
December 5, 2013
9 / 88
Solution (cont’d)
So, we have
I N σ2
=
1
N
+ 4
4
σ
σ
=
N
2N
+ 4
4
σ
σ
=
N
∑ Eσ 2
i =1
Xi2
σ2
N
σ4
Since the sample is i.i.d., the average Fisher information matrix is:
I σ2 =
Christophe Hurlin (University of Orléans)
1
1
I N σ2 = 4
N
σ
Advanced Econometrics - HEC Lausanne
December 5, 2013
10 / 88
Solution (cont’d)
The regularity conditions hold, and we have:
p
Here
b2
N σ
p
d
σ2 ! N 0, I
b2
N σ
1
σ2
d
σ2 ! N 0, σ4
where σ2 denotes the true value of the parameter. Or equivalently:
b2
σ
Christophe Hurlin (University of Orléans)
asy
N
σ2 ,
σ4
N
Advanced Econometrics - HEC Lausanne
December 5, 2013
11 / 88
Problem (cont’d)
Question 3: consider the test
H0 : σ2 = σ20
H1 : σ2 = σ21
with σ21 > σ20 . Determine the critical region of the UMP test of size α.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
12 / 88
Solution
Given the Neyman Pearson lemma, the rejection region is given by:
(
)
LN σ20 ; x1 , ., xN
W = x1 , .., xN j
<K
LN (σ21 ; x1 , ., xN )
where K is a constant determined by the level of the test α. So, we have
`N σ20 ; x
() ∑Ni=1 ln (xi )
`N σ21 ; x < ln (K )
N ln σ20
1
2σ20
2
∑N
i =1 xi
+N ln σ21 +
Christophe Hurlin (University of Orléans)
1
2σ21
∑N
i =1 ln (xi )
2
∑N
i =1 xi < ln (K )
Advanced Econometrics - HEC Lausanne
December 5, 2013
13 / 88
Solution (cont’d)
N ln σ21
ln σ20
()
with K1 = ln (K )
+
1
σ21
N log σ21
σ20 σ21
σ20 σ21
Christophe Hurlin (University of Orléans)
1
σ20
1
σ21
1
σ20
1 N 2
xi < ln (K )
2 i∑
=1
1 N 2
xi < K1
2 i∑
=1
log σ20
. or equivalently:
1 N 2
xi < K1
2 i∑
=1
Advanced Econometrics - HEC Lausanne
December 5, 2013
14 / 88
Solution (cont’d)
1 N 2
xi < K1
2 i∑
=1
σ20 σ21
σ20 σ21
Since σ21 > σ20 , we have:
1
2N
where A =
K1 σ20 σ21 /
Christophe Hurlin (University of Orléans)
σ20
σ21
N
∑ xi2 > A
i =1
N is a constant determined by α.
Advanced Econometrics - HEC Lausanne
December 5, 2013
15 / 88
Solution (cont’d)
The rejection region of the UMP test of size α
H0 : σ2 = σ20
with σ21 > σ20 is:
H1 : σ2 = σ21
n
o
b 2 (x ) > A
W= x :σ
where the critical value A is a constant determined by the size α and
b2 (x ) is the realisation of the ML estimator σ
b2 (the test statistic):
σ
b2 =
σ
Christophe Hurlin (University of Orléans)
1
2N
N
∑ Xi2
i =1
Advanced Econometrics - HEC Lausanne
December 5, 2013
16 / 88
Solution (cont’d)
Given the de…nition of the size:
b 2 > A H0
α = Pr ( Wj H0 ) = Pr σ
Under the null, for N large, we have:
b2
σ
Then
1
α = Pr
Christophe Hurlin (University of Orléans)
asy
H0
N
σ20 ,
σ40
N
b2
σ
σ2
A σ20
p0 <
p H0
σ20 / N
σ20 / N
Advanced Econometrics - HEC Lausanne
!
December 5, 2013
17 / 88
Solution (cont’d)
1
α = Pr
b2
σ
A σ20
σ2
p0 <
p H0
σ20 / N
σ20 / N
!
Denote by Φ (.) the cdf of the standard normal distribution:
σ2
A = σ20 + p 0 Φ
N
1
(1
α)
The rejection region of the UMP test of size α
H0 : σ2 = σ20
H1 : σ2 = σ21
with σ21 > σ20 is:
W=
σ2
b2 (x ) > σ20 + p 0 Φ
x:σ
N
Christophe Hurlin (University of Orléans)
1
(1
Advanced Econometrics - HEC Lausanne
α)
December 5, 2013
18 / 88
Problem (cont’d)
Question 4: consider the test
H0 : σ 2 = 2
H1 : σ 2 > 2
For a sample of size N = 100, we have
N
∑ xi2 = 470
i =1
What is the conclusion of the test for a size of 10%?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
19 / 88
Solution
Consider the test
H0 : σ2 = σ20
H1 : σ2 = σ21
with σ21 > σ20 . The rejection region of the UMP test of size α is given by:
W=
σ2
b2 (x ) > σ20 + p 0 Φ
x:σ
N
1
(1
α)
This region does not depend on the value of σ21 . So, it corresponds to the
rejection region of the one-sided UMP of size α :
H0 : σ2 = σ20
Christophe Hurlin (University of Orléans)
H1 : σ2 > σ20
Advanced Econometrics - HEC Lausanne
December 5, 2013
20 / 88
Solution (cont’d)
H0 : σ 2 = 2
W=
H1 : σ 2 > 2
σ2
b2 (x ) > σ20 + p 0 Φ
x:σ
N
1
(1
1
(0.9)
α)
NA: N = 100, α = 10% :
W=
b 2 (x ) > 2 +
x:σ
2
Φ
10
n
o
b2 (x ) > 2.2563
W= x :σ
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
21 / 88
Solution (cont’d)
o
n
b2 (x ) > 2.2563
W= x :σ
2
For this sample (N = 100) we have ∑N
i =1 xi = 470, and as a consequence
b 2 (x ) =
σ
1
2N
N
470
∑ xi2 = 200 = 2.35
i =1
For a signi…cance level of 10%, we reject the null H0 : σ2 = 2.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
22 / 88
Problem (cont’d)
Question 5: determine the power of the one-sided UMP test of size α for:
H0 : σ2 = σ20
H1 : σ2 > σ20
Numerical application: N = 100, σ20 = 2 and α = 10%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
23 / 88
Solution
The rejection region of the UMP of size α is:
o
n
b 2 (x ) > A
W= x :σ
with σ20 + Φ
1
(1
p
α) σ20 / N. By of the power, we have:
power = Pr ( Wj H1 ) = Pr
σ2
b2 > σ20 + p 0 Φ
σ
N
1
(1
α ) H1
Under the alternative hypothesis, for N large, we have:
b2
σ
Christophe Hurlin (University of Orléans)
asy
H1
N
σ2 ,
σ4
N
σ2 > σ20
Advanced Econometrics - HEC Lausanne
December 5, 2013
24 / 88
Solution (cont’d)
Then, the power is equal to:
power = 1
b2
σ
A σ2
σ2
p <
p H1
σ2 / N
σ2 / N
Pr
!
=1
Given the de…nition of the critical value A = σ20 + Φ
we have:
power = 1
Φ
σ20
σ2
σ2
p + 02 Φ
σ
σ2 / N
1
1
Φ
σ2
p
σ2 / N
(1
p
α) σ20 / N,
A
α)
8σ2 > σ20
(0.9)
8σ2 > σ20
(1
NA: σ20 = 2, N = 100 and α = 10%
power = 1
Christophe Hurlin (University of Orléans)
Φ
2 σ2
2
+ 2Φ
σ2 /10
σ
1
Advanced Econometrics - HEC Lausanne
December 5, 2013
25 / 88
Solution (cont’d)
1
0.9
0.8
0.7
power
0.6
0.5
0.4
0.3
0.2
0.1
0
2
2.2
2.4
2.6
σ
Christophe Hurlin (University of Orléans)
2.8
3
2
Advanced Econometrics - HEC Lausanne
December 5, 2013
26 / 88
Solution (cont’d)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
27 / 88
Problem (cont’d)
Question 6: consider the two-sided test
H0 : σ2 = σ20
H1 : σ2 6= σ20
What is the critical region of the test of size α?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
28 / 88
Solution
Consider the one-sided tests:
Test A: H0 : σ2 = σ20 against H1 : σ2 < σ20
Test B: H0 : σ2 = σ20 against H1 : σ2 > σ20
The non-rejection regions of the UMP one-sided tests of size α/2 are:
WA =
WB =
σ2
b2 (x ) > σ20 + p 0 Φ
x:σ
N
σ2
b2 (x ) < σ20 + p 0 Φ
x:σ
N
1
1
1
α
2
α
2
The non rejection region of the two-sided test corresponds to the
intersection of these two regions:
W = WA \WB
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
29 / 88
Solution (cont’d)
So, the non rejection region of the two-sided test of size α is:
W=
Since, Φ
σ2
x : σ20 + p 0 Φ
N
1
(α/2) =
W=
Φ
1
1
α
2
(1
b 2 (x )
x: σ
σ2
b2 (x ) < σ20 + p 0 Φ
<σ
N
1
1
α
2
α/2) , this region can be rewritten as:
σ2
σ20 > p 0 Φ
N
1
α
2
1
The rejection region of the two-sided of size α is:
W=
b 2 (x )
x: σ
Christophe Hurlin (University of Orléans)
σ2
σ20 < p 0 Φ
N
1
Advanced Econometrics - HEC Lausanne
1
α
2
December 5, 2013
30 / 88
Problem (cont’d)
Question 7: determine the power of the two-sided test of size α for:
H0 : σ2 = σ20
H1 : σ2 6= σ20
Numerical application: N = 100, σ20 = 2 and α = 10%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
31 / 88
Solution
The non rejection region of the two-sided test of size α is:
o
n
b 2 (x ) < B
W= x :A<σ
σ2
A = σ20 + p 0 Φ
N
1
α
2
σ2
B = σ20 + p 0 Φ
N
1
1
α
2
By de…nition of the power:
power = Pr ( Wj H1 ) = 1
Pr W H1
So, we have:
power = 1
Christophe Hurlin (University of Orléans)
b2 < B + Pr σ
b2 < A
Pr σ
Advanced Econometrics - HEC Lausanne
December 5, 2013
32 / 88
Solution (cont’d)
power = 1
Under the alternative
So, we have
b2
σ
power = 1
Christophe Hurlin (University of Orléans)
asy
H1
b2 < B + Pr σ
b2 < A
Pr σ
N
Φ
σ2 ,
σ4
N
σ2
p
σ2 / N
B
σ2 6= σ20
+Φ
Advanced Econometrics - HEC Lausanne
σ2
p
σ2 / N
A
December 5, 2013
33 / 88
Solution (cont’d)
We have
Φ
power = 1
σ2
A = σ20 + p 0 Φ
N
1
σ2
p
σ2 / N
B
+Φ
σ2
p
σ2 / N
A
σ2
B = σ20 + p 0 Φ
N
α
2
1
1
α
2
So 8σ2 6= σ20 , the power function of the two sided test is de…ned by:
Φ
power = 1
+Φ
Christophe Hurlin (University of Orléans)
σ2
σ2
p + 02 Φ 1 1
σ
σ2 / N
2
2
2
σ0 σ
σ
α
p + 02 Φ 1
2
σ
2
σ / N
σ20
Advanced Econometrics - HEC Lausanne
α
2
December 5, 2013
34 / 88
Solution (cont’d)
NA: N = 100, α = 10% and σ20 = 2. 8σ2 6= 2
power = 1
+Φ
Christophe Hurlin (University of Orléans)
Φ
2 σ2
2
+ 2 Φ 1 (0.95)
σ2 /10
σ
2
2 σ2
+ 2 Φ 1 (0.05)
2
σ /10
σ
Advanced Econometrics - HEC Lausanne
December 5, 2013
35 / 88
Solution (cont’d)
1
0.9
0.8
0.7
power
0.6
0.5
0.4
0.3
0.2
0.1
0
1
1.5
2
σ
Christophe Hurlin (University of Orléans)
2.5
3
2
Advanced Econometrics - HEC Lausanne
December 5, 2013
36 / 88
Solution (cont’d)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
37 / 88
Problem (cont’d)
Question 8: show that the two-sided test is unbiased and consistent.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
38 / 88
Solution
The power function is de…ned as to be:
P σ2
Φ
= 1
+Φ
σ2
σ2
p + 02 Φ 1 1
σ
σ2 / N
2
2
2
σ0 σ
σ
α
p + 02 Φ 1
2
σ
2
σ / N
σ20
α
2
If σ2 < σ20 , then:
lim P σ2 = 1
N !∞
Φ (+∞) + Φ (+∞) = 1
1+1 = 1
Φ ( ∞) + Φ ( ∞) = 1
0+0 = 1
If σ2 > σ20 , then:
lim P σ2 = 1
N !∞
The test is consistent.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
39 / 88
Solution (cont’d)
The power function is de…ned as to be:
P σ2
= 1
+Φ
Φ
σ20
σ2
σ2
p + 02 Φ 1 1
σ
σ2 / N
2
2
2
σ0 σ
σ
α
p + 02 Φ 1
2
σ
2
σ / N
α
2
This function reaches a minimum when σ2 tends to σ20 .
lim P σ2
σ2 !σ20
= 1
= 1
= α
Φ Φ
1
1
1
α
2
+Φ Φ
1
α
2
α
α
+
2
2
The test is unbiased.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
40 / 88
Subsection 4.2
The trilogy: LRT, Wald, and LM tests
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
41 / 88
Problem (Greene, 2007, page 531)
We consider two random variables Y and X such that the pdf of the
conditional distribution Y j X = x is given by
f Y jX ( y j x; β) =
1
exp
β+x
For convenience, let
βi =
y
β+x
1
β + xi
This exponential density is a restricted form of a more general gamma
distribution,
ρ
f Y jX
β
ρ
( y j x; β, ρ) = i yi
Γ (ρ)
1
exp ( yi βi )
The restriction is ρ = 1. We want to test the hypothesis
H0 : ρ = 1 versus
Christophe Hurlin (University of Orléans)
H1 : ρ 6= 1
Advanced Econometrics - HEC Lausanne
December 5, 2013
42 / 88
Reminder: the gamma function
The gamma function Γ (p ) is de…ned as to be:
Γ (p ) =
Z∞
tp
1
8p > 0
exp ( t ) dt
0
The gamma function obeys the recursion
Γ (p ) = (p
Γ
1
2
1) Γ (p
=
p
1)
π
So for integer values of p, we have
Γ (p ) = (p
Christophe Hurlin (University of Orléans)
1) !
Advanced Econometrics - HEC Lausanne
December 5, 2013
43 / 88
Reminder: the gamma function (cont’d)
The derivatives of the gamma function are
∂k Γ (p )
=
∂p k
Z∞
(ln (t ))k t p
1
exp ( t ) dt
0
The …rst two derivatives of ln (Γ (p )) are denoted
∂ ln (Γ (p ))
Γ0
=
= Ψ (p )
∂p
Γ
∂2 ln (Γ (p ))
=
∂p 2
ΓΓ"
Γ2
0
Γ2
= Ψ 0 (p )
where Ψ (p ) and Ψ0 (p ) are the digamma and trigamma functions (see
polygamma function and function psy in Matlab).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
44 / 88
Problem (cont’d)
Question 1: consider an i.i.d. sample fXi , Yi gN
i =1 and write its
log-likelihood under H1 (unconstrained model) and under H0 (constrained
model).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
45 / 88
Solution
Under H1 , with θ = ( β : ρ)> , we have
ρ
f Y i jX i
β
ρ
( yi j xi ; θ) = i yi
Γ (ρ)
1
exp ( yi βi )
with
βi =
1
β + xi
N
`N ( y j x; θ) =
∑ ln f Y jX
i
i =1
i
( yi j xi ; θ)
The log-likelihood under H1 (unconstrained model) is:
N
`N ( y j x; θ) = ρ ∑ ln ( βi )
N ln (Γ (ρ)) + (ρ
i =1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
N
1) ∑ ln (yi )
i =1
N
∑ yi βi
i =1
December 5, 2013
46 / 88
Solution (cont’d)
Under H0 : ρ = 1, we have
f Y i jX i ( yi j xi ; β) = βi exp ( yi βi )
with
βi =
1
β + xi
N
`N ( y j x; β) =
∑ ln f Y jX
i
i =1
i
( yi j xi ; β)
The log-likelihood under H0 (constrained model) is:
`N ( y j x; β) =
Christophe Hurlin (University of Orléans)
N
N
i =1
i =1
∑ ln ( βi ) ∑ yi βi
Advanced Econometrics - HEC Lausanne
December 5, 2013
47 / 88
Problem (cont’d)
Question 2: write the gradient vectors and the Hessian matrices
associated to the unconstrained log-likelihood (under H1 ) and to the
constrained log-likelihood (under H0 ).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
48 / 88
Solution
Under H1 :
N
`N ( y j x; θ) = ρ ∑ ln ( βi )
N
1) ∑ ln (yi )
N ln (Γ (ρ)) + (ρ
i =1
i =1
N
∑ yi βi
i =1
Remarks:
∂βi
∂ (1/ ( β + xi ))
=
=
∂β
∂β
∂ ln ( βi )
∂(
=
∂β
1
( β + xi )2
ln ( β + xi ))
=
∂β
=
1
=
β + xi
β2i
βi
∂ ln (Γ (ρ))
= Ψ (ρ)
∂ρ
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
49 / 88
Solution (cont’d)
N
`N ( y j x; θ) = ρ ∑ ln ( βi )
N
1) ∑ ln (yi )
N ln (Γ (ρ)) + (ρ
i =1
i =1
N
∑ yi βi
i =1
The gradient vector under H1 is:
gN ( y j x; θ) =
with
0
∂`N ( y j x; θ)
=@
∂θ
∂`N ( y j x; θ)
=
∂β
∂`N ( y jx ;θ)
∂ρ
N
N
i =1
i =1
1
A
ρ ∑ βi + ∑ yi β2i
N
∂`N ( y j x; θ)
= ∑ ln ( βi )
∂ρ
i =1
Christophe Hurlin (University of Orléans)
∂`N ( y jx ;θ)
∂β
N
NΨ (ρ) + ∑ ln (yi )
Advanced Econometrics - HEC Lausanne
i =1
December 5, 2013
50 / 88
Solution (cont’d)
∂`N ( y j x; θ)
=
∂β
N
N
i =1
i =1
ρ ∑ βi + ∑ yi β2i
So, we have:
N
N
∂2 `N ( y j x; θ)
β2i
=
ρ
∑
2
∂β
i =1
∂2 `N ( y j x; θ)
=
∂β∂ρ
Christophe Hurlin (University of Orléans)
2 ∑ yi β3i
i =1
N
∑ βi
i =1
Advanced Econometrics - HEC Lausanne
December 5, 2013
51 / 88
Solution (cont’d)
N
∂`N ( y j x; θ)
= ∑ ln ( βi )
∂ρ
i =1
So, we have
∂2 `N ( y j x; θ)
=
∂ρ2
∂2 `N ( y j x; θ)
=
∂ρ∂β
Christophe Hurlin (University of Orléans)
N
NΨ (ρ) + ∑ ln (yi )
i =1
NΨ0 (ρ)
N
∑ βi
i =1
Advanced Econometrics - HEC Lausanne
December 5, 2013
52 / 88
Solution (cont’d)
The Hessian matrix associated to the log-likelihood under H1 is:
0 2
1
2
with
∂2 `N ( y j x; θ)
B
=@
HN ( y j x; θ) =
>
∂θ∂θ
HN ( y j x; θ) =
Christophe Hurlin (University of Orléans)
2
ρ ∑N
i =1 β i
∂ `N ( y jx ;θ)
∂β2
∂ `N ( y jx ;θ)
∂β∂ρ
∂2 `
∂2 `
N ( y jx ;θ)
∂ρ∂β
3
2 ∑N
i =1 yi βi
∑N
i =1 β i
Advanced Econometrics - HEC Lausanne
N ( y jx ;θ)
∂ρ2
∑N
i =1 β i
NΨ0 (ρ)
C
A
!
December 5, 2013
53 / 88
Solution (cont’d)
Under H0 : ρ = 1, the gradient (scalar) is
gN ( y j x; β) =
∂`N ( y j x; β)
=
∂β
N
N
i =1
i =1
∑ βi + ∑ yi β2i
The Hessian (scalar) is:
HN ( y j x; β) =
∂2 `N ( y j x; β)
= ∑Ni=1 β2i
∂β2
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
3
2 ∑N
i =1 yi βi
December 5, 2013
54 / 88
Problem (cont’d)
Question 3: write the average Fisher information matrices under H1 and
under H0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
55 / 88
Solution
Under H1 (unconstrained model), the Hessian (stochastic) is
!
ρβ2i 2Yi β3i
βi
Hi ( Yi j xi ; θ) =
βi
Ψ0 (ρ)
The average Fisher information matrice can be de…ned (one of the three
de…nitions) as:
I (θ) = EX Eθ ( Hi ( Yi j xi ; θ))
I (θ) = EX
ρβ2i + 2Eθ (Yi ) β3i
βi
βi
Ψ0 (ρ)
!
since βi = 1/ ( β + Xi ) depends on the random variable Xi .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
56 / 88
Solution
ρβ2i + 2Eθ (Yi ) β3i
I (θ) = EX
βi
Ψ0 (ρ)
βi
!
Consider the score of the unit i. By de…nition, we have:
Eθ (si ( Yi j xi ; θ)) =
ρβi + Yi β2i
ln ( βi )
Ψ (ρ) + ln (Yi )
!
= 02
1
So, we have
Eθ (Yi ) =
ρ
βi
where Eθ denotes the expectation with respect to the conditional
distribution of Y given X = x.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
57 / 88
Solution (cont’d)
I (θ) = EX
ρβ2i + 2Eθ (Yi ) β3i
Ψ0 (ρ)
βi
Eθ (Yi ) =
βi
!
ρ
βi
Under H1 (unconstrained model), the average Fisher information is de…ned
as to be:
!
βi
ρβ2i
I (θ) = EX
βi Ψ 0 ( ρ )
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
58 / 88
Solution (cont’d)
Under H0 (constrained model), we have:
Hi ( Yi j xi ; β) = β2i
Eβ (si ( Yi j xi ; β)) = Eβ
2Yi β3i
βi + Yi β2i = 0
The average Fisher information number is de…ned by:
I ( β) = EX Eβ ( Hi ( Yi j xi ; β))
= EX
β2i + 2Eβ (Yi ) β3i
= EX β2i
The average Fisher information number is equal to:
I ( β) = EX β2i
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
59 / 88
Problem (cont’d)
Question 4: denote b
θH 1 the ML estimator of θ = ( β : ρ)> obtained under
H1 and b
θH 0 = b
βH 0 the ML estimator of β obtained under H0 : ρ = 1.
Determine the asymptotic distribution and the asymptotic variance
covariance matrix of b
θH 1 and b
θH 0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
60 / 88
Solution
The regularity conditions hold. Under H1 (unconstrainded model) we have:
p
N b
θH 1
d
θ1 ! N 0, I
1
(θ1 )
where θ1 denotes the true value of the parameters (under H1 ), or
equivalently:
asy
1
b
θH 1
N θ1 , I 1 (θ1 )
H1
N
with
I (θ1 ) = EX
Christophe Hurlin (University of Orléans)
ρβ2i
βi
βi
Ψ0 (ρ)
Advanced Econometrics - HEC Lausanne
!
December 5, 2013
61 / 88
Solution (cont’d)
The regularity conditions hold. Under H0 (constrainded model) we have:
p
θH 0
N b
d
θ0 ! N 0, I
1
(θ0 )
where θ0 = β0 denotes the true value of the parameter (under H0 ), or
equivalently:
asy
1
b
θH 0
N θ0 , I 1 (θ0 )
H0
N
with
I (θ0 ) = EX β2i
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
62 / 88
Problem (cont’d)
Question 5: Propose three alternative estimators of the average Fisher
information matrices under H1 and under H0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
63 / 88
Solution
Three alternative estimators of the average Fisher information matrix I (θ)
can be used:
1 N b b
bI A b
θ =
Ii θ
N i∑
=1
>!
N
1
∂
`
θ;
y
∂
`
θ;
y
x
x
(
(
)
)
j
j
i
i
i
i
i
i
bI B b
θ =
N i∑
∂θ
∂θ
b
b
θ
θ
=1
1
bI c b
θ =
N
Christophe Hurlin (University of Orléans)
N
∑
i =1
∂2 `i (θ; yi j xi )
∂θ∂θ>
Advanced Econometrics - HEC Lausanne
b
θ
December 5, 2013
64 / 88
Solution (cont’d)
First estimator: actual Fisher information matrix
1
bI A b
θ =
N
Under H1 :
Under H0 :
bI A
N
∑ bI i
i =1
b
θ
1
0
2
N b
b
b
β
β
ρ ∑N
∑
1
i =1 i A
i =1 i
b
θ = @
N b
N
ρ)
∑i =1 βi NΨ0 (b
1
bI A b
θ =
N
N
∑ bI i
i =1
1
b2
b
θ = ∑N
i =1 β i
N
where b
βi = 1/ b
β + xi and where the estimators b
β and b
ρ are obtained
under H1 (unconstrained model) or H0 (constrained model) given the case.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
65 / 88
Solution (cont’d)
Second estimator: BHHH estimator
Under H1 :
bI B
bI B b
θ
1
b
θ =
N
=
1
N
N
∂`i (θ; yi j xi )
∂θ
∑
i =1
N
0
∑@
i =1
ln b
βi
2
b
ρb
βi + yi b
βi
2
b
ρb
βi + yi b
βi
Under H0 :
1
bI B b
θ =
N
Christophe Hurlin (University of Orléans)
b
θ
∂`i (θ; yi j xi )
∂θ
N
∑
i =1
Ψ (b
ρ) + ln (yi )
ln b
βi
b
θ
!
1
A
Ψ (b
ρ) + ln (yi )
2
b
βi + yi b
βi
Advanced Econometrics - HEC Lausanne
>
2
December 5, 2013
66 / 88
Solution (cont’d)
Third estimator: Hessian
Under H1 :
Under H0 :
1
bI C b
θ =
N
1
bI C b
θ =
N
N
N
∑
i =1
0
1
bI C b
θ =
N
Christophe Hurlin (University of Orléans)
2
3
b
ρb
βi + 2yi b
βi
∑@
i =1
∂2 `i (θ; yi j xi )
∂θ∂θ>
N
∑
i =1
b
βi
b
θ
b
βi
Ψ0 (b
ρ)
1
A
2
3
b
βi + 2yi b
βi
Advanced Econometrics - HEC Lausanne
December 5, 2013
67 / 88
Problem (cont’d)
Question 6: Consider the dataset provided by Greene (2007) in the …le
Chapter4_Exercise2.xls. Write a Matlab code (1) to estimate the
parameters of model under H1 ( unconstrained model) by MLE, and (2)
to compute three alternative estimates of the asymptotic variance
covariance matrix.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
68 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
69 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
70 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
71 / 88
Remarks
1
Asymptotically the three estimators of the asymptotic variance
covariance matrix are equivalent.
2
But, this exercise con…rms that these estimators can give very
di¤erent results for small samples
3
The striking di¤erence of the BHHH estimator is typical of its erratic
performance in small samples
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
72 / 88
Problem (cont’d)
Question 7: Write a Matlab code (1) to estimate the parameters of
model under H0 ( constrained model) by MLE, and (2) to compute three
alternative estimates of the asymptotic variance covariance matrix.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
73 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
74 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
75 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
76 / 88
Problem (cont’d)
Question 8: test the hypothesis
H0 : ρ = 1 versus
H1 : ρ 6= 1
with a likelihood ratio (LR) test for a signi…cance level of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
77 / 88
Solution
The likelihood ratio (LR) test-statistic is de…ned by:
LR =
θH 0 ; y j x
`N b
2
θH 1 ; y j x
`N b
In this sample, we have a realisation equal to
LR (y ) =
2
( 88.4363 + 82.9160) = 11.0406
The critical region is
W = y : LR (y ) > χ20.95 (1) = 3.8415
where χ20.95 (1) is the critical value of the chi-squared distribution with
p = 1 degrees of freedom. Conclusion: for a signi…cance level of 5%, we
reject the null H0 : ρ = 1.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
78 / 88
Problem (cont’d)
Question 9: test the hypothesis
H0 : ρ = 1 versus
H1 : ρ 6= 1
with a Wald test for a signi…cance level of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
79 / 88
Solution
The null hypothesis H0 : ρ = 1 can be expressed as
H0 : c (θ) = 0
with c (θ) = ρ
θH 1
Wald = c b
1. The Wald test-statistic is de…ned by:
>
∂c
∂θ>
Here, we have
b asy b
b
θH 1
θH 1 V
∂c
b
θH 1
=
Wald (y ) = b
ρH 1
1
∂θ>
Then, we get
Christophe Hurlin (University of Orléans)
∂c
∂θ>
b
θH 1
>
1
θH 1
c b
0 1
2
Vasy1 b
ρH 1
Advanced Econometrics - HEC Lausanne
December 5, 2013
80 / 88
Solution (cont’d)
ρH 1
Wald (y ) = b
1
2
ρH 1
Vasy1 b
Given the estimator chosen for the asymptotic variance, we get:
WaldA (y ) =
(3.1509 1)2
= 8.0214
0.5768
WaldB (y ) =
(3.1509 1)2
= 3.0096
1.5372
WaldC (y ) =
(3.1509 1)2
= 7.3335
0.6309
The critical region is
W = y : Wald (y ) > χ20.95 (1) = 3.8415
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
81 / 88
Solution (cont’d)
Conclusion:
1
For a signi…cance level of 5%, the Wald test-statistic based on the
estimators A and C (actual Fisher matrix and Hessian) of the
asymptotic variance covariance matrix lead to reject the null
H0 : ρ = 1.
2
For a signi…cance level of 5%, the Wald test-statistic based on the
estimators B (BHHH estimator) of the asymptotic variance covariance
matrix fails to reject the null H0 : ρ = 1.
3
In most of software, the Hessian (estimator C) is preferred and the
Wald test-statistics are computed with this estimator.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
82 / 88
Problem (cont’d)
Question 10: test the hypothesis
H0 : ρ = 1 versus
H1 : ρ 6= 1
with a Lagrange Multiplier test for a signi…cance level of 5%. Write a
Matlab code to compute the three possibles values of the LM test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
83 / 88
Solution
The Lagrange multiplier test is based on the restricted estimators. The
LM test-statistic is de…ned by:
θH 0 ; yi j xi
LM = sN b
>
or equivalently
LM = sN b
θH 0 ; yi j xi
Christophe Hurlin (University of Orléans)
>
bI N b
θH 0
N bI b
θH 0
1
θH 0 ; yi j xi
sN b
1
sN b
θH 0 ; yi j xi
Advanced Econometrics - HEC Lausanne
December 5, 2013
84 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
85 / 88
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
86 / 88
Solution (cont’d)
θH 0 , we have:
So, given the estimator chosen for bI b
LMA (y ) = 4.7825
LmB (y ) = 15.6868
LMC (y ) = 5.1162
The critical region is
W = y : LM (y ) > χ20.95 (1) = 3.8415
Conclusion: for a signi…cance level of 5%, we reject the null H0 : ρ = 1,
whatever the choice of the estimator.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
87 / 88
End of Exercices - Chapter 4
Christophe Hurlin (University of Orléans)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 5, 2013
88 / 88