Chapter 4: Statistical Hypothesis Testing
Christophe Hurlin
November 20, 2015
Christophe Hurlin ()
Advanced Econometrics - Master ESA
November 20, 2015
1 / 225
Section 1
Introduction
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
2 / 225
1. Introduction
The outline of this chapter is the following:
Section 2. Statistical hypothesis testing
Section 3. Tests in the multiple linear regression model
Subsection 3.1. The Student test
Subsection 3.2. The Fisher test
Section 4. MLE and Inference
Subsection 4.1. The Likelihood Ratio (LR) test
Subsection 4.2.The Wald test
Subsection 4.3. The Lagrange Multiplier (LM) test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
3 / 225
1. Introduction
References
Amemiya T. (1985), Advanced Econometrics. Harvard University Press.
Greene W. (2007), Econometric Analysis, sixth edition, Pearson - Prentice
Hil (recommended)
Ruud P., (2000) An introduction to Classical Econometric Theory, Oxford
University Press.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
4 / 225
1. Introduction
Notations: In this chapter, I will (try to...) follow some conventions of
notation.
fY ( y )
probability density or mass function
FY ( y )
cumulative distribution function
Pr ()
probability
y
vector
Y
matrix
Be careful: in this chapter, I don’t distinguish between a random vector
(matrix) and a vector (matrix) of deterministic elements (except in section
2). For more appropriate notations, see:
Abadir and Magnus (2002), Notation in econometrics: a proposal for a
standard, Econometrics Journal.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
5 / 225
Section 2
Statistical hypothesis testing
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
6 / 225
2. Statistical hypothesis testing
Objectives
The objective of this section is to de…ne the following concepts:
1
Null and alternative hypotheses
2
One-sided and two-sided tests
3
Rejection region, test statistic and critical value
4
Size, power and power function
5
Uniformly most powerful (UMP) test
6
Neyman Pearson lemma
7
Consistent test and unbiased test
8
p-value
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
7 / 225
2. Statistical hypothesis testing
Introduction
1
A statistical hypothesis test is a method of making decisions or a
rule of decision (as concerned a statement about a population
parameter) using the data of sample.
2
Statistical hypothesis tests de…ne a procedure that controls (…xes)
the probability of incorrectly deciding that a default position (null
hypothesis) is incorrect based on how likely it would be for a set of
observations to occur if the null hypothesis were true.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
8 / 225
2. Statistical hypothesis testing
Introduction (cont’d)
In general we distinguish two types of tests:
1
The parametric tests assume that the data have come from a type
of probability distribution and makes inferences about the parameters
of the distribution
2
The non-parametric tests refer to tests that do not assume the data
or population have any characteristic structure or parameters.
In this course, we only consider the parametric tests.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
9 / 225
2. Statistical hypothesis testing
Introduction (cont’d)
A statistical test is based on three elements:
1
A null hypothesis and an alternative hypothesis
2
A rejection region based on a test statistic and a critical value
3
A type I error and a type II error
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
10 / 225
2. Statistical hypothesis testing
Introduction (cont’d)
A statistical test is based on three elements:
1
A null hypothesis and an alternative hypothesis
2
A rejection region based on a test statistic and a critical value
3
A type I error and a type II error
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
11 / 225
2. Statistical hypothesis testing
De…nition (Hypothesis)
A hypothesis is a statement about a population parameter. The formal
testing procedure involves a statement of the hypothesis, usually in terms
of a “null” or maintained hypothesis and an “alternative,” conventionally
denoted H0 and H1 , respectively.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
12 / 225
2. Statistical hypothesis testing
Introduction
1
The null hypothesis refers to a general or default position: that there
is no relationship between two measured phenomena or that a
potential medical treatment has no e¤ect.
2
The costs associated to the violation of the null must be higher than
the cost of a violation of the alternative.
Example (Choice of the null hypothesis)
In a credit scoring problem, in general we have: H0 : the client is not
risky(acceptance of the loan) versus H1 : the client is risky (refusal of the
loan).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
13 / 225
2. Statistical hypothesis testing
De…nition (Simple and composite hypotheses)
A simple hypothesis speci…es the population distribution completely. A
composite hypothesis does not specify the population distribution
completely.
Example (Simple and composite hypotheses)
If X
t (θ ) , H0 : θ = θ 0 is a simple hypothesis. H1 : θ > θ 0 , H1 : θ < θ 0 ,
and H1 : θ 6= θ 0 are composite hypotheses.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
14 / 225
2. Statistical hypothesis testing
De…nition (One-sided test)
A one-sided test has the general form:
H0 : θ = θ 0
H1 : θ < θ 0
Christophe Hurlin (University of Orléans)
or
H0 : θ = θ 0
H1 : θ > θ 0
Advanced Econometrics - Master ESA
November 20, 2015
15 / 225
2. Statistical hypothesis testing
De…nition (Two-sided test)
A two-sided test has the general form:
H0 : θ = θ 0
H1 : θ 6 = θ 0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
16 / 225
2. Statistical hypothesis testing
Introduction (cont’d)
A statistical test is based on three elements:
1
A null hypothesis and an alternative hypothesis
2
A rejection region based on a test statistic and a critical value
3
A type I error and a type II error
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
17 / 225
2. Statistical hypothesis testing
De…nition (Rejection region)
The rejection region is the set of values of the test statistic (or
equivalently the set of samples) for which the null hypothesis is rejected.
The rejection region is denoted W. For example, a standard rejection
region W is of the form:
W = fx : T (x ) > c g
or equivalently
W = fx1 , .., xN : T (x1 , .., xN ) > c g
where x denotes a sample fx1 , .., xN g , T (x ) the realisation of a test
statistic and c the critical value.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
18 / 225
2. Statistical hypothesis testing
Remarks
1
2
A (hypothesis) test is thus a rule that speci…es:
1
For which sample values the decision is made to ”fail to reject H0” as
true;
2
For which sample values the decision is made to ”reject H0”.
3
Never say "Accept H1", "fail to reject H1" etc..
The complement of the rejection region is the non-rejection region.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
19 / 225
2. Statistical hypothesis testing
Remark
The rejection region is de…ned as to be:
W = fx :
T (x )
| {z }
test statistic
7
c g
|{z}
critical value
T (x ) is the realisation of the statistic (random variable):
T (X ) = T (X1 , .., XN )
The test statistic T (X ) has an exact or an asymptotic distribution D
under the null H0 .
T (X )
Christophe Hurlin (University of Orléans)
H0
D
d
or T (X ) ! D
H0
Advanced Econometrics - Master ESA
November 20, 2015
20 / 225
2. Statistical hypothesis testing
Introduction (cont’d)
A statistical test is based on three elements:
1
A null hypothesis and an alternative hypothesis
2
A rejection region based on a test statistic and a critical value
3
A type I error and a type II error
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
21 / 225
2. Statistical hypothesis testing
Decision
Truth
Fail to reject H0
Reject H0
H0
Correct decision
Type I error
H1
Type II error
Correct decision
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
22 / 225
2. Statistical hypothesis testing
De…nition (Size)
The probability of a type I error is the (nominal) size of the test. This is
conventionally denoted α and is also called the signi…cance level.
α = Pr ( Wj H0 )
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
23 / 225
2. Statistical hypothesis testing
Remark
For a simple null hypothesis:
α = Pr ( Wj H0 )
For a composite null hypothesis:
α = sup Pr ( Wj H0 )
θ 0 2H 0
A test is said to have level if its size is less than or equal to α.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
24 / 225
2. Statistical hypothesis testing
De…nition (Power)
The power of a test is the probability that it will correctly lead to
rejection of a false null hypothesis:
power = Pr ( Wj H1 ) = 1
β
where β denotes the probability of type II error, i.e. β = Pr W H1 and
W denotes the non-rejection region.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
25 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 where σ2 is known. We want to test
H0 : m = m 0
H1 : m = m 1
with m1 < m0 . An econometrician propose the following rule of decision:
W = fx : x N < c g
where X N = N 1 ∑N
i =1 Xi denotes the sample mean and c is a constant
(critical value). Question: calculate the size and the power of this test.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
26 / 225
2. Statistical hypothesis testing
Solution
The rejection region is W= fx : x N < c g . Under the null H0 : m = m0 :
XN
H0
N
m0 ,
σ2
N
So, the size of the test is equal to:
α = Pr ( Wj H0 )
= Pr X N < c H0
= Pr
= Φ
Christophe Hurlin (University of Orléans)
XN
p
m0
σ/ N
m
p0
σ/ N
<
c
m
p 0 H0
σ/ N
c
Advanced Econometrics - Master ESA
November 20, 2015
27 / 225
2. Statistical hypothesis testing
Solution (cont’d)
The rejection region is W= fx : x N < c g . Under the alternative
H1 : m = m 1 :
σ2
XN
N m1 ,
H1
N
So, the power of the test is equal to:
power = Pr ( Wj H1 )
= Pr
= Φ
Christophe Hurlin (University of Orléans)
XN
p
m1
σ/ N
m
p1
σ/ N
<
c
m
p 1 H1
σ/ N
c
Advanced Econometrics - Master ESA
November 20, 2015
28 / 225
2. Statistical hypothesis testing
Solution (cont’d)
In conclusion:
α=Φ
c
m
p0
σ/ N
m
p1
σ/ N
We have a system of two equations with three parameters: α, β (or power)
and the critical value c.
β=1
power = 1
Φ
c
1
There is a trade-o¤ between the probabilities of the errors of type I
and II, i.e. α and β : if c decreases, α decreases but β increases.
2
A solution is to impose a size α and determine the critical value and
the power.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
29 / 225
2. Statistical hypothesis testing
Solution (cont’d)
In order to illustrate the tradeo¤ between α and β given the critical value
c, take an example with σ2 = 1 and N = 100:
H0 : m = m0 = 1.2
XN
H0
N
m0 ,
H1 : m = m1 = 1
σ2
N
XN
H1
N
m1 ,
σ2
N
We have
W = fx : x N < c g
α = Pr ( Wj H0 ) = Φ
β = Pr W H1 = 1
Christophe Hurlin (University of Orléans)
Φ
c
m
p0
σ/ N
c
m
p1
σ/ N
= Φ (10 (c
=1
Advanced Econometrics - Master ESA
1.2))
Φ (10 (c
1))
November 20, 2015
30 / 225
2. Statistical hypothesis testing
4.5
Density under H0
Density under H1
4
3.5
3
2.5
β=1-Pr(W|H1)=36.12%
α=Pr(W[H0)=5%
2
1.5
1
0.5
0
0.7
0.8
Christophe Hurlin (University of Orléans)
0.9
1
1.1
1.2
1.3
1.4
Advanced Econometrics - Master ESA
1.5
1.6
November 20, 2015
31 / 225
2. Statistical hypothesis testing
Click me!
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
32 / 225
2. Statistical hypothesis testing
Fact (Critical value)
The (nominal) size α is …xed by the analyst and the critical value is
deduced from α.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
33 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 , N = 100 and σ2 = 1 . We want to test
H0 : m = 1.2
H1 : m = 1
An econometrician propose the following rule of decision:
W = fx : x N < c g
where X N = N 1 ∑N
i =1 Xi denotes the sample mean and c is a constant
(critical value). Questions: (1) what is the critical value of the test of
size α = 5%? (2) what is the power of the test?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
34 / 225
2. Statistical hypothesis testing
Solution
We know that:
m
p0
σ/ N
So, the critical value that corresponds to a signi…cance level of α is:
α = Pr ( Wj H0 ) = Φ
σ
c = m0 + p Φ
N
c
1
(α)
NA: if m0 = 1.2, m1 = 1, N = 100, σ2 = 1 and α = 5%, then the
rejection region is
W = fx : x N < 1.0355g
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
35 / 225
2. Statistical hypothesis testing
Solution (cont’d)
W=
σ
x : x N < m0 + p Φ
N
1
(α)
The power of the test is:
power = Pr ( Wj H1 ) = Φ
c
m
p1
σ/ N
Given the critical value, we have:
power = Φ
m0
m
p 1 +Φ
σ/ N
1
(α)
NA: if m0 = 1.2, m1 = 1, N = 100, σ2 = 1 and α = 5%:
power = Φ
Christophe Hurlin (University of Orléans)
1.2 1
p
+Φ
1/ 100
1
(0.05)
Advanced Econometrics - Master ESA
= 0.6388
November 20, 2015
36 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 with σ2 = 1 and N = 100. We want to test
H0 : m = 1.2
H1 : m = 1
The rejection region for a signi…cance level α = 5% is:
W = fx : x N < 1.0355g
where X N = N 1 ∑N
i =1 Xi denotes the sample mean. Question: if the
realisation of the sample mean is equal to 1.13, what is the conclusion of
the test?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
37 / 225
2. Statistical hypothesis testing
Solution (cont’d)
For a nominal size α = 5%, the rejection region is:
W = fx : x N < 1.0355g
If we observe
x N = 1.13
This realisation does not belong to the rejection region:
xN 2
/W
For a level of 5%, we do not reject the null hypothesis H0 : m = 1.2.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
38 / 225
2. Statistical hypothesis testing
De…nition (Power function)
In general, the alternative hypothesis is composite. In this case, the power
is a function of the value of the parameter under the alternative.
power = P (θ )
Christophe Hurlin (University of Orléans)
8 θ 2 H1
Advanced Econometrics - Master ESA
November 20, 2015
39 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 where σ2 is known. We want to test
H0 : m = m 0
H1 : m < m 0
Consider the following rule of decision:
W=
σ
x : x N < m0 + p Φ
N
1
(α)
Questions: What is the power function of the test?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
40 / 225
2. Statistical hypothesis testing
Solution
As in the previous case, we have:
power = P (m ) = Φ
m0
p
m
σ/ N
+Φ
1
(α)
with m < m0
NA: if m0 = 1.2, N = 100, σ2 = 1 and α = 5%.
P (m ) = Φ
Christophe Hurlin (University of Orléans)
1.2 m
1/10
1.6449
with m < m0
Advanced Econometrics - Master ESA
November 20, 2015
41 / 225
2. Statistical hypothesis testing
Power function P (m )
1.2
1
0.8
0.6
0.4
0.2
0
0.7
Christophe Hurlin (University of Orléans)
0.8
0.9
1
Advanced Econometrics - Master ESA
1.1
1.2
November 20, 2015
42 / 225
2. Statistical hypothesis testing
Example (Power function)
Consider a test H0 : θ = θ 0 versus H1 : θ 6= θ 0 , the power function has this
general form:
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
43 / 225
2. Statistical hypothesis testing
De…nition (Most powerful test)
A test (denoted A) is uniformly most powerful (UMP) if it has greater
power than any other test of the same size for all admissible values of the
parameter.
αA = αB = α
βA
βB
for any test B of size α.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
44 / 225
2. Statistical hypothesis testing
UMP tests
How to derive the rejection region of the UMP test of size α ?
=> the Neyman–Pearson lemma
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
45 / 225
2. Statistical hypothesis testing
Lemma (Neyman Pearson)
Consider a hypothesis test between two point hypotheses H0 : θ = θ 0 and
H1 : θ = θ 1 . The uniformly most powerful (UMP) test has a rejection
region de…ned by:
LN (θ 0 ; x )
<K
W = xj
LN (θ 1 ; x )
where LN (θ 0 ; x ) denotes the likelihood of the sample x and K is a
constant determined by the size α such that:
Pr
Christophe Hurlin (University of Orléans)
LN (θ 0 ; X )
< K H0
LN (θ 1 ; X )
Advanced Econometrics - Master ESA
=α
November 20, 2015
46 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 where σ2 is known. We want to test
H0 : m = m 0
H1 : m = m 1
with m1 > m0 . Question: What is the rejection region of the UMP test of
size α?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
47 / 225
2. Statistical hypothesis testing
Solution
Since X1 , .., XN are N .i.d. m , σ2 , the likelihood of the sample
fx1 , .., xN g is de…ned as to be (cf. chapter 2):
!
1 N
1
2
exp
LN (θ; x ) =
(xi m)
2σ2 i∑
σN (2π )N /2
=1
Given the Neyman Pearson lemma the rejection region of the UMP test of
size α is given by:
LN (θ 0 ; x )
<K
LN (θ 1 ; x )
where K is a constant determined by the size α.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
48 / 225
2. Statistical hypothesis testing
Solution (cont’d)
1
σN (2π )N /2
exp
1
2σ2
∑N
i =1 (xi
m0 ) 2
1
σN (2π )N /2
exp
1
2σ2
∑N
i =1 (xi
m1 ) 2
<K
This expression can rewritten as:
exp
1
∑N
i =1 (xi
2σ2
() ∑Ni=1 (xi
m1 ) 2
m1 ) 2
∑N
i =1 (xi
∑N
i =1 (xi
m0 ) 2
<K
m0 ) 2 < K 1
where K1 = 2σ2 ln (K ) is a constant.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
49 / 225
2. Statistical hypothesis testing
Solution (cont’d)
∑N
i =1 (xi
() 2 (m0
m1 ) 2
N m12
Christophe Hurlin (University of Orléans)
m 0 ) 2 < K1
2
m1 ) ∑ N
i =1 xi + N m1
() (m0
where K2 = K1
∑N
i =1 (xi
m02
m02 < K1
m1 ) ∑ N
i =1 xi < K2
/2 is a constant.
Advanced Econometrics - Master ESA
November 20, 2015
50 / 225
2. Statistical hypothesis testing
Solution (cont’d)
( m0
m1 ) ∑ N
i =1 xi < K2
Since m1 > m0 , we have
1 N
∑ xi > K3
N i =1
where K3 = K2 / (N (m0
m1 )) is a constant.
The rejection region of the UMP test for H0 : m = m0 against
H0 : m = m1 with m1 > m0 has the general form:
W = fx : x N > Ag
where A is a constant.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
51 / 225
2. Statistical hypothesis testing
Solution (cont’d)
W = fx : x N > Ag
Determine the critical value A from the nominal size:
α = Pr ( Wj H0 )
= Pr ( x N > Aj H0 )
X N m0
A m0
p
p H0
= 1 Pr
<
σ/ N
σ/ N
A m0
p
= 1 Φ
σ/ N
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
52 / 225
2. Statistical hypothesis testing
Solution (cont’d)
α=1
Φ
A
m
p 0
σ/ N
So, we have
σ
A = m0 + p Φ
N
1
(1
α)
The rejection region of the UMP test of size α for H0 : m = m0 against
H0 : m = m1 with m1 > m0 is:
W=
σ
x : x N > m0 + p Φ
N
Christophe Hurlin (University of Orléans)
1
(1
Advanced Econometrics - Master ESA
α)
November 20, 2015
53 / 225
2. Statistical hypothesis testing
Fact (UMP one-sided test)
For a one-sided test
H0 : θ = θ 0 against H1 : θ > θ 0 (or H1 : θ < θ 1 )
the rejection region W of the UMP test is equivalent to the rejection
region obtained for the test
H0 : θ = θ 0 against H1 : θ = θ 1
with for θ 1 > θ 0 (or θ 1 < θ 0 ) if this region does not depend on the value
of θ 1 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
54 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 where σ2 is known. We want to test
H0 : m = m 0
H1 : m > m 0
Question: What is the rejection region of the UMP test of size α?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
55 / 225
2. Statistical hypothesis testing
Solution
Consider the test:
H0 : m = m 0
H1 : m = m 1
with m1 > m0 . The rejection region of the UMP test of size α is:
W=
σ
x : x N > m0 + p Φ
N
1
(1
α)
W does not depend on m1 . It is also the rejection region of the UMP
one-sided test for
H0 : m = m 0
H1 : m > m 0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
56 / 225
2. Statistical hypothesis testing
Fact (Two-sided test)
For a two-sided test
H0 : θ = θ 0 against H1 : θ 6= θ 0
the non rejection region W of the test of size α is the intersection of the
non rejection regions of the corresponding one-sided UMP tests of size
α/2
Test A: H0 : θ = θ 0 against H1 : θ > θ 0
Test B:
H0 : θ = θ 0 against H1 : θ < θ 0
So, we have:
W = WA \WB
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
57 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 where σ2 is known. We want to test
H0 : m = m0
H1 : m 6 = m0
Question: What is the rejection region of the test of size α?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
58 / 225
2. Statistical hypothesis testing
Solution
Consider the one-sided tests:
Test A: H0 : m = m0 against H1 : m < m0
Test B: H0 : m = m0 against H1 : m > m0
The rejection regions of the UMP test of size α/2 are:
WA =
WB =
Christophe Hurlin (University of Orléans)
σ
x : x N < m0 + p Φ
N
σ
x : x N > m0 + p Φ
N
1
1
(α/2)
(1
Advanced Econometrics - Master ESA
α/2)
November 20, 2015
59 / 225
2. Statistical hypothesis testing
Solution (cont’d)
The non-rejection regions of the UMP test of size α/2 are:
WA =
WB =
x : xN
x : xN
σ
m0 + p Φ
N
σ
m0 + p Φ
N
1
1
(α/2)
(1
α/2)
The non rejection region of the two-sided test corresponds to the
intersection of these two regions:
W = WA \WB
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
60 / 225
2. Statistical hypothesis testing
Solution (cont’d)
So, non rejection region of the two-sided test of size α is:
W=
Since, Φ
σ
x : m0 + p Φ
N
1
(α/2) =
W=
Φ
1
1
(1
x : jx N
Christophe Hurlin (University of Orléans)
(α/2)
xN
σ
m0 + p Φ
N
1
(1
α/2)
α/2) , this region can be rewritten as:
m0 j
σ
p Φ
N
1
(1
Advanced Econometrics - Master ESA
α/2)
November 20, 2015
61 / 225
2. Statistical hypothesis testing
Solution (cont’d)
σ
p Φ 1 (1 α/2)
N
Finally, the rejection region of the two-sided test of size α is:
W=
W=
x : jx N
x : jx N
Christophe Hurlin (University of Orléans)
m0 j
σ
m0 j > p Φ
N
1
(1
Advanced Econometrics - Master ESA
α/2)
November 20, 2015
62 / 225
2. Statistical hypothesis testing
Solution (cont’d)
W=
x : jx N
σ
m0 j > p Φ
N
1
(1
α/2)
NA: if m0 = 1.2, N = 100, σ2 = 1 and α = 5%:
W=
x : jx N
1.2j >
W = fx : jx N
1
Φ
10
1
(0.975)
1.2j > 0.1960g
If the realisation of jx N 1.2j is larger than 0.1960, we reject the null
H0 : m = 1.2 for a signi…cance level of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
63 / 225
2. Statistical hypothesis testing
De…nition (Unbiased Test)
A test is unbiased if its power P (θ ) is greater than or equal to its size α
for all values of the parameter θ.
P (θ )
α
8 θ 2 H1
By construction, we have P (θ 0 ) = Pr ( Wj H j0 ) = α.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
64 / 225
2. Statistical hypothesis testing
De…nition (Consistent Test)
A test is consistent if its power goes to one as the sample size grows to
in…nity.
lim P (θ ) = 1 8θ 2 H1
N !∞
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
65 / 225
2. Statistical hypothesis testing
Example (Test on the mean)
Consider a sequence X1 , .., XN of i.i.d. continuous random variables with
Xi
N m, σ2 where σ2 is known. We want to test
H0 : m = m 0
H1 : m < m 0
The rejection region of the UMP test of size α is
W=
σ
x : x N < m0 + p Φ
N
1
(α)
Question: show that this test is (1) unbiased and (2) consistent.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
66 / 225
2. Statistical hypothesis testing
Solution
W=
σ
x : x N < m0 + p Φ
N
1
(α)
The power function of the test is de…ned as to be:
P (m ) = Pr ( Wj H1 )
= Pr
= Φ
Christophe Hurlin (University of Orléans)
σ
X N < m0 + p Φ
N
m0 m
p + Φ 1 (α)
σ/ N
1
Advanced Econometrics - Master ESA
( α ) m < m0
November 20, 2015
67 / 225
2. Statistical hypothesis testing
Solution
m0
P (m ) = Φ
p
m
σ/ N
+Φ
1
(α)
8 m < m0
The test is consistent since:
lim P (m ) = 1
N !∞
The test is unbiased since
P (m )
α
8 m < m0
lim P (m ) = Φ Φ
m !m 0
Christophe Hurlin (University of Orléans)
1
(α) = α
Advanced Econometrics - Master ESA
November 20, 2015
68 / 225
2. Statistical hypothesis testing
Solution
The decision ”Reject H0” or ”fail to reject H0” is not so informative!
Indeed, there is some ”arbitrariness” to the choice of α (level).
Another strategy is to ask, for every α, whether the test rejects at
that level.
Another alternative is to use the so-called p-value— the smallest level
of signi…cance at which H0 would be rejected given the value of the
test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
69 / 225
2. Statistical hypothesis testing
De…nition (p-value)
Suppose that for every α 2 [0, 1], one has a size α test with rejection
region Wα . Then, the p-value is de…ned to be:
p-value = inf fα : T (y ) 2 Wα g
The p-value is the smallest level at which one can reject H0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
70 / 225
2. Statistical hypothesis testing
The p-value is a measure of evidence against H0 :
p-value
evidence
< 0.01
Very strong evidence against H0
0.01
0.05
Strong evidence against H0
0.05
0.10
Weak evidence against H0
> 0.10
Christophe Hurlin (University of Orléans)
Little or no evidence against H0
Advanced Econometrics - Master ESA
November 20, 2015
71 / 225
2. Statistical hypothesis testing
Remarks
1
A large p-value does not mean ”strong evidence in favor of H0”.
2
A large p-value can occur for two reasons:
3
1
H0 is true;
2
H0 is false but the test has low power.
The p-value is not the probability that the null hypothesis is true!
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
72 / 225
2. Statistical hypothesis testing
For a nominal size of 5%, we reject the null H0 : βSP 500 = 0.
For a nominal size of 5%, we fail to reject the null H0 : βC = 0.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
73 / 225
2. Statistical hypothesis testing
Summary
Hypothesis testing is de…ned by the following general procedure
Step 1: State the relevant null and alternative hypotheses (misstating the
hypotheses muddies the rest of the procedure!);
Step 2: Consider the statistical assumptions being made about the sample
in doing the test (independence, distributions, etc.)— incorrect
assumptions mean that the test is invalid!
Step 3: Choose the appropriate test (exact or asymptotic tests) and thus
state the relevant test statistic (say, T).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
74 / 225
2. Statistical hypothesis testing
Summary (cont’d)
Step 4: Derive the distribution of the test statistic under the null
hypothesis (sometimes it is well-known, sometimes it is more tedious!)— for
example, the Student t-distribution or the Fisher distribution.
Step 5: Determine the critical value (and thus the critical region).
Step 6: Compute (using the observations!) the observed value of the test
statistic T , say tobs .
Step 7: Decide to either fail to reject the null hypothesis or reject in favor
of the alternative assumption— the decision rule is to reject the null
hypothesis H0 if the observed value of the test statistic, tobs is in the
critical region, and to ”fail to reject” the null hypothesis otherwise
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
75 / 225
2. Statistical hypothesis testing
Key concepts
1
Null and alternative hypotheses
2
Simple and composite hypotheses
3
One-sided and two-sided tests
4
Rejection region, test statistic and critical value
5
Type I and type II errors
6
Size, power and power function
7
Uniformly most powerful (UMP) test
8
Neyman Pearson lemma
9
Consistent test and unbiased test
10
p-value
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
76 / 225
Section 3
Tests in the multiple linear regression model
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
77 / 225
3. Tests in the multiple linear regression model
Objectives
In the context of the multiple linear regression model (cf. chapter 3), the
objective of this section is to present :
1
the Student test
2
the t-statistic and the z-statistic
3
the Fisher test
4
the global F-test
5
To distinguish the case with normality assumption and the case
without any assumption on the distribution of the error term
(semi-parametric speci…cation)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
78 / 225
3. Tests in the multiple linear regression model
Be careful: in this section, I don’t distinguish between a random vector
(matrix) and a vector (matrix) of deterministic elements. For more
appropriate notations, see:
Abadir and Magnus (2002), Notation in econometrics: a proposal for a
standard, Econometrics Journal.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
79 / 225
3. Tests in the multiple linear regression model
Model
Consider the (population) multiple linear regression model:
y = Xβ + ε
where (cf. chapter 3):
y is a N
1 vector of observations yi for i = 1, .., N
X is a N K matrix of K explicative variables xik for k = 1, ., K and
i = 1, .., N
ε is a N
1 vector of error terms εi .
β = ( β1 ..βK )> is a K
Christophe Hurlin (University of Orléans)
1 vector of parameters
Advanced Econometrics - Master ESA
November 20, 2015
80 / 225
3. Tests in the multiple linear regression model
Assumptions
Fact (Assumptions)
We assume that the multiple linear regression model satisfy the
assumptions A1-A5 (cf. chapter 3)
We distinguish two cases:
N 0, σ2 IN
1
Case 1: assumption A6 (Normality) holds and ε
2
Case 2: the distribution of ε is unknown (semi-parametric
speci…cation) and ε ??
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
81 / 225
3. Tests in the multiple linear regression model
Parametric tests
The βk are unknown features of the population, but:
1
One can formulate a hypothesis about their value;
2
One can construct a test statistic with a known …nite sample
distribution (case 1) or an asymptotic distribution (case 2);
3
One can take a ”decision” meaning ”reject H0” if the value of the
test statistic is too unlikely.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
82 / 225
3. Tests in the multiple linear regression model
Three tests of interest:
H0 : β k = a k
H1 : β k < a k
or
H0 : β k = a k
H1 : β k > a k
H0 : β k = a k
H1 : β k 6 = a k
H0 : Rβ = q
H1 : Rβ 6= q
where ak = 0 or ak 6= 0.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
83 / 225
3. Tests in the multiple linear regression model
For that, we introduce two types of test
1
The Student test or t-test
2
The Fisher test of F-test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
84 / 225
Subsection 3.1
The Student test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
85 / 225
3.1. The Student test
Case 1: Normality assumption A6
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
86 / 225
3.1. The Student test
Assumption 6 (normality): the disturbances are normally distributed.
εj X
Christophe Hurlin (University of Orléans)
N 0N
1, σ
2
IN
Advanced Econometrics - Master ESA
November 20, 2015
87 / 225
3.1. The Student test
Reminder (cf. chapter 3)
Fact (Linear regression model)
b and σ
b2 have a
Under the assumption A6 (normality), the estimators β
…nite sample distribution given by:
b
β
N
b2
σ
(N
σ2
β,σ2 X> X
K)
χ2 (N
1
K)
b and σ
b2 are independent. This result holds whether or not the
Moreover, β
b is
matrix X is considered as random. In this last case, the distribution of β
conditional to X.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
88 / 225
3.1. The Student test
Remarks
1
2
b is also normally distributed:
Any linear combination of β
b
Aβ
N
Aβ,σ2 A X> X
1
A>
b has a joint normal distribution.
Any subset of β
b
βk
N βk , σ2 mkk
where mkk is k th diagonal element of X> X
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
1
.
November 20, 2015
89 / 225
3.1. The Student test
Reminder
If X and Y are two independent random variables such that
N (0, 1)
X
Y
χ2 ( θ )
then the variable Z de…ned as to be
Z = p
X
Y /θ
has a Student’s t-distribution with θ degrees of freedom
Z
Christophe Hurlin (University of Orléans)
t( θ )
Advanced Econometrics - Master ESA
November 20, 2015
90 / 225
3.1. The Student test
Student test statistic
Consider a test with the null:
H0 : β k = a k
Under the null H0 :
b
β k ak
p
σ mkk
b2
σ
(N
σ2
K)
H0
H0
N (0, 1)
χ2 (N
K)
and these two variables are independent...
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
91 / 225
3.1. The Student test
Student test statistic (cont’d)
b
β k ak
p
σ mkk
b2
σ
(N
σ2
K)
H0
H0
N (0, 1)
χ2 (N
K)
So, under the null H0 we have:
q
Christophe Hurlin (University of Orléans)
b
βk a k
p
σ m kk
b 2 (N K )
σ
σ 2 (N K )
=
b
β k ak
p
b mkk
σ
H0
t(N
Advanced Econometrics - Master ESA
K)
November 20, 2015
92 / 225
3.1. The Student test
De…nition (Student t-statistic)
Under the null H0 : βk = ak , the Student test-statistic or t-statistic is
de…ned to be:
Tk =
b
βk
ak
se
b b
βk
H0
t(N
K)
where N is the number of observations, K is the number of explanatory
variables (including the constant term), t(N K ) is the Student
t-distribution with N K degrees of freedom and
se
b b
βk
p
b mkk
=σ
with mkk is k th diagonal element of X> X
Christophe Hurlin (University of Orléans)
1
.
Advanced Econometrics - Master ESA
November 20, 2015
93 / 225
3.1. The Student test
Remarks
1
Under the assumption A6 (normality) and under the null
H0 : βk = ak , the Student test-statistic has an exact (…nite sample)
distribution.
Tk
2
H0
t(N
K)
The term se
b b
βk denotes the estimator of the standard error of the
OLS estimator b
βk and it corresponds to the square root of the k th
b (cf. chapter 3):
b β
diagonal element of V
b =σ
b β
b 2 X> X
V
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
1
November 20, 2015
94 / 225
3.1. The Student test
Consider the one-sided test:
H0 : β k = a k
H1 : β k < a k
The rejection region is de…ned as to be:
W = f y : Tk ( y ) < A g
where A is a constant determined by the nominal size α.
α = Pr ( Wj H0 ) = Pr
Christophe Hurlin (University of Orléans)
Tk ( y ) < A j Tk
Advanced Econometrics - Master ESA
H0
t(N
K)
November 20, 2015
95 / 225
3.1. The Student test
α = Pr
Tk ( y ) < A j Tk
H0
t(N
K)
= FN
K
(A)
where FN K (.) denotes the cdf of the Student’s t-distribution with N
degrees of freedom. Denote cα the α-quantile of this distribution:.
K
A = FN 1 K ( α ) = c α
The rejection region of the test of size α is de…ned as to be:
W = f y : Tk ( y ) < c α g
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
96 / 225
3.1. The Student test
De…nition (One-sided Student test)
The critical region of the Student test is that H0 : βk = ak is rejected in
favor of H1 : βk < ak at the α (say, 5%) signi…cance level if:
W = f y : Tk ( y ) < c α g
where cα is the α (say, 5%) critical value of a Student t-distribution with
N K degrees of freedom and Tk (y ) is the realisation of the Student
test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
97 / 225
3.1. The Student test
Example (One-sided test)
Consider the CAPM model (cf. chapter 1) and the following results
(Eviews). We want to test the beta of MSFT as
H0 : βMSFT = 1
against
H1 : βMSFT < 1
Question: give a conclusion for a nominal size of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
98 / 225
3.1. The Student test
Solution
Step 1: compute the t-statistic
TMSFT (y ) =
b
βMSFT
1
se
b b
βMSFT
=
1.9898 1
= 3.1501
0.3142
Step 2: Determine the rejection region for a nominal size α = 5%.
TMSFT
H0
t(20
W = f y : Tk ( y ) <
2)
1.7341g
Conclusion: for a signi…cance level of 5%, we fail to reject the null
H0 : βMSFT = 1 against H1 : βMSFT < 1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
99 / 225
3.1. The Student test
Solution (cont’d)
0.5
0.45
Density of Ts under H0
0.4
0.35
0.3
0.25
0.2
α=5%
0.15
DF = 18
0.1
Critical value = -1.7341
0.05
0
-4
-3
Christophe Hurlin (University of Orléans)
-2
-1
0
1
2
Advanced Econometrics - Master ESA
3
4
November 20, 2015
100 / 225
3.1. The Student test
Consider the one-sided test
H0 : β k = a k
H1 : β k > a k
The rejection region is de…ned as to be:
W = f y : Tk ( y ) > A g
where A is a constant determined by the nominal size α.
α = Pr ( Wj H0 ) = Pr
Christophe Hurlin (University of Orléans)
Tk ( y ) > A j Tk
Advanced Econometrics - Master ESA
H0
t(N
K)
November 20, 2015
101 / 225
3.1. The Student test
α=1
Tk ( y ) < A j Tk
Pr
H0
t(N
K)
or equivalently
1
α = FN
K
(A)
where FN K (.) denotes the cdf of the Student’s t-distribution with N K
degrees of freedom. Denote c1 α the 1 α quantile of this distribution:
A = FN 1 K ( 1
α ) = c1
α
The rejection region of the test of size α is de…ned as to be:
W = f y : Tk ( y ) > c1
Christophe Hurlin (University of Orléans)
αg
Advanced Econometrics - Master ESA
November 20, 2015
102 / 225
3.1. The Student test
De…nition (One-sided Student test)
The critical region of the Student test is that H0 : βk = ak is rejected in
favor of H1 : βk > ak at the α (say, 5%) signi…cance level if:
W = f y : Tk ( y ) > c1
αg
where c1 α is the 1 α (say, 95%) critical value of a Student
t-distribution with N K degrees of freedom and Tk (y ) is the realisation
of the Student test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
103 / 225
3.1. The Student test
Example (One-sided test)
Consider the CAPM model (cf. chapter 1) and the following results
(Eviews). We want to test the beta of MSFT as
H0 : βMSFT = 1
against
H1 : βMSFT > 1
Question: give a conclusion for a nominal size of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
104 / 225
3.1. The Student test
Solution
Step 1: compute the t-statistic
TMSFT (y ) =
b
βMSFT
1
se
b b
βMSFT
=
1.9898 1
= 3.1501
0.3142
Step 2: Determine the rejection region for a nominal size α = 5%.
TMSFT
H0
t(20
2)
W = fy : Tk (y ) > 1.7341g
Conclusion: for a signi…cance level of 5%, we reject the null
H0 : βMSFT = 1 against H1 : βMSFT > 1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
105 / 225
3.1. The Student test
Solution (cont’d)
0.5
0.45
Density of Ts under H0
0.4
0.35
0.3
0.25
0.2
0.15
α=5%
DF = 18
0.1
Critical value = 1.7341
0.05
0
-4
-3
Christophe Hurlin (University of Orléans)
-2
-1
0
1
2
Advanced Econometrics - Master ESA
3
4
November 20, 2015
106 / 225
3.1. The Student test
Consider the two-sided test
H0 : β k = a k
H1 : β k 6 = a k
The non-rejection region is de…ned as the intersection of the two
non-rejection regions of the one-sided test of level α/2:
W = W A \ WB
H0 : βk = ak against H1 : βk < ak
H0 : βk = ak against H1 : βk > ak
Christophe Hurlin (University of Orléans)
WA = fy : Tk (y ) > cα/2 g
WB = f y : T k ( y ) < c1
Advanced Econometrics - Master ESA
November 20, 2015
α/2 g
107 / 225
3.1. The Student test
W = fy : cα/2 < Tk (y ) < c1
α/2 g
Since the Student’s t-distribution is symmetric, cα/2 =
W = fy :
c1
α/2
< T k ( y ) < c1
c1
α/2
α/2 g
The rejection region is then de…ned as to be:
W = fy : jTk (y )j > c1
Christophe Hurlin (University of Orléans)
α/2 g
Advanced Econometrics - Master ESA
November 20, 2015
108 / 225
3.1. The Student test
De…nition (Two-sided Student test)
The critical region of the Student test is that H0 : βk = ak is rejected in
favor of H1 : βk 6= ak at the α (say, 5%) signi…cance level if:
W = fy : jTk (y )j > c1
α/2 g
where c1 α/2 is the 1 α/2 (say, 97.5%) critical value of a Student
t-distribution with N K degrees of freedom and Tk (y ) is the realisation
of the Student test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
109 / 225
3.1. The Student test
Example (One-sided test)
Consider the CAPM model (cf. chapter 1) and the following results
(Eviews). We want to test the beta of MSFT as
H0 : βMSFT = 1
against
H1 : βMSFT 6= 1
Question: give a conclusion for a nominal size of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
110 / 225
3.1. The Student test
Solution
Step 1: compute the t-statistic
TMSFT (y ) =
b
βMSFT
1
se
b b
βMSFT
=
1.9898 1
= 3.1501
0.3142
Step 2: Determine the rejection region for a nominal size α = 5%.
TMSFT
H0
t(20
2)
W = fy : jTk (y )j > 2.1009g
Conclusion: for a signi…cance level of 5%, we reject the null
H0 : βMSFT = 1 against H1 : βMSFT 6= 1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
111 / 225
3.1. The Student test
0.5
Density of Ts under H0
0.45
0.4
0.35
0.3
0.25
0.2
0.15
α=2.5%
α=2.5%
DF = 18
0.1
|Critical value| = 2.1009
0.05
0
-4
-3
Christophe Hurlin (University of Orléans)
-2
-1
0
1
2
Advanced Econometrics - Master ESA
3
4
November 20, 2015
112 / 225
3.1. The Student test
Rejection regions
H0
H1
β k = ak
β k > ak
β k = ak
β k < ak
β k = ak
β k 6 = ak
Rejection region
W = f y : Tk ( y ) > c1
αg
W = f y : Tk ( y ) < c α g
W = fy : jTk (y )j > c1
α/2 g
where cβ denotes the β-quantile (critical value) of the Student
t-distribution with N K degrees of freedom.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
113 / 225
3.1. The Student test
De…nition (P-values)
The p-values of Student tests are equal to:
Two-sided test: p-value = 2
FN
Right tailed test: p-value = 1
Left tailed test: p-value = FN
K
FN
K
( jTk (y )j)
K
(Tk (y ))
( Tk (y ))
where Tk (y ) is the realisation of the Student test-statistic and FN K (.)
the cdf of the Student’s t-distribution with N K degrees of freedom.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
114 / 225
3.1. The Student test
Example (One-sided test)
Consider the previous CAPM model. We want to test:
H0 : c = 0
against
H1 : c 6= 0
H0 : βMSFT = 0
against
H1 : βMSFT 6= 0
Question: …nd the corresponding p-values.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
115 / 225
3.1. The Student test
Solution
Since we consider two-sided tests with N = 20 and K = 2:
p-valuec = 2
p-valuec = 2
F18 (
F18 (
Christophe Hurlin (University of Orléans)
jTc (y )j) = 2
jTMSFT (y )j) = 2
F18 ( 0.9868) = 0.3368
F18 ( 6.3326) = 5.7e
Advanced Econometrics - Master ESA
November 20, 2015
006
116 / 225
3.1. The Student test
Fact (Student test with large sample)
For a large sample size N
Tk
H0
t(N
K)
N (0, 1)
Then, the rejection region for a Student two-sided test becomes
W = y : jTk (y )j > Φ
1
(1
α/2)
where Φ (.) denotes the cdf of the standard normal distribution. For
α = 5%, Φ 1 (0.975) = 1.96, so we have:
W = fy : jTk (y )j > 1.96g
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
117 / 225
3.1. The Student test
Case 2: Semi-parametric model
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
118 / 225
3.1. The Student test
Assumption 6 (normality): the distribution of the disturbances is
unknown, but satisfy (assumptions A1-A5):
E ( εj X) = 0N
1
V ( ε j X ) = σ 2 IN
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
119 / 225
3.1. The Student test
Problem
1
b2 are unknown.
The exact (…nite sample) distribution of b
βk and σ
2
As a consequence the …nite sample distribution of Tk (y ) is also
unknown.
3
But, we can use the asymptotic properties of the OLS estimators (cf.
chapter 3). In particular, we have:
p
where
b
N β
Q = p lim
Christophe Hurlin (University of Orléans)
d
β ! N 0, σ2 Q
1
1 >
X X = EX xi xi>
N
Advanced Econometrics - Master ESA
November 20, 2015
120 / 225
3.1. The Student test
De…nition (Z-statistic)
Under the null H0 : βk = ak , if the assumptions A1-A5 hold (cf. chapter
3), the z-statistic de…ned by
Zk =
b
βk
ak
βk
se
b asy b
d
! N (0, 1)
H0
p
b mkk denotes the estimator of the asymptotic
=σ
standard error of the estimator b
βk and mkk is k th diagonal element of
where se
b asy b
βk
X> X
1
.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
121 / 225
3.1. The Student test
Rejection regions
The rejection regions have the same form as for the t-test (except for the
distribution)
H0
H1
β k = ak
β k > ak
β k = ak
β k < ak
β k = ak
β k 6 = ak
Rejection region
W = y : Zk (y ) > Φ
1
W = y : Zk (y ) < Φ
W = y : jZk (y )j > Φ
1
(1
1
(1
α)
(α)
α/2)
where Φ (.) denotes the cdf of the standard normal distribution.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
122 / 225
3.1. The Student test
De…nition (P-values)
The p-values of the Z-tests are equal to:
Two-sided test: p-value = 2
right tailed test: p-value = 1
Φ(
jZk (y )j)
Φ (Zk (y ))
left tailed test: p-value = Φ ( Zk (y ))
where Zk (y ) is the realisation of the Z-statistic and Φ (.) the cdf of the
standard normal distribution.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
123 / 225
3.1. The Student test
Summary
Test-statistic
Normality Assumption
Non Assumption
t-statistic
z-statistic
De…nition
Tk =
Exact distribution
Tk
Asymptotic distribution
Christophe Hurlin (University of Orléans)
H0
b
βk a k
p
b m kk
σ
t(N
Zk =
—
K)
—
Advanced Econometrics - Master ESA
b
βk a k
p
b m kk
σ
d
ZK ! N (0, 1)
H0
November 20, 2015
124 / 225
3.1. The Student test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
125 / 225
Subsection 3.2
The Fisher test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
126 / 225
3.2. The Fisher test
Consider the two-sided test associated to p linear constraints on the
parameters βk :
H0 : Rβ = q
H1 : Rβ 6= q
where R is a p
K matrix and q is a p
Christophe Hurlin (University of Orléans)
1 vector.
Advanced Econometrics - Master ESA
November 20, 2015
127 / 225
3.2. The Fisher test
Example (Linear constraints)
If K = 4 and if we want to test H0 : β1 + β2 = 0 and β2
we have p = 2 linear constraints with:
R
β
(2 4 ) (4,1 )
1 1
0 1
Christophe Hurlin (University of Orléans)
0 0
3 0
=
3β3 = 4, then
q
(2 1 )
0
1
β1
B β C
B 2 C=
@ β A
3
β4
Advanced Econometrics - Master ESA
0
4
November 20, 2015
128 / 225
3.2. The Fisher test
Example (Linear constraints)
If K = 4 and if we want to test H0 : β2 = β3 = β4 = 0, then we have
p = 3 linear constraints with:
R
β
(3 4 ) (4,1 )
=
q
(3 1 )
0
1
0 1
β1
0 1 0 0
0
B β C
@ 0 0 1 0 AB 2 C = @ 0 A
@ β A
3
0 0 0 1
0
β4
0
Christophe Hurlin (University of Orléans)
1
Advanced Econometrics - Master ESA
November 20, 2015
129 / 225
3.1. The Student test
Case 1: Normality assumption A6
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
130 / 225
3.2. The Fisher test
De…nition (Fisher test-statistic)
Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic is
de…ned as to be:
F=
1
b
Rβ
p
q
>
b 2 R X> X
σ
1
1
R>
b
Rβ
q
b denotes the OLS estimator. Under the null H0 : Rβ = q, the
where β
F -statistic has a Fisher exact (…nite sample) distribution
F
Christophe Hurlin (University of Orléans)
H0
F(p,N
K)
Advanced Econometrics - Master ESA
November 20, 2015
131 / 225
3.2. The Fisher test
Reminder
If X and Y are two independent random variables such that
X
χ2 ( θ 1 )
Y
χ2 ( θ 2 )
Z =
X /θ 1
Y /θ 2
then the variable Z de…ned by
has a Fisher distribution with θ 1 and θ 2 degrees of freedom
Z
Christophe Hurlin (University of Orléans)
F(θ 1 ,θ 2 )
Advanced Econometrics - Master ESA
November 20, 2015
132 / 225
3.2. The Fisher test
Proof
Under assumption A6, we have the following (conditional to X)
distribution
1
b N β,σ2 X> X
β
b2
σ
(N
σ2
Christophe Hurlin (University of Orléans)
K)
χ2 (N
K)
Advanced Econometrics - Master ESA
November 20, 2015
133 / 225
3.2. The Fisher test
Proof (cont’d)
b
Consider the vector m = R β
q. Under the null
H0 : Rβ = q
We have
b
E (m) = RE β
V (m) = E
q = Rβ
b
Rβ
b R>
= RV β
= σ 2 R X> X
Christophe Hurlin (University of Orléans)
b
Rβ
q
1
q=0
q
>
R>
Advanced Econometrics - Master ESA
November 20, 2015
134 / 225
3.2. The Fisher test
Proof (cont’d)
We can base the test of H0 on the Wald criterion:
W
(1 1 )
=
=
m> (V (m))
(1 p )
b
Rβ
1
q
>
m
p 1
p p
2
>
σ R X X
1
1
R
>
Under assumption A6 (normality)
W
b2
σ
(N
σ2
H0
K)
b
Rβ
q
χ2 (p )
χ2 (N
K)
These two variables are independent.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
135 / 225
3.2. The Fisher test
Proof (cont’d)
W
H0
χ2 (p )
b2
σ
(N K ) χ2 (N K )
σ2
So, the ratio of these two variables has a Fisher distribution
F=
Christophe Hurlin (University of Orléans)
W
p
b 2 (N K ) H 0
σ
σ 2 (N K )
F(p,N
K)
Advanced Econometrics - Master ESA
November 20, 2015
136 / 225
3.2. The Fisher test
Proof (cont’d)
F=
b
Rβ
q
>
1
σ 2 R X> X
b2
σ
σ2
(N
1
R>
K ) / (N
K)
b
Rβ
q /p
After simpli…cation, the F-statistic is de…ned by:
1
b
Rβ
F=
p
q
>
Under the null H0 : Rβ =q :
F
Christophe Hurlin (University of Orléans)
2
>
b R X X
σ
H0
F(p,N
1
1
R
>
b
Rβ
q
K)
Advanced Econometrics - Master ESA
November 20, 2015
137 / 225
3.2. The Fisher test
De…nition (Fisher test-statistic)
Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic can
be de…ned as a function of the SSR of the constrained (H0 ) and
unconstrained model (H1 ):
F=
SSR0 SSR1
SSR1
N
K
p
where SSR0 denotes the sum of squared residuals of the constrained model
estimated under H0 and SSR1 denotes the sum of squared residuals of the
unconstrained model estimated under H1 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
138 / 225
3.2. The Fisher test
De…nition (Fisher test-statistic)
Under assumptions A1-A6 (cf. chapter 3), the Fisher test-statistic can
be de…ned as to be:
F=
1 b
βH 1
b2 p
σ
b
β
H0
>
X> X
b
β
H1
b
β
H0
b denotes the OLS estimator obtained in the constrained model
where β
H0
b denotes the OLS estimator obtained in the
(under H0 ) and β
H1
unconstrained model (under H1 ).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
139 / 225
3.2. The Fisher test
De…nition (Constrained OLS estimator)
b of
Under suitable regularity conditions, the constrained OLS estimator β
C
β, obtained under the constraint Rβ = q, is given by:
b =β
b
β
C
UC
X> X
1
R> R X> X
1
1
R>
b
where β
UC is the unconstrained OLS estimator.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
b
Rβ
UC
q
November 20, 2015
140 / 225
3.2. The Fisher test
Example (Fisher test and CAPM model)
Consider the extended CAPM model (…le: Chapter4_data.xls):
rMSFT ,t = β1 + β2 rSP 500,t + β3 rFord ,t + β4 rGE ,t + εt
where rMSFT ,t is the excess return for Microsoft, rSP 500,t for the SP500,
rFord ,t for Ford and rGE ,t for general electric. We want to test the
following linear constraints:
H0 : β2 = 1 and β3 = β4
Question: write a Matlab code to compute the F-statistic according to
the three alternative de…nitions.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
141 / 225
3.2. The Fisher test
Solution
In this problem, the null H0 : β2 = 1 and β3 = β4 can be written as:
R
β
(2 4 ) (4,1 )
0 1 0
0 0 1
Christophe Hurlin (University of Orléans)
0
1
=
q
(2 1 )
0
1
β1
B β C
B 2 C=
@ β A
3
β4
Advanced Econometrics - Master ESA
1
0
November 20, 2015
142 / 225
3.2. The Fisher test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
143 / 225
3.2. The Fisher test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
144 / 225
3.2. The Fisher test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
145 / 225
3.2. The Fisher test
Consider the Fisher test
H0 : Rβ = q
H1 : Rβ 6= q
Since the Fisher test-statistic is always positive, the rejection region is
de…ned as to be:
W = fy : F (y ) > Ag
where A is a constant determined by the nominal size α.
α = Pr ( Wj H0 ) = Pr
Christophe Hurlin (University of Orléans)
F (y ) > Aj F
Advanced Econometrics - Master ESA
H0
F(p,N
K)
November 20, 2015
146 / 225
3.2. The Fisher test
α = Pr ( Wj H0 ) = Pr
F (y ) > Aj F
H0
F(p,N
K)
or equivalently
α=1
Pr
F (y ) < Aj F
H0
F(p,N
K)
Denote d1 α the 1 α quantile of the Fisher distribution with p and
N K degrees of freedom.
A = d1 α
The rejection region of the test of size α is de…ned as to be:
W = fy : F (y ) > d1
Christophe Hurlin (University of Orléans)
αg
Advanced Econometrics - Master ESA
November 20, 2015
147 / 225
3.2. The Fisher test
De…nition (Rejection region of a Fisher test)
The critical region of the Fisher test is that H0 : Rβ = q is rejected in
favor of H1 : Rβ 6= q at the α (say, 5%) signi…cance level if:
W = fy : F (y ) > d1
αg
where d1 α is the 1 α critical value (say 95%) of the Fisher distribution
with p and N K degrees of freedom and Fk (y ) is the realisation of the
Fisher test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
148 / 225
3.2. The Fisher test
Example (Fisher test and CAPM model)
Consider the extended CAPM model (…le: Chapter4_data.xls):
rMSFT ,t = β1 + β2 rSP 500,t + β3 rFord ,t + β4 rGE ,t + εt
where rMSFT ,t is the excess return for Microsoft, rSP 500,t for the SP500,
rFord ,t for Ford and rGE ,t for general electric. We want to test the
following linear constraints:
H0 : β2 = 1 and β3 = β4
Question: given the realisation of the Fisher test-statistic (cf. previous
example), conclude for a signi…cance level α = 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
149 / 225
3.2. The Fisher test
Solution
Step 1: compute the F-statistic (cf. Matlab code)
F (y ) = 4.3406
Step 2: Determine the rejection region for a nominal size α = 5% for
N = 24, K = 4 and p = 2
F F(2,20 )
H0
W = fy : F (y ) > 3.4928g
Conclusion: for a signi…cance level of 5%, we reject the null H0 : Rβ = q
against H1 : Rβ 6= q
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
150 / 225
3.2. The Fisher test
1.2
Density of F under H0
1
DF1 = 2
DF2 = 20
Critical value = 3.4928
0.8
0.6
0.4
α=5%
0.2
0
0
Christophe Hurlin (University of Orléans)
1
2
3
4
Advanced Econometrics - Master ESA
5
6
November 20, 2015
151 / 225
3.2. The Fisher test
De…nition (Student test-statistic and Fisher test-statistic )
Consider the test
H0 : β k = a k
versus
H1 : βk 6= ak
the Fisher test-statistic corresponds to the squared of the corresponding
Student’s test-statistic
F = T2k
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
152 / 225
3.2. The Fisher test
Proof
Consider the test H0 : βk = ak against H1 : βk 6= ak , then we have:
R=
0 0 ..
1
k th position
0 0
q = ak
As a consequence :
b
Rβ
b 2 R X> X
σ
Christophe Hurlin (University of Orléans)
q=b
βk
1
ak
b b
R> = V
βk
Advanced Econometrics - Master ESA
November 20, 2015
153 / 225
3.2. The Fisher test
Proof (cont’d)
So, for a test H0 : βk = ak against H1 : βk 6= ak , the Fisher test-statistic
becomes
b
F = Rβ
So, we have:
q
>
b 2 R X> X
σ
F=
b
βk
ak
1
1
R>
b
Rβ
q
2
b b
V
βk
and the F test-statistic is equal to the squared t-statistic:
F = T2k
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
154 / 225
3.2. The Fisher test
De…nition (P-values)
The p-value of the F-test is equal to:
p-value = 1
Fp,N
K
(F (y ))
where F(y ) is the realisation of the F-statistic and Fp,N K (.) the cdf of
the Fisher distribution with p and N K degrees of freedom.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
155 / 225
3.2. The Fisher test
De…nition (Global F-test)
In a multiple linear regression model with a constant term
yi = β1 + ∑K
k =2 βk xik + εi
the global F-test corresponds to the test of signi…cance of all the
explicative variables:
H0 : β2 = .. = βK = 0
Under the assumption A6 (normality), the global F-test-statistic satis…es:
F
Christophe Hurlin (University of Orléans)
H0
F(K
1,N K )
Advanced Econometrics - Master ESA
November 20, 2015
156 / 225
3.2. The Fisher test
Remarks
1
The global F-test is a test designed to see if the model is useful
overall.
2
The null H0 : β2 = .. = βK = 0 can be written as:
R
β
(K 1 K ) (K ,1 )
0
B
B
B
B
@
0
0
..
..
0
Christophe Hurlin (University of Orléans)
1
0
..
..
..
0
1
0
..
0
0
0
1
..
0
..
..
..
..
..
0
0
..
..
1
=
1
q
(K 1 1 )
0
1 0
β
0
1
C
C B .. C B ..
CB
C B
C @ .. A = @ ..
A
βK
0
Advanced Econometrics - Master ESA
1
C
C
A
November 20, 2015
157 / 225
3.2. The Fisher test
Corollary (Global F-test)
In a multiple linear regression model with a constant term
yi = β1 + ∑K
k =2 βk xik + εi
the global F-test-statistic can also be de…ned as:
F=
R2
1 R2
N
K
K
1
where R 2 denotes the (unadjusted) coe¢ cient of determination.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
158 / 225
3.2. The Fisher test
Example (Global F-test and CAPM model)
Consider the extended CAPM model (…le: Chapter4_data.xls):
rMSFT ,t = β1 + β2 rSP 500,t + β3 rFord ,t + β4 rGE ,t + εt
Question: write a Matlab code to compute the global F-test, the critical
value for α = 5% and the p-value. Compare your results with Eviews.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
159 / 225
3.2. The Fisher test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
160 / 225
3.2. The Fisher test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
161 / 225
3.2. The Fisher test
Case 2: Semi-parametric model
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
162 / 225
3.2. The Fisher test
Assumption 6 (normality): the distribution of the disturbances is
unknown, but satisfy (assumptions A1-A5):
E ( εj X) = 0N
1
V ( ε j X ) = σ 2 IN
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
163 / 225
3.2. The Fisher test
Problem
1
2
3
b2 are unknown. As a
The exact (…nite sample) distribution of b
βk and σ
consequence the …nite sample distribution of F(y ) is also unknown.
But, we can express the F-statistic as a linear function of the Wald
statistic.
The Wald statistic has a chi-squared asymptotic distribution (cf. next
section)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
164 / 225
3.2. The Fisher test
De…nition (F-test-statistic and Wald statistic)
The Fisher test-statistic can expressed as a linear function of the Wald
test-statistic as
1
F = Wald
p
Wald =
1
b
Rβ
p
q
>
b
R Vasy β
1
1
R>
b
Rβ
q
Under assumptions A1-A5, the Wald test-statistic converges to a
chi-squared distribution
d
Wald ! χ2 (p )
H0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
165 / 225
3. Tests in the multiple linear regression model
Key concepts of Section 3
1
Student test
2
Fisher test
3
t-statistic and z-statistic
4
Global F-test
5
Exact (…nite sample) distribution under the normality assumption
6
Asymptotic distribution
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
166 / 225
Section 4
MLE and Inference
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
167 / 225
4. MLE and inference
Introduction
Consider a parametric model, linear or nonlinear (GARCH, probit,
logit, etc.), with a vector of parameters θ = (θ 1 : .. : θ K )>
We assume that the problem is regular (cf. chapter 2) and we
consider a ML estimator b
θ
The …nite sample distribution of b
θ is unknown, but b
θ is
asymptotically normally distributed (cf. chapter 2).
We want to test a set of linear or nonlinear constraints on the true
parameters (population) θ 1 , .., θ K .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
168 / 225
4. MLE and inference
De…nition (Null hypothesis)
Consider a null hypothesis of p linear and/or nonlinear constraints
H0 : c (θ) = 0p
| {z }
1
p 1
where c (θ) is a vectorial function de…ned as:
c:
Christophe Hurlin (University of Orléans)
RK ! Rp
θ 7! c (θ)
Advanced Econometrics - Master ESA
November 20, 2015
169 / 225
4. MLE and inference
Notations
1
c (θ) is a p
1 vector of functions c1 (θ) , .., cp (θ):
0
1
c1 (θ)
B c2 (θ) C
C
c (θ) = B
@ .. A
cp (θ)
2
In the case of p linear constraints, we have:
H0 : c (θ) = Rθ
Christophe Hurlin (University of Orléans)
q=0
Advanced Econometrics - Master ESA
November 20, 2015
170 / 225
4. MLE and inference
Example (Linear constraints)
Consider the two linear constraints θ 1 = θ 2 + θ 3 and θ 2 + θ 4 = 1. We
have p = 2 constraints such that:
H0 : c (θ) =
(2,1 )
θ1 θ2
θ2 + θ4
The function c (θ) can be written as Rθ
= (θ 1 θ 2 θ 3 θ 4 )> , we have
c (θ) = Rθ
q=
Christophe Hurlin (University of Orléans)
1
0
1
1
θ3
1
=
0
0
q. For instance if K = 4 and θ
1 0
0 1
0
1
θ1
B θ2 C
B
C
@ θ3 A
θ4
Advanced Econometrics - Master ESA
0
1
November 20, 2015
171 / 225
4. MLE and inference
Example (Nonlinear constraints)
Consider the linear and nonlinear constraints
θ 21
θ2 = 0
θ1
θ3 = 0
We have p = 2 constraints such that:
H0 : c (θ) =
(2,1 )
Christophe Hurlin (University of Orléans)
θ1
θ 21
θ2
θ3
=
Advanced Econometrics - Master ESA
0
0
November 20, 2015
172 / 225
4. MLE and inference
Assumptions
1
The functions c1 (θ) , .., cp (θ) are di¤erentiable.
2
There is no redundant constraint (identi…cation assumption).
Formally, we have
(row) rank
with
∂c (θ)
∂θ>
(p,K )
Christophe Hurlin (University of Orléans)
0
B
B
B
=B
B
@
∂c (θ)
= p 8θ 2 Θ
∂θ>
∂c1 (θ)
∂θ K
∂c2 (θ)
∂θ K
∂c1 (θ)
∂θ 1
∂c2 (θ)
∂θ 1
∂c1 (θ)
∂θ 2
∂c2 (θ)
∂θ 2
..
..
..
..
..
∂cp (θ)
∂θ 1
∂cp (θ)
∂θ 2
..
∂cp (θ)
∂θ K
..
Advanced Econometrics - Master ESA
1
C
C
C
C
C
A
November 20, 2015
173 / 225
4. MLE and inference
Consider the two-sided test
H0 : c (θ) = 0
versus
H1 : c (θ) 6= 0
We introduce three di¤erent asymptotic tests (the trilogy..)
1
The Likelihood Ratio (LR) test
2
The Wald test
3
The Lagrance Multiplier (LM) test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
174 / 225
4. MLE and inference
For each of the three tests, we will present:
1
the test-statistic
2
its asymptotic distribution under the null
3
the (asymptotic) rejection region
4
the (asymptotic) p-value
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
175 / 225
Subsection 4.1
The Likelihood Ratio (LR) test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
176 / 225
4.1. The Likelihood Ratio (LR) test
De…nition (Likelihood Ratio (LR) test statistic)
The likelihood ratio (LR) test-statistic is de…ned by as to be:
LR =
θH 0 ; y j x
2 `N b
`N b
θH 1 ; y j x
where `N (θ; y j x ) denotes the (conditional) log-likelihood of the sample y ,
b
θH 0 and b
θH 1 are respectively the maximum likelihood estimator of θ under
the alternative and the null hypothesis.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
177 / 225
4.1. The Likelihood Ratio (LR) test
Comments
Consider the ratio of likelihoods under H1 (no constraint) and under H0
(with c (θ) = 0).
θH 0 ; y j x
LN b
λ=
θH 1 ; y j x
LN b
1
λ > 0 since both likelihood are positive.
2
λ < 1 since LN (H0 ) cannot be larger than LN (H1 ) . A restricted
optimum is never superior to an unrestricted one.
3
If λ is too small, then doubt is cast on the restrictions c (θ) = 0.
4
Consider the statistic LR= 2 ln (λ): if λ is "too small", then LR is
large (rejection of the null)...
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
178 / 225
4.1. The Likelihood Ratio (LR) test
De…nition (Asymptotic distribution and critical region)
Under some regularity conditions (cf. chapter 2) and under the null
H0 : c (θ) = 0, the LR test-statistic converges to a chi-squared
distribution with p degrees of freedom (the number of restrictions
imposed):
d
LR ! χ2 (p )
H0
The (asymptotic) critical region for a signi…cance level of α is:
W = y : LR (y ) > χ21
α
(p )
where χ21 α (p ) is the 1 α critical value of the chi-squared distribution
with p degrees of freedom and LR(y ) is the realisation of the LR
test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
179 / 225
4.1. The Likelihood Ratio (LR) test
De…nition (p-value of the LRT test)
The p-value of the LR test is equal to:
p-value = 1
Gp (LR (y ))
where LR(y ) is the realisation of the LR test-statistic and Gp (.) is the cdf
of the chi-squared distribution with p degrees of freedom.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
180 / 225
4.1. The Likelihood Ratio (LR) test
Example (LRT and Poisson distribution)
Suppose that X1 ,X2 ,
,XN are i.i.d. discrete random variables, such that
Xi
Pois (θ ) with a pmf (probability mass function) de…ned as:
Pr (Xi = xi ) =
exp ( θ ) θ xi
xi !
where θ is an unknown parameter to estimate. We have a sample
(realisation) of size N = 10 given by f5, 0, 1, 1, 0, 3, 2, 3, 4, 1g . Question:
use a LR test to test the null H0 : θ = 1.8 against H1 : θ 6= 1.8 and give a
conclusion for signi…cance level of 5%.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
181 / 225
4.1. The Likelihood Ratio (LR) test
Solution
The log-likelihood function is de…ned as to be:
N
`N (θ; x ) =
θN + ln (θ ) ∑ xi
i =1
N
ln
∏ xi !
i =1
In the chapter 2, we found that the ML estimator of θ is the sample mean:
1
b
θ=
N
N
∑ Xi
i =1
Given the sample f5, 0, 1, 1, 0, 3, 2, 3, 4, 1g , the estimate of θ (under H1 ,
with non constraint) is b
θ H 1 = 2, and the corresponding log-likelihood is
equal to:
`N bθ H1 ; x = ln (0.104)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
182 / 225
4.1. The Likelihood Ratio (LR) test
Solution (cont’d)
Under the null H0 : θ = 1.8, we don’t need to estimate θ and the
log-likelihood is equal to:
N
`N ( θ H 0 ; x ) =
1.8N + ln (1.8) ∑ xi
i =1
N
ln
∏ xi !
= ln (0.0936)
i =1
The LR test-statistic is equal to:
LR (y ) =
Christophe Hurlin (University of Orléans)
2 ln
0.0936
0.104
= 0.21072
Advanced Econometrics - Master ESA
November 20, 2015
183 / 225
4.1. The Likelihood Ratio (LR) test
Solution (cont’d)
LR (y ) = 0.21072
For N = 10, p = 1 (one restriction) and α = 0.05, the critical region is:
W = y : LR (y ) > χ20.95 (1) = 3.8415
and the p-value is
pvalue = 1
G1 (0.21072) = 0.6462
where G1 (.) is the cdf of the χ2 (1) distribution.
Conclusion: for a signi…cance level of 5%, we fail to reject the null
H0 : θ = 1.8.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
184 / 225
Subsection 4.2
The Wald test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
185 / 225
4.2. The Wald test
De…nition (Wald test-statistic)
The Wald test-statistic associated to the test of H0 : c (θ) = 0 is de…ned
as to be:
θH 1
Wald = c b
>
∂c
∂θ>
b asy b
b
θH 1
θH 1 V
∂c
∂θ>
b
θH 1
>
1
θH 1
c b
where b
θH 1 is the maximum likelihood estimator of θ under the alternative
b asy b
hypothesis (unconstrained model) and V
θH 1 is an estimator of its
asymptotic variance covariance matrix.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
186 / 225
4.2. The Wald test
Remark
0
1
>B
>C
∂c b
B ∂c b
C
b asy b
Wald = c b
θH 1
θH 1
θ
B > θH 1 V
C
H
1
>
A
@
∂θ
∂θ
{z
}|
| {z } | {z }|
{z
}
1 p
Christophe Hurlin (University of Orléans)
p K
K K
Advanced Econometrics - Master ESA
K p
1
c b
θH 1
| {z }
November 20, 2015
p 1
187 / 225
4.2. The Wald test
Example (Wald test-statistic)
Consider a model with K = 3 parameters θ = (θ 1 : θ 2 : θ 3 )> with
θ 21
θ2 = 0
θ1
θ3 = 0
We have two constraints (p = 2) and:
H0 : c (θ) =
(2,1 )
θ1
θ 21
θ2
θ3
=
0
0
Denote b
θH 1 = (θ 1 : θ 2 : θ 3 )> the ML estimator of θ under the alternative
b asy b
hypothesis and V
θH 1 the estimator of its asymptotic variance
covariance matrix. Question: write the Wald test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
188 / 225
4.2. The Wald test
Solution
Here we have K = 3 and p = 2
θH 1
c b
∂c
>
∂θ
Wald = c b
θH 1
>
∂c
b
θH 1
∂θ>
Christophe Hurlin (University of Orléans)
=
=
b
θ1
2
b
θ1
1
2b
θ1
b
θ2
b
θ3
!
1
0
b
b asy b
θH 1 V
θH 1
0
1
∂c
∂θ>
Advanced Econometrics - Master ESA
b
θH 1
>
1
c b
θH 1
November 20, 2015
189 / 225
4.2. The Wald test
Remark
In the case of linear constraints
H0 : Rθ
q=0
we have
H0 : c (θ) = 0
with
c (θ) = Rθ
∂c
∂θ>
Christophe Hurlin (University of Orléans)
q
(θ) = R
Advanced Econometrics - Master ESA
November 20, 2015
190 / 225
4.2. The Wald test
De…nition (Wald test-statistic and linear constraints)
Consider the test of linear constraints H0 : c (θ) = Rθ
q = 0. The
Wald test-statistic is de…ned as to be:
Wald = Rb
θH 1
q
>
b asy b
θH 1
RV
R>
1
Rb
θH 1
q
where b
θH 1 is the maximum likelihood estimator of θ under the alternative
b asy b
hypothesis (unconstrained model) and V
θH 1 is an estimator of its
asymptotic variance covariance matrix.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
191 / 225
4.2. The Wald test
De…nition (Asymptotic distribution and critical region)
Under some regularity conditions (cf. chapter 2) and under the null
H0 : c (θ) = 0, the Wald test-statistic converges to a chi-squared
distribution with p degrees of freedom (the number of restrictions
imposed):
d
Wald ! χ2 (p )
H0
The (asymptotic) critical region for a signi…cance level of α is:
W = y : Wald (y ) > χ21
α
(p )
where χ21 α (p ) is the 1 α critical value of the chi-squared distribution
with p degrees of freedom and Wald(y ) is the realisation of the Wald
test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
192 / 225
4.2. The Wald test
Proof
Under some regularity conditions, we have
p
d
θH 1
N b
θ0 ! N 0, I
1
(θ0 )
We use the delta method for the function c (.) . The function c (.) is a
continuous and continuously di¤erentiable function not involving N, then
p
d
N c b
θH 1
c (θ0 ) ! N
∂c
0,
>
(θ0 ) I
1
∂c
(θ0 )
∂θ
>
(θ0 )>
∂θ
Under the null H0 : c (θ0 ) = 0, we have
∂c
>
(θ0 ) I
1
(θ0 )
∂θ
∂c
>
(θ0 )>
∂θ
where Ip is the identity matrix of size p.
Christophe Hurlin (University of Orléans)
1/2
p
Nc b
θH 1
Advanced Econometrics - Master ESA
d
! N (0, Ip )
November 20, 2015
193 / 225
4.2. The Wald test
Proof (cont’d)
The Wald criteria is de…ned as to be:
Wald criteria
= N
θH 1
c b
∂c
∂θ>
= N
>
(θ0 ) I
∂θ>
(θ0 ) I
c b
θH 1
∂c
>
1
(θ0 )
∂c
∂θ>
∂c
∂θ>
1
(θ0 )
(θ0 ) I
∂c
(θ0 )
>
1
∂θ>
(θ0 )
>
1/2
!>
1/2
(θ0 )
∂c
∂θ>
So, under the null H0 : c (θ0 ) = 0,, we have
c b
θH 1
(θ0 )>
1
c b
θH 1
d
Wald criteria ! χ2 (p )
H0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
194 / 225
4.2. The Wald test
Proof (cont’d)
>
θH 1
c b
Wald Criteria = N
∂c
∂θ>
(θ0 ) I
1
(θ0 )
∂c
∂θ>
1
(θ0 )>
θH 1
c b
A feasible Wald test-statistic is given by
Wald = N
c b
θH 1
∂c
∂θ>
Christophe Hurlin (University of Orléans)
>
b
θH 1 bI
1
b
θH 1
∂c
∂θ>
Advanced Econometrics - Master ESA
b
θH 1
>
1
c b
θH 1
November 20, 2015
195 / 225
4.2. The Wald test
Proof (cont’d)
Since
b asy b
θH 1
V
We have …nally
Wald = c b
θH 1
>
∂c
∂θ>
and
=N
1b 1
I
b
b asy b
θH 1 V
θH 1
b
θH 1
∂c
∂θ>
b
θH 1
>
1
c b
θH 1
d
Wald ! χ2 (p )
H0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
196 / 225
4.2. The Wald test
De…nition (p-value of the Wald test)
The p-value of the Wald test is equal to:
p-value = 1
Gp (Wald (y ))
where Wald(y ) is the realisation of the Wald test-statistic and Gp (.) is
the cdf of the chi-squared distribution with p degrees of freedom.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
197 / 225
4.2. The Wald test
De…nition (z-statistic)
Consider the test H0 : θ k = ak versus H1 : θ k 6= ak . The z-statistic
corresponds to the square root of the Wald test-statistic and satis…es
Zk = r
b
θk
ak
b asy b
θk
V
d
! N (0, 1)
H0
where b
θ k is the ML estimator of θ k obtained under H1 (unconstrained
model). The critical region for a signi…cance level of α is:
n
α o
W = y : jZk (y )j > Φ 1 1
2
where Φ (.) denotes the cdf of the standard normal distribution.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
198 / 225
4.2. The Wald test
Computational issues
The Wald test-statistic depends on the estimator of the asymptotic
variance covariance matrix:
θH 1
Wald = c b
where I b
θH 1
>
∂c
∂θ>
b asy b
b
θH 1
θH 1 V
b asy b
V
θH 1
=N
1b 1
I
∂c
∂θ>
b
θH 1
b
θH 1
>
1
θH 1
c b
denotes the average Fisher information matrix.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
199 / 225
4.2. The Wald test
Computational issues (cont’d)
Three estimators are available for the average Fisher information matrix:
1
θ =
Actual Average Fisher Matrix: bI A b
N
BHHH estimator: bI B
1
b
θ =
N
N
∑
i =1
∂`i (θ; yi j xi )
∂θ
1
Hessian based estimator: bI c b
θ =
N
Christophe Hurlin (University of Orléans)
N
∑
i =1
Advanced Econometrics - Master ESA
b
θ
N
∑ bI i
i =1
b
θ
∂`i (θ; yi j xi )
∂θ
∂2 `i (θ; yi j xi )
∂θ∂θ>
November 20, 2015
>
b
θ
!
b
θ
200 / 225
4.2. The Wald test
Computational issues (cont’d)
1
These estimators are asymptotically equivalent, but the corresponding
estimates may be very di¤erent in small samples.
2
Thus, we can obtain three di¤erent values for the Wald statistic
θH 1 (cf. exercises).
given the choice of the estimator for Vasy b
3
4
In general, the estimator A is rarely available and the estimator B
(BHHH) gives erratic results.
Most of the software use the estimator C (Hessian based estimator).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
201 / 225
4.2. The Wald test
Computational issues (cont’d)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
202 / 225
Subsection 4.3
The Lagrange Multiplier (LM) test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
203 / 225
4.3. The Lagrange Multiplier (LM) test
Introduction
Consider the set of constraints c (θ) = 0. Let λ be a vector of Lagrange
multipliers and de…ne the Lagrangian function
`N (θ ; y j x ) = `N (θ; y j x ) + λc (θ)
The solution to the constrained maximization problem is the root of
∂ `N ( θ ; y j x )
∂` (θ; y j x )
= N
+
∂θ
∂θ
∂c (θ)
∂θ>
>
λ
∂ `N ( θ ; y j x )
= c (θ)
∂λ
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
204 / 225
4.3. The Lagrange Multiplier (LM) test
Introduction (cont’d)
∂ `N ( θ ; y j x )
∂` (θ; y j x )
= N
+
∂θ
∂θ
∂c (θ)
∂θ>
>
λ
1
If the restrictions are valid, then imposing them will not lead to a
signi…cant di¤erence in the maximized value of the likelihood function.
In the …rst-order conditions, the meaning is that the second term in
the derivative vector will be small. In particular, λ will be small.
2
We could test this directly, that is, test
H0 : λ = 0
which leads to the Lagrange multiplier test.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
205 / 225
4.3. The Lagrange Multiplier (LM) test
Introduction (cont’d)
There is an equivalent simpler formulation, however. If the restrictions
c (θ) = 0 are valid, the derivatives of the log-likelihood of the
unconstrained model evaluated at the restricted parameter vector will
be approximately zero.
∂`N (θ; y j x )
∂θ
=0
b
θH 0
The vector of …rst derivatives of the log-likelihood is the vector of
(e¢ cient) scores.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
206 / 225
4.3. The Lagrange Multiplier (LM) test
De…nition (LM or score test)
For these reasons, this test is called the score test as well as the
Lagrange multiplier test.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
207 / 225
4.3. The Lagrange Multiplier (LM) test
Guess
Let us assume that θ is scalar, i.e. K = 1, then the LM statistic is simply
de…ned as:
2
θH 0 ; Y j x
sN b
LM =
θH 0 ; Y j x
V sN b
Since bI N b
θH 0
= V sN b
θH 0 ; Y j x
LM =
Christophe Hurlin (University of Orléans)
, we have:
sN b
θH 0 ; Y j x
2
bI N b
θH 0
Advanced Econometrics - Master ESA
November 20, 2015
208 / 225
4.3. The Lagrange Multiplier (LM) test
De…nition (LM or score test)
The LM test-statistic or score test associated to the test of H0 :
c (θ) = 0 is de…ned as to be:
LM = sN b
θH 0 ; Y j x
>
bI N 1 b
θH 0 ; Y j x
θH 0 sN b
where b
θH 0 is the maximum likelihood estimator of θ under the null
hypothesis (constrained model), sN (θ; Y j x ) is the score vector of the
unconstrained model and bI N b
θH 0 is an estimator of the Fisher
information matrix of the sample evaluated at b
θH 0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
209 / 225
4.3. The Lagrange Multiplier (LM) test
Remark
Since:
b asy b
θH 0
V
= bI N
1
b
θH 0
there is another expression for the LM statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
210 / 225
4.3. The Lagrange Multiplier (LM) test
De…nition (LM or score test)
The LM test-statistic or score test associated to the test of H0 :
c (θ) = 0 is de…ned as to be:
LM = sN b
θH 0 ; Y j x
>
b asy b
θH 0 ; Y j x
θH 0 sN b
V
where b
θH 0 is the maximum likelihood estimator of θ under the null
hypothesis (constrained model), sN (θ; Y j x ) is the score vector of the
b asy b
unconstrained model and V
θH 0 is an estimator of the asymptotic
variance covariance matrix of b
θH 0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
211 / 225
4.3. The Lagrange Multiplier (LM) test
Remark
The LM test-statistic can also be de…ned by:
LM = λ>
∂c
∂θ>
b asy b
b
θH 0
θH 0 V
∂c
∂θ>
b
θH 0
>
λ
where λ denotes the Lagrange Multiplier associated to the constraints
c (θ) = 0.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
212 / 225
4.3. The Lagrange Multiplier (LM) test
The LM test-statistic can be obtained from the following auxiliary
procedure:
Step 1: Estimate the constrained model and obtain b
θH 0 .
Step 2: Form the gradients for each observation of the unrestricted
model evaluated at b
θH 0
gi b
θH 0 ; yi j xi
8i = 1, ..N
Step 3: Run the regression of a vector of 1 on the variables
gi b
θH 0 ; yi j xi 8i = 1, ..N, then
LM = N
R2
where R 2 denotes the (unadjusted) coe¢ cient of determination of this
auxiliary regression.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
213 / 225
4.3. The Lagrange Multiplier (LM) test
Computational issues
1
The LM test-statistic depends on the estimator of the asymptotic
variance covariance matrix:
θH 0 ; Y j x
LM = sN b
2
where I b
θH 0
b asy b
V
θH 0
>
b asy b
θH 0 ; Y j x
θH 0 sN b
V
=N
1b 1
I
b
θH 0
denotes the average Fisher information matrix.
Thus, we can obtain three di¤erent values for the LM statistic
given the choice of the estimator for Vasy b
θH 0 (cf. exercises).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
214 / 225
4.3. The Lagrange Multiplier (LM) test
De…nition (Asymptotic distribution and critical region)
Under some regularity conditions (cf. chapter 2) and under the null
H0 : c (θ) = 0, the LM test-statistic converges to a chi-squared
distribution with p degrees of freedom (the number of restrictions
imposed):
d
LM ! χ2 (p )
H0
The (asymptotic) critical region for a signi…cance level of α is:
W = y : LM (y ) > χ21
α
(p )
where χ21 α (p ) is the 1 α critical value of the chi-squared distribution
with p degrees of freedom and LM(y ) is the realisation of the LM
test-statistic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
215 / 225
4.3. The Lagrange Multiplier (LM) test
De…nition (p-value of the LM test)
The p-value of the LM test is equal to:
p-value = 1
Gp (LM (y ))
where LM(y ) is the realisation of the LM test-statistic and Gp (.) is the
cdf of the chi-squared distribution with p degrees of freedom.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
216 / 225
Subsection 4.4
A comparison of the three tests
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
217 / 225
4.4. A comparison of the three tests
Source: Pelgrin (2010), Lecture notes, Advanced Econometrics
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
218 / 225
4.4. A comparison of the three tests
Summary
Test
Requires estimation under
LRT
H0 and H1
Wald
H1
LM
H0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
219 / 225
4.4. A comparison of the three tests
Computational problems
If the ML maximisation problem is complex (with local extrema) and
if it has no closed form solution (nonlinear models: GARCH, Markov
Switching models etc.), it may be particularly di¢ cult to get a ML
estimates b
θ through a numerical optimisation of the log-likelihood.
If the constraints c (θ) = 0 are not valid in the data, the (numerical)
convergence of the optimisation algorithm may be very problematic
under the null H0 .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
220 / 225
4.4. A comparison of the three tests
Asymptotic comparison
The three tests have the same asymptotic distribution under the null
H0 : c (θ) = 0:
d
LRT ! χ2 (p )
H0
d
Wald ! χ2 (p )
H0
d
LM ! χ2 (p )
H0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
221 / 225
4.4. A comparison of the three tests
Theorem (Asymptotic comparison)
The three tests are asymptotically equivalent. Under some regularity
conditions and under the null H0 : c (θ) = 0, the di¤erences between the
three test statistics converge to 0 as N tends to in…nity:
LRT
Christophe Hurlin (University of Orléans)
p
LM ! 0
H0
p
LRT
Wald ! 0
LM
Wald ! 0
H0
p
H0
Advanced Econometrics - Master ESA
November 20, 2015
222 / 225
4.4. A comparison of the three tests
Fact (Finite sample properties)
The …nite sample properties of the three tests may be di¤erent,
especially in small samples. For small sample size, they can lead to
opposite conclusion about the rejection of the null hypothesis.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
223 / 225
4. MLE and inference
Key concepts of Section 4
1
Likelihood Ratio (LR) test
2
Wald test
3
Lagrange Multiplier (LM) test
4
Computational issues
5
Comparison of the three tests (the trilogy) in …nite samples
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
224 / 225
End of Chapter 4
Christophe Hurlin (University of Orléans)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - Master ESA
November 20, 2015
225 / 225