Non-Nested Hypothesis
Testing
Introduction
• Models are non-nested when one cannot be derived as a
special case of the other.
• Example:
• Model A: 𝑌𝑖 = 𝛼𝑖 + 𝛼2 𝑋2,𝑖 + 𝛼3 𝑋3,𝑖 + 𝛼4 𝑊𝑖 + 𝜀𝑖
• Model B: 𝑌𝑖 = 𝛽𝑖 + 𝛽2 𝑍2,𝑖 + 𝛽3 𝑍3,𝑖 + 𝛽4 𝑊𝑖 + 𝜐𝑖
Non-Nested Hypothesis Testing Approaches
• Discrimination Approach
• Choosing between models based on goodness of fit
• Also called model selection testing
• Discerning Approach
• While considering one model, take into account information from
other models
Discrimination Approach
• All models under consideration must have the same
dependent variable.
• This approach gives all competing hypotheses equal
standing.
• Four common testing criteria
• 𝑅2
• Adjusted 𝑅2
• Akaike’s Information Criterion (AIC)
• Schwarz’s Information Criterion (SIC)
𝑅
•
2
𝑅2
=
𝑆𝑆𝑅
𝑆𝑆𝑇
=1−
𝑆𝑆𝐸
𝑆𝑆𝑇
• Remember from prior lectures that
• 𝑆𝑆𝐸 = 𝑒 ′ 𝑒
0
′
′𝑀
• 𝑆𝑆𝑅 = 𝑏 𝑋 𝑋𝑏
• 𝑆𝑆𝑇 = 𝑌 ′ 𝑀0 𝑌 = (𝑌 − 𝑌)′(𝑌 − 𝑌)
0
•𝑀 =
1
𝐼𝑛 − 𝑖𝑖′
𝑛
Adjusted 𝑅
2
•𝑅 =1 −
2
𝑆𝑆𝐸
𝑆𝑆𝑇
(𝑛−𝑘)
(𝑛−1)
Akaike’s Information Criterion (AIC)
• 𝐴𝐼𝐶 = 𝑒
• where
2𝑘
𝑛
2𝑘
𝑛
𝑆𝑆𝐸
𝑛
is the penalty factor
Schwarz’s Information Criterion (SIC)
• 𝑆𝐼𝐶 = 𝑛
𝑘
• ln 𝑆𝐼𝐶 =
• where
𝑆𝑆𝐸
𝑛
𝑘
ln 𝑛
𝑛
𝑛
𝑘
ln(𝑛)
𝑛
+
𝑆𝑆𝐸
ln( )
𝑛
is the penalty factor
Discerning Approach
• This approach tests the ability of one model to explain
features of another.
• Two tests
• Encompassing F-Test
• Davidson-Mackinnon J-Test
Encompassing F-Test
1. Create a “nested” model of models A and B
Model A: 𝑌𝑖 = 𝛼𝑖 + 𝛼2 𝑋2,𝑖 + 𝛼3 𝑋3,𝑖 + 𝛼4 𝑊𝑖 + 𝜀𝑖
Model B: 𝑌𝑖 = 𝛽𝑖 + 𝛽2 𝑍2,𝑖 + 𝛽3 𝑍3,𝑖 + 𝛽4 𝑊𝑖 + 𝜐𝑖
Nested Model: 𝑌𝑖 = 𝜆1 + 𝜆2 𝑋2,𝑖 + 𝜆3 𝑋3,𝑖 + 𝜆4 𝑍2,𝑖 + 𝜆5 𝑍3,𝑖 + 𝜆6 𝑊𝑖 + 𝜀𝑖
2. Use the F-Test to find which model is the correct model
To test if Model A is the correct model, test if 𝜆4 = 𝜆5 = 0
To test if Model B is the correct model, test if 𝜆2 = 𝜆3 = 0
Encompassing F-Test Shortcomings
• High correlation between X’s and Z’s may prevent us from
determining which model is the correct one and may cause some
𝜆 values to be statistically insignificant
• The choice of reference (null) hypothesis tends to skew the outcome
of the test
• Lack of economic meaning of artificially nested model
Davidson-Mackinnon J-Test
1. Estimate Model B and obtain the estimated Y values, 𝑌𝑖𝐵
2. Add 𝑌𝑖𝐵 as an additional regressor to Model A and estimate
𝑌𝑖 = 𝛼1 + 𝛼2 𝑋2,𝑖 + 𝛼3 𝑋3,𝑖 + 𝛼4 𝑌𝑖𝐵 + 𝜀𝑖
3. Using the t-test, test the hypothesis that 𝛼4 = 0
4. If we fail to reject the hypothesis, we can accept Model A as the
true model. Model A encompasses Model B.
Davidson-Mackinnon J-Test continued
5. Estimate Model A first and use the estimated Y values from this
model as a regressor in Model B.
𝑌𝑖 = 𝛽1 + 𝛽2 𝑍2,𝑖 + 𝛽3 𝑍3,𝑖 + 𝛽4 𝑌𝑖𝐴 + 𝜐𝑖
6. Using the t-test, test the hypothesis that 𝛽4 = 0
7. If we fail to reject the hypothesis, we can accept Model B as the
true model. Model A encompasses Model B.
J-Test Shortcomings
• No clear answer of acceptance/rejection of models
• Both can be accepted
• Both can be rejected
• Not very powerful in small samples
• Tends to reject the true model frequently
Matlab Example
Questions?