Supplemental Material of Lecture 7 and 8:
1. Sampling Distributions of OLS Estimators
𝑉𝑎𝑟(𝑎𝑥)= 𝑎2 𝑉𝑎𝑟(𝑥) a is a constant.
𝑉𝑎𝑟(𝑥 + 𝑦) = var(x) + var(y) + 2cov(x, y)
𝑉𝑎𝑟(𝑥𝑖 )= 𝜎 2
𝑥
𝜎2
𝑉𝑎𝑟( 𝑁𝑖 )= 𝑁2
1
1
𝑁
𝑉𝑎𝑟(𝑥̅ ) = 𝑉𝑎𝑟 (𝑁 ∑𝑁
𝑖=1 𝑥𝑖 ) = 𝑁 2 𝑉𝑎𝑟(∑𝑖=1 𝑥𝑖 )
2
If 𝑥𝑖 is independent with other observations, 𝑉𝑎𝑟(∑𝑁
𝑖=1 𝑥𝑖 ) = 𝑁𝜎
1
1
2
2
Therefore 𝑉𝑎𝑟(𝑥̅ ) = 𝑁2 𝑉𝑎𝑟(∑𝑁
𝑖=1 𝑥𝑖 ) = 𝑁 2 ∗ 𝑁𝜎 = 𝜎 /𝑁
χ2-distribution: In probability theory and statistics, the chi-squared distribution (also chisquare or χ2-distribution) with k degrees of freedom is the distribution of a sum of the
squares of k independent standard normal random variables. The chi-squared distribution is a
special case of the gamma distribution and is one of the most widely used probability
distributions in inferential statistics, notably in hypothesis testing and in construction of
confidence intervals.
T-distribution--In probability and statistics, Student's t-distribution (or simply the tdistribution) is any member of a family of continuous probability distributions that arise
when estimating the mean of a normally-distributed population in situations where the sample
size is small and the population's standard deviation is unknown.
The variance of 𝛽̂𝑗 ：
𝑉𝑎𝑟(𝛽̂𝑗 |x) =
𝛽̂𝑗 − 𝛽𝑗
𝜎2
𝑆𝑆𝑇𝑗 (1 − 𝑅𝑗2 )
~𝑁(0,1)
√𝑉𝑎𝑟(𝛽̂𝑗 |x)
The estimated variance of 𝛽̂𝑗
̂
̂𝑗 |x) =
𝑉𝑎𝑟(𝛽
𝜎̂ 2
𝑆𝑆𝑇𝑗 (1 − 𝑅𝑗2 )
𝛽̂𝑗 − 𝛽𝑗
̂
̂𝑗 |x)
√𝑉𝑎𝑟(𝛽
~𝑡𝑛−𝑘−1
Since we are testing H0，𝛽𝑗 , it is only natural to look at our unbiased estimator of 𝛽𝑗 , 𝛽̂𝑗 . The
point estimate , 𝛽̂𝑗 will never exactly be zero, whether or not the H0 is true. The question is
̂ 𝒋 very far from zero provide evidence against
how far , 𝛽̂𝑗 is from zero. A sample value of 𝜷
H0. However, we must recognize that there is sampling error in our estimate 𝛽̂𝑗 , so the size of
𝛽̂𝑗 must be weighed against its sample error. Since the standard error of 𝛽̂𝑗 is an estimate of
̂ 𝒋 is away
standard deviation of 𝛽̂𝑗 , T-𝜷̂𝒋 measures how many estimated standard deviations 𝜷
from zero.
2. Testing hypothesis: t test
Define a rejection rule so that, if it is true, H0 is rejected only with a small probability (=
significance level, e.g. 5%)
2.1 Testing against one-sided alternatives (greater than zero)
For a right-tailed test, c is chosen to make the area in right tail of t distribution equal 5%. In
other words, c is the 95 th percentile in the t distribution with n-k-1 degrees of freedom.
A pattern in the critical value: as significance level falls, the critical value increases, so that
we require a larger and larger value of T-𝛽̂𝑗 in order to reject the H0.So, if H0 is rejected at,
say 5% level, then is automatically rejected at 10% level.
2.2 Testing against one-sided alternatives (less than zero)
For a left-tailed test, c is chosen to make the area in left tail of t distribution equal 5%. In
other words, c is the 5 the percentile in the t distribution with n-k-1 degrees of freedom.
2.3 Testing against two-sided alternatives
Using the regression estimates to help us formulate the null or alternative hypotheses is not
allowed, because classical statistical inference presumes that we state the null and alternative
about the population before looking at the data.
For a two-tailed test, c is chosen to make the area in each tail of t distribution equal 2.5%. In
other words, c is the 97.5 th percentile in the t distribution with n-k-1 degrees of freedom.
2.4 Normal approximation
If a regression coefficient is different from zero in a two-sided test, the corresponding
variable is said to be “statistically significant”
If the number of degrees of freedom is large enough so that the normal approximation
applies, the following rules of thumb apply:
2.5 Testing more general hypotheses
dlogY/dlogX=(dY/Y)/ (dX/X)
2.6 P-value
The smallest significance level at which the null hypothesis is still rejected, is called the pvalue of the hypothesis test. The p-value is the probability of observing a t statistic as
extreme as we did if the null hypothesis is true.
A small p-value is evidence against the null hypothesis because one would reject the null
hypothesis even at small significance levels
A large p-value is evidence in favor of the null hypothesis. P-values are more informative
than tests at fixed significance levels
2.7 Confidence interval
𝑃 {−𝑡𝛼,𝑛−𝑘−1 <
2
𝛽̂𝑗 − 𝛽𝑗
< 𝑡𝛼,𝑛−𝑘−1 } = 1 − 𝛼
2
𝑠𝑒(𝛽̂𝑗 )
𝑃 {𝛽̂𝑗 − 𝑡𝛼,𝑛−𝑘−1 𝑠𝑒(𝛽̂𝑗 ) < 𝛽𝑗 < 𝛽̂𝑗 + 𝑡𝛼,𝑛−𝑘−1 𝑠𝑒(𝛽̂𝑗 )} = 1 − 𝛼
2
2
𝑃 {𝛽𝑗 ∉ (𝛽̂𝑗 − 𝑡𝛼,𝑛−𝑘−1 𝑠𝑒(𝛽̂𝑗 ), 𝛽̂𝑗 + 𝑡𝛼,𝑛−𝑘−1 𝑠𝑒(𝛽̂𝑗 ))} = 𝛼
2
2
The probability that 𝛽𝑗 does not fall into the confidence interval is 𝛼.
If random sample were obtained over and over again, with 𝛽𝑗 and 𝛽̅𝑗 computed each time, then
the (unknown) population value 𝛽𝑗 would be in the interval (𝛽𝑗 , 𝛽̅𝑗 ) for 95% of the samples.
Unfortunately, for the single sample that we use to contruct CI, we do not know whether 𝛽𝑗 is
actually contained in the interval. We hope we have obtained sample that is one of the 95% of
all samples where the interval estimate contains 𝛽𝑗 , but we have no guarantee.
If the null hypothesis is H0: 𝛽𝑗 = 𝛼𝑗 , then H0 is rejected against at the 5% level if, and only if,
𝛼𝑗 is not in the 95% confidence interval.
2.8 Examples in academic paper
Gaurav, Liang, Mobarak and Song (2020)
2.9 Testing linear combination
3 F test
3.1 The definition of F distribution
3.2 The R-Squared Form of the F Statistic
2 )
SSR 𝑟 = SST(1 − 𝑅𝑟2 ) SSR 𝑢𝑟 = SST(1 − 𝑅𝑢𝑟
2
2
(𝑅𝑢𝑟
(𝑅𝑢𝑟
− 𝑅𝑟2 )/𝑞
− 𝑅𝑟2 )/𝑞
𝐹=
=
2 )/(𝑛 − 𝑘 − 1)
2 )/𝑑𝑓
(1 − 𝑅𝑢𝑟
(1 − 𝑅𝑢𝑟
𝑢𝑟
3.3 P values for F test
p − value = P(Ϝ̃ > F)
Ϝ̃ denotes the F random variable with (q, n-k-1) degrees of freedom. F is the actual value of
the test statistic.
Reference:
Khanna, G., Liang, W., Mobarak, A. M., & Song, R. (2019). The productivity consequences of
pollution-induced migration in China. Working Paper, Yale University.
Wooldridge, J. M. (2016). Introductory econometrics: A modern approach. Nelson Education.
Wikipedia