Chapter 8: Interval Estimates and Hypothesis Testing
Chapter 8 Outline
• Clint’s Assignment: Taking Stock
• Estimate Reliability: Interval Estimate Question
o Normal Distribution versus the Student t-Distribution: One Last
Complication
o Assessing the Reliability of a Coefficient Estimate: Applying the
Student t-Distribution
• Theory Assessment: Hypothesis Testing
o Motivating Hypothesis Testing: The Cynic
o Formalizing Hypothesis Testing: The Steps
• Summary: The Ordinary Least Squares (OLS) Estimation Procedure
o Regression Model and the Role of the Error Term
o Standard Ordinary Least Squares (OLS) Premises
o Ordinary Least Squares (OLS) Estimation Procedure: Three
Important Estimation Procedures
o Properties of the Ordinary Least Squares (OLS) Estimation
Procedure and the Standard Ordinary Least Square (OLS) Premises
ƒ Each estimation procedure is unbiased.
ƒ The estimation procedure for the coefficient value is the
best linear unbiased estimation procedure (BLUE).
• Causation versus Correlation
Chapter 8 Prep Questions
1. Run the following simulation and answer the questions posed. Summarize your
answers by filling in the following blanks:
[Link to MIT-Lab 8P.1 goes here.]
Actual
Values
Repetitions
From
To
Between
βx
Var[e] Value Value From and to Values
2
50
1.5
2.5
≈_____%
2
50
1.0
3.0
≈_____%
2
50
.5
3.5
≈_____%
2. In the simulation you just ran (Question 1):
2
a. Using the appropriate equation, compute the variance of the coefficient
estimate’s probability distribution? ______
b. What is the standard deviation of the coefficient estimate’s probability
distribution? ______
c. Using the Normal distribution’s “rules of thumb,” what is the
probability that the coefficient estimate in one repetition would lie
between:
1) 1.5 and 2.5? ______
2) 1.0 and 3.0? ______
3) .5 and 3.5? ______
d. Are your answers to part c consistent with your simulation results?
3. Recall the normal distribution.
a. What is the definition of the normal distribution’s z?
b. Consider the regression results from Professor Lord’s first quiz:
Ordinary Least Squares (OLS)
Dependent Variable: y
Estimate
SE
t-Statistic
Prob
Explanatory Variable(s):
x
1.200000 0.519615
2.309401
0.2601
Const
63.00000 8.874120
7.099296
0.0891
Number of Observations
3
Sum Squared Residuals
54.00000
SE of Regression
7.348469
4. Recall the regression results from Professor Lord’s first quiz:
Ordinary Least Squares (OLS)
Dependent Variable: y
Estimate
SE
t-Statistic
Explanatory Variable(s):
x
1.200000 0.519615
2.309401
Const
63.00000 8.874120
7.099296
Number of Observations
3
Sum Squared Residuals
54.00000
SE of Regression
7.348469
a. Does the positive coefficient estimate suggest that studying more will
improve a student’s quiz score? Explain.
Consider the views of a cynic:
Cynic’s view: Studying has no impact on a student’s quiz score; the
positive coefficient estimate obtained from the first quiz was just “the luck
of the draw.” In fact, studying does not affect quiz scores.
b. If the cynic were correct and studying has no impact on quiz scores,
what would the actual coefficient, βx, equal?
Prob
0.2601
0.0891
3
c. Is it possible that the cynic is correct? To help you answer this question,
run the following simulation:
[Link to MIT-Lab 8P.4 goes here.]
Clint’s Assignment: Taking Stock
We shall begin by taking stock of where Clint stands. Recall the theory he must
assess.
Theory: Additional studying increases quiz scores.
Clint’s assignment is to assess the effect of studying on quiz scores:
Project: Use data from Professor Lord’s first quiz to assess the effect of
studying on quiz scores.
Clint uses a simple regression model to assess the theory. Quiz score is the
dependent variable and number of minutes studied is the explanatory variable:
yt = βConst + βxxt + et where yt = Quiz score of student t
xt = Minutes studied by student t
et = Error term for student t
βConst and βx are the model’s parameters. They incorporate the view that Professor
Lord awards each student some points just for showing up; subsequently, the
number of additional points each student earns depends on how much he/she
studied:
• βConst represents the number of points Professor Lord gives a student just
for showing up.
• βx represents the number of additional points earned for each additional
minute of study.
Since the values of βConst and βx are not observable, Clint adopted the
econometrician’s philosophy:
Econometrician’s Philosophy: If you lack the information to determine the
value directly, estimate the value to the best of your ability using the
information you do have.
Clint used the results of the first quiz to estimate the values of βConst and βx
by applying the ordinary least squares (OLS) estimation procedure to find the best
fitting line:
T
First Quiz Data
( yt − y )( xt − x )
∑
Student
x
y
240 6
t =1
=
= = 1.2
1
5
66
→ bx =
T
200 5
2
( xt − x )
2
15
87
∑
t =1
3
25
90
6
bConst = y − bx x = 81 − × 15 = 81 − 18 = 63
5
4
Clint’s estimates suggest that Professor Lord gives each student 63 points for
showing up; subsequently, each student earns 1.2 additional points for each
additional minute studied.
Clint realizes that he cannot expect the coefficient estimate to equal the
actual value; in fact, he is all but certain that it will not. So now, Clint must
address two related issues:
• Estimate Reliability: How reliable is the coefficient estimate, 1.2,
calculated from the first quiz? That is, how confident should Clint be that
the coefficient estimate, 1.2, will be close to the actual value?
• Theory Assessment: How confident should Clint be that the theory is
correct, that studying improves quiz scores?
We shall address both of these issues in this chapter. First, we consider estimate
reliability.
Estimate Reliability: Interval Estimate Question
The interval estimate question quantifies the notion of reliability:
Interval Estimate Question: What is the probability that the estimate, 1.20,
lies within ____ of the actual value? ____
The general properties of the ordinary least squares (OLS) estimation procedure
allow us to address this question. It is important to distinguish between the
general properties and one specific application. Recall that the general properties
refer to what we know about the estimation procedure before the quiz is given; the
specific application refers to the numerical values of the estimates calculated from
the results of the first quiz:
5
General Properties
↓
OLS Estimation Procedure:
Estimate βConst and βx by
finding the bConst and bx that
minimize the sum of squared
residuals
↓
Before experiment
↓
Random Variable:
Probability Distribution
⏐
⏐
⏐
⏐
⏐
⏐
↓
versus
⎯⎯⎯⎯⎯⎯⎯→
Model:
yt = βConst + βxxt + et
OLS equations:
T
∑(y
t
bx =
− y )( xt − x )
t =1
One Specific Application
⏐
⏐
↓
Apply the estimation
procedure once to the first
quiz’s data:
⏐
⏐
↓
After experiment
↓
Estimate: Numerical Value
↓
T
∑ (x − x )
2
t
t =1
bConst = y − bx x
240 6
= = 1.2
200 5
6
= 81 − × 15 = 63
5
bx =
bConst
Mean[bx] = βx
Var[e ]
Var[bx ] = T
∑ ( xt − x )2
t =1
↓
Mean and variance describe the center and spread of the estimate’s probability distribution
The estimates are random variables and a quiz can be viewed as an
experiment. We cannot determine the numerical value of an estimate with
certainty before the experiment (quiz) is conducted. What then do we know
beforehand? We can describe the probability distribution of the estimate. We
know that the mean of the coefficient estimate’s probability distribution equals
the actual value of the coefficient and its variance equals the variance of the error
term’s probability distribution divided by the sum of squared x deviations:
Mean of Estimate’s Probability
Variance of Estimate’s
Distribution Equals Actual Value
Probability Distribution
↓
↓
Estimation Procedure Is
Determines the Reliability
As Variance Decreases
⎯→
⎯→
Unbiased
of the Estimate
Reliability Increases
6
Both the mean and variance of the coefficient estimate’s probability distribution
play a crucial role:
• Since the mean of the coefficient estimate’s probability distribution,
Mean[bx], equals the actual value of the coefficient, βx, the estimation
procedure is unbiased; the estimation procedure does not systematically
underestimate or overestimate the actual coefficient value.
• When the estimation procedure for the coefficient value is unbiased, the
variance of the estimate’s probability distribution, Var[bx], determines the
reliability of the estimate; as the variance decreases, the probability
distribution becomes more tightly cropped around the actual value;
consequently, it becomes more likely for the coefficient estimate to be
close to the actual coefficient value.
To assess his estimate’s reliability, Clint must consider the variance of the
coefficient estimate’s probability distribution. But we learned that Clint can never
determine the actual variance of the error term’s probability distribution, Var[e].
Instead, Clint adopts a two step strategy for estimating the variance of the
coefficient estimate’s probability distribution:
Step 1: Estimate the variance of the error term’s
probability distribution from the available
information – data from the first quiz
↓
EstVar[e ] = AdjVar[ Res's ]
Step 2: Apply the relationship between the
variances of coefficient estimate’s and
error term’s probability distributions
↓
Var[e ]
Var[bx ] = T
SSR
54
( xt − x ) 2
=
=
= 54
∑
Degrees of Freedom 1
t =1
é
ã
EstVar[e ]
54
EstVar[bx ] = T
=
= .27
200
2
∑ ( xt − x )
t =1
EstSD[bx ] = EstVar[bx ] = .27 = .5196
Unfortunately, there is one last complication before we can address the interval
estimate question.
7
Normal Distribution versus the Student t-Distribution: One Last Complication
We begin by reviewing the normal distribution. Recall that the variable z played a
critical role in using the normal distribution:
Value of Random Variable − Distribution Mean
z =
Distribution Standard Deviation
= Number of Standard Deviations from the Mean
In words, z equals the number of standard deviations the value lies from the mean.
But Clint does not know what the variance and standard deviation of the
coefficient estimate’s probability distribution equal. That is why he must estimate
them. Consequently, he cannot use the normal distribution to calculate
probabilities.
When the standard deviation is not known and must be estimated, the
Student t-distribution must be used. The variable t is similar to the variable z;
instead of equaling the number of standard deviations the value lies from the
mean, t equals the number of estimated standard deviations the value lies from the
mean:
Value of Random Variable − Distribution Mean
t =
Estimated Distribution Standard Deviation
= Number of Estimated Standard Deviations from the Mean
Recall that the estimated standard deviation is called the standard error; hence,
Value of Random Variable − Distribution Mean
t =
Standard Error
= Number of Standard Errors from the Distribution Mean
Probability Distribution of Random Variable
Normal
Student t
Distribution Mean
Value of Random Variable
Figure 8.1: Normal and Student t-Distributions
8
Like the normal distribution, the t-distribution is symmetric about its
mean. Since estimating the standard deviation introduces an additional element of
uncertainty, the Student t-distribution is more “spread out” than the normal
distribution as illustrated in Figure 8.1. The Student t-distribution’s “spread”
depends on the degrees of freedom. As the number of degrees of freedom
increase, we have more information; consequently, the t-distribution’s spread
decreases, moving it closer and closer to the normal distribution. Since the
“spread” of the Student t-distribution depends on the degrees of freedom, the table
describing the Student t-distribution is more cumbersome than the normal
distribution table. Fortunately, our Econometrics Lab allows us to avoid the
cumbersome Student t-distribution table.
Assessing the Reliability of a Coefficient Estimate
How reliable is the coefficient estimate, 1.2, calculated from the first quiz? That
is, how confident should Clint be that the coefficient estimate, 1.2, will be close to
the actual value? The interval estimate question to address this question:
Interval Estimate Question: What is the probability that the coefficient
estimate, 1.2, lies within ____ of the actual coefficient value? ____
We begin by filling in the first blank, choosing our “close to” value. The
value we choose depends on how demanding we are; that is, our “close to” value
depends on the range that we consider to be “close to” the actual value. For
purposes of illustration, we shall choose 1.5; so we write 1.5 in the first blank.
Interval Estimate Question: What is the probability that the coefficient
estimate, 1.2, lies within 1.5 of the actual coefficient value? ____
Figure 8.2 illustrates the probability distribution of the coefficient estimate and
the probability that we wish to calculate. The estimation procedure we used to
calculate the coefficient estimate, the ordinary least squares (OLS) estimation
procedure is unbiased:
Mean[bx] = βx
Consequently, we place the actual coefficient value, βx, at the center of the
probability distribution.
9
Probability that the
estimate is within 1.5
of the actual value
Student t-distribution
Mean[bx] = βx
1.5
βx−1.5
bx
1.5
Actual Value = βx
βx+1.5
Figure 8.2: Probability Distribution of Coefficient Estimate – “Close To” Value
Equals 1.5
As discussed above, we must use the Student t-distribution rather than the
normal distribution since we must estimate the standard deviation of the
probability distribution. The regression results from Professor Lord’s first quiz
provide the estimate:
Ordinary Least Squares (OLS)
Dependent Variable: y
Estimate
SE
t-Statistic
Explanatory Variable(s):
x
1.200000 0.519615
2.309401
Const
63.00000 8.874120
7.099296
Number of Observations
Sum Squared Residuals
SE of Regression
Prob
0.2601
0.0891
3
54.00000
7.348469
Estimated Equation: Esty = 63 + 1.2x
Interpretation of Estimates:
bConst = 63: Students receive 63 points for showing up.
bx = 1.2: Students receive 1.2 additional points for each additional minute
studied.
Critical Result: The coefficient estimate equals 1.2. The positive sign of the
coefficient estimate, suggests that additional studying increases
quiz scores. This evidence lends support to our theory.
Table 8.1: Quiz Scores Regression Results
10
The standard error equals the estimated standard deviation. t equals the number of
standard errors (estimated standard deviations) that the value lies from the
distribution mean:
Value of Random Variable − Distribution Mean
t =
Standard Error
= Number of Standard Errors from the Distribution Mean
Since the distribution mean equals the actual value, we can “translate” 1.5 below
and above the actual value into t’s. Since the standard error equals .5196, 1.5
below and above the actual value translates into 2.89 standard errors below and
above the actual value:
1.5 below actual value
1.5 above actual value
↓
↓
1.5
1.5
= 2.89 SE's below actual value
= 2.89 SE's above actual value
.5196
.5196
To summarize,
The probability that the
The probability that the
estimate lies within 1.5 of
= estimate lies within 2.89 SE’s of
the actual value.
the actual value.
↓
That is, between t’s of
−2.89 and 2.89
Figure 8.3 adds this information to the probability distribution graph.
Probability that the
estimate is within 1.5
of the actual value
1.5
βx−1.5
t = −2.89
Student t-distribution
Mean[bx] = βx
SE[bx] = .5196
bx
1.5
2.89 SE’s
2.89 SE’s
Actual Value = βx
βx+1.5
t = 2.89
Figure 8.3: Probability Distribution of Coefficient Estimate – “Close To” Value
Equals 1.5
11
Econometrics Lab 8.1: Calculate Prob[Results IF H0 True].
We can now use the Econometrics Lab to calculate the probability that the
estimate is within 1.5 of the actual value by computing probabilities the left and
right tails probabilities.1
• Left Tail:
[Link to MIT-Lab 8.1a goes here.]
•
o The following information has been entered:
Degrees of freedom: 1
t: −2.89
o Click Calculate. The left tail probability is approximately .11.
Right Tail:
[Link to MIT-Lab 8.1b goes here.]
o The following information has been entered:
Degrees of freedom: 1
t: 2.89
o Click Calculate. The right tail probability is approximately .11.
Since the Student t-distribution is symmetric, both the left and right tail
probabilities equal .11. Hence, the probability that the estimate is within 1.5 of the
actual value equals .78:
1.00 − (.11 + .11) = .78.
.78
Probability that the
estimate is within 1.5
of the actual value
Student t-distribution
Mean[bx] = βx
SE[bx] = .5196
.11
.11
1.5
βx−1.5
t = −2.89
bx
1.5
2.89 SE’s
2.89 SE’s
Actual Value = βx
βx+1.5
t = 2.89
Figure 8.4: Probability Distribution of Coefficient Estimate – Applying Student tDistribution
12
We can now fill in the second blank in the interval estimate question:
Interval Estimate Question: What is the probability that the coefficient
estimate, 1.2, lies within 1.5 of the actual coefficient value? .78
We shall turn our attention to assessing the theory.
Theory Assessment: Hypothesis Testing
Hypothesis testing allows Clint to assess how much confidence he should have in
the theory. We begin by motivating hypothesis testing using the same approach as
we took with Clint’s opinion poll. We shall play the role of the cynic. Then, we
shall formalize the process.
Motivating Hypothesis Testing: The Cynic
Recall that the “theory” suggests that a student’s score on the quiz depends on the
number of minutes he/she studies:
Theory: Additional studying increases scores.
Review the regression model:
yt = βConst + βxxt + et
The theory suggests that βx is positive. Review the regression results for the first
quiz:
Ordinary Least Squares (OLS)
Dependent Variable: y
Estimate
SE
t-Statistic
Explanatory Variable(s):
x
1.200000 0.519615
2.309401
Const
63.00000 8.874120
7.099296
Number of Observations
Sum Squared Residuals
SE of Regression
Prob
0.2601
0.0891
3
54.00000
7.348469
Estimated Equation: Esty = 63 + 1.2x
Interpretation of Estimates:
bConst = 63: Students receive 63 points for showing up.
bx = 1.2: Students receive 1.2 additional points for each additional minute
studied.
Critical Result: The coefficient estimate equals 1.2. The positive sign of the
coefficient estimate, suggests that additional studying increases
quiz scores. This evidence lends support to our theory.
Table 8.2: Quiz Scores Regression Results
13
The estimate for βx, 1.2, is positive. We estimate that an additional minute
of studying increases a student’s quiz score by 1.2 points. This lends support to
Clint’s theory. But, how much confidence should Clint have in the theory? Does
this provide definitive evidence that Clint’s theory is correct or should we be
skeptical?
If βx = 0
Prob[bx > 0] ≈ .50
bx
0
Figure 8.5: Probability Distribution of Coefficient Estimate – Could the Cynic Be
Correct?
To answer this question, recall our earlier hypothesis testing discussion and play
the cynic. What would a cynic’s view of our theory and the regression results be?
Cynic’s view: Studying has no impact on a student’s quiz score; the positive
coefficient estimate obtained from the first quiz was just “the luck of the
draw.” In fact, studying has no effect on quiz scores; the actual coefficient, βx,
equals 0.
Is it possible that our cynic is correct?
14
Econometrics Lab 8.1: Assessing the Cynic’s View
We shall use our Could the Cynic Be Correct? simulation to show that it is.
A positive coefficient estimate can arise in one repetition of the experiment even
when the actual coefficient is 0.
[Link to MIT-Lab 8.2 goes here.]
In the simulation, the default actual coefficient value is 0. Check the
From-To checkbox. Also, 0 is specified in the From list. In the To list, no value is
specified; consequently, there is no upper From-To bound. The From-To Percent
box will report the percent of repetitions in which the coefficient estimate equals
0 or more. Be certain that the “Pause” checkbox is cleared. Click Start and then
after many, many repetitions, click Stop. In about half the repetitions, the
coefficient estimate is positive; that is, when the actual coefficient, βx, equals 0,
the estimate is positive about half the time. The histogram illustrates this. Now,
we can apply the relative frequency interpretation of probability. If the actual
coefficient were 0, the probability of obtaining a positive coefficient from one
quiz would be about one-half. Consequently, we cannot dismiss the cynic’s view
as absurd.
To assess the cynic’s view, we pose the following question:
Question for the Cynic: What is the probability that the result would be like
the one obtained (or even stronger), if studying actually has no impact on quiz
scores? That is, what is the probability that the coefficient estimate from the
first quiz would be 1.2 or more, if studying had no impact on quiz scores (if
the actual coefficient, βx, equals 0)?
Answer: Prob[Results IF Cynic Correct].
15
The magnitude of the probability determines the likelihood that the cynic is
correct, the likelihood that studying has no impact on quiz scores:
Prob[Results IF Cynic Correct] small
Prob[Results IF Cynic Correct] large
↓
↓
Unlikely that the
Likely that the
cynic is correct
cynic is correct
↓
↓
Unlikely that the
Likely that the
studying has no impact
studying has no impact
To compute this probability let us review what we know about the
probability distribution of the coefficient estimate:
If H0
OLS estimation
Standard
Number of
Number of
procedure unbiased
true
error
observations parameters
é ã
é
ã
↓
SE[bx] = .5196
DF = 3 − 2 = 1
Mean[bx ] = β x = 0
Question for the Cynic: What is the probability that the coefficient estimate
from the first quiz would be 1.2 or more, if studying had no impact on quiz
scores (if the actual coefficient, βx, equaled 0)?
Student t-distribution
Mean = 0
SE = .5196
DF = 1
.13
bx
0
1.2
Figure 8.6: Probability Distribution of Coefficient Estimate – Prob[Results IF
Cynic Correct]
How can we answer this question? We turn to the Econometrics Lab.
16
Econometrics Lab 8.2: Using the Econometrics Lab to Calculate Prob[Results IF
Cynic Correct]
[Link to MIT-Lab 8.3 goes here.]
The appropriate information has been entered:
Mean: 0
Value: 1.2
Standard Error: .5196
Degrees of Freedom: 1
Click Calculate. The probability that the estimate lies in the right tail equals .13.
The answer to the question for the cynic is .13:
Answer: Prob[Results IF Cynic Correct] = .13
In fact, there is an even easier way to compute the probability. We do not
even need to use the Econometrics Lab to because the statistical software
calculates this probability automatically. To illustrate this, we shall first calculate
the t-statistic based on the premise that the cynic is correct, based on the premise
that the actual value of the coefficient equals 0:
Value of Random Variable − Distribution Mean 1.2 − 0
t =
=
= 2.309
Standard Error
.5196
= Number of Standard Errors from the Distribution Mean
1.2 lies 2.309 standard errors from 0. Next, return to the regression results and
focus attention on the row corresponding to the coefficient and on the “t-Statistic”
and “Prob” column.
Ordinary Least Squares (OLS)
Dependent Variable: y
Estimate
SE
t-Statistic
Prob
Explanatory Variable(s):
x
1.200000 0.519615
2.309401
0.2601
Const
63.00000 8.874120
7.099296
0.0891
Number of Observations
Sum Squared Residuals
SE of Regression
3
54.00000
7.348469
Estimated Equation: Esty = 63 + 1.2x
Interpretation of Estimates:
bConst = 63: Students receive 63 points for showing up.
bx = 1.2: Students receive 1.2 additional points for each additional minute
studied.
Critical Result: The coefficient estimate equals 1.2. The positive sign of the
coefficient estimate, suggests that additional studying increases
quiz scores. This evidence lends support to our theory.
Table 8.3: Quiz Scores Regression Results
17
•
•
Two interesting observations emerge:
First, the t-Statistic column equals 2.309, the value of the t-statistic we just
calculated; the t-statistic based on the premise that the cynic is correct and
the actual coefficient equals 0. The t-Statistic column reports the number
of standard errors the coefficient estimate based on the premise that the
actual coefficient equals 0.
Second, the Prob column equals .2601. This is just twice the probability
we just calculated using the Econometrics Lab:
= Prob Column
2 × Prob[Results IF Cynic Correct]
=
.26
2 × .13
Student t-distribution
Mean = 0
SE = .5196
DF = 1
.2601/2
.2601/2
bx
−1.2
1.2
0
1.2
1.2
Figure 8.7: Probability Distribution of Coefficient Estimate – Tails Probability
The Prob column is based on the premise that the actual coefficient equals 0 and
then focuses on the two tails of the probability distribution where each tail begins
1.2 (the numerical value of the coefficient estimate) from 0. As Figure 8.7
illustrates, the value in the Prob column equals the probability of lying in the tails;
the probability that the estimate resulting from one week’s quiz lies at least 1.2
from 0 assuming that the actual coefficient, βx, equals 0. That is, the Prob column
reports the tails probability:
Tails Probability: The probability that the coefficient estimate, bx, resulting
from one regression would lie at least 1.2 from 0 based on the premise that the
actual coefficient, βx, equals 0.
Consequently, we do not need to use the Econometrics Lab to answer the
question that we pose for the cynic:
Question for the Cynic: What is the probability that the coefficient estimate
from the first quiz is 1.2 or more, if studying had no impact on quiz scores (if
the actual coefficient, βx, equals 0)?
Answer: Prob[Results IF Cynic Correct]
18
Student t-distribution
Mean = 0
SE = .5196
DF = 1
.2601/2
bx
0
1.2
Figure 8.8: Probability Distribution of Coefficient Estimate – Prob[Results IF
Cynic Correct]
We can use the regression results to answer this question. From the Prob
column we know that the tails probability equals .2601. We are only interested in
the right tail, however, the probability that the coefficient estimate will equal 1.2
or more, if the actual coefficient equals 0. Since the Student t-distribution is
.2601
symmetric, the probability of lying in one of the tails is
. The answer to the
2
question we posed to assess the cynic’s view is .13:
Tails Probability .2601
=
≈ .13
Prob[Results IF Cynic Correct] =
2
2
19
Formalizing Hypothesis Testing: The Steps
We formalized hypothesis testing in Chapter 4 when we considered Clint’s public
opinion poll. We shall follow the same steps here, with one exception. We add a
Step 0 to construct an appropriate model to assess the theory.
Theory: Additional studying increases quiz scores.
Step 0: Formulate a model reflecting the theory to be tested.
We have already constructed this model:
yt = βConst + βxxt + et
yt = Quiz score
βConst reflects points for showing up
xt = Minutes studied
βx reflects points for each minute studied
et = Error term
The theory suggests that βx is positive.
Step 1: Collect data, run the regression, and interpret the estimates.
First Quiz Data
Student
x
y
bConst = Estimated points for showing up = 63
1
5
66
→
2
15
87
bx = Estimated points for each minute studied = 1.2
3
25
90
Ordinary Least Squares (OLS)
Dependent Variable: y
Explanatory Variable(s):
x
Const
Estimate
1.200000
63.00000
Number of Observations
Sum Squared Residuals
SE of Regression
3
54.00000
7.348469
SE
0.519615
8.874120
t-Statistic
2.309401
7.099296
Prob
0.2601
0.0891
Estimated Equation: Esty = 63 + 1.2x
Interpretation of Estimates:
bConst = 63: Students receive 63 points for showing up.
bx = 1.2: Students receive 1.2 additional points for each additional minute
studied.
Critical Result: The coefficient estimate equals 1.2. The positive sign of the
coefficient estimate, suggests that additional studying increases
quiz scores. This evidence lends support to our theory.
Table 8.4: Quiz Scores Regression Results
20
Step 2: Play the cynic and challenge the results; construct the null and alternative
hypotheses.
Cynic’s view: Despite the results, studying has no impact on quiz scores. The
results were just “the luck of the draw.”
Now, we construct the null and alternative hypotheses. Like the cynic, the null
hypothesis challenges the evidence; the alternative hypothesis is consistent
with the evidence:
Cynic is correct: Studying has no impact on a student’s quiz score.
H0: βx = 0
Cynic is incorrect: Additional studying increases quiz scores.
H1: βx > 0
Step 3: Formulate the question to assess the cynic’s view and the null hypothesis.
Question for the Cynic:
• Generic Question: What is the probability that the results would be
like those we actually obtained (or even stronger), if the cynic is
correct and studying actually has no impact?
• Specific Question: The regression’s coefficient estimate was 1.2:
What is the probability that the coefficient estimate in one regression
would be 1.2 or more if H0 were actually true (if the actual coefficient,
βx, equals 0)?
Answer: Prob[Results IF Cynic Correct] or Prob[Results IF H0 True]
The magnitude of this probability determines whether we reject the null
hypothesis:
Prob[Results IF H0 True] small
Prob[Results IF H0 True] large
↓
↓
Unlikely that H0 is true
Likely that H0 is true
↓
↓
Reject H0
Do not reject H0
21
Step 4: Use the general properties of the estimation procedure, the probability
distribution of the estimate, to calculate Prob[Results IF H0 True].
If H0
OLS estimation
Standard
Number of
Number of
procedure unbiased
true
error
observations parameters
é ã
é
ã
↓
SE[bx] = .5196
DF = 3 − 2 = 1
Mean[bx] = βx = 0
We have already calculated this probability. First, we did so using the
Econometrics Lab. Then, we noted that the statistical software had done so
automatically. We need only divide the tails probability, as reported in the
Prob column of the regression results, by 2:
.2601
Prob[Results IF H 0 True] =
≈ .13
2
The probability that the coefficient estimate in one regression would be 1.2 or
more if H0 were actually true (if the actual coefficient, βx, equals 0) is .13.
Step 5: Decide on the standard of proof, a significance level.
The significance level is the dividing line between the probability being small
and the probability being large.
Prob[Results IF H0 True]
Prob[Results IF H0 True]
less than significance level
greater than significance level
↓
↓
Prob[Results IF H0 True] small
Prob[Results IF H0 True] large
↓
↓
Unlikely that H0 is true
Likely that H0 is true
↓
↓
Reject H0
Do not reject H0
Recall that the traditional significant levels used in academia are 1, 5, and 10
percent. Obviously, .13 is greater than .10. Consequently, Clint would not
reject the null hypothesis that studying has no impact on quiz scores even with
a 10 percent significance level.
22
Summary: The Ordinary Least Squares (OLS) Estimation Procedure
Regression Model and the Role of the Error Term
Now, let us sum up what we have learned about the ordinary least squares (OLS)
estimation procedure:
yt = Dependent variable
xt = Explanatory variable
yt = βConst + βxxt + et
t = 1, 2, …, T
et = Error term
T = Sample size
The error term is a random variable; it represents random influences. The mean of
the each error term’s probability distribution equals 0:
For each t = 1, 2, …, T
Mean[et] = 0
Standard Ordinary Least Squares (OLS) Premises
• Error Term Equal Variance Premise: The variance of the error term’s
probability distribution for each observation is the same; all the variances
equal Var[e]:
Var[e1] = Var[e2] = … = Var[eT] = Var[e]
• Error Term/Error Term Independence Premise: The error terms are
independent: Cov[ei, ej] = 0.
Knowing the value of the error term from one observation does not
help us predict the value of the error term for any other observation.
• Explanatory Variable/Error Term Independence Premise: The
explanatory variables, the xt’s, and the error terms, the et’s, are not
correlated.
Knowing the value of an observation’s explanatory variable does not
help us predict the value of that observation’s error term.
23
Ordinary Least Squares (OLS) Estimation Procedure: Three Important
Estimation Procedure
There are three important estimation procedures embedded within the ordinary
least squares (OLS) estimation procedures. A procedure to estimate the
• Values of the regression parameters, βx and βConst:
T
∑(y
t
bx =
− y )( xt − x )
and bConst = y − bx x
t =1
T
∑ (x − x )
2
t
t =1
•
•
Variance of the error term’s probability distribution, Var[e]:
SSR
EstVar[e ] =
Degrees of Freedom
Variance of the coefficient estimate’s probability distribution, Var[bx]:
EstVar[e ]
EstVar[bx ] = T
∑ ( xt − x )2
t =1
Properties of the Ordinary Least Squares (OLS) Estimation Procedure and the
Standard Ordinary Least Square (OLS) Premises
When the standard ordinary least square (OLS) premises are met:
• Each estimation procedure is unbiased; each estimation procedure does
not systematically underestimate or overestimate the actual value.
• The ordinary least squares (OLS) estimation procedure for the coefficient
value is the best linear unbiased estimation procedure (BLUE).
24
Causation versus Correlation
Our theory and Step 0 illustrate the important distinction between causation and
correlation:
Theory: Additional studying increases quiz scores.
Step 0: Formulate a model reflecting the theory to be tested.
yt = βConst + βxxt + et
βConst reflects points for showing up
• yt = Quiz score
• xt = Minutes studied βx reflects points for each minute studied
• et = Error term
The theory suggests that βx is positive.
Our model is a causal model. An increase in studying causes a student’s
quiz score to increase:
Increase in studying (xt)
↓ Causes
Quiz score to increase (yt)
Correlation results whenever a causal relationship describes the reality accurately.
That is, when additional studying indeed increases quiz scores, studying and quiz
scores will be (positively) correlated:
• Knowing the number of minutes a student studies allows us to predict
his/her quiz score.
• Knowing a student’s quiz score helps us predict the number of minutes
he/she has studied.
More generally, a causal model that describes reality accurately implies
correlation:
Causation
Correlation
Implies
Beware that correlation need not imply causation, however. For example,
consider precipitation in the Twin Cities, precipitation in Minneapolis and
precipitation in St Paul. Since the cities are near each other precipitation in the
two cities are highly correlated. When it rains in Minneapolis it also always rains
in St Paul also and vice versa. But there is no causation involved here. Rain in
Minneapolis does not cause rain in St. Paul nor does rain in St. Paul cause rain in
Minneapolis. The rain is caused by the weather system moving over the cities. In
general, the correlation of two variables need not imply that a causal relationship
exists between the variables:
Correlation
Need Not Imply Causation
25
Appendix 8.1: Student t-Distribution Table – Right Tail Critical Values
α: Right Tail
Probability
t
0
Figure 8.9: Student t-distribution Right Tail Probabilities
Degrees of
Freedom
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
α = 0.10
3.078
1.886
1.638
1.533
1.476
1.440
1.415
1.397
1.383
1.372
1.363
1.356
1.350
1.345
1.341
1.337
1.333
1.330
1.328
1.325
1.323
1.321
1.319
1.318
α = 0.05
6.314
2.920
2.353
2.132
2.015
1.943
1.895
1.860
1.833
1.812
1.796
1.782
1.771
1.761
1.753
1.746
1.740
1.734
1.729
1.725
1.721
1.717
1.714
1.711
α = 0.025
12.706
4.303
3.182
2.776
2.571
2.447
2.365
2.306
2.262
2.228
2.201
2.179
2.160
2.145
2.131
2.120
2.110
2.101
2.093
2.086
2.080
2.074
2.069
2.064
α = 0.01
31.821
6.965
4.541
3.747
3.365
3.143
2.998
2.896
2.821
2.764
2.718
2.681
2.650
2.624
2.602
2.583
2.567
2.552
2.539
2.528
2.518
2.508
2.500
2.492
α = 0.005
63.657
9.925
5.841
4.604
4.032
3.707
3.499
3.355
3.250
3.169
3.106
3.055
3.012
2.977
2.947
2.921
2.898
2.878
2.861
2.845
2.831
2.819
2.807
2.797
26
1.316
1.708
2.060
2.485
2.787
25
1.315
1.706
2.056
2.479
2.779
26
1.314
1.703
2.052
2.473
2.771
27
1.313
1.701
2.048
2.467
2.763
28
1.311
1.699
2.045
2.462
2.756
29
1.310
1.697
2.042
2.457
2.750
30
1.309
1.696
2.040
2.453
2.744
31
1.309
1.694
2.037
2.449
2.738
32
1.308
1.692
2.035
2.445
2.733
33
1.307
1.691
2.032
2.441
2.728
34
1.306
1.690
2.030
2.438
2.724
35
1.306
1.688
2.028
2.434
2.719
36
1.305
1.687
2.026
2.431
2.715
37
1.304
1.686
2.024
2.429
2.712
38
1.304
1.685
2.023
2.426
2.708
39
1.303
1.684
2.021
2.423
2.704
40
1.299
1.676
2.009
2.403
2.678
50
1.296
1.671
2.000
2.390
2.660
60
1.294
1.667
1.994
2.381
2.648
70
1.292
1.664
1.990
2.374
2.639
80
1.291
1.662
1.987
2.368
2.632
90
1.290
1.660
1.984
2.364
2.626
100
1.289
1.659
1.982
2.361
2.621
110
1.289
1.658
1.980
2.358
2.617
120
Table 8.5: Right Tail Critical Values for the Student t-Distribution
27
Appendix 8.2 Assessing the Reliability of a Coefficient Estimate Using the
Student t-Distribution Table
We begin by describing the Student t-distribution table; a portion of it appears in
Table 8.6:
Degrees of
Freedom
α = 0.10
α = 0.05
α = 0.025
α = 0.01
α = 0.005
6.314
12.706
31.821
63.657
1
3.078
1.886
2.920
4.303
6.965
9.925
2
1.638
2.353
3.182
4.541
5.841
3
Table 8.6: Right Tail Critical Values for the Student t-Distribution
The first column represents the degrees of freedom. The numbers in the body of
the table are called the “critical values.” A critical value equals the number of
standard errors a value lies from the mean. The top row specifies α’s value of, the
“right tail probability.” Since the t-distribution is symmetric, the “left tail
probability” also equals α. The probability of lying within the tails, in the center
of the distribution, is 1 − 2α. This no doubt sounds confusing, but everything
should become clear after we show how Clint can use this table to answer the
interval estimate question.
Student t-distribution
1 − 2α
α
α
Critical Value × SE
Critical Value × SE
Distribution Mean
Estimate
Figure 8.10: Student t-distribution – Illustrating the Probabilities
Interval Estimate Question: What is the probability that the estimate, 1.2,
lies within ____ of the actual value? ____
Let us review the regression results from Professor Lord’s first quiz:
Coefficient Estimate = bx = 1.2
Standard Error of Coefficient Estimate = SE[bx] = .5196
28
Next, we shall modify Figure 8.10 to reflect our specific example. Focus
on Figure 8.11.
• We are interested in the coefficient estimate; consequently, we replace the
horizontal axis label by substituting bx for Estimate.
• Also, we know that the estimation procedure Clint uses, the ordinary least
squares (OLS) estimation procedure, is unbiased; hence, the distribution
mean equals the actual value. We can replace the Distribution Mean with
the actual coefficient value, βx.
Student t-distribution
1 − 2α
α
α
Critical Value × SE
Critical Value × SE
bx
βx
Figure 8.11: Student t-distribution – Illustrating the Probabilities for Coefficient
Estimate
Now, let us help Clint fill in the blanks. When using the table we begin by
filling in the second blank rather than the first.
• Second Blank: Choose α to specify the tail probability.
Clint must choose a value for α. As we shall see, the value he chooses
depends on how demanding he is. For example, suppose that Clint
believes that a .80 probability of the estimate lying in the center of the
distribution, close to the mean, is good enough. He would then choose
an α equal to .10. To understand why, note that when α equals .10, the
probability of the estimate lying in the right tail would be .10. Since
the t-distribution is symmetric, the probability of the estimate lying in
the left tail would be .10 also. Therefore, the probability that the
estimate lies in the center of the distribution would be .80;
accordingly, we write .80 in the second blank.
What is the probability that the estimate, 1.2, lies within _____ of the
actual value? .80
• First Blank: Calculate tail boundaries.
29
The first blank quantifies what “close to” means. The standard error
and the Student t-distribution table allow us to fill in the first blank. To
do so, we begin by calculating the degrees of freedom. Recall that the
degrees of freedom equal 1:
Degrees of
=
Sample
–
Number of Estimated Parameters
Freedom
Size
=
3
–
2
=
1
Degrees of
Freedom
α = 0.10
α = 0.05
α = 0.025
α = 0.01
α = 0.005
6.314
12.706
31.821
63.657
1
3.078
1.886
2.920
4.303
6.965
9.925
2
1.638
2.353
3.182
4.541
5.841
3
Table 8.7: Right Tail Critical Values for the Student t-Distribution – α Equals
0.10 and Degrees of Freedom Equals 1
Clint chose a value of α equal to .10. The table indicates that the critical
value for α = .10 with 1 degree of freedom is 3.078. The probability that the
estimate falls within 3.078 standard errors of the mean is .80. Next, the regression
results report that the standard error equals .5196:
SE[bx] = .5196
After multiplying the critical value given in the table, 3.078, by the standard error,
.5196, we can fill in the first blank:
3.078 × .5196 = 1.6
Student t-distribution
.80
.10
.10
Critical Value × SE
3.078 × .5196 = 1.6
βx−1.6
Critical Value × SE
bx
3.078 × .5196 = 1.6
βx
βx+1.6
Figure 8.12: Student t-distribution – Calculations for an α Equal to .10
30
What is the probability that the estimate, 1.2, lies within 1.6 of the
actual value? .80
1
Appendix 8.2 shows how we can use the Student t-distribution table to address
the interval estimate question. Since the table is cumbersome we shall use the
Econometrics Lab to do so.