ECONOMICS 351* -- NOTE 12
M.G. Abbott
ECON 351* -- NOTE 12
OLS Estimation of the Multiple (Three-Variable)
Linear Regression Model
This note derives the Ordinary Least Squares (OLS) coefficient estimators for the
three-variable multiple linear regression model.
•
The population regression equation, or PRE, takes the form:
Yi = β0 + β1X1i + β 2 X 2i + u i
(1)
where ui is an iid random error term.
•
The OLS sample regression equation (OLS-SRE) for equation (1) can be
written as
Yi = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i + û i = Ŷi + û i
(i = 1, ..., N).
(2)
where the β$ j are the OLS estimators of the corresponding population regression
coefficients βj (j = 0, 1, 2),
û i = Yi − Ŷi = Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i
(i = 1, ..., N)
are the OLS residuals, and
Ŷi = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i
(i = 1, ..., N)
are the OLS estimated (or predicted) values of Yi.
The function f (X1i , X 2i ) = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i is called the OLS sample
regression function (or OLS-SRF).
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 1 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
1. The OLS Estimation Criterion
The OLS coefficient estimators are those formulas (or expressions) for β̂0 , β̂1 ,
and β̂ 2 that minimize the sum of squared residuals RSS for any given sample of
size N.
The OLS estimation criterion is therefore:
(
)
Minimize RSS βˆ 0 , βˆ 1 , βˆ 2 =
N
∑ û i2
=
i =1
∑ (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i )
N
2
(3)
i =1
{ β$ j }
(
)
(
)
Interpretation of the RSS βˆ 0 , βˆ 1 , βˆ 2 function:
♦
The knowns in the RSS βˆ 0 , βˆ 1 , βˆ 2 function are the sample observations
(Yi , 1, X1i , X 2i ) for i = 1, …, N. In other words, the N sample values of the
observable variables Y, X1, X2 are taken as known (or given).
♦
The unknowns in the RSS βˆ 0 , βˆ 1 , βˆ 2 function are therefore the coefficient
estimators βˆ , βˆ , βˆ .
(
0
♦
1
)
2
(
For purposes of deriving the OLS coefficient estimators, the RSS βˆ 0 , βˆ 1 , βˆ 2
function is interpreted as a function of the three unknowns βˆ , βˆ , βˆ .
0
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
1
)
2
… Page 2 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
2. The OLS Normal Equations: Derivation of the FOCs
(
)
STEP 1: Re-write the RSS βˆ 0, βˆ 1, βˆ 2 function in (3) as follows:
(
)
RSS βˆ 0 , βˆ 1 , βˆ 2 =
N
∑ û i2 =
i =1
N
∑ f (û i )
where
i =1
f (û i ) = û i2
û i = Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i
(3)
Note: The function f (û i ) = û i2 is a function of û i , and û i is in turn a function
of β̂0 , β̂1 , and β̂ 2 .
(
)
STEP 2: Partially differentiate the RSS βˆ 0, βˆ 1, βˆ 2 function in (3) with respect
to β̂ , βˆ , and β̂ :
0
•
1
Using the chain rule of differentiation, each partial derivative of the
RSS βˆ 0, βˆ 1, βˆ 2 function takes the general form
(
∂RSS
=
∂βˆ j
•
2
)
N
∑
i =1
d f ∂û i
.
d û i ∂βˆ j
(4)
Using the power rule of differentiation, the derivative df d û i is
df
d (û i2 )
=
= 2 û i .
d û i
d û i
The partial derivatives ∂ RSS ∂βˆ j for j = 0, 1, 2 are therefore
N d f ∂ û
N
N
∂RSS
∂û
∂û
i
= ∑
= ∑ 2 û i i = 2 ∑ û i i
ˆ
i =1 d û i ∂β
i =1
i =1
∂βˆ j
∂βˆ j
∂βˆ j
j
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
j = 0, 1, 2.
(5)
… Page 3 of 17 pages
ECONOMICS 351* -- NOTE 12
•
Since the i-th residual is û i = Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i , the partial derivatives
∂ u$ ∂β$ for j = 0, 1, 2 are:
i
j
∂û i
= −1;
∂βˆ 0
•
M.G. Abbott
∂û i
= −X1i ;
∂βˆ 1
∂û i
= −X 2i .
∂βˆ 2
Substitute the partial derivatives ∂ u$ i ∂β$ j for j = 0, 1, 2 into equation (5):
N
∂RSS
∂û
= 2 ∑ û i i
i =1
∂βˆ j
∂βˆ j
j = 0, 1, 2.
(5)
The partial derivatives ∂RSS ∂β$ j for j = 0, 1, 2 thus take the form:
N
N
N
∂RSS
∂û i
= 2 ∑ û i
= 2 ∑ û i (−1) = − 2 ∑ û i
∂βˆ 0
∂βˆ 0
i =1
i =1
i =1
(6.1)
N
N
N
∂RSS
∂û i
= 2 ∑ û i
= 2 ∑ û i (−X1i ) = − 2 ∑ X1i û i
∂βˆ 1
∂βˆ 1
i =1
i =1
i =1
(6.2)
N
N
N
∂RSS
∂û
= 2 ∑ û i i = 2 ∑ û i (−X 2i ) = − 2 ∑ X 2i û i
∂βˆ 2
∂βˆ 2
i =1
i =1
i =1
(6.3)
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 4 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
STEP 3: Obtain the first-order conditions (FOCs) for a minimum of the RSS
function by setting the partial derivatives (6.1)-(6.3) equal to zero, then dividing
each equation by −2, and finally setting û i = Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i :
•
N
∂RSS
= − 2 ∑ û i
∂βˆ 0
i =1
∂RSS
=0
∂βˆ 0
⇒
(6.1)
N
− 2 ∑ û i = 0
N
∑ û i = 0
(7.1)
∑ (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i ) = 0
(8.1)
i =1
⇒
i =1
N
⇒
i =1
•
N
∂RSS
= − 2 ∑ X1i û i
∂βˆ 1
i =1
∂RSS
=0
∂βˆ
⇒
(6.2)
N
− 2 ∑ X1i û i = 0
⇒
∑ X1i û i
= 0
(7.2)
∑ X1i (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i ) = 0
(8.2)
i =1
1
N
i =1
N
⇒
i =1
•
N
∂RSS
= − 2 ∑ X 2i û i
i =1
∂βˆ 2
∂RSS
=0
∂βˆ 2
⇒
⇒
(6.3)
N
− 2 ∑ X 2i û i = 0
N
∑ X 2i û i = 0
(7.3)
∑ X 2i (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i ) = 0
(8.3)
i =1
⇒
i =1
N
i =1
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 5 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
STEP 4: Rearrange each of the equations (8.1)-(8.3) to put them in the
conventional form of the OLS normal equations. Thus, taking summations and
rearranging terms, we obtain the OLS normal equations:
•
∑ (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i ) = 0
N
i =1
N
(8.1)
N
N
i =1
N
i =1
N
i =1
N
i =1
N
N
i =1
i =1
i =1
∑ Yi − Nβˆ 0 − βˆ 1 ∑ X1i − βˆ 2 ∑ X 2i = 0
i =1
N
− Nβˆ 0 − βˆ 1 ∑ X1i − βˆ 2 ∑ X 2i = −∑ Yi
i =1
Nβˆ 0 + βˆ 1 ∑ X1i + βˆ 2 ∑ X 2i = ∑ Yi
•
(N1)
∑ X1i (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i ) = 0
N
(8.2)
i =1
N
∑ (X1i Yi − βˆ 0 X1i − βˆ 1X12i − βˆ 2 X1i X 2i ) = 0
i =1
N
N
N
N
i =1
N
i =1
N
i =1
N
∑ X1i Yi − βˆ 0 ∑ X1i − βˆ 1 ∑ X12i − βˆ 2 ∑ X1i X 2i = 0
i =1
N
− βˆ 0 ∑ X1i − βˆ 1 ∑ X12i − βˆ 2 ∑ X1i X 2i = −∑ X1i Yi
i =1
N
i =1
N
i =1
N
N
i =1
i =1
i =1
i =1
i =1
β̂ 0 ∑ X1i + β̂1 ∑ X12i + β̂ 2 ∑ X1i X 2i = ∑ X1i Yi
•
∑ X 2i (Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i ) = 0
N
(N2)
(8.3)
i =1
N
∑ (X 2i Yi − βˆ 0 X 2i − βˆ 1X 2i X1i − βˆ 2 X 22i ) = 0
i =1
N
N
N
N
i =1
N
i =1
N
i =1
N
N
i =1
N
i =1
N
i =1
N
i =1
N
i =1
i =1
i =1
i =1
∑ X 2i Yi − βˆ 0 ∑ X 2i − βˆ 1 ∑ X 2i X1i − βˆ 2 ∑ X 22i = 0
i =1
− βˆ 0 ∑ X 2i − βˆ 1 ∑ X 2i X1i − βˆ 2 ∑ X 22i = −∑ X 2i Yi
βˆ 0 ∑ X 2i + βˆ 1 ∑ X 2i X1i + βˆ 2 ∑ X 22i = ∑ X 2i Yi
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
(N3)
… Page 6 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
RESULT: Assemble the three OLS normal equations (N1)-(N3):
N
N
N
i =1
i =1
i =1
Nβˆ 0 + βˆ 1 ∑ X1i + βˆ 2 ∑ X 2i = ∑ Yi
(N1)
N
N
N
N
i =1
i =1
i =1
i =1
N
N
N
N
i =1
i =1
i =1
i =1
βˆ 0 ∑ X1i + βˆ 1 ∑ X12i + βˆ 2 ∑ X1i X 2i = ∑ X1i Yi
βˆ 0 ∑ X 2i + βˆ 1 ∑ X 2i X1i + βˆ 2 ∑ X 22i = ∑ X 2i Yi
(N2)
(N3)
•
The OLS normal equations (N1)-(N3) constitute three linear equations in the
three unknowns β̂0 , β̂1 , and β̂ 2 .
•
Solution of the OLS normal equations (N1)-(N3) yields explicit expressions
(or formulas) for β̂0 , β̂1 , and β̂ 2 ; these expressions are the OLS estimators β̂0 ,
β̂1 , and β̂ 2 of the partial regression coefficients β0, β1, and β2 respectively.
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 7 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
3. Expressions for the OLS Coefficient Estimators
•
The expressions (formulas) for the OLS estimators are most conveniently
written in deviation-from-means form, which uses lower case letters to denote
the deviations of the sample values of each observable variable from their
respective sample means. Thus, define the deviations-from-means of Yi, X1i,
and X2i as:
y i ≡ Yi − Y;
x 1i ≡ X1i − X1 ;
x 2i ≡ X 2i − X 2 ;
where
Σ i Yi
is the sample mean of the Yi values;
N
ΣX
X1 = Σ i X1i N = i 1i is the sample mean of the X1i values;
N
ΣX
X 2 = Σ i X 2i N = i 2i is the sample mean of the X2i values.
N
Y = Σ i Yi N =
‰
‰
The OLS slope coefficient estimators β̂1 and β̂2 in deviation-from-means
form are:
(Σ x 2 )(Σ x y ) − (Σi x1i x 2i )(Σi x 2i yi ) ;
βˆ 1 = i 2i i2 1i i 2
(Σi x1i )(Σi x 2i ) − (Σi x1i x 2i )2
(9.2)
2
ˆβ = (Σ i x1i )(Σ i x 2i y i ) − (Σ i x1i x 2i )(Σ i x1i y i ) .
2
(Σi x12i )(Σi x 22i ) − (Σi x1i x 2i )2
(9.3)
The OLS intercept coefficient estimator β̂0 is:
βˆ 0 = Y − βˆ 1X1 − βˆ 2 X 2 .
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
(9.1)
… Page 8 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
4. The OLS Variance-Covariance Estimators
‰
An unbiased estimator of the error variance σ2
•
For the general multiple linear regression model with K regression
coefficients, an unbiased estimator of the error variance σ2 is the degrees-offreedom-adjusted estimator
Σ i û i2
RSS
σˆ =
=
(N − K ) (N − K )
2
where K = k + 1 = the total number of regression coefficients in the PRF.
•
For the three-variable multiple linear regression model (such as regression
equation (1) above) for which K = 3, the unbiased estimator of the error
variance σ2 is therefore
Σ i u$ 2i
RSS
=
.
σ$ =
( N − 3) ( N − 3)
2
(10)
where (N − 3) is the degrees of freedom for the residual sum of squares RSS in
the OLS-SRE (2). σ$ 2 is an unbiased estimator of σ2 because it can be shown
that E( ∑ i u$ 2i ) = E(RSS) = (N − 3)σ2 .
•
The error variance estimator σ$ 2 is used to obtain unbiased estimators of the
variances and covariances of the OLS coefficient estimators β̂0 , β̂1 , and β̂ 2 .
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 9 of 17 pages
ECONOMICS 351* -- NOTE 12
‰
Formulas for the variances and covariances of the slope coefficient
estimators β̂1 and β̂ 2 in the three-variable multiple regression model
( )
σ 2 Σ i x 22i
;
(Σi x12i )(Σi x 22i ) − (Σi x1i x 2i )2
( )
σ 2 Σ i x 12i
;
(Σ i x12i )(Σ i x 22i ) − (Σ i x1i x 2i )2
(
)
Var βˆ 1 =
Var βˆ 2 =
Cov βˆ 1 , βˆ 2 =
‰
M.G. Abbott
σ 2 Σ i x1i x 2i
.
(Σi x12i )(Σi x 22i ) − (Σi x1i x 2i )2
Unbiased estimators of the variances of the slope coefficient estimators β̂1
and β̂ 2 are obtained by substituting the unbiased estimator σ$ 2 for the unknown
error variance σ 2 in the formulas for Var(βˆ ) and Var(βˆ ) :
1
( )
σˆ 2 Σ i x 22i
;
(Σ i x12i )(Σ i x 22i ) − (Σ i x1i x 2i )2
(11.1)
( )
σˆ 2 Σ i x 12i
;
(Σ i x12i )(Σ i x 22i ) − (Σ i x1i x 2i )2
(11.2)
Vâr βˆ 1 =
Vâr βˆ 2 =
‰
2
Similarly, an unbiased estimator of the covariance between the slope
coefficient estimators β̂1 and β̂ 2 is obtained by substituting the unbiased
estimator σ$ 2 for the unknown error variance σ 2 in the formula for
Cov(βˆ 1 , βˆ 2 ) :
(
)
Côv βˆ 1 , βˆ 2 =
σˆ 2 Σ i x 1i x 2i
.
(Σ i x12i )(Σ i x 22i ) − (Σ i x1i x 2i )2
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
(11.3)
… Page 10 of 17 pages
ECONOMICS 351* -- NOTE 12
‰
M.G. Abbott
Interpretive formula for the variances of the OLS slope coefficient
estimators β̂ j , j = 1, 2, …, k
Consider the general multiple linear regression equation given by the PRE
Yi = β0 + β1X1i + L + β jX ji + L + β k X ki + u i
(12.1)
OLS estimation of the PRE in (11) yields the OLS SRE
Yi = βˆ 0 + βˆ 1X1i + L + βˆ jX ji + L + βˆ k X ki + û i
•
(12.2)
The formula for Var(βˆ j ) for j = 1, 2, …, k can be written as
Var(βˆ j ) =
σ2
TSS j ( 1 − R 2j )
for j = 1, 2, …, k
(13)
where
TSS j ≡ ∑ x 2ji ≡ ∑ (X ji − X j ) ≡ the total sample variation in the regressor Xj;
N
N
i =1
i =1
2
R 2j ≡ the R2 from the OLS regression of regressor Xj on all the other K−1
regressors in (12.1), including the intercept. That is, R 2j measures the
proportion of the total sample variation in Xj that is explained by the
other regressors in the PRE. Alternatively, R 2j measures the degree of
linear dependence between the sample values Xji of the regressor Xj
and the sample values of the other regressors in regression equation
(12.1).
This formula for Var (βˆ j ) is given in J. Kmenta, Elements of Econometrics, 2nd
edition (1986), pp. 437-438.
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 11 of 17 pages
ECONOMICS 351* -- NOTE 12
•
M.G. Abbott
Determinants of Var(βˆ j ) .
Var(βˆ j ) =
σ2
TSS j ( 1 − R 2j )
for j = 1, 2, …, k
(13)
Three factors determine Var(βˆ j ) :
1. the error variance σ 2 ;
2. the total sample variation in Xj, TSSj;
3. the degree of linear dependence between Xj and the other regressors in
the model, as measured by R 2j .
•
Var(βˆ j ) is smaller:
(1) the smaller is σ2, the error variance in the true model;
(2) the larger is TSSj, the total sample variation in the regressor Xj;
TSS j ≡ ∑ x 2ji ≡ ∑ (X ji − X j ) is larger
•
N
N
i =1
i =1
2
the larger the values of x 2ji = ( X ji − X j ) 2 for i = 1, …, N, meaning the
greater the sample variation in the Xji values around their sample mean;
•
the larger is N, the size of the estimation sample;
(3) the smaller (closer to 0) is R 2j , the lower the degree of linear dependence
between the sample values Xji of the regressor Xj and the sample values of
the other regressors in the PRE.
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 12 of 17 pages
ECONOMICS 351* -- NOTE 12
•
M.G. Abbott
Conversely, Var(βˆ j ) is larger:
(1) the larger is σ2, the error variance in the true model;
(2) the smaller is TSSj, the total sample variation in the regressor Xj;
TSS j ≡ ∑ x 2ji ≡ ∑ (X ji − X j ) is smaller
•
N
N
i =1
i =1
2
the smaller the values of x 2ji = ( X ji − X j ) 2 for i = 1, …, N, meaning the
greater the sample variation in the Xji values around their sample mean;
•
the smaller is N, the size of the estimation sample;
(3) the larger (closer to 1) is R 2j , the greater the degree of linear dependence
between the sample values Xji of the regressor Xj and the sample values of
the other regressors in the PRE.
Note: Assumption A8, the absence of perfect multicollinearity, rules out the
value 1 for R 2j .
If R 2j = 1, then the sample values of the regressor Xj exhibit an exact linear
dependence -- are perfectly multicollinear -- with one or more of the other
regressors in the model, in which case it is impossible to compute either
(1) the OLS estimate β̂ j of the slope coefficient βj associated with the
regressor Xj, or
(2) the estimated value of Var(βˆ j ) , the estimated variance of β̂ j .
If R 2j < 1, then multicollinearity is simply a question of degree: the closer to
1 is the sample value of R 2 , the larger is Var (βˆ ) .
j
j
Var(βˆ j ) → ∞
as
R 2j → 1
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 13 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
5. Computational Properties of the OLS-SRE (2)
Yi = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i + û i = Ŷi + û i
(i = 1, ..., N).
(2)
5.1 The OLS-SRE passes through the point of sample means (Y, X1 , X 2 ) ; i.e.,
Y = βˆ 0 + βˆ 1X1 + βˆ 2 X 2 .
(C1)
Proof of (C1): Follows directly from dividing OLS normal equation (N1) by
N.
N
N
i =1
i =1
Nβˆ 0 + βˆ 1 ∑ X1i + βˆ 2 ∑ X 2i =
N
Nβˆ 0 ˆ
+ β1
N
∑ X1i
i =1
N
N
+ βˆ 2
∑ X 2i
i =1
N
N
∑ Yi
(N1)
i =1
N
=
∑ Yi
i =1
N
dividing (N1) by N
βˆ 0 + βˆ 1X1 + βˆ 2 X 2 = Y .
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 14 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
5.2 The sum of the estimated Yi's (the Ŷi 's) equals the sum of the observed Yi's;
or the sample mean of the estimated Yi's (the Ŷi 's) equals the sample mean
of the observed Yi's.
N
N
∑ Ŷi
N
∑ Ŷi = ∑ Yi
i =1
i =1
or
N
i =1
N
=
∑ Yi
i =1
Ŷ = Y
or
N
(C2)
Proof of (C2):
1. Substitute βˆ 0 = Y − βˆ 1X1 − βˆ 2 X 2 in the expression for Ŷi :
Ŷi = βˆ 0 + βˆ 1X1i + βˆ 2 X 2i
= Y − βˆ 1X1 − βˆ 2 X 2 + βˆ 1X1i + βˆ 2 X 2i
= Y + βˆ (X − X ) + βˆ (X − X )
(
)
1
1i
1
2
2i
2
= Y + βˆ 1x1i + βˆ 2 x 2i
since
x1i ≡ (X1i − X1 )
x 2i ≡ (X 2i − X 2 )
2. Now sum the above equation over the sample:
N
N
N
i =1
i =1
i =1
∑ Ŷi = NY + βˆ 1 ∑ x1i + βˆ 2 ∑ x 2i
= NY
because
N
N
i =1
i =1
∑ x1i = ∑ x 2i = 0.
3. Thus, dividing both sides of the above equation by N, we obtain the result
N
N
N
∑ Ŷi
i =1
N
∑ Ŷi = NY ⇒
i =1
= Y or
∑ Ŷi
i =1
N
N
=
∑ Yi
i =1
N
N
N
i =1
i =1
$ = Y or ∑ Ŷ = ∑ Y .
or Y
i
i
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 15 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
5.3 The sum of the OLS residuals û i (i = 1, …, N) equals zero, or the sample
mean of the OLS residuals û i equals zero.
N
N
∑ u$ i
=0
or
û =
i =1
∑ û i
i =1
N
= 0.
(C3)
Proof of (C3): An immediate implication of equation (7.1).
∑(
N
i =1
)
Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i = 0
N
∑ u$ i
⇔
=0
(7.1)
i =1
5.4 The OLS residuals û i (i = 1, …, N) are uncorrelated with the sample values
of the regressors X1i and X2i (i = 1, …, N): i.e.,
N
∑ X1i û i = 0
i =1
N
and
∑ X 2i û i = 0 .
(C4)
i =1
Proof of (C4): An immediate implication of equations (7.2) and (7.3).
N
(
)
(
)
∑ X1i Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i = 0
i =1
N
∑ X 2i Yi − βˆ 0 − βˆ 1X1i − βˆ 2 X 2i = 0
i =1
⇔
N
∑ X1i û i = 0
(7.2)
i =1
⇔
N
∑ X 2i û i = 0
(7.3)
i =1
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 16 of 17 pages
ECONOMICS 351* -- NOTE 12
M.G. Abbott
5.5 The OLS residuals û i (i = 1, …, N) are uncorrelated with the estimated
values of Yi, the Ŷi values (i = 1, …, N): i.e.,
N
∑ Ŷi û i = 0 .
(C5)
i =1
Proof of (C5): Follows from properties (C3) and (C4) above.
$ are given by the following expression for the OLS sample regression
1. The Y
i
function (the OLS-SRF):
Ŷi = βˆ 0 + β1X1i + βˆ 2 X 2i .
2. Multiply the above equation by u$ i :
Ŷi û i = βˆ 0 û i + βˆ 1X1i û i + βˆ 2 X 2i û i .
3. Summing both sides of the above equation over the sample gives the result
N
N
N
N
i =1
i =1
i =1
i =1
∑ Ŷi û i = βˆ 0 ∑ û i + βˆ 1 ∑ X1i û i + βˆ 2 ∑ X 2i û i
=0
because
N
∑ û i = 0
by property (C3)
i =1
and
N
∑ X1i û i =
i =1
N
∑ X 2i û i = 0
by property (C4).
i =1
ECON 351* -- Note 12: OLS Estimation in the Multiple CLRM
… Page 17 of 17 pages