UNC-Wilmington
Department of Economics and Finance
ECN 377
Dr. Chris Dumas
Appendix to Regression Handout
̂ 𝟎 and 𝜷
̂𝟏
Derivation of the OLS Regression Estimator Equations for 𝜷
(The material in this Appendix is not required for ECN377, but if you plan to go to grad school, you should
understand and be able to explain the derivation below. If you want to understand, but you don’t, come by my
office, and I’ll help you.)
The OLS Regression Estimator Equations are the solution to the problem:
“Find the β̂i’s that minimize the sum of the squared êi’s for all individuals in the sample.”
min ∑(𝑒̂𝑖2 )
̂0 ,𝛽
̂1
𝛽
𝑖
The problem above is a nonlinear optimization problem without constraints, so we can use the classical calculus
method of optimization to find a solution (recall the classical calculus method of optimization from your ECN321
course at UNCW).
To solve this problem, begin by replacing êi with 𝑌𝑖 − β̂0 − β̂1 ∙ 𝑋1𝑖
2
min ∑ ((𝑌𝑖 − β̂0 − β̂1 ∙ 𝑋1𝑖 ) )
̂0 ,𝛽
̂1
𝛽
𝑖
Next, square the expression within the parentheses:
2
min ∑[𝑌𝑖2 − 𝑌𝑖 𝛽̂0 − 𝑌𝑖 𝛽̂1 𝑋1𝑖 − 𝛽̂0 𝑌𝑖 + 𝛽̂02 + 𝛽̂0 𝛽̂1 𝑋1𝑖 − 𝛽̂1 𝑋1𝑖 𝑌𝑖 + 𝛽̂1 𝑋1𝑖 𝛽̂0 + 𝛽̂12 𝑋1𝑖
]
̂0 ,𝛽
̂1
𝛽
𝑖
Inside the brackets, collect similar terms together:
2
min ∑[𝑌𝑖2 − 2𝑌𝑖 𝛽̂0 − 2𝑌𝑖 𝛽̂1 𝑋1𝑖 + 𝛽̂02 + 2𝛽̂0 𝛽̂1 𝑋1𝑖 + 𝛽̂12 𝑋1𝑖
]
̂0 ,𝛽
̂1
𝛽
𝑖
Take partial derivatives and set each equal to zero to find the First Order Conditions for the minimization problem
(remember that the partial derivative operator can “move through” the summation operator):
F.O.C.’s
(1)
(2)
𝜕
̂
𝜕𝛽
0
𝜕
̂
𝜕𝛽
1
̂ + 2𝛽
̂ 𝑋 ]=0
= ∑𝑖 [−2𝑌𝑖 + 2𝛽
0
1 1𝑖
̂ 𝑋 + 2𝛽
̂ 𝑋2 ] = 0
= ∑𝑖 [−2𝑌𝑖 𝑋1𝑖 + 2𝛽
0 1𝑖
1 1𝑖
Now let’s focus on simplifying FOC (1):
̂ + 2𝛽
̂ 𝑋 ]=0
∑ [−2𝑌𝑖 + 2𝛽
0
1 1𝑖
𝑖
1
UNC-Wilmington
Department of Economics and Finance
ECN 377
Dr. Chris Dumas
Recall that the summation operator can distribute across terms that are added or subtracted:
̂ ] + ∑ [2𝛽
̂ 𝑋 ]=0
∑[−2𝑌𝑖 ] + ∑ [2𝛽
0
1 1𝑖
𝑖
𝑖
𝑖
Recall that constants can be “pulled through” summation operators:
̂ ∑ 1 + 2𝛽
̂ ∑𝑋 = 0
−2 ∑ 𝑌𝑖 + 2𝛽
1𝑖
0
1
𝑖
𝑖
𝑖
Next, assuming that our sample size is “n”, notice that the sum of n “one’s” is simply equal to n:
̂ 𝑛 + 2𝛽
̂ ∑𝑋 = 0
−2 ∑ 𝑌𝑖 + 2𝛽
1𝑖
0
1
𝑖
𝑖
Cancelling the “2’s” and moving the term with the sum of Y to the right side of the equation:
𝛽̂0 𝑛 + 𝛽̂1 ∑𝑖 𝑋1𝑖
=
∑𝑖 𝑌𝑖
call this Equation (3)
Turning to FOC (2), using methods similar to those that we used for FOC (1), we find that FOC (2) simplifies to:
2
] = ∑𝑖[𝑌𝑖 𝑋1𝑖 ]
𝛽̂0 ∑𝑖[𝑋1𝑖 ] + 𝛽̂1 ∑𝑖[𝑋1𝑖
call this Equation (4)
Importantly, notice that Equation (3) and Equation (4) are “two equations in two unknowns.” The two unknowns
are 𝛽̂0 and 𝛽̂1 (recall that we are trying to solve for 𝛽̂0 and 𝛽̂1 .) The X and Y variables are not unknowns,
because we have sample data on X and Y that we can insert into the equations; we also assume that we know n,
the sample size. Next, we solve these “two equations in two unknowns” for 𝛽̂0 and 𝛽̂1 (for example, we could
solve Equation (3) for 𝛽̂0 , substitute the result into Equation (4), solve Equation (4) for 𝛽̂1 , and then substitute
the result for 𝛽̂1 back into Equation (3) to find 𝛽̂0 ). When we solve Equation (3) and Equation (4) for 𝛽̂0 and
𝛽̂1 , we find:
𝛽̂0
𝛽̂1 =
=
∑𝑖 𝑌𝑖
𝑛
− 𝛽̂1
∑𝑖 𝑋1𝑖
𝑛
∑𝑖[𝑌𝑖 𝑋1𝑖 ] − (∑𝑖[𝑋1𝑖 ]) (
2 ] (∑ [𝑋 ])
∑𝑖[𝑋1𝑖
−
𝑖
1𝑖 (
∑𝑖[𝑌𝑖 ]
𝑛
∑𝑖[𝑋1𝑖 ]
𝑛
)
)
2
UNC-Wilmington
Department of Economics and Finance
ECN 377
Dr. Chris Dumas
We can simplify the last two equations above a bit more by noticing that
the dataset, or ̅̅̅
𝑋1 , and similarly,
∑𝑖 𝑌𝑖
𝑛
∑𝑖 𝑋1𝑖
𝑛
is simply the average value of X1 in
is equal to 𝑌̅. Making these substitutions:
𝛽̂0
𝛽̂1 =
̅ − 𝛽̂1
=𝑌
∙ ̅̅̅
𝑋1
∑𝑖[𝑌𝑖 𝑋1𝑖 ] − (∑𝑖[𝑋1𝑖 ]) ∙̅𝑌
2 ] (∑ [𝑋 ]) ̅̅̅̅
∑𝑖[𝑋1𝑖
−
𝑖
1𝑖 ∙ 𝑋1
As a last simplification, if we multiply each ∑𝑖 𝑋1𝑖 by
𝑛
𝑛
and notice again that
∑𝑖 𝑋1𝑖
𝑛
is equal to ̅̅̅
𝑋1 , we find:
̂ 𝟎 and 𝜷
̂𝟏
The OLS Regression Estimator Equations for 𝜷
𝛽̂0
𝛽̂1 =
̅ − 𝛽̂1
=𝑌
∙ ̅̅̅
𝑋1
∑𝑖[𝑌𝑖 𝑋1𝑖 ] − 𝑛 ∙ 𝑋̅1 ∙̅𝑌
2]
∑𝑖[𝑋1𝑖
− 𝑛 ∙ (𝑋̅1 )2
3