Minimizing e2: A Refresher in Calculus
•
•
•
•
•
Minimizing error
The derivative
Slope at a point
Differentiation
Rules of Derivation
–
–
–
–
Power rule
Constants & Sums
Products & Quotients
The Chain-rule
• Critical points
– Factoring equations
Week 2
Lecture 2b
Slide #1
What’s in e?
• Measurement error
– Imperfect operationalizations
– Imperfect measure application
• Bias and noise
• Modeling error/misspecification
– Missing model explanation
– Incorrect assumptions about associations
– Incorrect assumptions about distributions
• More bias and more noise
• Sum of errors affecting measurement and model?
– Ideally, lots of small and independent influence
• Results in random quality of error
– When error is systematic, may bias estimates
Week 2
Lecture 2b
Slide #2
Measuring Error: Residuals2
Objective: to estimate
model such that we
minimize ∑e.
Yi  0  1 X i
Y
For computational
reasons, we
minimize ∑e2.
 e   (Y  b
2
i
i
0
Yˆi  b0  b1 X i  ei
 b1 X i )
2
X
  Y 2  2b0  Y  2b1  XY  nb02  2b0b1  X  b12  X 2
Week 2
Lecture 2b
Slide #3
The Derivative
• Minimization requires specifying an estimator of b0
and b1 such that ∑e2 is at the lowest possible value.
– Easy in the linear case, but ∑e2 is curvilinear (quadratic)
• Need calculus of derivatives
– Allow identification of slope at a point
The function is f(X)
The derivative is
f´(X)
Said as “f-prime X” or
Y=f(X)
x1
Week 2
x2
x3
Lecture 2b
Y
X
Slide #4
More on Derivatives
• If we knew the formula for f´(X), we could
plug in the value of X to find the slope at
that point.
• That means differential calculus is
preoccupied with the rules for defining the
derivative, given the various possible
functional forms of f(X)
• Now take a deep breath…
Week 2
Lecture 2b
Slide #5
Power Rule & Constant Rule
• If f(X) = xn, then f´(X) = nxn-1
• Example: if f(X) = x6, then f´(X) = 6x5
• If f(X) = c, then f´(X) = 0
• Example: if f(X) = 5, then f´(X) = 0
Week 2
Lecture 2b
Slide #6
Calculating Slope for (x,y) Pairs
Y= f(X) =x2
f´(X) =2x
30
X
Y
-5
-2
0
2
5
25
4
0
4
25
25
20
15
Y
10
5
X
Slope
-5
-2
0
2
5
-10
-4
0
4
10
0
-6
Week 2
-4
-2
0
Lecture 2b
2
4
6
Slide #7
More Rules
• A derivative of a constant times a function
– If f(X) = c · u(X), then f´(X) = c · u´(x)
– Example: f(X)=5x2; f´(x)= 5 · 2x = 10x
• Example: plot Y= f(x) = x2 - 6x + 5
– Find f´(x).
Week 2
Lecture 2b
Slide #8
Plot and Derivative for
Y= f(x) = x2-6x+5
Y= f(X)
X
-1
0
1
2
3
4
5
6
7
Y= f´(X)
Y
14
12
12
5
0
-3
-4
-3
0
5
12
Week 2
10
8
6
Y
4
2
0
-2
0
2
-2
-4
-6
Lecture 2b
4
6
8
X
Slope
-1
0
1
2
3
4
5
6
7
-8
-6
-4
-2
0
2
4
6
8
Slide #9
More Rules
• Differentiating a sum
– If f(x) = u(x) + v(x) then
f´(x) = u´(x) + v´(x)
– Example: f(x) = 32x + 4x2;
f´(x) = 32 + 8x
• If f(x) = 4x2 - 3x, what is f´(x)?
Week 2
Lecture 2b
Slide #10
Product Rule
• If f(x) = u(x) · v(x) then
f´(x) = u(x) · v´(x) + u´(x) · v(x)
• Example: f(x) = x3(x - 5); find f´(x)
– f´(x) = [x3 · 1] + [3x2 · (x - 5)] =
x3 + 3x3 - 15x2 = 4x3 - 15x2
• You get the same result using only the power rule
But the product rule is easier when f(x) is complex
• Example: f(x) = (x4 + 3)(3x3 + 1); find f´(x)
– f´(x) = [(x4 + 3) · 9x2] + [4x3 · (3x3 + 1)]
Week 2
Lecture 2b
Slide #11
Quotient Rule
N(x)
If f (x) 
then
D(x)
D(x)  N (x)  D (x)  N(x)
f (x) 
2
[D(x)]
Example:
x
then
2
x 5
(x 2  5) 1 2x  x
f (x) 

2
2
(x  5)
If f (x) 
x 2  5  2x 2
 x2  5
 2
2
2
2
(x  5)
(x  5)
Week 2
Lecture 2b
Slide #12
Chain Rule
• If f(x)=[u(x)]n then
f´(x) = n[u(x)]n-1 · u´(x)
• Example: if f(x) = (7x2 - 2x + 13)5 then
f´(x) = 5(7x2 - 2x + 13)4 · (14x - 2)
• Try this: if f(x) = (3x + 1)10 then f´(x) = ?
Week 2
Lecture 2b
Slide #13
Critical Points
• Finding minima and maxima
• Key: where f´(x) = 0, slope is zero
• Example: if f(x) = x2 - 4x +5
then f´(x) = 2x - 4
Set (2x - 4) = 0, then x - 2 = 0;
so x = 2 when f´(x) = 0
• Y coordinate when x = 2 is 1 (calculate it!!)
• So: f(x) has a critical point at x = 2, y = 1
Week 2
Lecture 2b
Slide #14
Critical Points, Continued
• How do you determine if it’s a minima or a
maxima?
– How many humps in the functional form?
• At least exponent minus one
– Identify x,y coordinates and plot
– Identify the derivative: f´(x) on either side of
the critical point
Week 2
Lecture 2b
Slide #15
Identification of Critical Point for
f(x) = x2 - 4x + 5
60
50
40
30
Y
20
f'(X)
10
0
-6
-4
-2
0
2
4
6
-10
-20
Week 2
Lecture 2b
8
10
If f´(x) = 2x - 4,
then the value
of f´(x) is
negative (slope
is negative) up
to 2, and positive
(slope is positive)
above 2.
Therefore, the
critical point at
(2,0) is a minimum
Slide #16
Factoring Ugly Functions
Sometimes a quadratic functional form can be simplified by
factoring. When the equation can be written as:
ax2 + bx + c
The factors can be derived as follows:
b  b 2  4ac
x
2a
For example: f(x) = x3 - 6x2 + 9x; f´(x) = 3x2 - 12x + 9
To find the critical points, we’d need to find the values
at which f´(x) = 0. Factoring reduces f´(x) to:
f´(x) = (x - 3)(3x - 3); so both x = 3 and x = 1 are CP’s
Week 2
Lecture 2b
Slide #17
Partial Derivation
When an equation includes two variables, one can take a
“partial derivative” with respect to only one variable. The
other variable is simply treated as a constant:
Y  f (x,z)  x  4 xz  5z
3
2
f (x)  3x  4z
f (z)  4 x 10z
2
Week 2
Lecture 2b
Slide #18
Some Puzzles
Differentiate f(x) = x3 - 4x + 7
Differentiate f(x) = (x - 3)(x2 + 7x + 10)
Differentiate f(x) = (x2 - 5)/(x + 7)
Differentiate f(x) = (x2 - 3x + 5)12
Find all critical points for
Y = f(x) = x3 + 3x2 + 1
Plot the function, identify the CP’s.
6. Find all critical points for
Y= f(x) = x4 - 8x3 + 18x2 - 27
Plot, and identify CP’s.
7. Y = f(x,z) = x4 + 15z2 + 2xz2 - 456z12 Find f’(x)
1.
2.
3.
4.
5.
Week 2
Lecture 2b
Slide #19