CURVE FITTING
Student Notes
ENGR 351
Numerical Methods for Engineers
Southern Illinois University Carbondale
College of Engineering
Instructor: L.R. Chevalier, Ph.D., P.E.
Specific Growth Rate, m
Applications
Need to determine parameters
for saturation-growth rate
model to characterize
microbial kinetics
Food Available, S
m  m max
S
Ks  S
Applications
T ( o C)
0
10
20
30
0
Epilimnion
5
z (m )
10
Thermocline
15
20
25
30
Hypolimnion
Applications
Interpolation of
data
What is kinematic
viscosity at 7.5º C?
v, 10-2 cm 2/s
2
1.5
1
0.5
0
0
10
20
T(oC)
30
T (oC)
0
4
8
12
16
20
24
v, 10-2 (cm2/s)
1.7923
1.5615
1.3874
1.2396
1.1168
1.0105
0.9186
f(x)
Can you suggest another?
x
We want to find the best “fit” of a curve through the data.
Here we see :
a) Least squares fit
b) Linear interpolation
Material to be Covered in
Curve Fitting
 Linear Regression




Polynomial Regression
Multiple Regression
General linear least squares
Nonlinear regression
 Interpolation



Lagrange polynomial
Coefficients of polynomials (CollocationPolynomial Fit)
Splines
Specific Study Objectives
 Understand the fundamental difference
between regression and interpolation and
realize why confusing the two could lead to
serious problems
 Understand the derivation of linear least
squares regression and be able to assess the
reliability of the fit using graphical and
quantitative assessments.
Specific Study Objectives
 Know how to linearize data by
transformation
 Understand situations where polynomial,
multiple and nonlinear regression are
appropriate
 Understand the general matrix formulation
of linear least squares
 Understand that there is one and only one
polynomial of degree n or less that passes
exactly through n+1 points
Specific Study Objectives
 Realize that more accurate results are
obtained if data used for interpolation is
centered around and close to the unknown
point
 Recognize the liabilities and risks associated
with extrapolation
 Understand why spline functions have utility
for data with local areas of abrupt change
Least Squares Regression
 Simplest is fitting a straight line to a set of
paired observations

(x1,y1), (x2, y2).....(xn, yn)
 The resulting mathematical expression is

y = ao + a1 x + e
 We will consider the error introduced at
each data point to develop a strategy for
determining the “best fit” equations
Determining the Coefficients
n x   xi 
2
2
i
ao  y  a1 x
x
0
1
2
3
4
5
6
y
9
7
5
4
3
1
0
Let’s consider
where this
comes from.
10
9
8
7
f(x)
a1 
n xi yi   xi  y i
6
5
4
3
2
1
0
0
2
4
x
6
8
Sum of the Residual Error, Sr
n
n
i 1
i 1
Sr   e 2i    yi  a o  a1xi 
2
f(x)
x
Sum of the Residual Error, Sr
n
n
i 1
i 1
Sr   e 2i    yi  a o  a1xi 
2
Note:
In this equation yi
is the raw data
point (dependent
data) associated
with xi
(independent
data)
f(x)
x
Sum of the Residual Error, Sr
n
n
i 1
i 1
Sr   e 2i    yi  a o  a1xi 
f(x)
2
A line that models
this data is:
y = ao + a1 x
x
Sum of the Residual Error, Sr
n
n
i 1
i 1
Sr   e 2i    yi  a o  a1xi 
2
f(x)
x
Sum of the Residual Error, Sr
n
n
i 1
i 1
Sr   e 2i    yi  a o  a1xi 
2
f(x)
yi  a o  a1xi
x
Determining the Coefficients
To determine the values for ao and a1, differentiate
with respect to each coefficient
Sr
 2  yi  ao  a1xi 
ao
Sr
 2  yi  ao  a1xi  xi 
a1
Note: we have simplified the summation symbols.
What mathematics technique will minimize Sr?
Determining the Coefficients
Sr
 2  yi  ao  a1xi 
ao
Sr
 2  yi  ao  a1xi  xi 
a1
Setting the derivative equal to zero will minimizing Sr.
If this is done, the equations can be expressed as:
0   yi   ao   a1xi
0   yi xi   ao xi   a1xi2
Determining the Coefficients
0   yi   ao   a1xi
0   yi xi   ao xi   a1xi2
Note:
a
o
 nao
We have two simultaneous equations, with two
unknowns, ao and a1.
What are these equations? (hint: only place terms with
ao and a1 on the LHS of the equations)
What are the final equations for ao and a1?
Determining the Coefficients
nao   xi a1   yi
2
x
a

x
 i o  i a1   xi yi
a1 
n xi yi   xi  y i
n x    xi 
ao  y  a1x
2
i
2
These first two
equations are
called
the normal
equations
Example
Determine the linear equation for the
following data
y
9
7
5
4
3
1
0
8
7
f(x)
x
0
1
2
3
4
5
6
10
9
6
5
4
3
2
1
0
0
2
4
x
Strategy
6
8
Strategy
 Set up a table
x
y
xy
x2
Sx
Sy
Sxy
Sx2
 From these values determine the
average values of x and y (x- and ybar)
 Calculate a0 and a1
Error
f(x)
x
The most common measure of the “spread” of a sample is
the standard deviation about the mean:
St    yi  y 
2
St
sy 
n 1
Error
Coefficient of determination r2:
St  Sr
r 
St
2
r is the correlation coefficient
Error
St  Sr
r 
St
2
The following signifies that the line explains
100 percent of the variability of the data:
Sr = 0
r = r2 = 1
If r = r2 = 0, then Sr = St and the fit is invalid.
Example
Determine the R2 value for the following data
y
9
7
5
4
3
1
0
8
7
f(x)
x
0
1
2
3
4
5
6
10
9
6
5
4
3
2
1
0
0
2
4
x
Strategy
6
8
Strategy
 Complete table
x
y
xy
x2
Sx
Sy
Sxy
Sx2
 Calculate Sr, St
 Determine R2
ymodel
e2
(yyavg)2
SSr
SSt
Consider the following four sets of data
Data 1
10
8.04
8
6.95
13
7.58
9
8.81
11
8.33
14
9.96
6
7.24
4
4.26
12
10.84
7
4.82
5
5.68
Data 2
10
9.14
8
8.14
13
8.74
9
8.77
11
9.26
14
8.10
6
6.13
4
3.10
12
9.13
7
7.26
5
4.74
Data 3
10
7.46
8
6.77
13
12.74
9
7.11
11
7.81
14
8.84
6
6.08
4
5.39
12
8.15
7
6.42
5
5.73
Data 4
8
6.58
8
5.76
8
7.71
8
8.84
8
8.47
8
7.04
8
5.25
19
12.50
8
5.56
8
7.91
8
6.89
12
12
10
10
8
8
6
6
4
4
y = 0.5001x + 3.0001
R2 = 0.6665
2
y = 0.5x + 3.0009
2
0
R2 = 0.6662
0
0
5
10
15
0
5
x
10
15
x
14
14
12
12
10
10
8
y
y
8
6
6
4
4
y = 0.4997x + 3.0025
2
2
y = 0.4999x + 3.0017
R2 = 0.6667
2
R = 0.6663
0
0
0
5
10
x
15
0
5
10
x
15
20
Common Sense!!!!
y
14
y
14
Linearization of non-linear
relationships
Some data is simply
ill-suited for linear
least squares
regression....
or so it appears.
f(x)
x
EXPONENTIAL
EQUATIONS
P
Linearize
P  Po e
rt
t
ln P
intercept = ln P0
slope = r
why?
t
P  P0e rt
ln P  ln P0e rt 
 ln P0   lne rt 
 ln P0   rt
Can you see the similarity
with the equation for a line:
y = a o + a1 x
lnP
intercept = ln Po
slope = r
t
P  P0e rt
ln P  ln P0e rt 
 ln P0   lne rt 
 ln P0   rt
ln P
intercept = ln P0
After taking the natural log
of the y-data, perform linear
regression.
From this regression:
The value of ao will give us
ln (P0). Hence, P0 = eao
The value of a1 will give us r
directly.
slope = r
t
Q
POWER EQUATIONS
Q  cH
a
(Flow over a weir)
log Q
H
log H
Here we linearize
the equation by
taking the log of
H and Q data.
What is the resulting
intercept and slope?
Q  cH a
log Q  logcH a 
 log c  log H
a
 log c  a log H
log Q
slope = a
log H
intercept = log c
Q  cH a
log Q  logcH
a
So how do we get
c and a from
performing regression
on the log H vs log Q
data?
From : y = ao + a1x

 log c  log H
a
 log c  a log H
ao = log c
log Q
c = 10ao
slope = a
a1 = a
log H
intercept = log c
SATURATION-GROWTH
RATE EQUATION
m
m  m max
S
1/m
slope = Ks/mmax
intercept = 1/mmax
1/ S
S
Ks  S
Here, m is the growth
rate of a microbial
population,
mmax is the maximum
growth rate, S is the
substrate or food
concentration, Ks is the
substrate concentration
at a value of m = mmax/2
Example
 Given the data below, determine the
coefficients a and b for the equation
y=axb
300
250
y
3.1
26
110
250
200
y
x
1
2
3
4
Raw Data
150
100
50
0
0
1
2
3
4
5
x
Strategy
Strategy
 Start a table. For y=axb you need a
log-log table and graph.
x
y
log x
log y
 Perform linear regression on the loglog data
 Based on y = a0 + a1x, calculate
 log a = a0 therefore a = 10a0
 b=a1
Residual Error
Linear: y = ao + a1x
S r   e 2    yi  ymodel     yi  ao  a1 xi 
2
2
Power: y = axb

b 2

xi b 2
Sr   e    yi  ymodel    yi  ax i
2
2

Exponential: y=aexb
Sr   e    yi  ymodel    yi  ae
2
2

Example
X
Y
Ymodel
1
0.7
0.8
2
1.7
1.6
3
3.3
3.1
4
7.3
6.2
5
10.9
12.1
6
22.7
23.9
y = 0.407e
0.679x
Given the following results, determine Sr.
Strategy
Strategy
 Calculate e2 = (yi – ymodel)2 for each
(x,y) pair
 Determine Se2 = Sr
Demonstration with Excel
Y
1
2
3
4
5
6
0.7
1.7
3.3
7.3
10.9
22.7
25
20
15
y
X
10
5
0
0
2
4
6
8
x
See: 2012-L4-NonlinearRegressionExample.xlsx
General Comments of Linear
Regression
 You should be cognizant of the fact
that there are theoretical aspects of
regression that are of practical
importance but are beyond the scope
of this book
 Statistical assumptions are inherent in
the linear least squares procedure
General Comments of Linear
Regression
 x has a fixed value; it is not random
and is measured without error
 The y values are independent random
variable and all have the same
variance
 The y values for a given x must be
normally distributed
General Comments of Linear
Regression
 The regression of y versus x is not the
same as x versus y
 The error of y versus x is not the same
as x versus y
General Comments of Linear
Regression
 The regression of y versus x is not the
same as x versus y
 The error of y versus x is not the same
as x versus y
y-direction
f(x)
x-direction
x
Polynomial Regression
 One of the reasons you were presented
with the theory behind linear
regression was to allow you the insight
behind similar procedures for higher
order polynomials
 y = a0 + a1x
 mth - degree polynomial

y = a0 + a1x + a2x2 +....amxm + e
Polynomial Regression
Sr    yi  a o  a1xi  a x ...... a x
2
2 i

m 2
m i
Based on the sum of the squares of
the residuals
Polynomial Regression
Sr    yi  a o  a1xi  a x ...... a x
2
2 i

m 2
m i
1. Take the derivative of the above equation
with respect to each of the unknown
coefficients: i.e. the partial with respect to
a2
Sr
2
 2 xi  yi  ao  a1xi  a2 xi2 ..... amxim 
a2
Polynomial Regression
2. These equations are set to zero to minimize
Sr, i.e. minimize the error.
3. Set all unknowns values on the LHS of the
equation.
ao  xi  a1  xi3  a2  xi4 ..... am  xi
2
m 2
  xi yi
2
Polynomial Regression
4. This set of normal equations result in
m+1 simultaneous equations which can be
solved using matrix methods to determine
a0, a1, a2......am
Polynomial Regression
n a o   x i a1   x
a   y
 x a   x a   x a   x y
 x a   x a   x a   x y
i
2
i
o
2
i
o
3
i
2
i
2
i
1
3
i
1
4
i
2
i
2
2
i
i
i
Multiple Linear Regression
 A useful extension of linear regression
is the case where y is a linear function
of two or more variables

y = ao + a1x1 + a2x2
 We follow the same procedure

y = ao + a1x1 + a2x2 + e
Multiple Linear Regression
For two variables, we would solve a 3 x 3 matrix
in the following form:
 n

  x1i
 x2i
x
x
x x
1i
2
1i
1i 2 i
x
x x
x
  a 0    yi 

  
1i 2 i   a1     x1i yi 
2
   x y
2 i  a 2 

 2 i i 
2i
[A] and {c}are clearly based on data given for x1, x2
and y to solve for the unknowns in {x}.
Interpolation
 General formula for an n-th order
polynomial

y = a0 + a1x + a2x2 +....amxm
 For m+1 data points, there is one, and
only one polynomial of order m or less
that passes through all points
 Example: y = a0 + a1x


fits between 2 points
1st order
Linear Interpolation
Temperature, C
Density, kg/m3
0
999.9
5
1000.0
10
999.7
15
999.1
20
998.2
How would you approach estimating the
density at 17 C?
Temperature, C
Density, kg/m3
0
999.9
5
1000.0
10
999.7
15
999.1
20
998.2
???
999.1 > r > 998.2
998.2
999.1
r
T
15
20
998.2
999.1
r
T
15
998.2  999.1 r

20  15
T
20
Assume a straight line
between the known
data.
Then calculate the
slope.
998.2
999.1
r
15
r  999.1 r

17  15
T
T
17
20
Assuming this linear
relationship is constant,
the slope is the same
between the unknown point
and a known point.
998.2
999.1
r
15
T
17
998.2  999.1 r  999.1 r


20  15
17 15 T
20
Solve for r
Therefore, the slope of one interval will equal
the slope of the other interval.
998.2  999 .1 998.2  r

20  15
20  17
f  x1   f  x0 
f1  x   f  xo  
 x  x0 
x1  x0
Alternate interpretation
f1 x   a0  a1 x  x0 
the intercept is f(x0)
the slope is a finite difference
approx. of dy/dx
true solution
f(x)
1
2
smaller intervals
provide a better estimate
x
true solution
f(x)
1
2
Alternative approach would be to
include a third point and estimate
f(x) from a 2nd order polynomial.
x
true solution
f(x)
Alternative approach would be to
include a third point and estimate
f(x) from a 2nd order polynomial.
x
Lagrange Interpolating Polynomial
n
f n  x   Li  x f  xi 
i0
n
Li  x  
j0
ji
x  xj
xi  x j
where P designates the “product of”
The linear version of this expression is at n=1
Linear version: n=1
n
f n  x   Li  x f  xi 
i 0
n
Li  x  
j 0
j i
x  xj
xi  x j
x  x1
x  x0
f1 
f  x0  
f  x1 
x0  x1
x1  x0
Your text shows you how to do n=2 (second order).
What would third order be?
n
f n  x    Li  x  f  xi 
i 0
n
Li  x   
j 0
j i
x  xj
xi  x j
 x  x1  x  x2  x  x3 
f3 
f  x0 
 x0  x1  x0  x2  x0  x3 
.......
Note:
x0 is
not being subtracted
from the constant
term x
n
f n  x    Li  x  f  xi 
i 0
n
Li  x   
j 0
j i
x  xj
xi  x j
 x  x1  x  x2  x  x3 
f3 
f  x0 
 x0  x1  x0  x2  x0  x3 
.......
Note:
x0 is
not being subtracted
from the constant
term x
or xi = x0 in
the numerator
or the denominator
j= 0
n
f n  x    Li  x  f  xi 
i 0
n
Li  x   
j 0
j i
x  xj
xi  x j
 x  x1  x  x2  x  x3 
f3 
f  x0 
 x0  x1  x0  x2  x0  x3 
 x  x0  x  x2  x  x3 

f  x1 
 x1  x0  x1  x2  x1  x3 
.......
n
f n  x    Li  x  f  xi 
i 0
n
Li  x   
j 0
j i
x  xj
xi  x j
 x  x1  x  x2  x  x3 
f3 
f  x0 
 x0  x1  x0  x2  x0  x3 
 x  x0  x  x2  x  x3 

f  x1 
 x1  x0  x1  x2  x1  x3 
.......
Note:
x1 is
not being
subtracted
from the
constant
term x
or xi = x1 in
the numerator
or the
denominator
j= 1
n
f n  x    Li  x  f  xi 
i 0
n
Li  x   
j 0
j i
x  xj
xi  x j
 x  x1  x  x2  x  x3 
f3 
f  x0 
 x0  x1  x0  x2  x0  x3 
 x  x0  x  x2  x  x3 

f  x1 
 x1  x0  x1  x2  x1  x3 
 x  x0  x  x1  x  x3 

f  x2 
 x2  x0  x2  x1  x2  x3 
......
Note:
x2 is
not being
subtracted
from the
constant
term x or
xi = x2 in
the numerator
or the
denominator
j= 2
n
f n  x   Li  x f  xi 
i 0
n
Li  x  
j 0
j i
x  xj
xi  x j
x  x1  x  x2  x  x3 

f3 
f  x0 
 x0  x1  x0  x2  x0  x3 
x  x0  x  x2  x  x3 


f  x1 
 x1  x0  x1  x2  x1  x3 
x  x0  x  x1  x  x3 


f  x2 
 x2  x0  x2  x1  x2  x3 
x  x0  x  x1  x  x2 


f  x3 
 x3  x0  x3  x1  x3  x2 
Note:
x3 is
not being
subtracted
from the
constant
term x or
xi = x3 in
the numerator
or the
denominator
j= 3
Example
Determine the density
at 17o C using a 2nd order
Lagrange Interpolating
Polynomial
1000.5
Density
1000
999.5
999
998.5
998
0
5
10
15
Temp
20
25
Temperature, C
Density, kg/m3
0
999.9
5
1000.0
10
999.7
15
999.1
20
998.2
Strategy
Strategy
 Determine the number of data points
needed


2nd Order – 3 points
3rd Order – 4 points
 Chose the data points for the formula
 Write out the formula without the data
 Add the data
 Solve
Coefficients of an Interpolating
Polynomial
y = a0 + a1x + a2x2 +....amxm
HOW CAN WE BE MORE STRAIGHT
FORWARD IN GETTING VALUES?
f x0   a0  a1 x0  a2 x02
f x1   a0  a1 x1  a2 x12
f x2   a0  a1 x2  a2 x22
This is a 2nd order polynomial.
We need three data points.
Plug the value of xi and f(xi)
directly into equations.
This gives three simultaneous equations
to solve for a0 , a1 , and a2
Example
Determine the density
at 17o C using the method of
“Coefficient of Interpolating
Polynominals”. Base your solution on
a 2nd order polynomial.
1000.5
Density
1000
999.5
999
998.5
998
0
10
20
Temp
30
Temperature, C
Density, kg/m3
0
999.9
5
1000.0
10
999.7
15
999.1
20
998.2
Strategy
Strategy
 Recognize that you are solving for a0, a1 and a2 for
the formula

y = a0 + a1x + a2x2
 Determine the three points you need for the
problem
 Recognize that the x-value associate with a0 is x0 = 1
 Set up the following matrix
1 x1

1 x2
1 x3

x12  a0   y1 
  
2 
x2   a1    y2 
x32  a2   y3 
Spline Interpolation
 Our previous approach was to derive an
nth order polynomial for n+1 data
points.
 An alternative approach is to apply
polynomials to subset of data points
 Such connecting polynomials are called
spline functions
 Adaptation of drafting techniques
Spline interpolation is an adaptation of the drafting
technique of using a spline to draw smooth curves
through a series of points
Spline interpolation is an adaptation of the drafting
technique of using a spline to draw smooth curves
through a series of points
Spline interpolation is an adaptation of the drafting
technique of using a spline to draw smooth curves
through a series of points
Spline interpolation is an adaptation of the drafting
technique of using a spline to draw smooth curves
through a series of points
Linear Splines
f  x  f  x0   m0  x  x0 
f  x  f  x1   m1  x  x1 
x0  x  x1
x1  x  x2

f  x  f  xn  1   mn  1  x  xn  1 
where
f  xi  1   f  xi 
mi 
xi  1  xi
xn  1  x  xn
Quadratic Spline
a1 x 2  b1 x  c1
Quadratic Spline
a 2 x 2  b2 x  c 2
Quadratic Spline
a3 x 2  b3 x  c3
Quadratic Spline
a4 x 2  b4 x  c4
Quadratic Spline
a1 x 2  b1 x  c1
a2 x 2  b2 x  c2
a4 x 2  b4 x  c4
a3 x 2  b3 x  c3
APPROACH
SHOW EXAMPLE
PRESENT THEORY
Example
A well pumping at 250 gallons per minute has
observation wells located at 15, 42, 128, 317 and 433 ft
away along a straight line from the well.After three
hours of pumping, the following drawdowns in the five
wells were observed: 14.6, 10.7, 4.8,1.7 and 0.3 ft
respectively. Derive equations of each quadratic spline.
16
Draw dow n (ft)
14
12
10
8
6
4
2
0
0
100
200
300
Distanc e from w ell (ft)
400
500
Strategy
Strategy
ai 1 xi21  bi 1 xi 1  ci 1  f xi 1 
ai xi21  bi xi 1  ci  f xi 1 
 Determine how many segments
14.6
10.7
4.8
1.7
0.3
16
14
Draw dow n (ft)
and equations you have
 Determine the two equations that
go to each internal point
 Determine the equation that goes
to each end point
 Determine the equations for the
first derivatives at each internal
point
15
42
128
317
433
12
10
8
6
4
2
0
0
100
200
300
Distanc e from w ell (ft)
400
500
What are the equations?
a1x2 + b1x + c1 = y
16
Draw dow n (ft)
14
a2x2 + b2x + c2 = y
12
10
8
a3x2 + b3x + c3 = y
6
4
2
a4x2 + b4x + c4 = y
0
0
100
200
300
Distanc e from w ell (ft)
400
500
ai  1xi2 1  bi  1xi  1  ci  1  f  xi  1 
Solution
ai xi2 1  bi xi  1  ci  f  xi  1 
(42)2 a1 + 42b1 + c1 = 10.7
(42)2 a2 + 42 b2 + c2 = 10.7
16
Draw dow n (ft)
14
12
10
8
6
4
2
0
0
100
200
300
400
500
Distanc e from w ell (ft)
15
42
128
317
433
14.6
10.7
4.8
1.7
0.3
ai  1xi2 1  bi  1xi  1  ci  1  f  xi  1 
Solution
ai xi2 1  bi xi  1  ci  f  xi  1 
16,384a2 + 128b2 + c2 = 4.8
16,384 a3 + 128 b3 + c3 = 4.8
16
Draw dow n (ft)
14
12
10
8
6
4
2
0
0
100
200
300
400
500
Distanc e from w ell (ft)
15
42
128
317
433
14.6
10.7
4.8
1.7
0.3
ai  1xi2 1  bi  1xi  1  ci  1  f  xi  1 
Solution
ai xi2 1  bi xi  1  ci  f  xi  1 
16
Draw dow n (ft)
14
100,489a3 + 317b3 + c3 = 1.7
100,489a4 + 317b4 + c4 = 1.7
12
10
8
6
4
2
0
0
100
200
300
400
500
Distanc e from w ell (ft)
15
42
128
317
433
14.6
10.7
4.8
1.7
0.3
15
42
128
317
433
Solution
(15)2a1 + 15 b1 + c1 = 14.6
14.6
10.7
4.8
1.7
0.3
187,489a4 + 433b4 + c4 = 0.3
16
14
Draw dow n (ft)
Similarly,
the equations
include
the end points
12
10
8
6
4
2
0
0
100
200
300
Distanc e from w ell (ft)
400
500
15
42
128
317
433
Solution
2ai  1xi  1  bi  1  2ai xi  1  b
14.6
10.7
4.8
1.7
0.3
The first derivative at the interior
knots must be equal.
16
2a2 (128) + b2 = 2a3 (128) + b3
Draw dow n (ft)
2a1 (42) + b1 = 2a2 (42) + b2
14
12
10
8
6
4
2
0
2a3 (317) + b3 = 2a4 (317) + b4
0
100
200
300
Distanc e from w ell (ft)
400
500
15
42
128
317
433
Solution
14.6
10.7
4.8
1.7
0.3
Addition the last condition a1 = 0
You should be able to set these equations into a
matrix to solve for ai , bi, and ci for i = 1,3
....end of problem
Splines
 To ensure that the mth derivatives are
continuous at the “knots”, a spline of at
least m+1 order must be used
 3rd order polynomials or cubic splines that
ensure continuous first and second
derivatives are most frequently used in
practice
 Although third and higher derivatives may be
discontinuous when using cubic splines, they
usually cannot be detected visually and
consequently are ignored.
Splines
 The derivation of cubic splines is somewhat
involved
 First illustrate the concepts of spline
interpolation using second order
polynomials.
 These “quadratic splines” have continuous
first derivatives at the “knots”
 Note: This does not ensure equal second
derivatives at the “knots”
Quadratic Spline
1.The function must be equal at the
interior knots. This condition can be
represented as:
ai 1xi21  bi 1xi 1  ci 1  f  xi 1 
note: we are referencing the same x and f(x)
ai xi21  bi xi 1  ci  f  xi 1 
Quadratic Spline
ai  1xi2 1  bi  1xi  1  ci  1  f  xi  1 
ai xi2 1  bi xi  1  ci  f  xi  1 
This occurs between i = 2, n
Using the interior knots (n-1) this will provide
2n -2 equations.
Quadratic Spline
2. The first and last functions must pass
through the end points.
This will add two more equations.
a1x02  b1x0  c1  f  x0 
an xn2  bn xn  cn  f  xn 
We now have 2n - 2 +2 = 2n equations.
How many do we need?
Quadratic Spline
3. The first derivative at the interior knots must
be equal.
This provides another n-1 equations for
2n + n-1 =3n -1.
We need 3n
2ai  1xi  1  bi  1  2ai xi  1  b
Quadratic Spline
4. Unless we have some additional information
regarding the functions or their derivatives,
we must make an arbitrary choice in order
to successfully compute the constants.
5. Assume the second derivative is zero
at the first point. The visual interpretation
of this condition is that the first
two points will be connected by a
straight line.
a1 = 0
Cubic Splines
 Third order polynomial
 Need n+1 = 3+1 = 4 intervals
 Consequently there are 4n unknown
constants to evaluate
 What are these equations?
f i  x  ai x 3  bi x 2  ci x  di
Cubic Splines
 The function values must be equal at the




interior knots (2n -2)
The first and last functions must pass
through the end points (2)
The first derivatives at the interior knots
must be equal (n-1)
The second derivatives at the interior knots
must be equal (n-1)
The second derivative at the end knots are
zero (2)
Cubic Splines
 The function values must be equal at the




interior knots (2n -2)
The first and last functions must pass
through the end points (2)
The first derivatives at the interior knots
must be equal (n-1)
The second derivatives at the interior knots
must be equal (n-1)
The second derivative at the end knots are
zero (2)
Previous Exam Question
Given the following data, develop the
simultaneous equations for a quadratic spline.
Express your final answers in matrix form.
7.00
f(x)
0.50
4.60
1.50
3.00
6.00
5.00
4.00
f(x)
x
1
4
6
7
3.00
2.00
1.00
0.00
0
1
2
3
4
x
5
6
7
8
(4, 4.6)
7.00
Interior knots:
16a1 + 4b1 + c1 = 4.6
16a2 + 4b2 + c2 = 4.6
6.00
5.00
f(x)
4.00
3.00
2.00
1.00
0.00
0
1
2
3
4
5
6
7
8
36a2 + 6b2 + c2 = 1.5
36a3 + 6b3 + c3 = 1.5
x
x
1
4
6
7
f(x)
0.50
4.60
1.50
3.00
End conditions
a1 + b1 + c1 = 0.5
49a3 + 7b3 + c3 = 3.0
First derivative continuous at interior knots
8a1 + b1 = 8a2 + b2
12a2 + b2 = 12a3 + b3
Extra equation
a1 =0
Interior knots:
16a1 + 4b1 + c1 = 4.6
36a2 + 6b2 + c2 = 1.5
4
0

0
0

1

0
1

0
1
End conditions
a1 + b1 + c1 = 0.5 49a3 + 7b3 + c3 = 3.0
First derivative cont. at interior knots
8a1 + b1 = 8a2 + b2 12a2 + b2 = 12a3 + b3
Extra equation
a1 =0
16a2 + 4b2 + c2 = 4.6
36a3 + 6b3 + c3 = 1.5
0
0
0
0 16
4
1
0 36
6
1
0
0
0
0
1
0
0
0
0
0
0
0
0 8 1 0
0 12
1
0
0  b1  4.6
0
0 0  c1  4.6
   
0
0 0 a2   15
. 
36 6 1 b2   15
. 
    
0
0 0  c2   0.5

49 7 1  a3   3.0
   
0
0 0  b3   0 

12 1 0  c3   0 
0
0
SPECIAL NOTE
On the surface it may appear that a third order
approximation using splines would be inferior to higher
order polynomials.
Consider a situation where a spline may perform better:
A generally smooth function undergoes an abrupt change
in a region of interest.
The abrupt change
induces oscillations
in interpolating
polynomials.
In contrast,
the cubic spline
provides a
much more
acceptable
approximation
Specific Study Objectives
 Understand the fundamental difference
between regression and interpolation and
realize why confusing the two could lead to
serious problems
 Understand the derivation of linear least
squares regression and be able to assess the
reliability of the fit using graphical and
quantitative assessments.
Specific Study Objectives
 Know how to linearize data by
transformation
 Understand situations where polynomial,
multiple and nonlinear regression are
appropriate
 Understand the general matrix formulation
of linear least squares
 Understand that there is one and only one
polynomial of degree n or less that passes
exactly through n+1 points
Specific Study Objectives
 Realize that more accurate results are
obtained if data used for interpolation
is centered around and close to the
unknown point
 Recognize the liabilities and risks
associated with extrapolation
 Understand why spline functions have
utility for data with local areas of
abrupt change