Lecture Notes for Math-CSE 451:
Introduction to Numerical Computation
Wen Shen
2011
2
These notes are used by myself in the course. They are provided to students. These
notes are self-contained, and covers almost everything we go through in class. However,
if a student shall be interested in more detailed discussion, he/she should consult the
recommended textbook.
Text: Guide to Scientific Computing, 2nd edition, by Peter Turner. CRC Press, ISBN
0-8493-1242-6.
Chapter 3
Piece-wise polynomial
interpolation. Splines
3.1
Introduction
Usage:
• visualization of discrete data
• graphic design –VW car design
Requirement:
• interpolation
• certain degree of smoothness
Disadvantages of polynomial interpolation Pn (x)
• n-time differentiable. We do not need such high smoothness;
• big error in certain intervals (esp. near the ends);
• no convergence result;
• Heavy to compute for large n
Suggestion: use piecewise polynomial interpolation.
Problem setting : Given a set of data
x t0 t1 · · ·
y y0 y1 · · ·
3
tn
yn
4
CHAPTER 3. SPLINES
Find a function S(x) which interpolates the points (ti , yi )ni=0 .
The set t0 < t1 < · · · < tn are called knots. Note that they need to be ordered.
S(x) consists of piecewise polynomials

S0 (x),




S1 (x),
S(x)=
˙
..

.



Sn−1 (x),
t0 ≤ x ≤ t1
t1 ≤ x ≤ t2
tn−1 ≤ x ≤ tn
S(x) is called a spline of degree n, if
• Si (x) is a polynomial of degree n;
• S(x) is (n − 1) times continuous differentiable, i.e., for i = 1, 2, · · · , n − 1 we have
Si−1 (ti ) = Si (ti ),
′
(ti ) = Si′ (ti ),
Si−1
..
.
(n−1)
(n−1)
Si−1 (ti ) = Si
(ti ),
Commonly used ones:
• n = 1: linear spline (simplest)
• n = 2: quadratic spline (less popular)
• n = 3: cubic spline (most used)
If you are given a function which is piecewise polynomial, could you check if it is a spline
of certain degree? All you need to do is to check whether all the conditions are satisfied.
Example 1. Determine whether this function is a first-degree spline function:

x ∈ [−1, 0]
 x
S(x) =
1−x
x ∈ (0, 1)

2x − 2
x ∈ [1, 2]
Answer. Check all the properties of a linear spline.
• Linear polynomial for each piece: OK.
• S(x) is continuous at inner knots: At x = 0, S(x) is discontinuous, because from
the left we get 0 and from the right we get 1.
5
3.2. LINEAR SPLINE
Therefore this is NOT a linear spline.
Example 2. Determine whether the following function is a quadratic spline:
 2
x ∈ [−10, 0]
 x
2
S(x) =
−x
x ∈ (0, 1)

1 − 2x
x≥1
Answer. Let’s label them:
Q0 (x) = x2 ,
Q1 (x) = −x2 ,
Q2 (x) = 1 − 2x.
We now check all the conditions, i.e, the continuity of Q and Q′ at inner knots 0, 1:
Q0 (0) = 0,
Q1 (0) = 0,
OK
Q1 (1) = −1,
Q2 (1) = −1,
OK
Q′0 (0)
Q′1 (0) = 0,
Q′2 (1) = −2,
OK
Q′1 (1)
= 0,
= −2,
OK
It passes all the test, so it is a quadratic spline.
3.2
Linear Spline
We consider the case when n = 1. Piecewise linear interpolation, i.e., straight line
between 2 neighboring points. See Figure 3.1.
S(x) 6
y1
×
y2
×
y0
×
y3
×
-
t0
t1
t2
t3
Figure 3.1: Linear splines
So
Si (x) = ai + bi x,
i = 0, 1, ·, n − 1
x
6
CHAPTER 3. SPLINES
Requirements:
S0 (t0 ) = y0
Si−1 (ti ) = Si (ti ) = yi ,
i = 1, 2, · · · , n − 1
Sn−1 (tn ) = yn .
Easy to find: write the equation for a line through two points: (ti , yi ) and (ti+1 , yi+1 ),
Si (x) = yi +
yi+1 − yi
(x − ti ),
ti+1 − ti
Accuracy Theorem for linear spline:
i = 0, 1, · · · , n − 1.
Assume t0 < t1 < t2 < · · · < tn , and let
h = max(ti+1 − ti )
i
Let f (x) be a given function, and let S(x) be a linear spline that interpolates f (x) s.t.
S(ti ) = f (ti ),
i = 0, 1, · · · , n
We have the following, for x ∈ [t0 , tn ],
(1) If f ′ exists and is continuous, then
|f (x) − S(x)| ≤
1
h max f ′ (x) .
x
2
(2) If f ′′ exits and is continuous, then
1
|f (x) − S(x)| ≤ h2 max f ′′ (x) .
x
8
To minimize error, it is obvious that one should add more knots where the function has
large first or second derivative.
3.3
Quadratics spline
This type of splines is not much used. Cubic splines is usually favored for its minimum
curvature property (see next section).
Given a set of knots t0 , t1 , · · · , tn , and the data y0 , y1 , · · · , yn , we seek piecewise polynomial representation

Q0 (x)
t0 ≤ x ≤ t1



 Q2 (x)
t1 ≤ x ≤ t2
Q(x) =
..

.



Qn−1 (x) tn−1 ≤ x ≤ tn
7
3.4. NATURAL CUBIC SPLINE
where Qi (x) (i = 0, 1, · · · , n − 1) are quadratic polynomials. In general, Qi (x) = ai x2 +
bi x + ci . Total number of unknowns= 3n.
Conditions we impose on Qi :
Qi (ti ) = yi ,
Qi (ti+1 ) = yi+1 ,
Q′i (ti )
=
Q′i+1 (ti ),
i = 0, 1, · · · , n − 1 :
2n conditions
i = 1, 2, · · · , n − 1 :
n − 1 conditions.
Total number of conditions: 2n + (n − 1) = 3n − 1.
An extra condition could be imposed. For example: Q′0 (t0 ) = 0 or Q′′0 (t0 ) = 0, depending
on the specific problem.
Construction of Qi (t): Since Q′ is continuous, we set
zi = Q′ (ti )
We don’t know these zi ’s, they are the unknowns, and will be computed later.
Then, each Qi must satisfy the conditions:
Qi (ti ) = yi ,
Q′i (ti ) = zi ,
Q′i (ti+1 ) = zi+1 ,
Qi (ti+1 ) = yi+1 .
(3.1)
0 ≤ i ≤ n − 1.
(3.2)
Using the first 3 conditions, we obtain the polynomials
Qi (x) =
zi+1 − zi
(x − ti )2 + zi (x − ti ) + yi ,
2(ti+1 − ti )
It is easy to verify the first 3 conditions in (3.1).
To find the values for zi , we now use the 4th condition in (3.1). This gives us
yi+1 − yi
,
0 ≤ i ≤ n − 1.
zi+1 = −zi + 2
ti+1 − ti
(3.3)
Given a z0 , all the zi ’s can now be constructed.
We now summarize the algorithm:
• Given z0 , compute zi using (3.3).
• Compute Qi by using (3.2).
3.4
Natural cubic spline
Given t0 < t1 < · · · < tn , we define the cubic spline S(x) = Si (x) for ti ≤ x ≤ ti+1 . We
′′ (t ) = 0,
require that S, S ′ , S ′′ are all continuous. If in addition we require S0′′ (t0 ) = Sn−1
n
then it is called natural cubic spline.
Write
Si (x) = ai x3 + bi x2 + ci x + di ,
i = 0, 1, · · · , n − 1
8
CHAPTER 3. SPLINES
Total number of unknowns= 4 · n.
Equations we have
equation
(1) Si (ti ) = yi ,
i = 0, 1, · · · , n − 1
(2) Si (ti+1 ) = yi+1 ,
i = 0, 1, · · · , n − 1
′ (t ),
(3) Si′ (ti+1 ) = Si+1
i
i = 0, 1, · · · , n − 2
′′ (t ),
(4) Si′′ (ti+1 ) = Si+1
i
i = 0, 1, · · · , n − 2
(5) S0′′ (t0 ) = 0,
′′ (t ) = 0,
(6) Sn−1
n
How to compute Si (x)? We know:
Si
Si′
Si′′
:
:
:

number 





n





n



n−1



n−1 






1




1.
total = 4n.
polynomial of degree 3
polynomial of degree 2
polynomial of degree 1
procedure:
• Start with Si′′ (x), they are all linear, one can use Lagrange form,
• Integrate Si′′ (x) twice to get Si (x), you will get 2 integration constant
• Determine these constants by (2) and (1). Various tricks on the way...
Details: Define zi as
zi = S ′′ (ti ),
i = 1, 2, · · · , n − 1,
z0 = zn = 0
NB! These zi ’s are our unknowns.
Introduce the notation hi = ti+1 − ti .
Lagrange form
Si′′ (x) =
zi+1
zi
(x − ti ) − (x − ti+1 ).
hi
hi
Then
Si′ (x) =
Si (x) =
zi+1
(x − ti )2 −
2hi
zi+1
(x − ti )3 −
6hi
zi
(x − ti+1 )2 + Ci − Di
2hi
zi
(x − ti+1 )3 + Ci (x − ti ) − Di (x − ti+1 ).
6hi
(You can check by yourself that these Si , Si′ are correct.)
Interpolating properties:
(1). Si (ti ) = yi gives
yi = −
1
zi
(−hi )3 − Di (−hi ) = zi h2i + Di hi
6hi
6
⇒
Di =
yi
hi
− zi
hi
6
9
3.4. NATURAL CUBIC SPLINE
(2). Si (ti+1 ) = yi+1 gives
yi+1 =
zi+1 3
h + Ci hi ,
6hi i
⇒
Ci =
yi+1 hi
− zi+1 .
hi
6
We see that, once zi ’s are known, then (Ci , Di )’s are known, and so Si , Si′ are known.
yi+1 hi
zi+1
zi
(x − ti )3 −
(x − ti+1 )3 +
− zi+1 (x − ti )
6hi
6hi
hi
6
yi
hi
−
− zi (x − ti+1 ).
hi
6
zi
yi+1 − yi zi+1 − zi
z
i+1
hi .
(x − ti )2 −
(x − ti+1 )2
−
Si′ (x) =
2hi
2hi
hi
6
Si (x) =
How to compute zi ’s? Last condition that’s not used yet: continuity of S ′ (x), i.e.,
′
Si−1
(ti ) = Si′ (ti ),
i = 1, 2, · · · , n − 1
We have
yi+1 − yi zi+1 − zi
zi
−
hi
(−hi )2 +
2hi
hi
6
| {z }
bi
1
1
= − hi zi+1 − hi zi + bi
6
3
1
1
′
Si−1
(ti ) =
zi−1 hi−1 + zi hi−1 + bi−1
6
3
Si′ (ti ) = −
Set them equal to each other, we get
hi−1 zi−1 + 2(hi−1 + hi )zi + hi zi+1 = 6(bi − bi−1 ),
z0 = zn = 0.
i = 1, 2, · · · , n − 1
In matrix-vector form:
H · ~z = ~b
where


2(h0 + h1 )
h


h1
2(h1 + h2 )
h2




h2
2(h2 + h3 )
h3


H=

..
..
..


.
.
.




hn−3
2(hn−3 + hn−2 )
hn−2
hn−2
2(hn−2 + hn−1 )
10
CHAPTER 3. SPLINES
and

z1
z2
z3
..
.










~z = 
,




zn−2 
zn−1

6(b1 − b0 )
6(b2 − b1 )
6(b3 − b2 )
..
.








~b = 
.




6(bn−2 − bn−3 )
6(bn−1 − bn−2 )
Here, H is a tri-diagonal matrix, symmetric, and diagonal dominant
2 |hi−1 + hi | > |hi | + |hi−1 |
which implies unique solution for ~z.
See slides for Matlab codes and solution graphs.
Theorem on smoothness of cubic splines. If S is the natural cubic spline function
that interpolates a twice-continuously differentiable function f at knots
a = t0 < t1 < · · · < tn = b
then
Z
a
Note that
R
b
2
S (x) dx ≤
′′
Z
a
b
2
f ′′ (x) dx.
(f ′′ )2 is related to the curvature of f .
Cubic spline gives the least curvature, ⇒ most smooth, so best choice.
Proof. Let
g(x) = f (x) − cS(x)
Then
g(ti ) = 0,
i = 0, 1, · · · , n
and f ′′ = S ′′ + g ′′ , so
⇒
(f ′′ )2 = (S ′′ )2 + (g′′ )2 + 2S ′′ g′′
Z b
Z b
Z b
Z b
′′ 2
′′ 2
′′ 2
2S ′′ g′′ dx
(g ) dx +
(S ) dx +
(f ) dx =
a
a
a
Claim that
Z
b
S ′′ g′′ dx = 0
a
then this would imply
Z
b
(f ′′ )2 dx ≥
a
Z
b
(S ′′ )2 dx
a
a
11
3.4. NATURAL CUBIC SPLINE
and we are done.
Proof of the claim: Using integration-by-parts,
Z
b
a
b Z b
S ′′′ g′ dx
S ′′ g′′ dx = S ′′ g′ −
a
a
Since g(a) = g(b) = 0, so the first term is 0. For the second term, since S ′′′ is piecewise
constant. Call
ci = S ′′′ (x), for x ∈ [ti , ti+1 ].
Then
Z
b
′′′ ′
S g dx =
a
(b/c g(ti ) = 0).
n−1
X
i=0
ci
Z
ti+1
′
g (x) dx =
ti
n−1
X
i=0
ci [g(ti+1 ) − g(ti )] = 0,