Numerical Methods I: Orthogonal polynomials
and interpolation
Georg Stadler
Courant Institute, NYU
stadler@cims.nyu.edu
Nov 5/12, 2015
1 / 34
Sources/reading
Main reference:
Sections 6 and 7 in Deuflhard/Hohmann; Further reading: Section
8 in Quarteroni/Sacco/Saleri
2 / 34
Overview
Orthogonal polynomials and 3-term recurrence
relations
3 / 34
Orthogonal polynomials
Choices for inner products between functions:
Z π
(f, g) =
f (x)g(x) dx.
−π
Orthogonal functions w.r. to this inner product are cos(kx),
sin(kx).
They satisfy a three-term recurrence relation:
Tk (x) = 2 cos(x)Tk−1 (x) − Tk−2 (x),
where Tk (x) = cos(kx) or Tk (x) = sin(kx).
4 / 34
Orthogonal polynomials
Define an inner products between functions:
b
Z
ω(x)f (x)g(x) dx,
(f, g) =
a
where ω(x) > 0p
for a < x < b is a weight function. The induced
norm is kf k := (f, f ).
Examples for ω(x):
ω(x) ≡ 1,
or
ω(x) = √
1
for a = −1, b = 1.
1 − x2
5 / 34
Orthogonal polynomials
Denote by P k the polynomials of degree k.
Theorem: There exist uniquely determined orthogonal polynomials
Pk ∈ P k with leading coefficient 1. These polynomials satisfy the
3-term recurrence relation:
Pk (t) = (t + ak )Pk−1 (t) + bk Pk−2 (t),
k = 2, 3
with starting values P0 = 1, P1 = t + a1 , where
ak = −
(tPk−1 , Pk−1 )
,
(Pk−1 , Pk−1 )
bk = −
(Pk−1 , Pk−1 )
(Pk−2 , Pk−2 )
6 / 34
Orthogonal polynomials
Theorem: The orthogonal polynomials Pk ∈ P k have exactly k
simple roots in (a, b).
Sketch of proof: Denote the distinct roots
Q where Pk changes its
sign by t1 , . . . , tm , and consider Q = m
i=1 (t − ti ). Then
(Q, Pk ) 6= 0, and since Pk is orthogonal to all polynomials in
P k−1 , deg(Q) = m ≥ k.
7 / 34
Orthogonal polynomials
Theorem: The orthogonal polynomials Pk ∈ P k have exactly k
simple roots in (a, b).
Sketch of proof: Denote the distinct roots
Q where Pk changes its
sign by t1 , . . . , tm , and consider Q = m
i=1 (t − ti ). Then
(Q, Pk ) 6= 0, and since Pk is orthogonal to all polynomials in
P k−1 , deg(Q) = m ≥ k.
Note that 3-term recurrence can be use to
I
Compute polynomials Pk completely, or
I
Evaluate Pk at a point x0 .
7 / 34
Orthogonal polynomials
Chebyshev polynomials
Chebyshev polynomials, for −1 ≤ x ≤ 1:
Tk (x) = cos(k arccos(x))
3-term recursion: T0 (x) = 1, T1 (x) = x,
Tk (x) = 2xTk−1 (x) − Tk−2 (x)
I
I
for k ≥ 2.
Tk are polynomials,
Tk can be defined for all x ∈ R.
Source: Wikipedia.
8 / 34
Orthogonal polynomials
Chebyshev polynomials
In what inner product are Chebyshev polynomials orthogonal?


n 6= m
Z 1
0
1
√
Tn (x)Tm (x) dx = π
n=m=0

1 − x2
−1

π/2 n = m 6= 0
The roots of Chebyshev polynomials play an important role in
interpolation.
9 / 34
Orthogonal polynomials
Legendre polynomials
Orthogonal polynomials with for weight function ω ≡ 1, satisfy
L0 = 1, L1 = x, and
Lk+1 (x) =
k
2k + 1
xLk (x) −
L
k+1
k + 1 k−1(x)
Source: Wikipedia.
10 / 34
Orthogonal polynomials
Spherical harmonics
Orthogonal polynomials on the sphere (defined in spherical
coordinates (θ, ϕ); also satisfy a 3-term recursion
Source: Wikipedia.
11 / 34
Numerical aspects
I
Simple application of 3-term recurrence might not always be
stable due to cancellation.
I
Adjoint summation (Sec 6.3 in Deuflhard/Hohmann) can
avoid cancellation errors.
12 / 34
Overview
Interpolation and Approximation
13 / 34
Classical polynomial interpolation
Given fi := f (ti ), i = 0, . . . , n, we would like to find a polynomial
P ∈ P n such that
P (ti ) = fi .
Interpolation is thus a map from Rn+1 → P n .
Theorem: Given nodes (ti , fi ), 0 ≤ i ≤ n, with pairwise distinct
nodes ti , then there exists a unique interpolating polynomial
P ∈ P n.
To compute that polynomial, we have to choose a basis in P n .
14 / 34
Classical polynomial interpolation
Monomial basis: 1, t, t2 . . . leads to system with Vandermonde
matrix Vn .
Qn Qn
I det(Vn ) =
i=0
j=i+1 (ti − tj ) 6= 0
I
For large n, this can be a poorly conditioned system.
15 / 34
Classical polynomial interpolation
Monomial basis: 1, t, t2 . . . leads to system with Vandermonde
matrix Vn .
Qn Qn
I det(Vn ) =
i=0
j=i+1 (ti − tj ) 6= 0
I
For large n, this can be a poorly conditioned system.
Lagrange basis Li defined by Li (tj ) = δij .
I
Lagrange polynomials form an orthogonal basis in P n w.r. to
the inner product
(P, Q) :=
n
X
P (ti )Q(ti )
i=0
Pn
I
Simple interpolant: P (t) =
I
Not always useful in practice.
i=0 fi Li (t)
15 / 34
Classical polynomial interpolation
Conditioning
Theorem: Let a ≤ t0 < . . . < tn ≤ b be pairwise distinct and Li be
the corresponding Lagrange polynomials. Then the absolute
condition number of the polynomial interpolation:
C([a, b]) → P n
w.r. to the supremum norm is the Lebesgue constant
κabs = Λn = max
t∈[a,b]
n
X
|Li (t)|.
i=1
Note that the Lebesgue constant depends on n and the location of
the ti .
16 / 34
constants for the Chebyshev nodes (see Section 7.1.4)
Classical polynomial
interpolation
2i + 1) for z.= 0, ... ,n
ti = cos (
--7["
2n+ 2
Conditioning
(where the maximum was taken over [-1,1]). They grow only very slowly.
Lebesgue constants for different orders:
Table 7.1. Lebesgue constant An for equidistant and for Chebyshev nodes.
n
5
10
15
20
An for equidistant nodes
3.106292
29.890695
512.052451
10986.533993
An for Chebyshev nodes
2.104398
2.489430
2.727778
2.900825
Chebyshev nodes are the roots of the Chebyshev polynomials:
7.1.2 Hermite Interpolation
and
Divided Differences
2i + 1
If one is only interested
ti = cosin the interpolating
π , forpolynomial
i = 0, . . . ,Pnat a single po2n + 2
sition t, then the recursive computation
of P(t) turns out to be the most
effective method. It is based on the following simple observation, the Aitken
lemma.
Lemma 7.4 The interpolating polynomial P
the recurrence relation
=
P(f Ito,· .. , tn) satisfies
17 / 34
Classical polynomial interpolation
Conditioning
Lebesgue constant for n = 10, uniform vs. Chebyshev nodes:
18 / 34
Classical polynomial interpolation
Conditioning
Lebesgue constant for n = 40, uniform vs. Chebyshev nodes:
19 / 34
Classical polynomial interpolation
Aitken Lemma
The interpolating polynomial P = P (f |t0 , . . . , tn ) satisfies the
recurrence relation:
P (f |t0 , . . . , tn ) =
(t0 − t)P (f |t1 , . . . , tn ) − (tn − t)P (f |t0 , . . . , tn−1 )
t0 − tn
Proof: The rhs (say ϕ(t)) is a polynomial of order n, and it satisfies
the interpolation conditions f (ti ) = ϕ(ti ) for all i = 0, . . . , n.
20 / 34
P lO
->
Pll
Classical polynomial interpolation
Neville scheme
Pn-1,o
->
->
Pn-1,n-l
PnO
->
->
Pn,n-l
->
Pnn
Neville scheme allows to compute value of P (f |t0 , . . . , tn )(t)
given by
iteratively.
Pia
Ii for i = 0, ... ,n
Define Pik := P (f |ti−k , . .t .-, titi )(t) for i ≥ k, and use (Neville
Pi k-l +
(Pi k-l - Pi- 1 k-d for i k. (7.4)
algorithm):
,
ti - ti-k'
,
I Pi0 = 7.5
fi Computation of sin 62° from the nodes
Example
I
t−ti
(Pi,k−1
− P...i−1,k−1
) for
i ≥ k.
Pik = Pi,k−1
(50°, +
sin t 50°),
(55°,
sin 55°),
, (70°, sin
70°)
i −ti−k
by the Aitken-Neville algorithm.
ti
50°
55°
60°
65°
70°
sin ti
0.7660444
0.8191520
0.8660254
0.9063078
0.9396926
0.8830292
0.8829293
0.8829661
0.8829493
0.8829465
0.8829476
The recursive structure of the interpolation polynomials according to
Lemma 7.4 can also be utilized for the determination of the entire polynomial PC! I to, ... ,tn). This is also true for the generalized interpolation
21 / 34
Hermite interpolation
Assume
a = t0 ≤ t1 ≤ . . . ≤ tn = b
with possibly duplicated nodes. If the node ti occurs k times, the
corresponding node values correspond to f (ti ), f 0 (ti ), . . . , f k−1 (ti ).
The Hermite interpolation polynomial p(x) is a polynomial of order
n, which coincides with the nodal values (and, for duplicated
nodes, derivatives at nodal values) at the nodes.
22 / 34
Hermite interpolation
Theorem: (somewhat loosely formulated version) Given n + 1
nodes and nodal values (possibly of derivatives), then there exists a
unique Hermite polynomial p ∈ P n .
23 / 34
Hermite interpolation
Theorem: (somewhat loosely formulated version) Given n + 1
nodes and nodal values (possibly of derivatives), then there exists a
unique Hermite polynomial p ∈ P n .
Examples:
I
All t0 = . . . = tn .
23 / 34
Hermite interpolation
Theorem: (somewhat loosely formulated version) Given n + 1
nodes and nodal values (possibly of derivatives), then there exists a
unique Hermite polynomial p ∈ P n .
Examples:
I
All t0 = . . . = tn .
I
Cubic Hermite interpolation: Nodes: t0 = t1 < t2 = t3 ,
Values: f (t0 ), f 0 (t0 ), f (t1 ), f 0 (t1 ).
23 / 34
Hermite interpolation
Theorem: (somewhat loosely formulated version) Given n + 1
nodes and nodal values (possibly of derivatives), then there exists a
unique Hermite polynomial p ∈ P n .
Examples:
I
All t0 = . . . = tn .
I
Cubic Hermite interpolation: Nodes: t0 = t1 < t2 = t3 ,
Values: f (t0 ), f 0 (t0 ), f (t1 ), f 0 (t1 ).
I
locally cubic Hermite interpolation.
23 / 34
Classical polynomial interpolation
Version of Aitken lemma for Hermite interpolation
The (Hermite) interpolating polynomial P = P (f |t0 , . . . , tn )
satisfies, if ti 6= tj , the recurrence relation:
P (f |t0 , . . . , tn ) =
(ti − t)P (f |t0 , . . . , tˆj , . . . , tn ) − (tj − t)P (f |t0 , . . . , tˆi , . . . , tn )
,
ti − tj
where the hat symbol indicates that the corresponding node is
omitted.
24 / 34
Classical polynomial interpolation
Newton polynomial basis
The Newton basis ω0 , . . . , ωn is given by
ωi (t) :=
i−1
Y
(t − tj ) ∈ P i .
j=0
The leading coefficient an of the interpolation polynomial of f
P (f |t0 , . . . , tn ) = an xn + . . .
is called the n-th divided difference, [t0 , . . . , tn ]f := an .
25 / 34
Classical polynomial interpolation
Newton polynomial basis
Theorem: For f ∈ C n , the interpolation polynomial
P (f |t0 , . . . , tn ) is given by
P (t) =
n
X
[t0 , . . . , ti ]f ωi (t).
i=0
If f ∈ C n+1 , then
f (t) = P (t) + [t0 , . . . , tn ]f ωn+1 (t).
This property allows to estimate the interpolation error.
26 / 34
Classical polynomial interpolation
Divided differences
The divided differences [t0 , . . . , tn ]f satisfy the following
properties:
I
[t0 , . . . , tn ]P = 0 for all P ∈ P n−1 .
I
If t0 = . . . = tn :
[t0 , . . . , tn ]f =
I
f (n) (t0 )
n!
The following recurrence relation holds for ti 6= tj :
[t0 , . . . , tˆi , . . . , tn ]f − [t0 , . . . , tˆj , . . . , tn ]f
[t0 , . . . , tn ]f =
tj − ti
I
1 (n)
[t0 , . . . , tn ]f = n!
f (τ ) with an a ≤ τ ≤ b, and the divided
differences depend continuously on the nodes.
27 / 34
Classical polynomial interpolation
Divided differences
Let us use divided differences to compute the coefficients for the
Newton basis for the cubic interpolation polynomial p that satisfies
p(0) = 1, p(0.5) = 2, p(1) = 0, p(2) = 3.
ti
0
0.5
1
2
[t0 ]f
[t1 ]f
[t2 ]f
[t3 ]f
=1
=2
=0
=3
−[t0 ]f
[t0 t1 ]f = [t1 ]f
=2
t1 −t0
[t2 ]f −[t1 ]f
[t1 t2 ]f = t2 −t1 = −4
−[t2 ]f
[t2 t3 ]f = [t3 ]f
=3
t3 −t2
[t0 t1 t2 ]f = −6
[t1 t2 t3 ]f = 14
3
16
3
Thus, the interpolating polynomial is
p(t) = 1 + 2t + (−6)t(t − 0.5) +
16
t(t − 0.5)(t − 1).
3
28 / 34
Classical polynomial interpolation
Divided differences
Let us now use divided differences to compute the coefficients for
the Newton basis for the cubic interpolation polynomial p that
satisfies p(0) = 1, p0 (0) = 2, p00 (0) = 1, p(1) = 3.
ti
0
0
0
1
[t0 ]f
[t0 ]f
[t0 ]f
[t3 ]f
=1
=1
=1
=3
[t0 t1 ]f = p0 (0) = 2
[t1 t2 ]f = p0 (0) = 2
−[t0 ]f
=2
[t2 t3 ]f = [t3 ]f
t3 −t0
00
[t0 t1 t2 ]f = p 2!(0) =
0
1
2
− 12
Thus, the interpolating polynomial is
1
1
p(t) = 1 + 2t + t2 + (− )t3
2
2
29 / 34
Classical polynomial interpolation
Approximation error
If f ∈ C (n+1) , then
f (t) − P (f |t0 , . . . , tn )(t) =
f (n+1) (τ )
ωn+1 (t)
(n + 1)!
for an appropriate τ = τ (t), a < τ < b.
In particular, the error depends on the choice of the nodes.
Thus, one can try to minimize
max |ωn+1 (t)|,
a≤t≤b
which is achieved by choosing the nodes as the roots of the
Chebyshev polynomial of order (n + 1).
30 / 34
Splines
Assume (l + 2) pairwise disjoint nodes:
a = t0 < t1 < . . . < tl+1 = b.
A spline of degree k − 1 (order k) is a function in C k−2 which on
each interval [ti , ti+1 ] coincides with a polynomial in P k−1 .
Most important examples:
I
linear splines, k = 2
I
cubic splines, k = 4
31 / 34
Splines
220
Cubic
splines lookand
smooth:
7. Interpolation
Approximation
So
a
=
s
to
Figure 7.13. Cubic splines, order k = 4.
32 / 34
Splines
B-splines
B-splines are a basis in the spline space that:
222
I
has local support
I
satisfies a 3-term recursion
I
non-negative
7. Interpolation and Approximation
Figure 7.14. B-splines of order k = 1,2,3.
33 / 34
Splines
B-splines
I
Coefficients for interpolation with the B-spline basis can be
computed efficiently using the De Boor algorithm.
I
Splines are essential in Computer Aided Design (CAD).
I
Also important in CAD: Bezier curves (these do not
interpolate points and have useful geometrical properties).
34 / 34