Numerical Methods I: Numerical
integration/quadrature
Georg Stadler
Courant Institute, NYU
stadler@cims.nyu.edu
Nov 19/Dec 3, 2015
1 / 34
Overview
Review of interpolation and approximation
2 / 34
Classic Polynomial interpolation
Function values: Given fi := f (ti ), i = 0, . . . , n, we would like to
find a polynomial P ∈ P n such that
P (ti ) = fi .
Interpolation is thus a map from Rn+1 → P n .
Function values and derivatives (Hermite interpolation): If a node
tj appears more than once, the convention is that besides function
values fj , we also want to interpolate higher-order derivatives.
Theorem: Given nodes (ti , fi ), 0 ≤ i ≤ n, then there exists a
unique interpolating polynomial P ∈ P n .
3 / 34
Classical polynomial interpolation
To compute that polynomial, we have to choose a basis in P n .
I
I
I
Monomial basis: 1, t, t2 . . .
(
1 if i = j,
Lagrange basis: Li (t) such that Li (tj ) = δij :=
0 if i 6= j.
Note that a Lagrange basis also exists for Hermite
interpolation; the basis polynomials then satisfy, e.g., for
ti = ti+1 : Li (tj ) = δij and L0i+1 (tj ) = δij .
Q
Newton basis: ωi (t) = i−1
j=0 (t − tj ) ∈ P i .
4 / 34
Classical polynomial interpolation
Divided differences are the coefficients for the Newton basis
The divided differences [t0 , . . . , tn ]f satisfy the following
properties:
I
[t0 , . . . , tn ]P = 0 for all P ∈ P n−1 .
I
If t0 = . . . = tn :
[t0 , . . . , tn ]f =
I
f (n) (t0 )
n!
The following recurrence relation holds for ti 6= tj :
[t0 , . . . , tˆi , . . . , tn ]f − [t0 , . . . , tˆj , . . . , tn ]f
[t0 , . . . , tn ]f =
tj − ti
I
1 (n)
[t0 , . . . , tn ]f = n!
f (τ ) with an a ≤ τ ≤ b, and the divided
differences depend continuously on the nodes.
5 / 34
Classical polynomial interpolation
Divided differences
Let us use divided differences to compute the coefficients for the
Newton basis for the cubic interpolation polynomial p that satisfies
p(0) = 1, p(0.5) = 2, p(1) = 0, p(2) = 3.
ti
0
0.5
1
2
[t0 ]f
[t1 ]f
[t2 ]f
[t3 ]f
=1
=2
=0
=3
−[t0 ]f
[t0 t1 ]f = [t1 ]f
=2
t1 −t0
[t2 ]f −[t1 ]f
[t1 t2 ]f = t2 −t1 = −4
−[t2 ]f
[t2 t3 ]f = [t3 ]f
=3
t3 −t2
[t0 t1 t2 ]f = −6
[t1 t2 t3 ]f = 14
3
16
3
Thus, the interpolating polynomial is
p(t) = 1 + 2t + (−6)t(t − 0.5) +
16
t(t − 0.5)(t − 1).
3
6 / 34
Classical polynomial interpolation
Divided differences
Let us now use divided differences to compute the coefficients for
the Newton basis for the cubic interpolation polynomial p that
satisfies p(0) = 1, p0 (0) = 2, p00 (0) = 1, p(1) = 3.
ti
0
0
0
1
[t0 ]f
[t0 ]f
[t0 ]f
[t3 ]f
=1
=1
=1
=3
[t0 t1 ]f = p0 (0) = 2
[t1 t2 ]f = p0 (0) = 2
−[t0 ]f
=2
[t2 t3 ]f = [t3 ]f
t3 −t0
00
[t0 t1 t2 ]f = p 2!(0) =
0
1
2
− 12
Thus, the interpolating polynomial is
1
1
p(t) = 1 + 2t + t2 + (− )t3
2
2
7 / 34
Classical polynomial interpolation
Divided differences
Theorem: For f ∈ C n , the interpolation polynomial
P (f |t0 , . . . , tn ) is given by
P (t) =
n
X
[t0 , . . . , ti ]f ωi (t).
i=0
If f ∈ C n+1 , then
f (t) = P (t) + [t0 , . . . , tn ]f ωn+1 (t).
This property allows to estimate the interpolation error.
8 / 34
Classical polynomial interpolation
Approximation error
If f ∈ C (n+1) , then
f (t) − P (f |t0 , . . . , tn )(t) =
f (n+1) (τ )
ωn+1 (t)
(n + 1)!
for an appropriate τ = τ (t), a < τ < b.
In particular, the error depends on the choice of the nodes.
Thus, one can try to minimize
max |ωn+1 (t)|,
a≤t≤b
which is achieved by choosing the nodes as the roots of the
Chebyshev polynomial of order (n + 1).
9 / 34
Classical polynomial interpolation
Conditioning
Theorem: Let a ≤ t0 < . . . < tn ≤ b be pairwise distinct and Li be
the corresponding Lagrange polynomials. Then the absolute
condition number of the polynomial interpolation:
C([a, b]) → P n
w.r. to the supremum norm is the Lebesgue constant
κabs = Λn = max
t∈[a,b]
n
X
|Li (t)|.
i=1
Note that the Lebesgue constant depends on n and the location of
the ti .
10 / 34
constants for the Chebyshev nodes (see Section 7.1.4)
Classical polynomial
interpolation
2i + 1) for z.= 0, ... ,n
ti = cos (
--7["
2n+ 2
Conditioning
(where the maximum was taken over [-1,1]). They grow only very slowly.
Lebesgue constants for different orders:
Table 7.1. Lebesgue constant An for equidistant and for Chebyshev nodes.
n
5
10
15
20
An for equidistant nodes
3.106292
29.890695
512.052451
10986.533993
An for Chebyshev nodes
2.104398
2.489430
2.727778
2.900825
Chebyshev nodes are the roots of the Chebyshev polynomials:
7.1.2 Hermite Interpolation
and
Divided Differences
2i + 1
If one is only interested
ti = cosin the interpolating
π , forpolynomial
i = 0, . . . ,Pnat a single po2n + 2
sition t, then the recursive computation
of P(t) turns out to be the most
effective method. It is based on the following simple observation, the Aitken
lemma.
Lemma 7.4 The interpolating polynomial P
the recurrence relation
=
P(f Ito,· .. , tn) satisfies
11 / 34
Classical polynomial interpolation
Conditioning and Approximation
I
Condition number reflects the sensitivity with respect to
perturbations; since it’s based on the supremum norm, this is
the “worst case” sensitivity.
I
Approximation depends on actual function f . Approximation
can be good even if the condition number is large (depends on
higher derivatives of f )
Online interpolation demo:
http://martin-thoma.com/html5/polynom-interpolation.htm
12 / 34
Splines
Assume (l + 2) pairwise disjoint nodes:
a = t0 < t1 < . . . < tl+1 = b.
A spline of degree k − 1 (order k) is a function in C k−2 which on
each interval [ti , ti+1 ] coincides with a polynomial in P k−1 .
Most important examples:
I
linear splines, k = 2
I
cubic splines, k = 4
13 / 34
Splines
220
Cubic
splines lookand
smooth:
7. Interpolation
Approximation
So
a
=
s
to
Figure 7.13. Cubic splines, order k = 4.
14 / 34
Overview
Numerical integration
15 / 34
Sources/reading
Main reference:
Section 9 in Deuflhard/Hohmann;
16 / 34
Numerical integration
We want to approximate the definite integral
I(f ) =
Iab (f )
Z
=
b
f (t) dt
a
numerically. Properties of the Riemann integral:
I
I is linear
I
positive, i.e., if f is nonnegative, then I(f ) is nonnegative.
I
additive w.r. to the interval bounds: Iac = Iab + Ibc
17 / 34
Numerical integration
Conditioning
Lets study the map
Z
([a, b], f ) →
b
f (t) dt,
a
where we use the L1 -norm for f . The absolute and relative
condition numbers of integration are:
κabs = 1,
I(|f |)
.
κrel =
I(f )
So, integration is harmless w.r. to the absolute condition number,
and problematic w.r. to the relative condition number if I(f ) is
small and f changes sign.
18 / 34
Numerical integration
We are looking for a map
(
C([a, b]) → R
Iˆ :
ˆ )
f
7→ I(f
ˆ )| is small.
such that |I(f ) − I(f
Example: Trapezoidal rule.
General quadrature formula:
ˆ )=
I(f
n
X
λi f (ti ),
i=0
with weights λi and nodal points ti , i = 0, 1, . . . , n.
19 / 34
Numerical integration
Newton-Cotes formulas
Replace f by easy-to-integrate approximation fˆ, and set
ˆ ) := I(fˆ).
I(f
Given fixed nodes t0 , . . . , tn , use polynomial approximation
fˆ = P (f |t0 , . . . , tn ) =
n
X
f (ti )Lin (t)
i=0
Thus:
ˆ ) = (b − a)
I(f
n
X
λin f (ti ),
i=0
where λin =
1
b−a
Rb
a
Lin (t) dt
20 / 34
lb
1
n t - tj
1 in n
j
Numerical integration
Ain = - II-dt= II-ds.
b- a
ti - tj
n
i - j
S -
a
Newton-Cotes formulas
0
j=O
j#i
j=O
j#i
Theorem:
For Ain,
(n +which
1) pairwise
distinct
points,
there
exists only
exactly
The weights
are independent
of the
interval
boundaries,
to be computed
once,
respectively,
one have
quadrature
formula
that
is exactgiven
for once.
all p In
∈ Table
P n . 9.1, we have
listed them up to order n
= 4. The weights, and therefore also the quadra-
Uniformly
spaced
ture formulas,
are points:
always positive for the orders n = 1, ... , 7. Higher orders
Z bstarting
Z weights may occur.
are less attractive, since
with n = 8, negative
Y t−
1 nY
ti the characteristic
s − j (9.3), up
In this case, the 1
Lebesgue constant
is
λin =
ds
dt =that we havequantity
to the normalization
factor
(b
a)-I.
Note
already
b−a a
ti − tj
n 0
i − j encouni6=jin Section 7.1 as the condition
i6=j number of the
tered the Lebesgue constant
polynomial interpolation.
These weights are independent of a and b.
Table 9.1. Newton-Cotes weights
n
AOn , ... , Ann
1
"2 "2
1
1
(;
2
1
3
4
Error
1" (,)
1
4
(;
3
1
3:0
1
5
8 8 8 8
7
90
32
90
12
90
for n = 1, ... ,4.
Trapezoidal rule
Simpson's rule, Kepler's barrel rule
(;
3
Ain
Name
32
90
7
90
f(4)(,)
Newton's 3/8-rule
f(6)(,)
Milne's rule
In Table 9.1, we have already assigned the respective approximation erWeights
are positive up to order 7, then some become negative.
rors to the Newton-Cotes formulas (for sufficiently smooth integrands),
21 / 34
Numerical integration
Newton-Cotes formulas
Proof of error terms is based on
I
a variant of the mean value theorem for integrals
I
the Newton remainder form from interpolation, e.g., for linear
interpolation
f (t) − P (t) = [t, a, b]f · (t − a)(t − b) =
f 00 (τ )
(t − a)(t − b).
2
For piece-wise polynomial approximations (e.g., the trapezoidal
rule), analogous estimates as shown in the table hold.
22 / 34
Numerical integration
Gauss-Christoffel quadrature
Now, let’s allow the nodes t0 , . . . , tn to vary. The best we can
hope for is exact interpolation up to polynomials of degree 2n + 1
(based on a non-rigorous counting argument).
Also, for generalization, we consider quadrature of weighted
integrals, with a positive weight function ω(t):
Z
I(f ) =
b
ω(t)f (t) dt
a
Theorem: If Iˆ is exact for polynomials p ∈ P 2n+1 (for ω-weighted
polynomials), then
Pj+1 (t) = (t − τ0j ) · . . . · (t − τjj ) ∈ P n+1
are orthogonal with respect to the scalar product induced by ω(t).
23 / 34
Numerical integration
Gauss-Christoffel quadrature
Thus:
I
one must choose the roots of the orthogonal polynomials
(which are single roots)
I
the weights are uniquely determined and yield exact
integration for polynomials up to degree n. . . but:
Theorem: Let τ0n , . . . , τnn be the roots of the (n + 1)st orthogonal
polynomial for the weight ω. Then any quadrature formula Iˆ is
exact for polynomials up to order n if and only if it is exact up to
order 2n + 1.
24 / 34
Numerical integration
Gauss-Christoffel quadrature
9.3. Gauss-Christoffel Quadrature
285
Table 9.3. Commonly occurring classes of orthogonal polynomials.
w(t)
Interval I
=
[a, b]
Orthogonal polynomials
1
v'1-t 2
[-1,1]
Chebyshev polynomials Tn
e- t
[0,00]
Laguerre polynomials Ln
e- t2
[-00,00]
Hermite polynomials Hn
1
[-1,1]
Legendre polynomials P n
quadrature
rules
are usually For
prefixed
withintegrands,
“Gauss-”, the
(wCorresponding
== 1) is only used
in special
applications.
general
trapezoidal
sum extrapolation,
about which
we shall learn in the next seci.e., “Gauss-Legendre
quadrature”,
or “Gauss-Chebyshev
tion,
is superior. However, the weight function of the Gauss-Chebyshev
quadrature”.
quadrature is weakly singular at t = ±1, so that the trapezoidal rule
is not applicable. Of particular interest are the Gauss-Hermite and the
Gauss-Laguerre quadrature, which allow the approximation of integrals
over infinite intervals (and even solve exactly for polynomials P E P 2n +1)'
25 / 34
Numerical integration
Gauss-Legendre points/weights for interval [−1, 1]
26 / 34
Numerical integration
Gauss points in 2D
Tensor-product Gauss points. Weights are products of 1D-weights.
27 / 34
Numerical integration
I
I
I
Accuracy in Gauss-(Chebyshev, Laguerre, Hermite,
Legendre,. . . ) can only be improved by increasing number of
points
Of particular interest are quadrature points for infinite
intervals (Laguerre, Hermite)
Interval partitioning superior, but only possible for ω ≡ 1
(Gauss-Legendre or Gauss-Lobatto)
3
4
1
2
0
2D-Gauss-Lobatto integration points (also used as interpolation points).
28 / 34
Numerical integration
Interval partitioning
Split interval [a, b] into subintervals of size h = (b − a)/n. Basic
trapezoidal sum:
!
n−1
X
1
n
(f (a) − f (b)) +
f (a + ih)
T (h) = T = h
2
i=1
We can now think of what happens as h → 0 (i.e., more
subintervals).
Theorem: For f ∈ C 2m+1 holds:
Z
T (h) −
b
f (t) dt = τ2 h2 + τ4 h4 + . . . + R2m+2 (h)h2m+2 ,
a
which coefficients τi that depend on the derivatives of f at a and
b, and on the Bernoulli numbers B2k and R2m+2 (h) is a remainder
term that involves f (2m) .
29 / 34
Numerical integration
Interval partitioning
Extrapolation:
lim T (h) := lim T n = I(f )
h→0
n→∞
Extrapolation uses h1 , . . . , hn to estimate the limit. Efficient
computation of extrapolation through Neville-Aitken scheme.
I
One can estimate the order gained by extrapolation
theoretically (Thm. 9.22 in Deuflhard/Hohmann).
I
Existence of an asymptotic expansion with a certain order hp
(p ≥ 1) can be used to improve the extrapolation.
I
Extrapolation ideas with Trapezoidal rule leads to Romberg
quadrature.
30 / 34
Numerical
integration
Table
9.5. Romberg
quadrature for the needle impulse f(t) = 1/ (10- 4 +x 2 ). (ckk'
Interval precision,
partitioningA k , cost in terms of f evaluations.)
relative
k
1
2
3
4
5
6
7
8
9
10
11
12
13
Tkk
ckk
Ak
1.999800
13333.999933
2672.664361
1551.888793
792.293096
441.756664
307.642217
293.006708
309.850398
312.382805
312.160140
312.159253
312.159332
9.9. 10- 1
4.2.10 1
7.6.10°
4.0.10°
1.5.10°
4.2.10- 1
1.4 . 10- 2
6.1.10- 2
7.4.10- 3
7.2.10- 4
2.6.10- 6
2.5.10- 7
1.1. 10- 9
2
3
5
9
17
33
65
129
257
513
1025
2049
4097
31 / 34
Numerical integration
Adaptive interval partitioning
Idea: On each sub-interval, estimate the quadrature error by
either:
I
Using a higher-order quadrature (e.g., Simpson rule), or
I
Comparing the error on a subinterval with the error on a
refinement
Then, subdivide the interval depending on the error estimation,
and repeat. Main challenge: Derive an error estimator ¯ that
estimates the true error in the following way:
c1 ≤ ¯ ≤ c2 with c1 ≤ 1 ≤ c2 . This method uses an a posteriori estimate of the
error.
32 / 34
Numerical integration
Difficult cases for quadrature:
I
(Unknown) discontinuities in f : adaptive quadrature continues
to refine, which can be used to localize discontinuities
I
Highly oscillating integrals
I
(Weakly) singular integrals (as required, e.g., in integral
methods (the fast multipole method)
33 / 34
Implementation examples in MATLAB
Demo time
34 / 34