Numerical Methods for ODEs
17
PART II
Review of numerical methods for
Ordinary Differential Equations
Numerical Methods for ODEs
18
B OUNDARY VALUE P ROBLEMS (I)
Solving a two–point boundary value problem
y 2π
0 2π
for x
g
0
d2y
dx2
y0
Finite–difference approximation:
– Second–order Central Difference formula for the interior nodes:
– Endpoint nodes:
jh
yN
1
2yN
h2 g1
– Tridiagonal algebraic system
0
2y1
1
yN
y2
0
y0
N
1
g j for j
and x j
1
2π
N 1
yj
where h
2y j
h2
1
yj
h2 gN
Numerical Methods for ODEs
19
B OUNDARY VALUE P ROBLEMS (II)
Solving a two–point boundary value problem
0 2π
for x
d2y
g
dx2
dy
dy
0
2π
dx
dx
0
Finite–difference approximation:
– Second–order Central Difference formula for the interior nodes:
yj
1
N
1
g j for j
2y j
h2
1
yj
– First–order Forward/Backward Difference formulae to re–express
endpoint values:
y1 y0
y0
y1
yN
1
0
h
yN
1
yN
0
h
yN
First–order only — degraded accuracy!
– Tridiagonal algebraic system — Where is the problem?
Numerical Methods for ODEs
20
B OUNDARY VALUE P ROBLEMS (III)
In order to retain second–order accuracy in the approximation of the
Neumann problem need to use higher-order formulae at endpoints, e.g.
y2
3y0
h2 g1
4y1
Second–order accuracy recovered!
2
y1
3
y2
2
y2
3
The first row thus becomes
1
3
y0
0
4y1
2h
y0
Numerical Methods for ODEs
21
B OUNDARY VALUE P ROBLEMS (IV)
Compact Stencils — stencils based on three grid points only:
x j 1 x j x j 1 at the j th node
Is is possible to obtain higher (then second) order of accuracy on compact
stencils? — YES !
1
O h4
h2 iv
y
12 j
gj
using the equation in question:
gj
1
Inserting into the original finite–difference equation:
1
gj
1
O h4
2g j
12
gj
Slight modification of the RHS
gj
1
yj
2y j
h2
1
yj
O h4
h2 iv
g
12 j
2g j
h2
1
h2 g j
12
h2
g
12 j
h2 iv
12 y j
Re-express the error term
h2 iv
y
12 j
yj
2y j
h2
1
yj
d2 y
dy2
Consider the central difference approximation to the equation
fourth—order accuracy!!!
g
Numerical Methods for ODEs
B OUNDARY VALUE P ROBLEMS (V)
Compact Finite Difference Schemes — drawbacks:
– need to be tailored to the specific equation solved
– can get fairly complicated for more complex equations
22
Numerical Methods for ODEs
23
I NITIAL VALUE P ROBLEMS —
G ENERAL R EMARKS
Consider the following Cauchy problem :
f y t with y t0
dy
dt
y0
The independent variable t is usually referred to as time .
Equations with higher–order derivatives can be reduced to systems of
first–order equations
Generalizations to systems of ODEs straightforward
Assume uniform time–steps ( h is constant )
f t ,
When the RHS function doesn’t depend on y, i.e., f y t
solution obtained via quadrature
Numerical Methods for ODEs
24
I NITIAL VALUE P ROBLEMS —
C HARACTERIZATION OF I NTEGRATION
M ETHODS
— unlike in the Boundary Value Problems, there is no terminal
condition and approximation errors may accumulate in time; consequently, a
relevant characterization of accuracy is provided by the global error
ACCURACY
local error
# of time steps
global error
rather than the local error .
— unlike in the Boundary Value Problems, where boundedness
of the solution at final time is enforced via a suitable terminal condition , in
Initial Value Problems there is a priori no guarantee that the solution will
remain bounded.
STABILITY
Numerical Methods for ODEs
25
I NITIAL VALUE P ROBLEMS —
M ODEL P ROBLEM
Stability of various numerical schemes is usually analyzed by applying these
schemes to the following linear model:
y0
iλi y with y t0
λt
y0
λh
λ3 h3
6
λ2 h2
2
1
y0 e
λr
0.
Exact solution: y t
which is stable when λr
λy
dy
dt
Motivation — consider the following advection–diffusion PDE:
∂2 u
a 2
∂x
∂u
c
∂x
∂u
∂t
0
Taking Fourier transform yields (k is the wavenumber):
a k2 ûk
c i k ûk
d ûk
dt
0
where
– the real term a k2 ûk represents diffusion
– the imaginary term c i k ûk represents advection
Numerical Methods for ODEs
26
I NITIAL VALUE P ROBLEMS —
E XPLICIT E ULER S CHEME (I)
Consider a Taylor series expansion
hy tn
y tn
1
y tn
h2
y tn
2
Using the ODE we obtain
df
dt
ft
f fy
f
y
dy
dt
dy
dt
y
Neglecting terms proportional to second and higher powers of h yields the
Explicit Euler Method
h f y n tn
yn
1
yn
Retaining higher–order terms is inconvenient, as it requires differentiation of
f and does not lead to schemes with desirable stability properties.
Numerical Methods for ODEs
27
I NITIAL VALUE P ROBLEMS —
E XPLICIT E ULER S CHEME (II)
Local error analysis:
O h2
λh yn
1
1
yn
Global error analysis:
T
h
Ch2
Ch2 N
global error
Ch
Thus, the scheme is
– locally second–order accurate
Nh ) first–order accurate
– globally (over the interval t0 t0
Numerical Methods for ODEs
28
I NITIAL VALUE P ROBLEMS —
E XPLICIT E ULER S CHEME (III)
Stability (for the model problem)
1
λh yn
λhyn
yn
1
yn
Thus, the solution after n time steps
1
σ
σn y 0
λh n y0
1
yn
λh
For large n, the numerical solution remains stable iff
2
λi h
– unstable stable for imaginary λ
2
– conditionally stable for real λ
λr h
1
1
σ
1
Numerical Methods for ODEs
29
I NITIAL VALUE P ROBLEMS —
I MPLICIT E ULER S CHEME (I)
Implicit Schemes — based on approximation of the RHS that involve
f yn 1 t , where yn 1 is the unknown to be determined
Implicit Euler Scheme — obtained by neglecting second and higher–order
terms in the expansion:
we obtain
1 tn 1
1
1
hy tn
1
f yn
dy
dt t
n
Upon substitution
y tn
y tn
h2
y tn
2
1
1 tn 1
h f yn
yn
1
yn
The scheme is
– locally second–order accurate
Nh ) first–order accurate
– globally (over the interval t0 t0
Numerical Methods for ODEs
30
I NITIAL VALUE P ROBLEMS —
I MPLICIT E ULER S CHEME (II)
Stability (for the model problem):
yn
λh
1
2
1
λi h
2
λr h
1
1
1
1
σ
λh
1
1
λh
σ
1
σn y 0
y0
yn
n
1
1
λhyn
yn
1
yn
1
yn
Implicit Euler scheme is thus stable for
– all stable model problems
– most unstable model problems
A t y, each implicit step
I hA 1 yn
When solving systems of ODEs of the form y
requires solution of an algebraic system: yn 1
Implicit schemes are generally hard to implement for nonlinear problems
Numerical Methods for ODEs
31
I NITIAL VALUE P ROBLEMS —
C RANK –N ICHOLSON S CHEME (I)
Obtained by approximating the formal solution of the ODE
tn 1
f y t dt using the trapezoidal quadrature:
yn 1 yn
tn
1 tn 1
f yn
h
f y n tn
2
yn
1
yn
The scheme is
– locally third–order accurate
Nh ) second–order accurate
– globally (over the interval t0 t0
Stability (for the model problem):
yn
1
1
1
n
σ
y0
1
σ y0
n
1
λh
2
λh
2
1
1
1
yn
1
yn
yn
1
yn
λh
yn
2
Stable for all model ODEs with stable solutions
ℜ λh
1
σ
λh
2
λh
2
0
λh
2
λh
2
yn
Numerical Methods for ODEs
32
I NITIAL VALUE P ROBLEMS —
L EAPFROG S CHEME (I)
Leapfrog as an example of a two–step method :
yn
2 h λ yn
1
1
yn
1
σn y 0 )
0
2 h λσ
σ2
Characteristic equation for the amplification factor (yn
where roots give the amplification factors:
eλh
λh
e
O h3
Thus, the scheme is
– locally third–order accurate
Nh ) second–order accurate
– globally (over the interval t0 t0
O h3
λh
1
λ2 h2
2
λ2 h2
2
λh
1
λ 2 h2
1
λh
1
λh
σ2
σ1
λ 2 h2
Numerical Methods for ODEs
33
I NITIAL VALUE P ROBLEMS —
L EAPFROG S CHEME (II)
λh
1
σ1
Stability for diffusion problems ( λ
λr ):
λ2r h2
1 for all h
0
Thus the scheme is unconditionally unstable for diffusion problem!
iλi ):
1 (!!!) for h
2
σ21
Stability for advection problems ( λ
1
λi
Thus the scheme is conditionally unstable and non–diffusive for advection
problems!
Question — analyze dispersive (i.e., related to arg σ ) errors of the leapfrog
scheme.
Numerical Methods for ODEs
34
I NITIAL VALUE P ROBLEMS —
M ULTISTEP P ROCEDURES
General form of a multistep procedure :
q
p
j 1
j tn j
h ∑ β j f yn
j
∑ α j yn
j 1
with characteristic polynomials
α0
β0
q 1
q — implicit scheme
q — explicit scheme
1z
p 1
βq
1z
β q zq
αp
– if p
αpzp
ζq z
– if p
ξp z
A ξ ζ –procedure converges uniformly in a b , i.e.,
limh 0 maxtn a b yn y tn
0 if:
0 and
– the following consistency conditions are verified: ξ 1
ξ 1
ζ 1 ( consistency condition )
– all roots of the polynomial ξ z are such that zi
zk
1 are simple ( stability condition )
1 and the roots with
Numerical Methods for ODEs
35
I NITIAL VALUE P ROBLEMS —
RUNGE –K UTTA M ETHODS (I)
General form of a fractional step method :
γ3 h k3
γ2 h k2
γ1 h k1
yn
1
yn
where
f yn tn
k1
α1 h
β2 hk1
β3 hk2 tn
β1 hk1 tn
α2 h
f yn
k3
f yn
k2
.
.
.
Choose γi , βi and αi to match as many expansion coefficients as possible in
tn
hy tn
ft
f fy
f 2 fyt
ft fy 2 f fyt
ftt
y
y
y tn
h3
y
6
f
y
1
y tn
h2
y tn
2
f 2 fyy
Runge—Kutta methods are self–starting with fairly good stability and
accuracy properties.
Numerical Methods for ODEs
36
I NITIAL VALUE P ROBLEMS —
RUNGE –K UTTA M ETHODS (II)
RK4 — an ODE workhorse:
k3
h
k4
6
1
1 2
hk3 tn
f yn
h
k1 t n
2
k4
1 2
h
k2 t n
2
f yn
k3
f yn
k2
f yn tn
h
k2
3
k1
h
k1
6
yn
1
yn
The amplification factor:
λh
1
λ3 h3
6
λ4 h4
24
1
Thus, stability iff σ
σ
λ2 h2
2
Accuracy:
O h5
σ
eλh
Thus, the scheme is
– locally fifth–order accurate
Nh ) fourth–order accurate
– globally (over the interval t0 t0
Numerical Methods for ODEs
37
I NITIAL VALUE P ROBLEMS —
RUNGE ’ S P RINCIPLE
Let k 1 be the local truncation error; denote Y t h an approximation of
the exact solution y t computed with the step size h; then at t t0 2nh:
k 1
Ct
t 0 hk
C n 2h
Ct
t 0 2k hk
Y t 2h
1
yt
C 2 n hk
Y t h
yt
Subtracting:
2 k hk
t0 1
Ct
Y t h
Y t 2h
Thus we can obtain an estimate of the absolute error based on solution with
two step–sizes only:
Y t h Y t 2h
2k 1
Y t h
yt
Runge’s principle is very useful for adaptive step size refinement