3469 — Practical Numerical Simulations
Mike Peardon
School of Mathematics
Trinity College Dublin
Michaelmas Term 2015-16
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
1 / 22
Numerical solutions to ODEs
1. Initial-value problems
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
2 / 22
Classifying the problem
Finding a full solution to an ordinary differential equation
requires analysing two features:
1
2
The differential equation(s).
The boundary condition(s).
To find a solution numerically, it is the second set of defining
properties that determine the algorithm to apply.
The simplest types of boundary conditions to consider are
initial value problems. We will start with these.
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
3 / 22
Initial value problems
A general N th -order differential equation (in a single
variable) is
f (0) (x, t) +
N
X
f (k) (x, t)
k=1
dk x
=0
dtk
In an initial value problem the values of x and all its
(N −1)
derivatives up to ddt(N −1)x are all specified for some value of
t = t0 .
Aim is to compute x for all t (or perhaps t > t0 ) and all
numerical methods achieve this by evolving x away from t0
in small time increments.
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
4 / 22
Re-expressing an N th -order ODE
An N th -order ODE such as
f (0) (x, t) +
N
X
f (k) (x, t)
k=1
dk x
=0
dtk
can be re-expressed as a set of N coupled first-order ODEs
N th -order
→ N × 1st order
(N −2)
(k−1)
(k) = dx
(N −1) = dx
Defining x(1) = dx
gives N − 1
dt , . . . , x
dt , . . . , x
dt
coupled equations, and then one more comes from the ODE itself
f (N ) (x, t)
N
−1
X
dx(N −1)
+ f (0) (x, t) +
f (k) (x, t)x(k) = 0
dt
k=1
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
5 / 22
Euler’s method
Euler’s method gives a simple introduction - not used much
in practical simulations.
Consider solving a simple initial-value ODE:
ẋ = f (x, t) with x(t0 ) = x0
Assuming a solution exists near t0 , a Taylor expansion gives
x(t) in the neighbourhood of t near t0 :
x(t) = x(t0 ) + (t − t0 )ẋ(t0 ) +
(t − t0 )2
ẍ(t0 ) + . . .
2!
Substituting the ODE and truncating gives first term in Euler
method to evaluate x at t1 = t0 + h:
x1 = x0 + hf (x0 , t0 )
x(t + h) = x1 + O(h2 )
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
6 / 22
Euler’s method (2)
Now with x1 , apply the same rule to estimate x(t0 + 2h), using
x(t + 2h) ≈ x2 = x1 + hf (x1 , t1 )
Euler’s method to evaluate x(tN )
Given x0 , for k = 1...N = (t1 − t0 )/h and tk = t0 + kh evaluate in
sequence
xk = xk−1 + hf (xk , tk )
and use the approximation
x(tN ) = xN + O(h)
Euler’s method is not used in practise since it is:
not accurate (O(h))
not efficient (there are better schemes)
not stable - for many ODEs numerical solution diverges quickly
from true solution
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
7 / 22
Euler’s method (3)
Solve ẋ = x, x(0) = 1 using Euler
3
x(t)
2.5
2
1.5
1
0
0.2
0.4
0.6
0.8
1
t
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
8 / 22
Euler’s method (4)
Error from finite h - solve ẋ = x, x(0) = 1 using Euler
1
|x_N - x(t)|
0.1
0.01
0.001
0.001
0.01
0.1
1
h
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
9 / 22
Euler’s method (5)
Define local discretisation error for one step of Euler’s
method:
Local discretisation error
τ=
x(t0 + h) − x0 x1 − x0
−
h
h
See that τ = O(h)
Can prove (see Stoer and Bulirsch) that the global
discretisation error has the same order as the local
discretisation error, so that xN = x(t) + O(h) for Euler.
Euler’s method is order 1
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
10 / 22
Modified Euler method
By evaluating f more often per step, and matching more
terms in the Taylor expansion, can build higher-order scheme
Modified Euler
Given x0 , for j = 1 . . . N , N = (t1 − t0 )/h and tj = t0 + hj, evaluate
in sequence
k1 = hf (xj , tj )
h
k1
k2 = hf (xj + , tj + )
2
2
xj+1 = xj + k2
Now xn = x(t) + O(h2 )
This method is second order, and requires two evaluations of
f per step.
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
11 / 22
Modified Euler method (2)
3
2.5
2.5
x(t)
x(t)
Solve ẋ = x, x(0) = 1 using modified Euler
3
2
1.5
1.5
1
1
0
2
0.2
0.4
0.6
0.8
1
Euler
Mike Peardon (TCD)
0
0.2
0.4
0.6
0.8
1
t
t
Modified Euler
3469
Michaelmas Term 2015-16
12 / 22
Modified Euler method (3)
Error from finite h using Euler and modified Euler
0
10
-1
10
-2
|x_N - x(t)|
10
-3
10
-4
10
-5
10
-6
10
Euler
modified Euler
-7
10
-8
100.001
0.01
0.1
1
h
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
13 / 22
Fourth-order Runge-Kutta
Runge-Kutta algorithm
Given x0 , for j = 1 . . . N , N = (t1 − t0 )/h and tj = t0 + hj, evaluate
in sequence
k1 = hf (xj , tj )
h
k1
k2 = hf (xj + , tj + )
2
2
k2
h
k3 = hf (xj + , tj + )
2
2
k4 = hf (xj + k3 , tj + h)
1
xj+1 = xj + (k1 + 2k2 + 2k3 + k4 )
6
Now xn = x(t) + O(h4 )
This method is fourth-order, and requires four evaluations of f
per step.
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
14 / 22
Runge-Kutta RK4 (2)
Error from finite h using Euler, modified Euler and RK4
0
10
-1
10
-2
|x_N - x(t)|
10
-3
10
-4
10
-5
10
-6
10
-7
10
Euler
modified Euler
4th-order Runge Kutta
-8
100.001
0.01
0.1
1
h
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
15 / 22
2nd-order ODE solved using Euler’s method
Solve ẍ = −x, x(0) = 0, ẋ(0) = 1 using Euler
4
2
0
-2
-4
0
1
Mike Peardon (TCD)
2
3
4
3469
5
6
7
Michaelmas Term 2015-16
16 / 22
Controlling finite step-size errors
To find solution x(t1 ) given x(t0 ) it is usually a bad idea to
make a single step with h = t1 − t0 . Better to make n > 1 steps
with h = (t1 − t0 )/n.
How should n be determined if a particular accuracy is
needed?
Need to balance work required ∝ h−1 against error ∝ hp .
No need to keep h fixed throughout the integration. Can use
a smaller value for h where required for good precision.
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
17 / 22
Error in one-step
Error lemma
If f has bounded partial derivatives up to and including N + 2 for
t ∈ [a, b] and if x̄(t; h) is the numerical solution obtained by a p − 1th
global (pth local) method then:
x̄(t; h) = x(t) + hp ep (t) + hp+1 ep+1 (t) + . . .
· · · + hN eN (t) + hN +1 EN +1 (t; h)
ek (t0 ) = 0 for k = p . . . N . These functions are independent of
h
EN +1 is bounded.
Lemma hold for all h =
Mike Peardon (TCD)
b−a
n ,n
= 1, 2, . . .
3469
Michaelmas Term 2015-16
18 / 22
Error in one-step (2)
The error due to finite-step-sizes is
(t; h) = x̄(t; h) − x(t)
= hp ep (t) + O(hp+1 )
Since x(t) is not known, estimate by:
Estimating 1
Compute x̄(t; h) (one step, step size h)
2
Compute x̄(t; h/2) (two steps, step size h/2)
3
Then estimate using
(t; h/2) ≈
x̄(t; h) − x̄(t; h/2)
2p − 1
This approximation can be used to build an adaptive
step-size algorithm
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
19 / 22
Adaptive step-size methods
The estimate of can be used to determine h needed for a
certain level of accuracy.
Ensure
|(t; h)| < σ
for every step.
h now depends on t and σ
After one step, will be
|(t + h; h)| ≈ |hp ep (t + h)|
≈ |hp+1 e0p (t)|
since ep (t) = 0.
How can e0 be computed? As before, estimate using h—h/2
method
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
20 / 22
(t + h; h/2) ≈
and also
x̄(t + h; h) − x̄(t + h; h/2)
2p − 1
h
(t + h; h/2) ≈ ( )p he0p (t)
2
so
e0p (t) ≈
2p
(x̄(t + h; h) − x̄(t + h; h/2))
hp+1 2p − 1
1
Now consider an algorithm that uses data from one step to
adapt h such that || ≤ σ (with σ some defined tolerance).
Solve |(t + h; h)| = σ to find h̃, largest possible step that keeps
(approximately) inside the bound.
Adaptive step-size for an O(p) update
h̃p+1 = hp+1
Mike Peardon (TCD)
2p − 1
σ
p
2
|x̄(t + h; h) − x̄(t + h; h/2)|
3469
Michaelmas Term 2015-16
21 / 22
Example: Euler+adaptive step-size
Solve ẋ =
1
(t−5)2 +0.1
using Euler with adaptive step-size
12
10
x(t)
8
6
4
2
0
0
1
2
3
4
5
6
7
8
9
10
t
Algorithm focuses on region near t = 5 where changes are
most rapid
Mike Peardon (TCD)
3469
Michaelmas Term 2015-16
22 / 22