Introduction to Simulation - Lecture 14
Multistep Methods II
Jacob White
Thanks to Deepak Ramaswamy, Michal Rewienski, and
Karen Veroy
Outline
Small Timestep issues for Multistep Methods
Reminder about LTE minimization
A nonconverging example
Stability + Consistency implies convergence
Investigate Large Timestep Issues
Absolute Stability for two time-scale examples.
Oscillators.
Basic Equations
Multistep Methods
General Notation
Nonlinear Differential Equation:
k-Step Multistep Approach:
xˆ
xˆ
l −k
l −2
ˆx l −1 xˆ l
d
x(t ) = f ( x(t ), u (t ))
dt
k
k
j =0
j =0
(
l− j
l− j
ˆ
ˆ
α
x
=
∆
t
β
f
x
, u ( tl − j )
∑ j
∑ j
Multistep coefficients
Solution at discrete points
tl − k
tl −3 tl − 2 tl −1 tl
Time discretization
)
Simplified Problem for
Analysis
Multistep Methods
Scalar ODE:
d
v ( t ) = λ v(t ), v ( 0 ) = v0
dt
k
k
Scalar Multistep formula:
λ ∈^
l− j
l− j
ˆ
ˆ
α
v
=
∆
t
β
λ
v
∑ j
∑ j
j =0
j =0
Must Consider ALL λ ∈ ^
Im ( λ )
Decaying
Solutions
O
s
c
i
l
l
a
t
i
o
n
s
Growing
Solutions
Re ( λ )
Multistep Methods
Convergence Analysis
Convergence Definition
Definition: A multistep method for solving initial value
problems on [0,T] is said to be convergent if given any
initial condition
max
⎡ T ⎤
l∈⎢0, ⎥
⎣ ∆t ⎦
ˆvl − v ( l ∆t ) → 0 as ∆t → 0
ˆv l computed with ∆t
∆t
l
vˆ computed with
2
vexact
Multistep Methods
Convergence Analysis
Two Conditions for
Convergence
1) Local Condition: “One step” errors are small
(consistency)
Typically verified using Taylor Series
2) Global Condition: The single step errors do not grow
too quickly (stability)
Multi-step (k > 1) methods require careful analysis.
Convergence Analysis
Multistep Methods
Global Error Equation
k
Multistep formula:
Exact solution Almost
satisfies Multistep Formula:
∑α j vˆ
l− j
j =0
k
k
− ∆t ∑ β j λ vˆl − j = 0
j =0
k
d
l
α
−
∆
β
=
v
t
t
v
t
e
(
∑
∑
j ( l− j )
j
l− j )
dt
j =0
j =0
Global Error: E l ≡ v ( tl ) − vˆ l
Local Truncation Error
(LTE)
Difference equation relates LTE to Global error
l
l −1
l −k
l
α
−
λ
∆
t
β
E
+
α
−
λ
∆
t
β
E
+
"
+
α
−
λ
∆
t
β
E
=
e
( 0
( 1
( k
0)
1)
k)
Making LTE Small
Multistep Methods
Exactness Constraints
k
k
d
l
α
−
∆
β
=
v
t
t
v
t
e
(
∑
Local Truncation Error: ∑ j ( l − j )
j
l− j )
dt
j =0
j =0
Can't be from
d
v (t ) = λv (t )
dt
LTE
d
p −1
If v ( t ) = t ⇒ v ( t ) = pt
dt
p
k
j ) ∆t )
( k −
∑ α (
v (t )
j =0
j
k− j
p
k
p −1
− ∆t ∑ β j p ( ( k − j ) ∆ t )
=e
j =0
d
v ( tk − j )
dt
k
Multistep Methods
Making LTE Small
Exactness Constraint k=2
Example
k
⎛ k
p
p −1 ⎞
Exactness Constraints: ⎜ ∑ α j ( k − j ) − ∑ β j p ( k − j ) ⎟ = 0
j =0
⎝ j =0
⎠
For k=2, yields a 5x6 system of equations for Coefficients
p=0
p=1
p=2
p=3
p=4
⎡1
⎢2
⎢
⎢4
⎢
⎢8
⎢⎣16
1
1
1
1
1
1 0
0 −1
0 −4
0 −12
0 −32
0
−1
−2
−3
−4
⎡α 0 ⎤
0 ⎤ ⎢ ⎥ ⎡0⎤
α1 ⎥ ⎢ ⎥
⎥
⎢
−1⎥
0
⎢α 2 ⎥ ⎢ ⎥
0 ⎥ ⎢ ⎥ = ⎢0⎥
⎥ ⎢ β0 ⎥ ⎢ ⎥
0⎥
0⎥
⎢
⎢ β1 ⎥
0 ⎥⎦ ⎢ ⎥ ⎢⎣0 ⎥⎦
⎢⎣ β 2 ⎥⎦
Note
∑αi = 0
Always
Multistep Methods
Making LTE Small
Exactness Constraint k=2
example, generating methods
First introduce a normalization, for example α 0 = 1
⎡1
⎢1
⎢
⎢1
⎢
⎢1
⎢⎣1
0 0 ⎤ ⎡α1 ⎤ ⎡ −1 ⎤
−1 −1⎥⎥ ⎢⎢α 2 ⎥⎥ ⎢⎢ −2 ⎥⎥
0 −4 −2 0 ⎥ ⎢ β 0 ⎥ = ⎢ −4 ⎥
⎥⎢ ⎥ ⎢
⎥
0 −12 −3 0 ⎥ ⎢ β1 ⎥ ⎢ −8 ⎥
0 −32 −4 0 ⎥⎦ ⎢⎣ β 2 ⎥⎦ ⎢⎣ −16 ⎥⎦
1
0
0
−1
Solve for the 2-step method with lowest LTE
α 0 = 1, α1 = 0, α 2 = −1, β 0 = 1/ 3, β1 = 4 / 3, β 2 = 1/ 3
Satisfies all five exactness constraints LTE = C ( ∆t )
5
Solve for the 2-step explicit method with lowest LTE
α 0 = 1, α1 = 4, α 2 = −5, β 0 = 0, β1 = 4, β 2 = 2
Can only satisfy four exactness constraints LTE = C ( ∆t )
4
Making LTE Small
Multistep Methods
LTE Plots for the FE, Trap, and
“Best” Explicit (BESTE).
0
10
d
v (t ) = v (t )
d
FE
-5
L 10
T
E
Trap
-10
10
Beste
Best Explicit Method
has highest one-step
accurate
-15
10
-4
10
-3
10
Timestep
-2
10
-1
10
0
10
Making LTE Small
Multistep Methods
Global Error for the FE, Trap,
and “Best” Explicit (BESTE).
0
10
M
d
a -2 d v (t ) = v (t )
10
x
E -4
r 10
r
-6
10
o
r
t ∈ [0,1]
FE
Where’s BESTE?
Trap
-8
10 -4
10
-3
10
-2
10
Timestep
-1
10
0
10
Multistep Methods
Global Error for the FE, Trap,
and “Best” Explicit (BESTE).
worrysome
200
M 10
a
x 100
E 10
r
r 0
10
o
r
10
Making LTE Small
Best Explicit Method has
lowest one-step error but
global errror increases as
timestep decreases
d
v (t ) = v (t )
d
Beste
FE
Trap
-100
10
-4
10
-3
10
Timestep
-2
10
-1
10
0
Multistep Methods
Stability of the method
Difference Equation
Why did the “best” 2-step explicit method fail to
Converge?
Multistep Method Difference Equation
l
l −1
l −k
l
α
−
λ
∆
t
β
E
+
α
−
λ
∆
t
β
E
+
"
+
α
−
λ
∆
t
β
E
=
e
( 0
( 1
( k
0)
1)
k)
v ( l ∆t ) − vˆ
l
LTE
Global Error
We made the LTE so small, how come the Global
error is so large?
Multistep Methods
Stability of the method
Stability Definition
Multistep Method Difference Equation
l
l −1
l −k
l
α
−
λ
∆
t
β
E
+
α
−
λ
∆
t
β
E
+
"
+
α
−
λ
∆
t
β
E
=
e
( 0
( 1
( k
0)
1)
k)
Definition: A multistep method is stable if as ∆t → 0
T
l
l
max ⎡ T ⎤ E ≤ C (T ) max ⎡ T ⎤ e
N ∆t
l∈⎢0, ⎥
l∈⎢0, ⎥
t
∆
⎣
⎦
⎣ ∆t ⎦
interval
dependent
Global Error is bounded by a
Stability means:
constant times the sum of the LTE’s
Convolution Sum
Aside on difference
Equations
Root Relation
Given a kth order difference eqn with zero initial conditions
a0 x + " + ak x
l
l −k
−1
= u , x = 0, " , x
l
l
x
x can be related to the input u by =
Q M q −1
Root multiplicity
m
q =1 m=0
k −1
l− j j
h
u
∑
j =0
l
Roots of
a0 z + a1 z
k
l
convolution sum
h = ∑ ∑ γ q ,m ( l ) ( ς q )
l
−k
+ " + ak = 0
=0
Aside on difference
Equations
Convolution Sum
Bounding Terms
Q M q −1
l
⎛
l− j
m
l
j ⎞
x = ∑ ∑ ⎜ ∑ γ q ,m ( l − j ) ( ς q ) u ⎟
q =1 m =0 ⎝ j =0
⎠
Rq ,m
If ς q <1, then Rq ,m ≤ C max j u
j
Independent of l
If ς q < (1+ε ) , then Rq ,0 ≤ C
e
εl
ε
Bounds distinct Roots
max j u
j
Multistep Methods
Stability of the method
Stability Theorem
Theorem: A multistep method is stable if and only if
Roots of α 0 z k + α1 z k −1 + " + α k = 0 either:
1. Have magnitude less than one
2. Have magnitude equal to one and are distinct
Multistep Methods
Stability of the method
Stability Theorem “Proof”
Given the Multistep Method Difference Equation
(α 0 − λ∆t β 0 ) E l + (α1 − λ∆t β1 ) E l −1 + " + (α k − λ∆t β k ) E l −k = el
If, as ∆t → 0, roots of (α 0 − λ∆t β 0 ) z l + " + (α k − λ∆t β k ) = 0
• less than one in magnitude or
• are distinct and bounded by 1 + κ∆t , κ > 0
Then from the aside on difference equations
κ l ∆t
κT
e
C
e
T
l
l
l
max ⎡ T ⎤ E ≤ C
max ⎡ T ⎤ e ≤
max ⎡ T ⎤ e
l∈⎢0, ⎥
l∈⎢0, ⎥
l∈⎢0, ⎥
t
T
t
∆
∆
N
t
t
∆
∆
⎣
⎦
⎣
⎦
⎣ ∆t ⎦
C (T )
Multistep Methods
Stability of the method
Stability Theorem Picture
roots of
k
∑α
j =0
j
z
k− j
=0
-1
roots of
Im
As ∆t → 0, roots
move inward to
match α polynomial
1
k
Re
k− j
α
−
λ
∆
t
β
z
= 0 for a nonzero ∆t
∑( j
j)
j =0
Stability of the method
Multistep Methods
The BESTE Method
Best explicit 2-step method
α 0 = 1, α1 = 4, α 2 = −5, β 0 = 0, β1 = 4, β 2 = 2
Im
roots of z + 4 z − 5 = 0
2
-5
-1
1
Method is Wildly unstable!
Re
Multistep Methods
Stability of the method
Dahlquist’s First Stability
Barrier
For a stable, explicit k-step multistep method, the
maximum number of exactness constraints that can be
satisfied is less than or equal to k (note there are 2k-1
coefficients). For implicit methods, the number of
constraints that can be satisfied is either k+2 if k is even
or k+1 if k is odd.
Convergence Analysis
Multistep Methods
Conditions for convergence,
stability and consistency
1) Local Condition: One step errors are small (consistency)
Exactness Constraints up to p0 (p0 must be > 0)
⇒ max
⎡ T ⎤
l∈⎢0, ⎥
⎣ ∆t ⎦
e
≤ C1 ( ∆t )
l
p0 +1
for ∆t < ∆t0
2) Global Condition: One step errors grow slowly (stability)
roots of
k
k− j
Inside the unit circle or on the
α
z
=
0
∑ j
unit circle and distinct
j =0
T
⇒ max ⎡ T ⎤ E ≤ C2 max ⎡ T ⎤ el
l∈⎢0, ⎥
l∈⎢0, ⎥
t
∆
∆
t
⎣
⎦
⎣ ∆t ⎦
p0
l
Convergence Result: max ⎡ T ⎤ E ≤ CT ( ∆t )
l
l∈⎢ 0, ⎥
⎣ ∆t ⎦
Multistep Methods
Large timestep stability
Two time-constant circuit
small ∆t
Backward-Euler
Computed Solution
Circuit Example
d
x(t ) = Ax(t )
dt
eig ( A) = −2.1, −0.1
large ∆t
With Backward-Euler it is easy to use small timesteps for
the fast dynamics and then switch to large timesteps for the
slow decay
Large Timestep Stability
Multistep Methods
FE on two time-constant
circuit?
Forward-Euler Computed
Solution
The Forward-Euler is accurate for small timesteps, but goes
unstable when the timestep is enlarged
Large Timestep Stability
Multistep Methods
Scalar ODE:
FE, BE and Trap on the scalar
ode problem
d
v ( t ) = λ v(t ), v ( 0 ) = v0
dt
λ ∈^
l +1
l
l
l
Forward-Euler: vˆ = vˆ + ∆t λ vˆ = (1 + ∆t λ ) vˆ
If 1 + ∆t λ > 1 the solution grows even if λ <0
1
l +1
l
l +1
l +1
Backward-Euler: vˆ = vˆ + ∆t λ vˆ ⇒ vˆ =
(1 − ∆tλ )
vˆl
1
If
< 1 the solution decays even if λ > 0
1 − ∆t λ
Trap Rule: vˆ
l +1
= vˆ + 0.5∆t λ ( vˆ
l
l +1
+ vˆ ) ⇒ vˆ
l
l +1
1 + 0.5∆t λ ) l
(
=
vˆ
(1 − 0.5∆tλ )
Large Timestep Stability
Multistep Methods
Forward Euler
z = (1 + ∆t λ )
-1
Im ( λ )
ODE stability
region
Im(z)
Difference Eqn
Stability region
FE large timestep region of
absolute stability
1
Re(z)
−2
∆t
Region of
Absolute
Stability
Re ( λ )
Large Timestep Stability
Multistep Methods
FE large timestep stability,
circuit example
Circuit example with ∆t = 0.1, λ = −2.1, − 0.1
Im ( λ )
ODE stability
region
Im(z)
-1
Difference Eqn
Stability region
1
Re(z)
−2
∆t
Region of
Absolute
Stability
Re ( λ )
Large Timestep Stability
Multistep Methods
FE large timestep stability,
circuit example
Circuit example with ∆t=1.0, λ = −2.1, − 0.1
Im ( λ )
Unstable
Difference
Equation
-1
ODE stability
region
Im(z)
Difference Eqn
Stability region
1
Re(z)
−2
∆t
Region of
Absolute
Stability
Re ( λ )
Large Timestep Stability
Multistep Methods
Backward Euler
BE large timestep region of
absolute stability
z = (1 − ∆t λ )
−1
Im(z)
-1
Difference Eqn
Stability region
1
Re(z)
Region of
Absolute
Stability
Im ( λ )
Large Timestep Stability
Multistep Methods
BE large timestep stability,
circuit example
Circuit example with ∆t = 0.1, λ = −2.1, − 0.1
Im ( λ )
Im(z)
-1
Difference Eqn
Stability region
1
Re(z)
Region of
Absolute
Stability
Large Timestep Stability
Multistep Methods
BE large timestep stability,
circuit example
Circuit example with ∆t =1.0, λ = −2.1, − 0.1
Im ( λ )
Stable Difference
Equation Im(z)
-1
Difference Eqn
Stability region
1
Re(z)
Region of
Absolute
Stability
Large Timestep Stability
Multistep Methods
Stability Definitions
Region of Absolute Stability for ak Multistep method:
Values of λ∆t where roots of ∑ (α j − λ∆t β j ) z k − j = 0
are inside the unit circle.
j =0
A-stable:
A method is A-stable if its region of absolute stability
includes the entire left-half of the complex plane
Dahlquist’s second Stability barrier:
There are no A-stable multistep methods of convergence
order greater than 2, and the trap rule is the most accurate.
Multistep methods
4
2
Numerical Experiments
Oscillating Strut and Mass
∆t = 0.1
Forward-Euler
0
-2
-4
-6
Trap rule
0
5
10
Backward-Euler
15
20
25
30
Why does FE result grow, BE result decay and the
Trap rule preserve oscillations
Large Timestep Stability
Multistep Methods
Forward Euler
FE large timestep oscillator
example
z = (1 + ∆t λ )
Im ( λ )
ODE stability
region
Im(z)
oscillating
unstable
-1
Difference Eqn
Stability region
1
Re(z)
−2
∆t
Region of
Absolute
Stability
Re ( λ )
Large Timestep Stability
Multistep Methods
Backward Euler
BE large timestep oscillator
example
z = (1 − ∆t λ )
−1
Im ( λ )
Im(z)
decaying
-1
Difference Eqn
Stability region
1
oscillating
Re(z)
Region of
Absolute
Stability
Large Timestep Stability
Multistep Methods
Trap large timestep oscillator
example
1 + 0.5∆t λ )
(
Trap Rule z = (1 − 0.5∆t λ )
Im ( λ )
Im(z)
oscillating
oscillating
-1
Difference Eqn
Stability region
1
Re(z)
Region of
Absolute
Stability
Multistep Methods
Large Timestep Issues
Two Time-Constant Stable problem (Circuit)
FE: stability, not accuracy, limited timestep size.
BE was A-stable, any timestep could be used.
Trap Rule most accurate A-stable m-step method
Oscillator Problem
Forward-Euler generated an unstable difference
equation regardless of timestep size.
Backward-Euler generated a stable (decaying)
difference equation regardless of timestep size.
Trapezoidal rule mapped the imaginary axis
Summary
Small Timestep issues for Multistep Methods
Local truncation error and Exactness.
Difference equation stability.
Stability + Consistency implies convergence.
Investigate Large Timestep Issues
Absolute Stability for two time-scale examples.
Oscillators.
Didn’t talk about
Runge-Kutta schemes, higher order A-stable
methods.