ColoState
Notes on Stiffness and A-stability of ODE Solvers
MATH451
Jiangguo (James) Liu
∗
March 6, 2016
1
Examples of Stiffness
Stiffness refers to a wide disparity in spatial or temporal scales existing in a mathematical, physical, chemical, or biological problem. Stiff ODEs/PDEs arise in a variety of applications. Some
satisfactory numerical methods might perform poorly on stiff problems.
Example 1
Consider a very simple ODE IVP
x0 (t) = −30x,
x(0) = 1.
(1)
The exact solution
x(t) = e−30t
(2)
exhibits an initial dramatic drop and then becomes extremely flat, e.g., for t = 1, x(1) = e−30 =
9.357 × 10−14 , any changes after that are negligible! This indicates that different time scales exist
in this problem. To resolve the initial steep drop (transiency) in the solution, a numerical method
might have to proceed with a very small time step, which is a waste for the later stage when the
solution changes negligibly.
Example 2
Consider a 2-component linear system
( 0
x (t) = −11x − 10y,
y 0 (t) = −10x − 11y,
The exact solution is
(
x(0) = 2,
(3)
y(0) = 0.
x(t) = e−21t + e−t
y(t) = e−21t − e−t
or
x(t)
y(t)
−21t
=e
1
1
+e
−t
1
−1
.
(4)
The solution is a combination of e−21t and e−t . The two terms evolve at different time scales. Note
that the coefficient matrix
−11 −10
−10 −11
∗
Department of Mathematics,
(liu@math.colostate.edu)
Colorado
State
University,
1
Fort
Collins,
CO
80523-1874,
USA
has two quite different eigenvalues
λ1 = −21,
λ2 = −1.
(5)
x0 (t) = λx,
x(0) = 1
(6)
Now consider the model problem
with λ < 0 but |λ| being large. Obviously, the exact solution x(t) = eλt exhibits an immediate
dramatic drop in the early stage and extremely slow changes in the later stage.
Applying the explicit Euler method with a step size h, we obtain
yn = (1 + λh)n ,
n = 0, 1, 2, . . .
For the numerical solution to reflect the asymptotic decay, we must have
|1 + λh| < 1,
which implies that
2
h≤− .
λ
This is prohibitive when |λ| is large.
Applying the implicit Euler method with a step size h, we obtain
yn =
1
,
(1 − λh)n
n = 0, 1, 2, . . .
For the numerical solution to reflect the asymptotic decay, we should have
|1 − λh| > 1.
This holds for any positive h if λ < 0. There is no any restriction on h.
2
A-stability of ODE solvers
I didn’t like all these "strong", "perfect",
"absolute", "generalized", "super", "hyper",
"complete" and so on in mathematical definitions.
I wanted something neutral; and I having been
impressed by David Young’s "property A", I chose
the term "A-stable".
[G.Dahlquist, 1979]
There are at least two ways to combat stiffness.
One is to design a better computer, the other,
to design a better algorithm.
[H.Lomax, 1985]
2
(7)
JL Remark:
People still use the term "absolute stability".
Definition (Domains of A-stability, A-stable) The domain of A-stability DA of an ODE
solver is a subset of the complex plane C consisting of w = hλ such that the numerical solution
obtained by applying the ODE solver to the model problem (6) exhibits exponential decay. An
ODE solver is called A-stable if and only if C− ⊆ DA .
Remark The implication is that when a method is A-stable, we can choose the step size based
on accuracy considerations only, without worrying about stability constraints.
Explicit Euler method As seen in the previous discussion, the numerical solution of the model
problem (6) generated by the explicit Euler method is
yn = (1 + λh)n ,
n = 0, 1, 2, . . .
Then lim yn = 0 is true if and only if |1 + λh| < 1. Setting w = λh, we obtain the domain of
n→∞
A-stability of the explicit Euler scheme
DEE = {w ∈ C : |w + 1| < 1}.
This is the open unit disk centered at (−1, 0). Clearly, C− 6⊆ DEE . Therefore, the explicit Euler
scheme is not A-stable.
Implicit Euler method The numerical solution obtained by applying the implicit Euler method
to the model problem (6) is
1
yn =
, n = 0, 1, 2, . . .
(1 − λh)n
Setting w = λh, we obtain the domain of A-stability of the implicit Euler scheme
DIE = {w ∈ C : |w − 1| > 1}.
This is the exterior of the unit disk centered at (1, 0). Clearly, C− ⊆ DIE . Therefore, the implicit
Euler scheme is A-stable.
Trapezoid method
Applying the trapezoid scheme to the model problem (6), we obtain
1 yn+1 − yn = h λyn + λyn+1 .
2
Let w = λh, we have
w
w
yn+1 − 1 +
yn = 0.
2
2
We can immediately derive a solution formula
1 + w2 n
yn =
, n = 0, 1, 2, . . .
1 − w2
1−
(8)
However, we would like to generalize the analysis. Equation (8) is a linear difference equation with
the characteristic polynomial
w
w
1−
z− 1+
,
2
2
3
which can be obtained by replacing yn+1 by z and yn by 1. Obviously, its root is
z=
1+
1−
w
2
w
2
.
Setting |z| < 1, we obtain the domain of A-stability of the (implicit) trapezoid scheme
2 + w
DTZ = w ∈ C : <1 .
2 − w
(9)
2+w
It is known from Complex Analysis that W = 2−w
is a fractional linear conformal mapping from
the w-plane to the W -plane. Usually we consider complex mappings from the z-plane (source)
to the w-plane (image). But z is already used for the characteristic polynomial, now we use
w (for source) and W (for image) for complex planes. Anyhow, the fractional linear conformal
transformation maps a circle on the w-plane to a circle on the W -plane, and vice versa. Straight
lines on the complex plane are viewed as circles with a radius of infinity. It can be verified that this
fractional linear mapping maps the negative half of the w-plane to the interior of the unit circle on
the W -plane. In other words, one has
2 + w
w∈C:
< 1 = C− ,
2 − w
i.e., DTZ = C− . Therefore, the trapezoid scheme is A-stable.
It becomes increasingly difficult to analyze the domains of A-stability for higher order multistep
methods. We need new tools.
3
Root Locus Curves
To obtain the domains of A-stability for more ODE solvers, we utilize the tool of root locus
curves [3].
A general multistep method takes the form
a1 yn+1 + a0 yn + . . . + a−k yn−k = h b1 fn+1 + b0 fn + . . . + b−l fn−l .
Applying the method to the model problem (6), we obtain
(a1 − λhb1 )yn+1 + (a0 − λhb0 )yn + (a−1 − λhb−1 )yn−1 + . . . = 0.
Again, this is a homogenous linear difference equation. Setting w = hλ, we formulate its “characteristic polynomial” as
1
(a1 − wb1 )z + (a0 − wb0 ) + (a−1 − wb−1 ) + . . .
z
If |z| < 1, then the numerical solution of this method exhibits the asymptotic decay of the exact
solution to the model problem (6).
4
Note that |z| = 1, which means z = eiθ for θ ∈ [0, 2π), is an important “divide” based on the
“characteristic polynomial”. We introduce two finite Laurent series
r(z) = a1 z + a0 + a−1 z −1 + . . . + a−k z −k ,
s(z) = b1 z + b0 + b−1 z −1 + . . . + b−l z −l ,
(10)
and then a parameterized curve on the complex plane
w=
r(z)
,
s(z)
z = eiθ ,
θ ∈ [0, 2π).
(11)
This is the main idea of a root locus curve. The domain of A-stability should be either the interior
or the exterior of this closed curve.
Adams−Bashforth
1.5
1
0.5
0
−0.5
−1
−1.5
−2.5
−2
−1.5
−1
−0.5
0
0.5
Figure 1: The domain of A-stability of an Adams-Bashforth scheme is the interior of the corresponding closed curve: blue: order 1; red: order 2; green: order 3
Domains of A-stability of the Adams-Bashforth schemes
Exercise: Apply these schemes to the model problem (6) and derive their characteristic polynomials
and Laurent series.
Their root locus curves are
• For AB1,
z = eiθ ,
w = z − 1,
• For AB2,
z−1
,
w= 1
1
3
−
2
z
• For AB3,
w=
z = eiθ ,
z−1
1
12
23 −
16
z
+
5
z2
,
• Higher orders (Exercise).
5
θ ∈ [0, 2π).
θ ∈ [0, 2π).
z = eiθ ,
θ ∈ [0, 2π).
Adams−Moulton
4
3
2
1
0
−1
−2
−3
−4
−6
−4
−2
0
Figure 2: The domain of A-stability of an Adams-Moulton scheme is the interior of the corresponding closed curve: blue: order 3; red: order 4; green: order 5
The domain of A-stability of an Adams-Bashforth scheme is the interior of the corresponding closed
curve.
Domains of A-stability of the Adams-Moulton schemes
Exercise: Apply these schemes to the model problem and derive their characteristic polynomials
and Laurent series.
Their root locus curves are
• For AM1 and AM2, Exercise.
• For AM3,
w=
z−1
1
12
• For AM4,
w=
• For AM5,
w=
5z + 8 −
1
z
z−1
1
24
9z + 19 −
5
z
z = eiθ ,
,
,
+
1
z2
+
106
z2
z = eiθ ,
z−1
1
720
251z + 646 −
264
z
−
θ ∈ [0, 2π).
19
z3
,
θ ∈ [0, 2π).
z = eiθ ,
θ ∈ [0, 2π).
The domain of A-stability of an Adams-Moulton scheme is the interior of the corresponding closed
curve.
Observations Neither the (explicit) Adams-Bashforth nor the (implicit) Adams-Moulton methods are A-stable, although the A-stability domains for the latter are slightly larger.
6
References
[1] K.E. Atkinson, An introduction to Numerical Analysis, John Wiley & Sons, 2nd ed., 1989.
[2] A. Iserles, A First Course in the Numerical Analysis of Differential Equations, Cambridge
University Press, 1996.
[3] E. Hairer and G. Wanner, Solving Ordinary Differential Equations II: Stiff and DifferentialAlgebraic Problems, Springer, 1996.
[4] D. Kincaid and Cheney, Numerical Analysis:
Brooks/Cole, 3rd ed., 2002.
7
Mathematics of Scientific Computing,