Multistage Methods I: Runge-Kutta Methods
Varun Shankar
January 12, 2016
1
Introduction
Previously, we saw that explicit multistep methods (AB methods) have shrinking stability regions as their orders are increased. Further, the second Dalhquist
barrier stopped us from generating high-order A-stable multistep methods. In
this chapter, we will introduce Runge-Kutta methods. Unlike the multistep
methods, these methods generate information within a time-step to get highorder convergence; these methods are called multistage methods due to the
generation of these intermediate stages.
We will show how both explicit and implicit Runge-Kutta (RK) methods can be
generated from quadrature rules. We will then discuss and analyze the general
form of RK methods, abstracting away the polynomial interpolation idea.
2
Explicit RK Methods from Quadrature
Let us now derive some popular explicit RK methods. Unlike multistep methods,
we will only use information over a single time-step to generate these. For this,
we will return to the integral form of the simple ODE problem. This integral
form is given by:
y
n+1
n
tZ
n+1
=y +
f (t, y(t))dt.
(1)
tn
2.1
Forward Euler
Using the left endpoint rule to approximate the integral, we get
tn+1 − tn
f (tn , y n ),
1
≈ y n + ∆tf (tn , y n ).
y n+1 ≈ y n +
=⇒ y n+1
1
(2)
(3)
Forward Euler is thus both AB1 and RK1.
2.2
Second-order RK methods
We now use the trapezoidal rule to approximate the integral. This gives us
y n+1 ≈ y n +
∆t
[f (tn , y n ) + f (tn+1 , y n+1 )].
2
(4)
This, however, is an implicit method. To make it explicit, we will simply approximate the y n+1 on the right hand side with Forward Euler! This gives
y n+1 ≈ y n +
∆t
∆t
f (tn , y n ) +
f (tn+1 , y n + ∆tf (tn , y n )).
2
2
(5)
This is typically written as
∆t
(k1 + k2 ),
2
k1 = f (tn , y n ),
y n+1 ≈ y n +
(6)
(7)
k2 = f (tn + h, yn + ∆tk1 ).
(8)
This is the classical second-order Runge-Kutta method, referred to as RK2. It
is also known as “Improved Euler” or “Heun’s Method”. This is not the only
RK2 method.
To generate a second RK2 method, all we need to do is apply a different quadrature rule of the same order to approximate the integral. For example, if we use
the Midpoint rule, we get
1
y n+1 ≈ y n + ∆tf tn+ 21 , y n+ 2 .
(9)
1
To make this a fully-explicit method, we simply replace y n+ 2 by a Forward
Euler approximation. This yields
∆t
y n+1 ≈ y n + ∆tf tn+ 21 , y n +
f (tn , y n ) .
(10)
2
This is called the “Midpoint” method. There is no real consistent naming
scheme. This is simply because there were (and continue to be) armies of researchers developing RK methods of different orders and properties.
2.3
RK4
If f is not dependent on y, RK4 can be derived from Simpson’s rule. You will
show this in your assignment.
2
3
General derivations of explicit RK methods
As we go to higher-order explicit RK methods, the quadrature approach becomes
ridiculously cumbersome. Instead, it is fairly common to generate explicit RK
methods using the general form of RK methods. This is typically written as
y n+1 = y n + ∆t
s
X
bi k i ,
(11)
i=1
k1 = f (tn , y n ),
(12)
k2 = f (tn + c2 ∆t, y n + ∆t(a21 k1 )),
n
k3 = f (tn + c3 ∆t, y + ∆t(a31 k1 + a32 k2 )),
..
.
n
ks = f (tn + cs ∆t, y + ∆t (as1 k1 + as2 k2 + . . . + as,s−1 ks−1 )) .
(13)
(14)
(15)
(16)
It is straightforward to collect the coefficients aij into the Runge-Kutta matrix
A. We also have the condition
s
X
aij .
(17)
ci =
j=1
For explicit RK methods, A is strictly lower-triangular. This implies that the
first row of A is always filled with zeros! Consequently, for explicit RK methods,
c1 = 0.
3.1
Example: Generating explicit RK2 methods
Let’s see how one goes about generating explicit RK2 methods with the above
general formula. To do this, we first write down the general form of an RK2
method:
y n+1 = y n + ∆t(b1 k1 + b2 k2 ) + O(∆t3 ),
n
k1 = f (tn , y ),
(18)
(19)
n
k2 = f (tn + c2 ∆t, y + ∆ta21 k1 ).
(20)
Our strategy to find b1 , b2 , c2 and a21 is straightforward: we will match the
Taylor expansion of y n+1 to the Taylor expansion of the general RK2 method.
This will give us conditions to find the unknown coefficients.
Taylor Expansion for y n+1
First, let’s Taylor expand y n+1 .
y n+1 = y n + ∆t
∆t2 ∂ 2 n
∂ n
y (t) +
y (t) + O(∆t3 ).
∂t
2 ∂t2
3
(21)
From the ODE, we have y 0 (t) = f (t, y). We can use this to obtain an expression
for y 00 (t):
y 00 (t) = ft (t, y) + fy (t, y)y 0 (t),
= ft (t, y) + fy (t, y)f (t, y).
(22)
(23)
Plugging back into the Taylor series for y n+1 , we get
y n+1 = y n + ∆tf (tn , y n ) +
∆t2
[ft (tn , y n ) + fy (tn , y n )f (tn , y n )] + O(∆t3 ),
2
(24)
∆t
∆t
f (tn , y n ) +
[f (tn , y n ) + ∆tft (tn , y n ) + ∆tfy (tn , y n )f (tn , y n )] + O(∆t3 ).
= yn +
2
2
(25)
Taylor Expansion for the general RK2 scheme
Now that we have an exact Taylor expansion for y n+1 , we can expand the RK2
scheme, and compare. First, we Taylor expand f (tn + c2 ∆t, y n + ∆ta21 k1 ):
f (tn + c2 ∆t, y n + ∆ta21 k1 ) = f (tn , y n ) + c2 ∆tft (tn , y n ) + ∆ta21 fy (tn , y n )k1 + O(∆t2 ),
(26)
= f (tn , y n ) + c2 ∆tft (tn , y n ) + ∆ta21 fy (tn , y n )f (tn , y n ) + O(∆t2 ).
(27)
We then plug in this expression into the general RK2 scheme and simplify:
y n+1 = y n + (b1 + b2 )∆tf (tn , y n ) + b2 ∆t2 [c2 ft (tn , y n ) + a21 fy (tn , y n ), f (tn , y n )] + O(∆t3 ).
(28)
Comparing Taylor expansions
We can now compare the Taylor expansion of y n+1 to the Taylor-expanded
general RK2 scheme, and require the terms to match up to O(∆t3 ). This gives
us three conditions:
b1 + b2 = 1,
1
c 2 b2 = ,
2
1
a21 b2 = .
2
(29)
(30)
(31)
These are three nonlinear equations for four unknowns! There are thus multiple
choices of the unknowns that satisfy these equations.
4
In practice, we typically pick the c values, then solve the b values, and then
solve for the a values. Remember, the c values dictate where between tn and
tn+1 we place our stages.
b1 = b2 = 12 , c2 = a21 = 1 yields the classical RK2 method.
b1 = 0, b2 = 1, c2 = a21 =
1
2
leads to the previously seen Midpoint method.
b1 = 14 , b2 = 43 , c2 = a21 = 32 describes the so-called Ralston method, which has
been shown to minimize the local truncation error for the class of RK2 schemes.
In general, the higher the number of stages s of the RK method, the more the
number of conditions and unknowns, and the trickier it becomes to solve for the
unknowns.
3.2
Butcher Tableaux
It becomes extremely tedious to write down the different a, b, c values for RK
methods. John Butcher designed an elegant solution to this problem: it is now
common to collect these coefficients in a table, also called a Butcher tableau.
c1 a11 a12 . . . a1s
..
..
..
..
..
.
.
.
.
.
Butcher tableaux are represented as follows: cs as1 as2 . . . ass
b1
b2
...
bs
A huge list of Butcher tableaux for both explicit and implicit RK methods is
given at https://en.wikipedia.org/wiki/List_of_Runge%E2%80%93Kutta_
methods.
4
Implicit RK Methods From Quadrature
If you recall the derivation of explicit RK methods from quadrature, we initially
ended up with something implicit when applying the quadrature rule, then
threw out the “implicitness” by replacing appropriate terms with a Forward
Euler approximation. To generate implicit RK methods (IRK methods), we
simply apply the quadrature rule over the interval, and do no replacing.
Why use implict RK methods at all? For one thing, we explicit implicit RK
methods to be more stable than explicit methods, just like AM and BDF methods were more stable than AB methods. For another, explicit RK methods
have a non-monotonic relationship between order p and number of stages s. For
p > 4, s > p. An eighth order explicit RK method actually requires 11 stages!
However, for implicit RK methods, it is possible to get a much higher order
than the number of stages used.
5
Here, we will discuss a few IRK methods and how to generate them. First, we
present a definition and theorem that reassure us about the connection between
RK methods and polynomial interpolation.
Definition: Let c1 , . . . , cs be distinct real numbers in [0, 1]. The collocation
polynomial p(t) is a polynomial of degree s satisfying
p(t0 ) = y0 ,
(32)
pt (t0 + ci ∆t) = f (t0 + ci ∆t, p(t0 + ci ∆t)), i = 1, . . . , s,
(33)
and the numerical solution of the corresponding collocation method is defined
by y1 = p(t0 + ∆t).
Theorem, Wright 1970: The collocation method of the above definition is
equivalent to the s-stage RK method with coefficients
Zci
aij =
`j (t)dt,
(34)
`i (t)dt,
(35)
0
Z1
bi =
0
Y t − ck
.
`i (t) =
ci − ck
(36)
k6=i
Practically speaking, this theorem simply tells us that we can generate the RK
coefficients for an IRK method by picking the c values, then using the Lagrange
interpolating polynomial over the interval to generate a and b values. Recall
that the c values are where the intermediate stages are located in time between
tn and tn+1 .
4.1
Gauss methods
Gauss methods are based on Gaussian quadrature. The idea is simple: we select
c1 . . . cs to be the zeros of some orthogonal polynomial. Some common choices
are Legendre polynomials, Chebyshev polynomials of the first kind and Chebyshev polynomials of the second kind. We will briefly discuss Gauss-Legendre
IRK methods.
A Gauss-Legendre method based on s stages has an order of p = 2s. Further,
all Gauss-Legendre methods are A-stable! These methods therefore can go
to arbitrarily high order, making them superior in one way to BDF methods.
Unfortunately, they can be very expensive to compute; recall that for an implicit
method, one has to solve a linear system. In the case of an s−step implicit
multistep method that solves an ODE system of M components, the linear
system is M × M in size. However, for an s−stage IRK method, the linear
6
system is M s × M s in size. Thus, in practice, Gauss-Legendre methods of p > 6
are rarely used. This is why even BDF3 and BDF4 are popular despite their
lack of A-stability; on large systems of ODEs, they are very efficient to solve (at
the cost of extra storage).
Here is the Butcher tableau for the two-stage Gauss-Legendre method of order 4.
1
2
1
2
√
−
+
3
√6
3
6
1
4
1
4√
+
3
6
1
2
1
4
√
−
3
6
1
4
1
2
Expanding the stages from the Butcher tableau, we have
∆t
(k1 + k2 ),
2
√ !
3
1
−
∆t, yn + ∆t
tn +
2
6
√ !
1
3
+
∆t, yn + ∆t
tn +
2
6
y n+1 = y n +
k1 = f
k2 = f
(37)
√ ! !!
1
1
3
k1 +
−
k2
,
4
4
6
!!
√ !
1
1
3
+
k1 + k2
.
4
6
4
(38)
(39)
This is obviously implicit! It also doesn’t look as “nice” as an explicit RK
method!
4.2
Radau methods
Radau methods have the highest possible order among quadrature formulae
with either c1 = 0 or cs = 1. In these cases, p = 2s − 1. Radau formulae
allow either the first or last RK stage to be explicit, thereby offering lower costs
than Gauss-Legendre RK methods while also allowing for a high-order A-stable
method. If cs = 1, the corresponding collocation method is called a Radau IIA
method.
4.3
Lobatto methods
Lobatto methods have the highest possible order with c1 = 0 and cs = 1. For
this to be possible, the c values must be zeros of
ds−2
xs−1 (x − 1)s−1 .
dxs−2
(40)
The order is p = 2s − 2. When s = 2, we recover the implicit Trapezoidal rule.
7