Runge–Kutta methods for
ordinary differential equations
John Butcher
The University of Auckland
New Zealand
COE Workshop on Numerical Analysis
Kyushu University
May 2005
Runge–Kutta methods for ordinary differential equations – p. 1/48
Contents
Introduction to Runge–Kutta methods
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Construction of low order explicit methods
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Construction of low order explicit methods
Order barriers
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Construction of low order explicit methods
Order barriers
Algebraic interpretation
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Construction of low order explicit methods
Order barriers
Algebraic interpretation
Effective order
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Construction of low order explicit methods
Order barriers
Algebraic interpretation
Effective order
Implicit Runge–Kutta methods
Runge–Kutta methods for ordinary differential equations – p. 2/48
Contents
Introduction to Runge–Kutta methods
Formulation of method
Taylor expansion of exact solution
Taylor expansion for numerical approximation
Order conditions
Construction of low order explicit methods
Order barriers
Algebraic interpretation
Effective order
Implicit Runge–Kutta methods
Singly-implicit methods
Runge–Kutta methods for ordinary differential equations – p. 2/48
Introduction to Runge–Kutta methods
It will be convenient to consider only autonomous initial
value problems
y 0 (x) = f (y(x)),
y(x0 ) = y0 ,
f : R N → RN .
Runge–Kutta methods for ordinary differential equations – p. 3/48
Introduction to Runge–Kutta methods
It will be convenient to consider only autonomous initial
value problems
y 0 (x) = f (y(x)),
y(x0 ) = y0 ,
f : R N → RN .
The simple Euler method:
yn = yn−1 + hf (yn−1 ),
h = xn − xn−1
can be made more accurate by using
the mid-point
quadrature formula:
1
yn = yn−1 + hf yn−1 + 2 hf (yn−1 ) .
Runge–Kutta methods for ordinary differential equations – p. 3/48
Introduction to Runge–Kutta methods
It will be convenient to consider only autonomous initial
value problems
y 0 (x) = f (y(x)),
y(x0 ) = y0 ,
f : R N → RN .
The simple Euler method:
yn = yn−1 + hf (yn−1 ),
h = xn − xn−1
can be made more accurate by using either the mid-point
or the trapezoidal rule quadrature formula:
1
yn = yn−1 + hf yn−1 + 2 hf (yn−1 ) .
1
1
yn = yn−1 + 2 hf (yn−1 ) + 2 hf yn−1 + hf (yn−1 ) .
Runge–Kutta methods for ordinary differential equations – p. 3/48
These methods from Runge’s 1895 paper are “second
order” because the error in a single step behaves like
O(h3 ).
Runge–Kutta methods for ordinary differential equations – p. 4/48
These methods from Runge’s 1895 paper are “second
order” because the error in a single step behaves like
O(h3 ).
A few years later, Heun gave a full explanation of order 3
methods and Kutta gave a detailed analysis of order 4
methods.
Runge–Kutta methods for ordinary differential equations – p. 4/48
These methods from Runge’s 1895 paper are “second
order” because the error in a single step behaves like
O(h3 ).
A few years later, Heun gave a full explanation of order 3
methods and Kutta gave a detailed analysis of order 4
methods.
In the early days of Runge–Kutta methods the aim
seemed to be to find explicit methods of higher and
higher order.
Runge–Kutta methods for ordinary differential equations – p. 4/48
These methods from Runge’s 1895 paper are “second
order” because the error in a single step behaves like
O(h3 ).
A few years later, Heun gave a full explanation of order 3
methods and Kutta gave a detailed analysis of order 4
methods.
In the early days of Runge–Kutta methods the aim
seemed to be to find explicit methods of higher and
higher order.
Later the aim shifted to finding methods that seemed to
be optimal in terms of local truncation error and to
finding built-in error estimators.
Runge–Kutta methods for ordinary differential equations – p. 4/48
With the emergence of stiff problems as an important
application area, attention moved to implicit methods.
Runge–Kutta methods for ordinary differential equations – p. 5/48
With the emergence of stiff problems as an important
application area, attention moved to implicit methods.
Methods have been found based on Gaussian quadrature.
Runge–Kutta methods for ordinary differential equations – p. 5/48
With the emergence of stiff problems as an important
application area, attention moved to implicit methods.
Methods have been found based on Gaussian quadrature.
Later this extended to methods related to Radau and
Lobatto quadrature.
Runge–Kutta methods for ordinary differential equations – p. 5/48
With the emergence of stiff problems as an important
application area, attention moved to implicit methods.
Methods have been found based on Gaussian quadrature.
Later this extended to methods related to Radau and
Lobatto quadrature.
A-stable methods exist in these classes.
Runge–Kutta methods for ordinary differential equations – p. 5/48
With the emergence of stiff problems as an important
application area, attention moved to implicit methods.
Methods have been found based on Gaussian quadrature.
Later this extended to methods related to Radau and
Lobatto quadrature.
A-stable methods exist in these classes.
Because of the high cost of these methods, attention
moved to diagonally and singly implicit methods.
Runge–Kutta methods for ordinary differential equations – p. 5/48
Formulation of method
In carrying out a step we evaluate s stage values
Y1 ,
Y2 ,
...,
Ys
and s stage derivatives
F1 , F 2 ,
...,
Fs ,
using the formula Fi = f (Yi ).
Runge–Kutta methods for ordinary differential equations – p. 6/48
Formulation of method
In carrying out a step we evaluate s stage values
Y1 ,
Y2 ,
...,
Ys
and s stage derivatives
F1 , F 2 ,
...,
Fs ,
using the formula Fi = f (Yi ).
Each Yi is found as a linear combination of the Fj added
on to y0 :
s
X
aij Fj
Yi = y 0 + h
j=1
Runge–Kutta methods for ordinary differential equations – p. 6/48
Formulation of method
In carrying out a step we evaluate s stage values
Y1 ,
Y2 ,
...,
Ys
and s stage derivatives
F1 , F 2 ,
...,
Fs ,
using the formula Fi = f (Yi ).
Each Yi is found as a linear combination of the Fj added
on to y0 :
s
X
aij Fj
Yi = y 0 + h
j=1
and the approximation at x1 = x0 + h is found from
s
X
b i Fi
y1 = y0 + h
i=1
Runge–Kutta methods for ordinary differential equations – p. 6/48
We represent the method by a tableau:
c1 a11
c2 a21
.. ..
. .
cs as1
b1
a12
a22
..
.
as2
b2
· · · a1s
· · · a2s
..
.
· · · ass
· · · bs
Runge–Kutta methods for ordinary differential equations – p. 7/48
We represent the method by a tableau:
c1 a11
c2 a21
.. ..
. .
cs as1
b1
a12
a22
..
.
as2
b2
· · · a1s
· · · a2s
..
.
· · · ass
· · · bs
or, if the method is explicit, by the simplified tableau
0
c2 a21
.. ..
.. . . .
. .
.
cs as1 as2 · · · as,s−1
b1 b2 · · · bs−1 bs
Runge–Kutta methods for ordinary differential equations – p. 7/48
Examples:
1
y1 = y0 + 0hf (y0 ) + 1hf y0 + 2 hf (y0 )
0
1
2
1
2
0 1
Runge–Kutta methods for ordinary differential equations – p. 8/48
Examples:
1
y1 = y0 + 0hf (y0 ) + 1hf y0 + 2 hf (y0 )
Y1
0
1
2
1
2
Y2
0 1
Runge–Kutta methods for ordinary differential equations – p. 8/48
Examples:
1
y1 = y0 + 0hf (y0 ) + 1hf y0 + 2 hf (y0 )
0
Y1
1
2
Y2
1
2
0 1
y1 = y0 +
1
hf (y0 )
2
+
1
hf
2
y0 + 1hf (y0 )
0
1 1
1
2
1
2
Runge–Kutta methods for ordinary differential equations – p. 8/48
Examples:
1
y1 = y0 + 0hf (y0 ) + 1hf y0 + 2 hf (y0 )
0
Y1
1
2
Y2
1
2
0 1
y1 = y0 +
Y1
1
hf (y0 )
2
+
1
hf
2
y0 + 1hf (y0 )
0
1 1
1
2
Y2
1
2
Runge–Kutta methods for ordinary differential equations – p. 8/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
Runge–Kutta methods for ordinary differential equations – p. 9/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
y 0 (x) = f (y(x))
Runge–Kutta methods for ordinary differential equations – p. 9/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
y 0 (x) = f (y(x))
y 00 (x) = f 0 (y(x))y 0 (x)
= f 0 (y(x))f (y(x)))
Runge–Kutta methods for ordinary differential equations – p. 9/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
y 0 (x) = f (y(x))
y 00 (x) = f 0 (y(x))y 0 (x)
= f 0 (y(x))f (y(x)))
y 000 (x) = f 00 (y(x))(f (y(x)), y 0 (x)) + f 0 (y(x))f 0 (y(x))y 0 (x)
=f 00 (y(x))(f (y(x)), f (y(x)))+f 0 (y(x))f 0 (y(x))f (y(x))
Runge–Kutta methods for ordinary differential equations – p. 9/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
y 0 (x) = f (y(x))
y 00 (x) = f 0 (y(x))y 0 (x)
= f 0 (y(x))f (y(x)))
y 000 (x) = f 00 (y(x))(f (y(x)), y 0 (x)) + f 0 (y(x))f 0 (y(x))y 0 (x)
=f 00 (y(x))(f (y(x)), f (y(x)))+f 0 (y(x))f 0 (y(x))f (y(x))
This will become increasingly complicated as we
evaluate higher derivatives.
Runge–Kutta methods for ordinary differential equations – p. 9/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
y 0 (x) = f (y(x))
y 00 (x) = f 0 (y(x))y 0 (x)
= f 0 (y(x))f (y(x)))
y 000 (x) = f 00 (y(x))(f (y(x)), y 0 (x)) + f 0 (y(x))f 0 (y(x))y 0 (x)
=f 00 (y(x))(f (y(x)), f (y(x)))+f 0 (y(x))f 0 (y(x))f (y(x))
This will become increasingly complicated as we
evaluate higher derivatives.
Hence we look for a systematic pattern.
Runge–Kutta methods for ordinary differential equations – p. 9/48
Taylor expansion of exact solution
We need formulae for the second, third, . . . , derivatives.
y 0 (x) = f (y(x))
y 00 (x) = f 0 (y(x))y 0 (x)
= f 0 (y(x))f (y(x)))
y 000 (x) = f 00 (y(x))(f (y(x)), y 0 (x)) + f 0 (y(x))f 0 (y(x))y 0 (x)
=f 00 (y(x))(f (y(x)), f (y(x)))+f 0 (y(x))f 0 (y(x))f (y(x))
This will become increasingly complicated as we
evaluate higher derivatives.
Hence we look for a systematic pattern.
Write f = f (y(x)), f0 = f 0 (y(x)), f00 = f 00 (y(x)), . . . .
Runge–Kutta methods for ordinary differential equations – p. 9/48
y 0 (x) = f
f
00
0
f
f0
000
00
y (x) = f f
y (x) = f (f, f)
+ f0 f0 f
f
f
f00
f
f0
f0
Runge–Kutta methods for ordinary differential equations – p. 10/48
y 0 (x) = f
f
00
0
f
f0
000
00
y (x) = f f
y (x) = f (f, f)
+ f0 f0 f
f
f
f00
f
f0
f0
The various terms have a structure related to rooted-trees.
Runge–Kutta methods for ordinary differential equations – p. 10/48
y 0 (x) = f
f
00
0
f
f0
000
00
y (x) = f f
y (x) = f (f, f)
+ f0 f0 f
f
f
f00
f
f0
f0
The various terms have a structure related to rooted-trees.
Hence, we introduce the set of all rooted trees and some
functions on this set.
Runge–Kutta methods for ordinary differential equations – p. 10/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
σ(t)
symmetry of t = order of automorphism group
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
σ(t)
symmetry of t = order of automorphism group
γ(t)
density of t
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
σ(t)
symmetry of t = order of automorphism group
γ(t)
density of t
α(t)
number of ways of labelling with an ordered set
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
σ(t)
symmetry of t = order of automorphism group
γ(t)
density of t
α(t)
number of ways of labelling with an ordered set
β(t)
number of ways of labelling with an unordered set
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
σ(t)
symmetry of t = order of automorphism group
γ(t)
density of t
α(t)
number of ways of labelling with an ordered set
β(t)
number of ways of labelling with an unordered set
F (t)(y0 ) elementary differential
Runge–Kutta methods for ordinary differential equations – p. 11/48
Let T denote the set of rooted trees:
n
T =
,
,
,
,
,
,
,
,
...
o
We identify the following functions on T .
In this table, t will denote a typical tree
r(t)
order of t = number of vertices
σ(t)
symmetry of t = order of automorphism group
γ(t)
density of t
α(t)
number of ways of labelling with an ordered set
β(t)
number of ways of labelling with an unordered set
F (t)(y0 ) elementary differential
We will give examples of these functions based on t =
Runge–Kutta methods for ordinary differential equations – p. 11/48
t =
Runge–Kutta methods for ordinary differential equations – p. 12/48
t =
1 23 4
r(t) = 7
5
6
7
Runge–Kutta methods for ordinary differential equations – p. 12/48
t =
1 23 4
r(t) = 7
5
6
7
σ(t) = 8
Runge–Kutta methods for ordinary differential equations – p. 12/48
t =
1 23 4
r(t) = 7
5
6
7
σ(t) = 8
1 11 1
γ(t) = 63
3
3
7
Runge–Kutta methods for ordinary differential equations – p. 12/48
t =
1 23 4
5
r(t) = 7
6
7
σ(t) = 8
1 11 1
3
γ(t) = 63
α(t) =
r(t)!
σ(t)γ(t)
3
7
= 10
Runge–Kutta methods for ordinary differential equations – p. 12/48
t =
1 23 4
5
r(t) = 7
6
7
σ(t) = 8
1 11 1
3
γ(t) = 63
3
7
α(t) =
r(t)!
σ(t)γ(t)
= 10
β(t) =
r(t)!
σ(t)
= 630
Runge–Kutta methods for ordinary differential equations – p. 12/48
t =
1 23 4
5
r(t) = 7
6
7
σ(t) = 8
1 11 1
3
γ(t) = 63
7
α(t) =
r(t)!
σ(t)γ(t)
= 10
β(t) =
r(t)!
σ(t)
= 630
00
3
00
00
F (t) = f f (f, f), f (f, f)
f f f f
f00
f00
f00
Runge–Kutta methods for ordinary differential equations – p. 12/48
These functions are easy to compute up to order 4 trees:
t
r(t) 1 2
3
3
4
4
4
4
σ(t) 1 1
2
1
6
1
2
1
γ(t) 1 2
3
6
4
8
12
24
α(t) 1 1
1
1
1
3
1
1
β(t) 1 2
3
6
4
24
12
24
F (t) f f0 f f00 (f, f) f0 f0 f f(3) (f, f, f) f00 (f, f0 f) f0 f00 (f, f) f0 f0 f0 f
Runge–Kutta methods for ordinary differential equations – p. 13/48
The formal Taylor expansion of the solution at x0 + h is
y(x0 + h) = y0 +
X α(t)hr(t)
t∈T
r(t)!
F (t)(y0 )
Runge–Kutta methods for ordinary differential equations – p. 14/48
The formal Taylor expansion of the solution at x0 + h is
y(x0 + h) = y0 +
X α(t)hr(t)
t∈T
r(t)!
F (t)(y0 )
Using the known formula for α(t), we can write this as
y(x0 + h) = y0 +
X
t∈T
hr(t)
F (t)(y0 )
σ(t)γ(t)
Runge–Kutta methods for ordinary differential equations – p. 14/48
The formal Taylor expansion of the solution at x0 + h is
y(x0 + h) = y0 +
X α(t)hr(t)
t∈T
r(t)!
F (t)(y0 )
Using the known formula for α(t), we can write this as
y(x0 + h) = y0 +
X
t∈T
hr(t)
F (t)(y0 )
σ(t)γ(t)
Our aim will now be to find a corresponding formula for
the result computed by one step of a Runge-Kutta
method.
Runge–Kutta methods for ordinary differential equations – p. 14/48
The formal Taylor expansion of the solution at x0 + h is
y(x0 + h) = y0 +
X α(t)hr(t)
t∈T
r(t)!
F (t)(y0 )
Using the known formula for α(t), we can write this as
y(x0 + h) = y0 +
X
t∈T
hr(t)
F (t)(y0 )
σ(t)γ(t)
Our aim will now be to find a corresponding formula for
the result computed by one step of a Runge-Kutta
method.
By comparing these formulae term by term, we will
obtain conditions for a specific order of accuracy.
Runge–Kutta methods for ordinary differential equations – p. 14/48
Taylor expansion for numerical approximation
We need to evaluate various expressions which depend
on the tableau for a particular method.
Runge–Kutta methods for ordinary differential equations – p. 15/48
Taylor expansion for numerical approximation
We need to evaluate various expressions which depend
on the tableau for a particular method.
These are known as “elementary weights”.
Runge–Kutta methods for ordinary differential equations – p. 15/48
Taylor expansion for numerical approximation
We need to evaluate various expressions which depend
on the tableau for a particular method.
These are known as “elementary weights”.
We use the example tree we have already considered to
illustrate the construction of the elementary weight Φ(t).
l
t=
m n
j
o
k
i
Runge–Kutta methods for ordinary differential equations – p. 15/48
Taylor expansion for numerical approximation
We need to evaluate various expressions which depend
on the tableau for a particular method.
These are known as “elementary weights”.
We use the example tree we have already considered to
illustrate the construction of the elementary weight Φ(t).
l
t=
Φ(t) =
s
X
m n
j
o
k
i
bi aij aik ajl ajmaknako
i,j,k,l,m,n,o=1
Runge–Kutta methods for ordinary differential equations – p. 15/48
Taylor expansion for numerical approximation
We need to evaluate various expressions which depend
on the tableau for a particular method.
These are known as “elementary weights”.
We use the example tree we have already considered to
illustrate the construction of the elementary weight Φ(t).
l
t=
Φ(t) =
m n
j
s
X
o
k
i
bi aij aik ajl ajmaknako
i,j,k,l,m,n,o=1
Simplify by summing over l, m, n, o:
Φ(t) =
s
X
2
2
biaij cj aik ck
i,j,k=1
Runge–Kutta methods for ordinary differential equations – p. 15/48
Now add Φ(t) to the table of functions:
t
r(t)
α(t)
β(t)
Φ(t)
1
1
1
P
bi
2
1
2
P
bi ci
3
1
3
P 2
bi ci
3
1
6
P
bi aij cj
t
r(t)
4
4
4
4
α(t)
1
3
1
1
4
24
12
24
β(t)
P 3 P
P
P
2
Φ(t)
bi ci
bi ci aij cj
bi aij cj
bi aij ajk ck
Runge–Kutta methods for ordinary differential equations – p. 16/48
The formal Taylor expansion of the solution at x0 + h is
y1 = y0 +
X β(t)hr(t)
t∈T
r(t)!
Φ(t)F (t)(y0 )
Runge–Kutta methods for ordinary differential equations – p. 17/48
The formal Taylor expansion of the solution at x0 + h is
y1 = y0 +
X β(t)hr(t)
t∈T
r(t)!
Φ(t)F (t)(y0 )
Using the known formula for β(t), we can write this as
y1 = y 0 +
X hr(t)
t∈T
σ(t)
Φ(t)F (t)(y0 )
Runge–Kutta methods for ordinary differential equations – p. 17/48
Order conditions
To match the Taylor series
X hr(t)
F (t)(y0 )
y(x0 + h) = y0 +
σ(t)γ(t)
t∈T
y1 = y0 +
X hr(t)
t∈T
σ(t)
Φ(t)F (t)(y0 )
up to hp terms we need to ensure that
1
,
Φ(t) =
γ(t)
Runge–Kutta methods for ordinary differential equations – p. 18/48
Order conditions
To match the Taylor series
X hr(t)
F (t)(y0 )
y(x0 + h) = y0 +
σ(t)γ(t)
t∈T
y1 = y0 +
X hr(t)
t∈T
σ(t)
Φ(t)F (t)(y0 )
up to hp terms we need to ensure that
for all trees such that
1
,
Φ(t) =
γ(t)
r(t) ≤ p.
Runge–Kutta methods for ordinary differential equations – p. 18/48
Order conditions
To match the Taylor series
X hr(t)
F (t)(y0 )
y(x0 + h) = y0 +
σ(t)γ(t)
t∈T
y1 = y0 +
X hr(t)
t∈T
σ(t)
Φ(t)F (t)(y0 )
up to hp terms we need to ensure that
for all trees such that
1
,
Φ(t) =
γ(t)
r(t) ≤ p.
These are the “order conditions”.
Runge–Kutta methods for ordinary differential equations – p. 18/48
Construction of low order explicit methods
We will attempt to construct methods of order p = s with
s stages for s = 1, 2, . . . .
Runge–Kutta methods for ordinary differential equations – p. 19/48
Construction of low order explicit methods
We will attempt to construct methods of order p = s with
s stages for s = 1, 2, . . . .
We will find that this is possible up to order 4 but not for
p ≥ 5.
Runge–Kutta methods for ordinary differential equations – p. 19/48
Construction of low order explicit methods
We will attempt to construct methods of order p = s with
s stages for s = 1, 2, . . . .
We will find that this is possible up to order 4 but not for
p ≥ 5.
The usual approach will be to first choose c2 , c3 , . . . , cs
and then solve for b1 , b2 , . . . , bs .
Runge–Kutta methods for ordinary differential equations – p. 19/48
Construction of low order explicit methods
We will attempt to construct methods of order p = s with
s stages for s = 1, 2, . . . .
We will find that this is possible up to order 4 but not for
p ≥ 5.
The usual approach will be to first choose c2 , c3 , . . . , cs
and then solve for b1 , b2 , . . . , bs .
After this solve for those of the aij which can be found as
solutions to linear equations.
Runge–Kutta methods for ordinary differential equations – p. 19/48
Construction of low order explicit methods
We will attempt to construct methods of order p = s with
s stages for s = 1, 2, . . . .
We will find that this is possible up to order 4 but not for
p ≥ 5.
The usual approach will be to first choose c2 , c3 , . . . , cs
and then solve for b1 , b2 , . . . , bs .
After this solve for those of the aij which can be found as
solutions to linear equations.
b1 + b 2 = 1
Order 2:
b2 c2 = 12
0
c2
0
c2
1 − 2c12
1
2c2
1
2
1
2
0 1
0
1 1
1
2
1
2
Runge–Kutta methods for ordinary differential equations – p. 19/48
Order 3:
b1 + b 2 + b 3
b2 c2 + b 3 c3
b2 c22 + b3 c23
b3 a32 c2
=1
= 12
= 13
= 16
Runge–Kutta methods for ordinary differential equations – p. 20/48
b1 + b 2 + b 3
b2 c2 + b 3 c3
b2 c22 + b3 c23
b3 a32 c2
Order 3:
0
1
2
=1
= 12
= 13
= 16
0
1
2
1 −1 2
1
6
2
3
1
6
2
3
2
3
0
2
3
0
1
4
2
3
3
8
2
3
3
8
2
3
0 −1 1
0 34
1
4
Runge–Kutta methods for ordinary differential equations – p. 20/48
b1 + b 2 + b 3 + b 4
b2 c2 + b 3 c3 + b 4 c4
b2 c22 + b3 c23 + b4 c24
b3 a32 c2 + b4 a42 c2 + b4 a43 c3
b2 c32 + b3 c33 + b4 c34
b3 c3 a32 c2 + b4 c4 a42 c2 + b4 c4 a43 c3
2
2
2
b3 a32 c2 + b4 a42 c2 + b4 a43 c3
b4 a43 a32 c2
Order 4:
=1
= 12
= 13
= 16
= 14
= 18
1
= 12
1
= 24
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
Runge–Kutta methods for ordinary differential equations – p. 21/48
b1 + b 2 + b 3 + b 4 = 1
(1)
(2)
b2 c2 + b3 c3 + b4 c4 = 12
(3)
b2 c22 + b3 c23 + b4 c24 = 13
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 16
(4)
b2 c32 + b3 c33 + b4 c34 = 14
(5)
b3 c3 a32 c2 + b4 c4 a42 c2 + b4 c4 a43 c3 = 18
(6)
2
2
2
1
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 12
(7)
1
b4 a43 a32 c2 = 24
(8)
To solve these equations, treat c2 , c3 , c4 as parameters,
and solve for b1 , b2 , b3 , b4 from (1), (2), (3), (5).
Order 4:
Runge–Kutta methods for ordinary differential equations – p. 21/48
b1 + b 2 + b 3 + b 4 = 1
(1)
(2)
b2 c2 + b3 c3 + b4 c4 = 12
(3)
b2 c22 + b3 c23 + b4 c24 = 13
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 16
(4)
b2 c32 + b3 c33 + b4 c34 = 14
(5)
b3 c3 a32 c2 + b4 c4 a42 c2 + b4 c4 a43 c3 = 18
(6)
2
2
2
1
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 12
(7)
1
b4 a43 a32 c2 = 24
(8)
To solve these equations, treat c2 , c3 , c4 as parameters,
and solve for b1 , b2 , b3 , b4 from (1), (2), (3), (5).
Now solve for a32 , a42 , a43 from (4). (6), (7).
Order 4:
Runge–Kutta methods for ordinary differential equations – p. 21/48
b1 + b 2 + b 3 + b 4 = 1
(1)
(2)
b2 c2 + b3 c3 + b4 c4 = 12
(3)
b2 c22 + b3 c23 + b4 c24 = 13
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 16
(4)
b2 c32 + b3 c33 + b4 c34 = 14
(5)
b3 c3 a32 c2 + b4 c4 a42 c2 + b4 c4 a43 c3 = 18
(6)
2
2
2
1
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 12
(7)
1
b4 a43 a32 c2 = 24
(8)
To solve these equations, treat c2 , c3 , c4 as parameters,
and solve for b1 , b2 , b3 , b4 from (1), (2), (3), (5).
Now solve for a32 , a42 , a43 from (4). (6), (7).
Use (8) to obtain consistency condition on c2 , c3 , c4 .
Order 4:
Runge–Kutta methods for ordinary differential equations – p. 21/48
b1 + b 2 + b 3 + b 4 = 1
(1)
(2)
b2 c2 + b3 c3 + b4 c4 = 12
(3)
b2 c22 + b3 c23 + b4 c24 = 13
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 16
(4)
b2 c32 + b3 c33 + b4 c34 = 14
(5)
b3 c3 a32 c2 + b4 c4 a42 c2 + b4 c4 a43 c3 = 18
(6)
2
2
2
1
b3 a32 c2 + b4 a42 c2 + b4 a43 c3 = 12
(7)
1
b4 a43 a32 c2 = 24
(8)
To solve these equations, treat c2 , c3 , c4 as parameters,
and solve for b1 , b2 , b3 , b4 from (1), (2), (3), (5).
Now solve for a32 , a42 , a43 from (4). (6), (7).
Use (8) to obtain consistency condition on c2 , c3 , c4 .
Result: c4 = 1.
Order 4:
Runge–Kutta methods for ordinary differential equations – p. 21/48
We will prove a stronger result in another way.
Runge–Kutta methods for ordinary differential equations – p. 22/48
We will prove a stronger result in another way.
Lemma 1 Let U and V be 3 × 3 matrices such that


w11 w12 0
w11 w12


U V = w21 w22 0 where
is non-singular
w21 w22
0
0 0
then either the last row of U is zero or the last column of
V is zero.
Runge–Kutta methods for ordinary differential equations – p. 22/48
We will prove a stronger result in another way.
Lemma 1 Let U and V be 3 × 3 matrices such that


w11 w12 0
w11 w12


U V = w21 w22 0 where
is non-singular
w21 w22
0
0 0
then either the last row of U is zero or the last column of
V is zero.
Proof Let W = U V . Either U or V is singular. If U is singular, let
x be a non-zero vector such that xT U = 0. Therefore xT W = 0.
Therefore the first two components of x are zero. Hence, the last row
of U is zero. The second case follows similarly if V is singular.
Runge–Kutta methods for ordinary differential equations – p. 22/48
We will prove a stronger result in another way.
Lemma 1 Let U and V be 3 × 3 matrices such that


w11 w12 0
w11 w12


U V = w21 w22 0 where
is non-singular
w21 w22
0
0 0
then either the last row of U is zero or the last column of
V is zero.
Proof Let W = U V . Either U or V is singular. If U is singular, let
x be a non-zero vector such that xT U = 0. Therefore xT W = 0.
Therefore the first two components of x are zero. Hence, the last row
of U is zero. The second case follows similarly if V is singular.
We will apply this result with a specific choice of U and
V.
Runge–Kutta methods for ordinary differential equations – p. 22/48
Let

b2

b2 c2

P
U =
i bi ai2
−b2 (1 − c2 )
b3
bc
P 3 3
i bi ai3
−b3 (1 − c3 )

b4

b4 c4

P

b
a
i i i4
−b4 (1 − c4 )
Runge–Kutta methods for ordinary differential equations – p. 23/48
Let


b2
b3
b4


b
c
b
c
b
c
2 2
3 3
4 4

P
P
P
U =


b
a
b
a
b
a
i i i2
i i i3
i i i4
−b2 (1 − c2 )
−b3 (1 − c3 )
−b4 (1 − c4 )
P

1 2 
2
c2 c2
a2j cj − 2 c2
j
P
1 2 
2
a
c
−
c3
V =  c3 c3
3j j
j
2
P
1 2
2
c4 c4
a
c
−
j 4j j
2 c4
Runge–Kutta methods for ordinary differential equations – p. 23/48
Let


b2
b3
b4


b
c
b
c
b
c
2 2
3 3
4 4

P
P
P
U =


b
a
b
a
b
a
i i i2
i i i3
i i i4
−b2 (1 − c2 )
−b3 (1 − c3 )
−b4 (1 − c4 )
P

1 2 
2
c2 c2
a2j cj − 2 c2
j
P
1 2 
2
a
c
−
c3
V =  c3 c3
3j j
j
2
P
1 2
2
c4 c4
a
c
−
j 4j j
2 c4
then

 1 1
2 3 0
U V =  13 41 0 
0 0 0
Runge–Kutta methods for ordinary differential equations – p. 23/48
It follows that b4 = 0, c2 = 0 or c4 = 1.
Runge–Kutta methods for ordinary differential equations – p. 24/48
It follows that b4 = 0, c2 = 0 or c4 = 1.
The first two options are impossible because
1
b4 a43 a32 c2 = 24
.
Runge–Kutta methods for ordinary differential equations – p. 24/48
It follows that b4 = 0, c2 = 0 or c4 = 1.
The first two options are impossible because
1
b4 a43 a32 c2 = 24
.
Hence, c4 = 1 and the last row of U is zero.
Runge–Kutta methods for ordinary differential equations – p. 24/48
It follows that b4 = 0, c2 = 0 or c4 = 1.
The first two options are impossible because
1
b4 a43 a32 c2 = 24
.
Hence, c4 = 1 and the last row of U is zero.
The construction of fourth order Runge–Kutta methods
now becomes straightforward.
Runge–Kutta methods for ordinary differential equations – p. 24/48
It follows that b4 = 0, c2 = 0 or c4 = 1.
The first two options are impossible because
1
b4 a43 a32 c2 = 24
.
Hence, c4 = 1 and the last row of U is zero.
The construction of fourth order Runge–Kutta methods
now becomes straightforward.
Kutta classified all solutions to the fourth order
conditions.
Runge–Kutta methods for ordinary differential equations – p. 24/48
It follows that b4 = 0, c2 = 0 or c4 = 1.
The first two options are impossible because
1
b4 a43 a32 c2 = 24
.
Hence, c4 = 1 and the last row of U is zero.
The construction of fourth order Runge–Kutta methods
now becomes straightforward.
Kutta classified all solutions to the fourth order
conditions.
In particular we have his famous method:
0
1
2
1
2
1
2
0 12
1 0 0 1
1
6
1
3
1
3
1
6
Runge–Kutta methods for ordinary differential equations – p. 24/48
Order barriers
We will review what is achievable up to order 8.
Runge–Kutta methods for ordinary differential equations – p. 25/48
Order barriers
We will review what is achievable up to order 8.
In the table below, Np is the number of order conditions
to achieve this order.
Runge–Kutta methods for ordinary differential equations – p. 25/48
Order barriers
We will review what is achievable up to order 8.
In the table below, Np is the number of order conditions
to achieve this order.
Ms = s(s + 1)/2 is the number of free parameters to
satisfy the order conditions for the required s stages.
Runge–Kutta methods for ordinary differential equations – p. 25/48
Order barriers
We will review what is achievable up to order 8.
In the table below, Np is the number of order conditions
to achieve this order.
Ms = s(s + 1)/2 is the number of free parameters to
satisfy the order conditions for the required s stages.
p Np s M s
1
1 1
1
2
2 2
3
3
4 3
6
4
8 4 10
5 17 6 21
6 37 7 28
7 115 9 45
8 200 11 66
Runge–Kutta methods for ordinary differential equations – p. 25/48
We will now prove that s = p = 5 is impossible.
Runge–Kutta methods for ordinary differential equations – p. 26/48
We will now prove that s = p = 5 is impossible.
P
5
b
Let bj = i=1 bi aij , j = 1, 2, 3, 4 and let


bb3
bb4
bb2


b
b
b


b2 c2
b3 c3
b4 c4
U = P

P
P
bbi ai2
bbi ai3
bbi ai4


i
i
i
− 12bb2 (1 − c2 )
− 12bb3 (1 − c3 )
− 12bb4 (1 − c4 )
Runge–Kutta methods for ordinary differential equations – p. 26/48
We will now prove that s = p = 5 is impossible.
P
5
b
Let bj = i=1 bi aij , j = 1, 2, 3, 4 and let


bb3
bb4
bb2


b
b
b


b2 c2
b3 c3
b4 c4
U = P

P
P
bbi ai2
bbi ai3
bbi ai4


i
i
i
− 12bb2 (1 − c2 )
− 12bb3 (1 − c3 )
− 12bb4 (1 − c4 )
P

1 2 
2
a2j cj − 2 c2
c2 c2
j
P
1 2 
2
a
c
−
c3
V =  c3 c3
3j j
j
2
P
1 2
2
c4 c4
a
c
−
j 4j j
2 c4
Runge–Kutta methods for ordinary differential equations – p. 26/48
We will now prove that s = p = 5 is impossible.
P
5
b
Let bj = i=1 bi aij , j = 1, 2, 3, 4 and let


bb3
bb4
bb2


b
b
b


b2 c2
b3 c3
b4 c4
U = P

P
P
bbi ai2
bbi ai3
bbi ai4


i
i
i
− 12bb2 (1 − c2 )
− 12bb3 (1 − c3 )
− 12bb4 (1 − c4 )
P

1 2 
2
a2j cj − 2 c2
c2 c2
j
P
1 2 
2
a
c
−
c3
V =  c3 c3
3j j
j
2
P
1 2
2
c4 c4
a
c
−
j 4j j
2 c4
then

 1 1
6 12 0
1
1

U V =  12
0
20
0 0 0
Runge–Kutta methods for ordinary differential equations – p. 26/48
Using Lemma 1, we deduce that c4 = 1.
Runge–Kutta methods for ordinary differential equations – p. 27/48
Using Lemma 1, we deduce that c4 = 1. Now use the
lemma
again
with


b2 (1 − c2 )
b3 (1 − c3 )
b5 (1 − c5 )
 b2 c2 (1 − c2 )

b
c
(1
−
c
)
b
c
(1
−
c
)
3
3
3
5
5
5

P
P
P
U =


b
a
(1
−
c
)
b
a
(1
−
c
)
b
a
(1
−
c
)
2
3
5
i i i2
i i i3
i i i5
−b2 (1 − c2 )2
−b3 (1 − c3 )2
−b5 (1 − c5 )2
Runge–Kutta methods for ordinary differential equations – p. 27/48
Using Lemma 1, we deduce that c4 = 1. Now use the
lemma
again
with


b2 (1 − c2 )
b3 (1 − c3 )
b5 (1 − c5 )
 b2 c2 (1 − c2 )

b
c
(1
−
c
)
b
c
(1
−
c
)
3
3
3
5
5
5

P
P
P
U =


b
a
(1
−
c
)
b
a
(1
−
c
)
b
a
(1
−
c
)
2
3
5
i i i2
i i i3
i i i5
−b2 (1 − c2 )2
−b3 (1 − c3 )2
−b5 (1 − c5 )2
P

1 2 
2
a2j cj − 2 c2
c2 c2
j
P
1 2 
2
a
c
−
c3
V =  c3 c3
3j j
j
2
P
1 2
2
c5 c5
a
c
−
j 5j j
2 c5
Runge–Kutta methods for ordinary differential equations – p. 27/48
Using Lemma 1, we deduce that c4 = 1. Now use the
lemma
again
with


b2 (1 − c2 )
b3 (1 − c3 )
b5 (1 − c5 )
 b2 c2 (1 − c2 )

b
c
(1
−
c
)
b
c
(1
−
c
)
3
3
3
5
5
5

P
P
P
U =


b
a
(1
−
c
)
b
a
(1
−
c
)
b
a
(1
−
c
)
2
3
5
i i i2
i i i3
i i i5
−b2 (1 − c2 )2
−b3 (1 − c3 )2
−b5 (1 − c5 )2
P

1 2 
2
a2j cj − 2 c2
c2 c2
j
P
1 2 
2
a
c
−
c3
V =  c3 c3
3j j
j
2
P
1 2
2
c5 c5
a
c
−
j 5j j
2 c5
then

 1 1
6 12 0
1
1
.
U V =  12
0
20
0 0 0
Runge–Kutta methods for ordinary differential equations – p. 27/48
It follows that c5 = 1.
Runge–Kutta methods for ordinary differential equations – p. 28/48
It follows that c5 = 1.
Since we already know that c4 = 1, we obtain a
contradiction from
X
bi (1 − ci )aij ajk ck
Runge–Kutta methods for ordinary differential equations – p. 28/48
It follows that c5 = 1.
Since we already know that c4 = 1, we obtain a
contradiction from
X
1
bi (1 − ci )aij ajk ck = 120
Runge–Kutta methods for ordinary differential equations – p. 28/48
It follows that c5 = 1.
Since we already know that c4 = 1, we obtain a
contradiction from
X
1
0=
bi (1 − ci )aij ajk ck = 120
Runge–Kutta methods for ordinary differential equations – p. 28/48
It follows that c5 = 1.
Since we already know that c4 = 1, we obtain a
contradiction from
X
1
0=
bi (1 − ci )aij ajk ck = 120
By modifying the details slightly, we can prove that
s = p > 5 is never possible.
Runge–Kutta methods for ordinary differential equations – p. 28/48
It follows that c5 = 1.
Since we already know that c4 = 1, we obtain a
contradiction from
X
1
0=
bi (1 − ci )aij ajk ck = 120
By modifying the details slightly, we can prove that
s = p > 5 is never possible.
The proof that s = p + 1 is impossible when p ≥ 7 is
more complicated.
Runge–Kutta methods for ordinary differential equations – p. 28/48
It follows that c5 = 1.
Since we already know that c4 = 1, we obtain a
contradiction from
X
1
0=
bi (1 − ci )aij ajk ck = 120
By modifying the details slightly, we can prove that
s = p > 5 is never possible.
The proof that s = p + 1 is impossible when p ≥ 7 is
more complicated.
The proof that s = p + 2 is impossible when p ≥ 8 is
much more complicated.
Runge–Kutta methods for ordinary differential equations – p. 28/48
Algebraic interpretation
We will introduce an algebraic system which represents
individual Runge-Kutta methods and also compositions
of methods.
Runge–Kutta methods for ordinary differential equations – p. 29/48
Algebraic interpretation
We will introduce an algebraic system which represents
individual Runge-Kutta methods and also compositions
of methods.
This centres on the meaning of order for Runge-Kutta
methods and leads to a possible generalisation to
“effective order”.
Runge–Kutta methods for ordinary differential equations – p. 29/48
Algebraic interpretation
We will introduce an algebraic system which represents
individual Runge-Kutta methods and also compositions
of methods.
This centres on the meaning of order for Runge-Kutta
methods and leads to a possible generalisation to
“effective order”.
Denote by G the group consisting of mappings of
(rooted) trees to real numbers where the group operation
is defined in the usual way
Runge–Kutta methods for ordinary differential equations – p. 29/48
Algebraic interpretation
We will introduce an algebraic system which represents
individual Runge-Kutta methods and also compositions
of methods.
This centres on the meaning of order for Runge-Kutta
methods and leads to a possible generalisation to
“effective order”.
Denote by G the group consisting of mappings of
(rooted) trees to real numbers where the group operation
is defined in the usual way, according to the algebraic
theory of Runge-Kutta methods or to the (equivalent)
theory of B-series.
Runge–Kutta methods for ordinary differential equations – p. 29/48
Algebraic interpretation
We will introduce an algebraic system which represents
individual Runge-Kutta methods and also compositions
of methods.
This centres on the meaning of order for Runge-Kutta
methods and leads to a possible generalisation to
“effective order”.
Denote by G the group consisting of mappings of
(rooted) trees to real numbers where the group operation
is defined in the usual way, according to the algebraic
theory of Runge-Kutta methods or to the (equivalent)
theory of B-series.
We will illustrate this operation in a table
Runge–Kutta methods for ordinary differential equations – p. 29/48
Algebraic interpretation
We will introduce an algebraic system which represents
individual Runge-Kutta methods and also compositions
of methods.
This centres on the meaning of order for Runge-Kutta
methods and leads to a possible generalisation to
“effective order”.
Denote by G the group consisting of mappings of
(rooted) trees to real numbers where the group operation
is defined in the usual way, according to the algebraic
theory of Runge-Kutta methods or to the (equivalent)
theory of B-series.
We will illustrate this operation in a table, where we also
introduce the special member E ∈ G.
Runge–Kutta methods for ordinary differential equations – p. 29/48
i ti
1
2
3
4
5
6
7
8
Runge–Kutta methods for ordinary differential equations – p. 30/48
r(ti )
1
2
3
i ti
1
2
3
3
4
4
5
4
6
4
7
4
8
Runge–Kutta methods for ordinary differential equations – p. 30/48
r(ti )
1
2
3
i ti α(ti ) β(ti )
1
α1
β1
2
α2
β2
3
α3
β3
3
4
4
5
α4
α5
β4
β5
4
6
α6
β6
4
7
α7
β7
4
8
α8
β8
Runge–Kutta methods for ordinary differential equations – p. 30/48
r(ti )
1
2
3
i ti α(ti ) β(ti )
1
α1
β1
2
α2
β2
3
α3
β3
(αβ)(ti )
α1 + β 1
α2 + α 1 β 1 + β 2
α3 + α12 β1 + 2α1 β2 + β3
3
4
4
5
α4
α5
4
6
α6
4
7
α7
β4
α4 + α 2 β 1 + α 1 β 2 + β 4
β5 α5 + α13 β1 + 3α12 β2 + 3α1 β3 + β5
2
α
+
α
α
β
+
(α
6
1 2 1
1 + α2 )β2
β6
+ α1 (β3 + β4 ) + β6
β7 α7 + α3 β1 + α12 β2 + 2α1 β4 + β7
4
8
α8
β8
α8 + α 4 β 1 + α 2 β 2 + α 1 β 4 + β 8
Runge–Kutta methods for ordinary differential equations – p. 30/48
r(ti )
1
2
3
i ti α(ti ) β(ti )
1
α1
β1
2
α2
β2
3
α3
β3
3
4
4
5
α4
α5
β4
β5
4
6
α6
β6
4
7
α7
β7
4
8
α8
β8
(αβ)(ti )
α1 + β 1
α2 + α 1 β 1 + β 2
α3 + α12 β1 + 2α1 β2 + β3
E(ti )
1
α8 + α 4 β 1 + α 2 β 2 + α 1 β 4 + β 8
1
24
1
2
1
3
1
α4 + α 2 β 1 + α 1 β 2 + β 4
6
α5 + α13 β1 + 3α12 β2 + 3α1 β3 + β5 14
1
α6 + α1 α2 β1 + (α12 + α2 )β2
8
+ α1 (β3 + β4 ) + β6
1
α7 + α3 β1 + α12 β2 + 2α1 β4 + β7 12
Runge–Kutta methods for ordinary differential equations – p. 30/48
Gp will denote the normal subgroup defined by t 7→ 0 for
r(t) ≤ p.
Runge–Kutta methods for ordinary differential equations – p. 31/48
Gp will denote the normal subgroup defined by t 7→ 0 for
r(t) ≤ p.
If α ∈ G then this maps canonically to αGp ∈ G/Gp.
Runge–Kutta methods for ordinary differential equations – p. 31/48
Gp will denote the normal subgroup defined by t 7→ 0 for
r(t) ≤ p.
If α ∈ G then this maps canonically to αGp ∈ G/Gp.
If α is defined from the elementary weights for a
Runge-Kutta method then order p can be written as
αGp = EGp.
Runge–Kutta methods for ordinary differential equations – p. 31/48
Gp will denote the normal subgroup defined by t 7→ 0 for
r(t) ≤ p.
If α ∈ G then this maps canonically to αGp ∈ G/Gp.
If α is defined from the elementary weights for a
Runge-Kutta method then order p can be written as
αGp = EGp.
Effective order p is defined by the existence of β such
that
βαGp = EβGp.
Runge–Kutta methods for ordinary differential equations – p. 31/48
The computational interpretation of this idea is that we
carry out a starting step corresponding to β
Runge–Kutta methods for ordinary differential equations – p. 32/48
The computational interpretation of this idea is that we
carry out a starting step corresponding to β and a
finishing step corresponding to β −1
Runge–Kutta methods for ordinary differential equations – p. 32/48
The computational interpretation of this idea is that we
carry out a starting step corresponding to β and a
finishing step corresponding to β −1 , with many steps in
between corresponding to α.
Runge–Kutta methods for ordinary differential equations – p. 32/48
The computational interpretation of this idea is that we
carry out a starting step corresponding to β and a
finishing step corresponding to β −1 , with many steps in
between corresponding to α.
This is equivalent to many steps all corresponding to
βαβ −1 .
Runge–Kutta methods for ordinary differential equations – p. 32/48
The computational interpretation of this idea is that we
carry out a starting step corresponding to β and a
finishing step corresponding to β −1 , with many steps in
between corresponding to α.
This is equivalent to many steps all corresponding to
βαβ −1 .
Thus, the benefits of high order can be enjoyed by high
effective order.
Runge–Kutta methods for ordinary differential equations – p. 32/48
We analyse the conditions for effective order 4.
Without loss of generality assume β(t1 ) = 0.
(Eβ)(ti )
i
(βα)(ti )
1
α1
1
1
2
β 2 + α2
2 + β2
1
3
β 3 + α3
3 + 2β2 + β3
1
4
β 4 + β 2 α1 + α 4
6 + β2 + β4
1
5
β 5 + α5
4 + 3β2 + 3β3 + β5
1
3
6
β 6 + β 2 α2 + α 6
+
8
2 β2 + β 3 + β 4 + β 6
1
7
β 7 + β 3 α1 + α 7
12 + β2 + 2β4 + β7
1
1
8 β 8 + β 4 α1 + β 2 α2 + α 8
+
24
2 β2 + β 4 + β 8
Runge–Kutta methods for ordinary differential equations – p. 33/48
Of these 8 conditions, only 5 are conditions on α.
Runge–Kutta methods for ordinary differential equations – p. 34/48
Of these 8 conditions, only 5 are conditions on α.
Once α is known, there remain 3 conditions on β.
Runge–Kutta methods for ordinary differential equations – p. 34/48
Of these 8 conditions, only 5 are conditions on α.
Once α is known, there remain 3 conditions on β.
The 5 order conditions, written in terms of the
Runge-Kutta tableau, are
X
bi = 1
X
bi ci = 12
X
bi aij cj = 16
X
1
bi aij ajk ck = 24
X
X
bi aij cj (2ci − cj ) = 14
bi c2i (1 − ci ) +
Runge–Kutta methods for ordinary differential equations – p. 34/48
Implicit Runge–Kutta methods
Since we have the order barriers, we might ask how to
get around them.
Runge–Kutta methods for ordinary differential equations – p. 35/48
Implicit Runge–Kutta methods
Since we have the order barriers, we might ask how to
get around them. For explicit methods, solving the order
conditions becomes increasingly difficult as the order
increases
Runge–Kutta methods for ordinary differential equations – p. 35/48
Implicit Runge–Kutta methods
Since we have the order barriers, we might ask how to
get around them. For explicit methods, solving the order
conditions becomes increasingly difficult as the order
increases but everything becomes simpler for implicit
methods.
Runge–Kutta methods for ordinary differential equations – p. 35/48
Implicit Runge–Kutta methods
Since we have the order barriers, we might ask how to
get around them. For explicit methods, solving the order
conditions becomes increasingly difficult as the order
increases but everything becomes simpler for implicit
methods.
For example the following method has order 5:
0
1
4
7
10
1
1
8
1
− 100
2
7
1
14
1
8
14
25
0
32
81
3
20
5
7
250
567
5
54
Runge–Kutta methods for ordinary differential equations – p. 35/48
This idea can be taken further by introducing a full lower
triangular A matrix.
Runge–Kutta methods for ordinary differential equations – p. 36/48
This idea can be taken further by introducing a full lower
triangular A matrix.
If all the diagonal elements are equal, we get the
Diagonally-Implicit methods of R. Alexander and the
Semi-Explicit methods of S. P. Nørsett.
Runge–Kutta methods for ordinary differential equations – p. 36/48
This idea can be taken further by introducing a full lower
triangular A matrix.
If all the diagonal elements are equal, we get the
Diagonally-Implicit methods of R. Alexander and the
Semi-Explicit methods of S. P. Nørsett.
The following third order L-stable method illustrates
what is possible for DIRK methods
Runge–Kutta methods for ordinary differential equations – p. 36/48
This idea can be taken further by introducing a full lower
triangular A matrix.
If all the diagonal elements are equal, we get the
Diagonally-Implicit methods of R. Alexander and the
Semi-Explicit methods of S. P. Nørsett.
The following third order L-stable method illustrates
what is possible for DIRK methods
λ
1
2 (1 + λ)
1
λ
1
2 (1 − λ)
1
2
(−6λ
+ 16λ − 1)
4
1
2
(−6λ
+ 16λ − 1)
4
λ
1
2
(6λ
− 20λ + 5) λ
4
1
2
(6λ
− 20λ + 5) λ
4
where λ ≈ 0.4358665215 satisfies 16 − 32 λ+3λ2 −λ3 = 0.
Runge–Kutta methods for ordinary differential equations – p. 36/48
Singly-implicit Runge-Kutta methods
A SIRK method is characterised by the equation
σ(A) = {λ}.
Runge–Kutta methods for ordinary differential equations – p. 37/48
Singly-implicit Runge-Kutta methods
A SIRK method is characterised by the equation
σ(A) = {λ}. That is A has a one-point spectrum.
Runge–Kutta methods for ordinary differential equations – p. 37/48
Singly-implicit Runge-Kutta methods
A SIRK method is characterised by the equation
σ(A) = {λ}. That is A has a one-point spectrum.
For DIRK methods the stages can be computed
independently and sequentially from equations of the
form
Yi − hλf (Yi ) = a known quantity
Runge–Kutta methods for ordinary differential equations – p. 37/48
Singly-implicit Runge-Kutta methods
A SIRK method is characterised by the equation
σ(A) = {λ}. That is A has a one-point spectrum.
For DIRK methods the stages can be computed
independently and sequentially from equations of the
form
Yi − hλf (Yi ) = a known quantity
Each stage requires the same factorised matrix I − hλJ
to permit solution by a modified Newton iteration
process (where J ≈ ∂f /∂y).
Runge–Kutta methods for ordinary differential equations – p. 37/48
Singly-implicit Runge-Kutta methods
A SIRK method is characterised by the equation
σ(A) = {λ}. That is A has a one-point spectrum.
For DIRK methods the stages can be computed
independently and sequentially from equations of the
form
Yi − hλf (Yi ) = a known quantity
Each stage requires the same factorised matrix I − hλJ
to permit solution by a modified Newton iteration
process (where J ≈ ∂f /∂y).
How then is it possible to implement SIRK methods in a
similarly efficient manner?
Runge–Kutta methods for ordinary differential equations – p. 37/48
Singly-implicit Runge-Kutta methods
A SIRK method is characterised by the equation
σ(A) = {λ}. That is A has a one-point spectrum.
For DIRK methods the stages can be computed
independently and sequentially from equations of the
form
Yi − hλf (Yi ) = a known quantity
Each stage requires the same factorised matrix I − hλJ
to permit solution by a modified Newton iteration
process (where J ≈ ∂f /∂y).
How then is it possible to implement SIRK methods in a
similarly efficient manner?
The answer lies in the inclusion of a transformation to
Jordan canonical form into the computation.
Runge–Kutta methods for ordinary differential equations – p. 37/48
Suppose the matrix T transforms A to canonical form as
follows
T −1 AT = A
Runge–Kutta methods for ordinary differential equations – p. 38/48
Suppose the matrix T transforms A to canonical form as
follows
T −1 AT = A
where
A = λ(I − J)
Runge–Kutta methods for ordinary differential equations – p. 38/48
Suppose the matrix T transforms A to canonical form as
follows
T −1 AT = A
where
A = λ(I − J) = λ









1
0
−1
1
0 −1
..
..
.
.
0
0
0
0
0
0
1
..
.
0
0
··· 0
··· 0
··· 0
..
.
··· 1
· · · −1
0
0
0
..
.
0
1









Runge–Kutta methods for ordinary differential equations – p. 38/48
Consider a single Newton iteration, simplified by the use
of the same approximate Jacobian J for each stage.
Runge–Kutta methods for ordinary differential equations – p. 39/48
Consider a single Newton iteration, simplified by the use
of the same approximate Jacobian J for each stage.
Assume the incoming approximation is y0 and that we
are attempting to evaluate
y1 = y0 + h(bT ⊗ I)F
Runge–Kutta methods for ordinary differential equations – p. 39/48
Consider a single Newton iteration, simplified by the use
of the same approximate Jacobian J for each stage.
Assume the incoming approximation is y0 and that we
are attempting to evaluate
y1 = y0 + h(bT ⊗ I)F
where F is made up from the s subvectors Fi = f (Yi ),
i = 1, 2, . . . , s.
Runge–Kutta methods for ordinary differential equations – p. 39/48
Consider a single Newton iteration, simplified by the use
of the same approximate Jacobian J for each stage.
Assume the incoming approximation is y0 and that we
are attempting to evaluate
y1 = y0 + h(bT ⊗ I)F
where F is made up from the s subvectors Fi = f (Yi ),
i = 1, 2, . . . , s.
The implicit equations to be solved are
Y = e ⊗ y0 + h(A ⊗ I)F
Runge–Kutta methods for ordinary differential equations – p. 39/48
Consider a single Newton iteration, simplified by the use
of the same approximate Jacobian J for each stage.
Assume the incoming approximation is y0 and that we
are attempting to evaluate
y1 = y0 + h(bT ⊗ I)F
where F is made up from the s subvectors Fi = f (Yi ),
i = 1, 2, . . . , s.
The implicit equations to be solved are
Y = e ⊗ y0 + h(A ⊗ I)F
where e is the vector in Rn with every component equal
to 1 and Y has subvectors Yi , i = 1, 2, . . . , s
Runge–Kutta methods for ordinary differential equations – p. 39/48
The Newton process consists of solving the linear system
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
Runge–Kutta methods for ordinary differential equations – p. 40/48
The Newton process consists of solving the linear system
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
and updating
Y →Y −D
Runge–Kutta methods for ordinary differential equations – p. 40/48
The Newton process consists of solving the linear system
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
and updating
Y →Y −D
To benefit from the SI property, write
Y = (T −1 ⊗I)Y,
F = (T −1 ⊗I)F,
D = (T −1 ⊗I)D,
Runge–Kutta methods for ordinary differential equations – p. 40/48
The Newton process consists of solving the linear system
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
and updating
Y →Y −D
To benefit from the SI property, write
Y = (T −1 ⊗I)Y,
F = (T −1 ⊗I)F,
D = (T −1 ⊗I)D,
so that
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
Runge–Kutta methods for ordinary differential equations – p. 40/48
The Newton process consists of solving the linear system
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
and updating
Y →Y −D
To benefit from the SI property, write
Y = (T −1 ⊗I)Y,
F = (T −1 ⊗I)F,
D = (T −1 ⊗I)D,
so that
(Is ⊗ I − hA ⊗ J )D = Y − e ⊗ y0 − h(A ⊗ I)F
The following table summarises the costs
Runge–Kutta methods for ordinary differential equations – p. 40/48
LU factorisation
s3 N 3
Backsolves
s2 N 2
Runge–Kutta methods for ordinary differential equations – p. 41/48
without
transformation
LU factorisation
s3 N 3
Backsolves
s2 N 2
Runge–Kutta methods for ordinary differential equations – p. 41/48
without
transformation
LU factorisation
s3 N 3
Backsolves
s2 N 2
with
transformation
Runge–Kutta methods for ordinary differential equations – p. 41/48
without
transformation
with
transformation
LU factorisation
s3 N 3
N3
Backsolves
s2 N 2
sN 2
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Also we reduce the back substitution cost
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Also we reduce the back substitution cost
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Also we reduce the back substitution cost to the same
work per stage as for DIRK or BDF
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Also we reduce the back substitution cost to the same
work per stage as for DIRK or BDF methods.
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Also we reduce the back substitution cost to the same
work per stage as for DIRK or BDF methods.
By comparison, the additional transformation costs are
insignificant for large problems
Runge–Kutta methods for ordinary differential equations – p. 41/48
LU factorisation
Transformation
Backsolves
Transformation
without
transformation
with
transformation
s3 N 3
N3
s2 N
sN 2
s2 N
s2 N 2
In summary, we reduce the very high LU factorisation
cost to a level comparable to BDF methods.
Also we reduce the back substitution cost to the same
work per stage as for DIRK or BDF methods.
By comparison, the additional transformation costs are
insignificant for large problems .
Runge–Kutta methods for ordinary differential equations – p. 41/48
Stage order s means that
Z
s
X
aij φ(ci ) =
j=1
ci
φ(t)dt,
0
Runge–Kutta methods for ordinary differential equations – p. 42/48
Stage order s means that
Z
s
X
aij φ(ci ) =
j=1
ci
φ(t)dt,
0
for φ any polynomial of degree s − 1.
Runge–Kutta methods for ordinary differential equations – p. 42/48
Stage order s means that
Z
s
X
aij φ(ci ) =
j=1
ci
φ(t)dt,
0
for φ any polynomial of degree s − 1. This implies that
Ack−1 = k1 ck ,
k = 1, 2, . . . , s,
Runge–Kutta methods for ordinary differential equations – p. 42/48
Stage order s means that
Z
s
X
aij φ(ci ) =
j=1
ci
φ(t)dt,
0
for φ any polynomial of degree s − 1. This implies that
Ack−1 = k1 ck ,
k = 1, 2, . . . , s,
where the vector powers are interpreted component by
component.
Runge–Kutta methods for ordinary differential equations – p. 42/48
Stage order s means that
Z
s
X
aij φ(ci ) =
j=1
ci
φ(t)dt,
0
for φ any polynomial of degree s − 1. This implies that
Ack−1 = k1 ck ,
k = 1, 2, . . . , s,
where the vector powers are interpreted component by
component.
This is equivalent to
1 k
A c = c ,
k!
k 0
k = 1, 2, . . . , s
(∗)
Runge–Kutta methods for ordinary differential equations – p. 42/48
From the Cayley-Hamilton theorem
(A − λI)s c0 = 0
Runge–Kutta methods for ordinary differential equations – p. 43/48
From the Cayley-Hamilton theorem
(A − λI)s c0 = 0
and hence
s
X
i=0
s
(−λ)s−i Ai c0 = 0.
i
Runge–Kutta methods for ordinary differential equations – p. 43/48
From the Cayley-Hamilton theorem
(A − λI)s c0 = 0
and hence
s
X
i=0
s
(−λ)s−i Ai c0 = 0.
i
Substitute from (∗) and it is found that
s
X
1 s
s−i i
(−λ) c = 0.
i! i
i=0
Runge–Kutta methods for ordinary differential equations – p. 43/48
Hence each component of c satisfies
s
X 1 s x i
−
=0
i
i!
λ
i=0
Runge–Kutta methods for ordinary differential equations – p. 44/48
Hence each component of c satisfies
s
X 1 s x i
−
=0
i
i!
λ
i=0
That is
x
=0
Ls
λ
where LS denotes the Laguerre polynomial of degree s.
Runge–Kutta methods for ordinary differential equations – p. 44/48
Hence each component of c satisfies
s
X 1 s x i
−
=0
i
i!
λ
i=0
That is
x
=0
Ls
λ
where LS denotes the Laguerre polynomial of degree s.
Let ξ1 , ξ2 , . . . , ξs denote the zeros of Ls so that
ci = λξi ,
i = 1, 2, . . . , s
Runge–Kutta methods for ordinary differential equations – p. 44/48
Hence each component of c satisfies
s
X 1 s x i
−
=0
i
i!
λ
i=0
That is
x
=0
Ls
λ
where LS denotes the Laguerre polynomial of degree s.
Let ξ1 , ξ2 , . . . , ξs denote the zeros of Ls so that
ci = λξi ,
i = 1, 2, . . . , s
The question now is, how should λ be chosen?
Runge–Kutta methods for ordinary differential equations – p. 44/48
Unfortunately, to obtain A-stability, at least for orders
p > 2, λ has to be chosen so that some of the ci are
outside the interval [0, 1].
Runge–Kutta methods for ordinary differential equations – p. 45/48
Unfortunately, to obtain A-stability, at least for orders
p > 2, λ has to be chosen so that some of the ci are
outside the interval [0, 1].
This effect becomes more severe for increasingly high
orders and can be seen as a major disadvantage of these
methods.
Runge–Kutta methods for ordinary differential equations – p. 45/48
Unfortunately, to obtain A-stability, at least for orders
p > 2, λ has to be chosen so that some of the ci are
outside the interval [0, 1].
This effect becomes more severe for increasingly high
orders and can be seen as a major disadvantage of these
methods.
We will look at two approaches for overcoming this
disadvantage.
Runge–Kutta methods for ordinary differential equations – p. 45/48
Unfortunately, to obtain A-stability, at least for orders
p > 2, λ has to be chosen so that some of the ci are
outside the interval [0, 1].
This effect becomes more severe for increasingly high
orders and can be seen as a major disadvantage of these
methods.
We will look at two approaches for overcoming this
disadvantage.
However, we first look at the transformation matrix T for
efficient implementation.
Runge–Kutta methods for ordinary differential equations – p. 45/48
Define the matrix T as follows:

L0 (ξ1 ) L1 (ξ1 ) L2 (ξ1 )
 L (ξ ) L (ξ ) L (ξ )
1 2
2 2
 0 2
 L (ξ ) L (ξ ) L (ξ )
T = 0 3
1 3
2 3

..
..
..

.
.
.
L0 (ξs ) L1 (ξs ) L2 (ξs )
· · · Ls−1 (ξ1 )
· · · Ls−1 (ξ2 )
· · · Ls−1 (ξ3 )
..
.
· · · Ls−1 (ξs )







Runge–Kutta methods for ordinary differential equations – p. 46/48
Define the matrix T as follows:

L0 (ξ1 ) L1 (ξ1 ) L2 (ξ1 )
 L (ξ ) L (ξ ) L (ξ )
1 2
2 2
 0 2
 L (ξ ) L (ξ ) L (ξ )
T = 0 3
1 3
2 3

..
..
..

.
.
.
L0 (ξs ) L1 (ξs ) L2 (ξs )
· · · Ls−1 (ξ1 )
· · · Ls−1 (ξ2 )
· · · Ls−1 (ξ3 )
..
.
· · · Ls−1 (ξs )







It can be shown that for a SIRK method
T −1 AT = λ(I − J)
Runge–Kutta methods for ordinary differential equations – p. 46/48
There are two ways in which SIRK methods can be
generalized
In the first of these we add extra diagonally implicit
stages so that the coefficient matrix looks like this:
b 0
A
,
W λI
b is
where the spectrum of the p × p submatrix A
b = {λ}
σ(A)
For s − p = 1, 2, 3, . . . we get improvements to the
behaviour of the methods
Runge–Kutta methods for ordinary differential equations – p. 47/48
A second generalization is to replace “order” by
“effective order”.
Runge–Kutta methods for ordinary differential equations – p. 48/48
A second generalization is to replace “order” by
“effective order”.
This allows us to locate the abscissae where we wish.
Runge–Kutta methods for ordinary differential equations – p. 48/48
A second generalization is to replace “order” by
“effective order”.
This allows us to locate the abscissae where we wish.
In “DESIRE” methods:
Diagonally Extended Singly Implicit Runge-Kutta
methods using Effective order
these two generalizations are combined.
Runge–Kutta methods for ordinary differential equations – p. 48/48
A second generalization is to replace “order” by
“effective order”.
This allows us to locate the abscissae where we wish.
In “DESIRE” methods:
Diagonally Extended Singly Implicit Runge-Kutta
methods using Effective order
these two generalizations are combined.
This seems to be as far as we can go in constructing
efficient and accurate singly-implicit Runge-Kutta
methods.
Runge–Kutta methods for ordinary differential equations – p. 48/48