ORDINARY DIFFERENTIAL
EQUATIONS
ENGR 351
Numerical Methods for Engineers
Southern Illinois University Carbondale
College of Engineering
Dr. L.R. Chevalier
Dr. B.A. DeVantier
Ordinary Differential
Equations
• A differential equation defines a
relationship between an unknown function
and one or more of its derivatives
• Physical problems using differential
equations
- electrical circuits
- heat transfer
- motion
Ordinary Differential
Equations
• The derivatives are of the dependent
variable with respect to the independent
variable
• First order differential equation with y as
the dependent variable and x as the
independent variable would be:
- dy/dx = f(x,y)
Ordinary Differential
Equations
• A second order differential equation would
have the form:
d2y
dy 

 f  x, y, 
2

dx 
dx
}
does not necessarily have to include
all of these variables
Ordinary Differential
Equations
• An ordinary differential equation is one
with a single independent variable.
• Thus, the previous two equations are
ordinary differential equations
• The following is not:
dy
 f  x1 , x2 , y
dx1
Ordinary Differential
Equations
• The analytical solution of ordinary
differential equation as well as partial
differential equations is called the “closed
form solution”
• This solution requires that the constants of
integration be evaluated using prescribed
values of the independent variable(s).
Ordinary Differential
Equations
• An ordinary differential equation of order n
requires that n conditions be specified.
• Boundary conditions
• Initial conditions
Ordinary Differential
Equations
• An ordinary differential equation of order n
requires that n conditions be specified.
• Boundary conditions
• Initial conditions
consider this beam where the
deflection is zero at the boundaries
x= 0 and x = L
These are boundary conditions
consider this beam where the
deflection is zero at the boundaries
x= 0 and x = L
These are boundary conditions
P
a
yo
In some cases, the specific behavior of a system(s)
is known at a particular time. Consider how the deflection of a
beam at x = a is shown at time t =0 to be equal to yo.
Being interested in the response for t > 0, this is called the
initial condition.
Ordinary Differential
Equations
• At best, only a few differential equations
can be solved analytically in a closed form.
• Solutions of most practical engineering
problems involving differential equations
require the use of numerical methods.
Scope of Lectures on
ODE
• One Step Methods
-
Euler’s Method
Heun’s Method
Improved Polygon
Runge Kutta
Systems of ODE
• Adaptive step size control
Scope of Lectures on
ODE
• Boundary value problems
• Case studies
Specific Study Objectives
• Understand the visual representation of Euler’s,
Heun’s and the improved polygon methods.
• Understand the difference between local and
global truncation errors
• Know the general form of the Runge-Kutta
methods.
• Understand the derivation of the second-order RK
method and how it relates to the Taylor series
expansion.
Specific Study Objectives
• Realize that there are an infinite number of
possible versions for second- and higher-order RK
methods
• Know how to apply any of the RK methods to
systems of equations
• Understand the difference between initial value
and boundary value problems
Review of Analytical
Solution
dy
 4x2
dx
2
dy

4
x
  dx
4x3
y
C
3
At this point lets consider
initial conditions.
y(0)=1
and
y(0)=2
4x3
y
C
3
for y0  1
40
1
C
3
then C  1
What we see are different
values of C for the two
different initial conditions.
3
for y0  2
40
2
C
3
and C  2
3
The resulting equations
are:
4x3
y
1
3
4x3
y
2
3
16
y(0)=1
12
y(0)=2
y
y(0)=3
8
y(0)=4
4
0
0
0.5
1
1.5
x
2
2.5
One Step Methods
• Focus is on solving ODE in the form
dy
 f  x , y
dx
yi 1  yi  h
h
y
yi+1
yi
slope = 
x
This is the same as saying:
new value = old value + slope x step size
Euler’s Method
• The first derivative provides a direct
estimate of the slope at xi
• The equation is applied iteratively, or one
step at a time, over small distance in order
to reduce the error
• Hence this is often referred to as Euler’s
One-Step Method
EXAMPLE
For the initial condition y(1)=1, determine
y for h = 0.1 analytically and usingEuler’s
method given:
dy
2
 4x
dx
Error Analysis of Euler’s
Method
• Truncation error - caused by the nature of the
techniques employed to approximate values of y
- local truncation error (from Taylor Series)
- propagated truncation error
- sum of the two = global truncation error
• Round off error - caused by the limited number of
significant digits that can be retained by a
computer or calculator
y
Example
12
10
8
6
4
2
0
Analytical
Solution
Numerical
Solution
0
1
2
x
....end of example
Higher Order Taylor
Series Methods
yi  1
f '  xi , yi  2
 yi  f  xi , yi  h 
h
2
• This is simple enough to implement with polynomials
• Not so trivial with more complicated ODE
• In particular, ODE that are functions of both dependent
and independent variables require chain-rule
differentiation
• Alternative one-step methods are needed
Modification of Euler’s
Methods
• A fundamental error in Euler’s method is
that the derivative at the beginning of the
interval is assumed to apply across the
entire interval
• Two simple modifications will be
demonstrated
• These modification actually belong to a
larger class of solution techniques called
Runge-Kutta which we will explore later.
Heun’s Method
• Consider our Taylor expansion
f ' x i , y i  2
yi 1  yi  f x i , yi  h 
h
2
• Approximate f’ as a simple forward difference
f x i 1 , y i 1   f x i , y i 
f ' x i , y i  
h
Heun’s Method
• Substituting into the expansion
2
f

f
h
 i 1 i 
 f i 1  f i 
yi 1  yi  f i h  
 yi  

h
 h  2
 2 
Heun’s Method
• Determine the derivatives for the interval @
- the initial point
- end point (based on Euler step from initial point)
• Use the average to obtain an improved
estimate of the slope for the entire interval
• We can think of the Euler step as a “test” step
Heun’s Method
• Determine the derivative for the interval
- the initial point
- end point
• Use the average to obtain an improved
estimate of the slope for the entire interval
y
f  xi , yi   f xi 1 , yi 1 
yi 1  yi 
h
2
y
xi
xi+1
xi
xi+1
x
Improved Polygon
Method
• Another modification of Euler’s Method
• Uses Euler’s to predict a value of y at the
midpoint of the interval
yi 1/ 2
h
 yi  f  xi , yi 
2
• This predicted value is used to estimate the
slope at the midpoint
y 'i 1/ 2  f  xi 1/ 2 , yi  1/ 2 
Improved Polygon
Method
• We then assume that this slope represents a valid
approximation of the average slope for the entire
interval
• Use this slope to extrapolate linearly from xi to
xi+1 using Euler’s algorithm
yi 1  yi  f x i 1/ 2 , yi 1/ 2  h
Improved Polygon
Method
• We could also get this algorithm from
substituting a forward difference in f to i+1/2 into
the Taylor expansion for f’, i.e.
2
f

f
h
 i 1/ 2 i 
yi 1  yi  f i h  
 yi  f i 1/ 2 h

 h/2  2
y
f(xi+1)
xi
xi+1/2
xi+1
x
Runge-Kutta Methods
• RK methods achieve the accuracy of a Taylor
series approach without requiring the calculation
of a higher derivative
• Many variations exist but all can be cast in the
generalized form:
{
yi 1  yi    xi , yi , hh
 is called the incremental function
 , Incremental Function
can be interpreted as a representative
slope over the interval
  a1k1  a2 k 2  an k n
where the a ' s are constant and the k ' s are:
k1  f  xi , yi 
k 2  f  xi  p1h , yi  q11k1h
k 3  f  xi  p2 h , yi  q21k1h  q22 k 2 h

k n  f  xi  pn h , yi  qn 1,1k1h  qn 1, 2 k 2 h  qn 1, n 1k n 1h
  a1k1  a2 k 2  an k n
where the a ' s are constant and the k ' s are:
k1  f  xi , yi 
k 2  f  xi  p1h , yi  q11k1h
k 3  f  xi  p2 h , yi  q21k1h  q22 k 2 h

k n  f  xi  pn h , yi  qn 1,1k1h  qn 1, 2 k 2 h  qn 1, n 1k n 1h
NOTE:
k’s are recurrence relationships,
that is k1 appears in the equation for k2
which appears in the equation for k3
This recurrence makes RK methods efficient for
computer calculations
Second Order RK Methods
yi  1  yi   a1k1  a2 k 2 h
where
k1  f  xi , yi 
k 2  f  xi  p1h, yi  q11k1h
  a1k1  a2 k 2  an k n
where the a ' s are constant and the k ' s are:
k1  f  xi , yi 
k 2  f  xi  p1h , yi  q11k1h
k 3  f  xi  p2 h , yi  q21k1h  q22 k 2 h

k n  f  xi  pn h , yi  qn 1,1k1h  qn 1, 2 k 2 h  qn 1, n 1k n 1h
Second Order RK Methods
• We have to determine values for the constants a1,
a2, p1 and q11
• To do this consider the Taylor series in terms of
yi+1 and f(xi,yi)
yi  1  yi  a1k1  a2 k 2 h
h2
yi  1  yi  f  xi , yi h  f '  xi , yi 
2
Now, f’(xi , yi ) must be determined by the
chain rule for differentiation
f f dy
f '  xi , yi  

x y dx
substituting
 f f dy  h 2
yi  1  yi  f  xi , yi h   

 x y dx  2
The basic strategy underlying Runge-Kutta methods
is to use algebraic manipulations to solve for values
of a1, a2, p1 and q11
yi 1  yi  a1k1  a2 k2 h
 f f dy  h 2
yi 1  yi  f  xi , yi h   

 x y dx  2
By setting these two equations equal to each other and
recalling:
k1  f  xi , yi 
k2  f  xi  p1h, yi  q11k1h
we derive three equations to evaluate the four unknown
constants
a1  a2  1
1
2
1
a2 q11 
2
a2 p1 
Because we have three equations with four unknowns,
we must assume a value of one of the unknowns.
Suppose we specify a value for a2.
What would the equations be?
a1  1  a2
1
p1  q11 
2a2
Because we can choose an infinite number of values
for a2 there are an infinite number of second order
RK methods.
Every solution would yield exactly the same result
if the solution to the ODE were quadratic, linear or a
constant.
Lets review three of the most commonly used and
preferred versions.
y i 1  y i   a 1 k 1  a 2 k 2  h
Consider the following:
where
k 1  f  xi , yi 
k 2  f  xi  p1h, yi  q11 k1h
a1  a 2  1
1
a 2 p1 
2
1
a 2 q11 
2
Case 1: a2 = 1/2
Case 2: a2 = 1
These two methods
have been previously
studied.
What are they?
a1  1  a2  1  1 / 2  1 / 2
1
a2 p1 
2
1
a2 q11 
2
1
p1  q11 
1
2a2
1 
1
yi  1  yi   k1  k 2  h
2
2 
where
k1  f  xi , yi 
k 2  f  xi  h, yi  k1h
Case 1: a2 = 1/2
This is Heun’s Method with
a single corrector.
Note that k1 is the slope at
the beginning of the interval
and k2 is the slope at the
end of the interval.
yi  1  yi  a1k1  a2 k 2 h
where
k1  f  xi , yi 
k 2  f  xi  p1h, yi  q11k1h
a1  1  a2  1  1  0
Case 2: a2 = 1
1
2
1
a2 q11 
2
This is the Improved Polygon
Method.
a2 p1 
1
1
p1  q11 

2 a2 2
yi  1  yi  k 2 h
where
k1  f  xi , yi 
1
1 

k 2  f  xi  h, yi  k1h

2
2 
yi  1  yi  a1k1  a2 k 2 h
where
k1  f  xi , yi 
k 2  f  xi  p1h, yi  q11k1h
Ralston’s Method
Ralston (1962) and Ralston and Rabinowitiz (1978)
determined that choosing a2 = 2/3 provides a minimum
bound on the truncation error for the second order RK
algorithms.
This results in a1 = 1/3 and p1 = q11 = 3/4
1
2 

yi 1  yi   k1  k 2  h
3
3 
where
k1  f  xi , yi 
3
3 

k 2  f  xi  h, yi  k1h

4
4 
EXAMPLE
As a class problem, lets
consider two steps.
dy
 4x 2 y
dx
I .C. y  1 at x  1 i.e. y 1  1
step size h  0.1
Some of you folks do the
analytical solution,
others do either:
•Ralstons’s
•Heun’s
•Improved Polygon
Third Order Runge-Kutta Methods
• Derivation is similar to the one for the secondorder
• Results in six equations and eight unknowns.
 1version results

• One ycommon
 y   k  4 k  k  h in the following
i 1
i
 6
1
2
3

where
k 1  f  xi , yi 
Note the third term
1
1 

k 2  f  xi  h, yi  k1h


2
2
k 3  f  xi  h, yi  hk1  2hk 2 
NOTE: if the derivative is a function of x only, this reduces to Simpson’s 1/3 Rule
Fourth Order Runge Kutta
• The most popular
• The following is sometimes called the classical
fourth-order RK method
1

yi  1  yi    k 1  2 k 2  2 k 3  k 4   h
6

where
k 1  f  xi , yi 
1
1


k 2  f  xi  h, yi  k1h


2
2
1
1


k 3  f  xi  h, yi  hk 2 


2
2
k 4  f  xi  h, yi  hk 3 
• Note that for ODE that are a function of x alone
that this is also the equivalent of Simpson’s 1/3
Rule
1

yi 1  yi   k1  2k 2  2k3  k 4  h
6

where
k1  f  xi , yi 
1
1
k 2  f  xi  h, yi  k1h 
2
2 

1
1


k3  f  xi  h, yi  hk2 
2
2


k 4  f  xi  h, yi  hk3 
EXAMPLE
Use 4th Order RK to solve the following differential equation:
dy
xy

dx 1  x 2
I . C. y1  1
using an interval of h = 0.1
Higher Order RK Methods
• When more accurate results are required,
Bucher’s (1964) fifth order RK method is
recommended
• There is a similarity to Boole’s Rule
• The gain in accuracy is offset by added
computational effort and complexity
Systems of Equations
• Many practical problems in engineering and
science require the solution of a system of
simultaneous differential equations
dy1
 f 1  x , y1 , y2 , , yn 
dx
dy2
 f 2  x , y1 , y2 , , yn 
dx

dyn
 f n  x , y1 , y2 , , yn 
dx
• Solution requires n initial conditions
• All the methods for single equations can be used
• The procedure involves applying the one-step
technique for every equation at each step before
proceeding to the next step
dy1
 f 1  x , y1 , y 2 , , y n 
dx
dy2
 f 2  x , y1 , y2 , , yn 
dx

dyn
 f n  x , y1 , y2 , , yn 
dx
Adaptive Step Size
Control
• Methods evaluated up to now have employed a
constant stepsize
• This can introduce serious limitations
y
apply larger step size
where change is gradual
apply smaller step size
where change is abrupt
x
Adaptive Runge-Kutta
Method
• Error is estimated as the difference between
two predictions using the same order RK
method but with different step sizes (also
called step doubling)
• Local truncation error is estimated as the
difference between two prediction using
different order RK methods
Step Doubling
• Step doubling involves taking each step
twice
- one full step
- two half steps
• The difference in the results represents an
estimate of the local truncation error.
The error is:
  y2  y1
prediction at two
half steps
prediction with a
single step
We can also use this error to provide a criterion for
step size control, as shown here for a fourth order RK:

y2  y2 
15
Note: this estimate is fifth order
accurate!
Step Size Control
• Let’s further consider the use of the error
estimate to adjust the step size
• General strategy
- increase step size if error is small
- decrease step size if error is large
Runge-Kutta Fehlberg
• Employs the strategy where we obtain an error
estimate be computing two RK methods of
different orders
• Subtract to obtain an estimate of the local
truncation error
• Computations increased
• Runge-Kutta Fehlberg circumvents this problem
• Uses a 4th and 5th order RK method with the
same values of k1 - k5
1408
2197
1 
 25
yi 1  yi  
k1 
k3 
k 4  k5  h
2565
4104
5 
 216
6656
28,561
9
2 
 16
yi 1  yi  
k1 
k3 
k 4  k5  k 6  h
12,825
56,430
50
55 
 135
Formulas for k’s are given on p. 617.
The error is estimated by subtracting the two equations.
Boundary Value Problems
• Recall that the solution to an nth order ODE
requires n conditions
• If all the conditions are specified at the same value
of the independent variable, then we are dealing
with an initial value problem
• Problems so far have been devoted to this type of
problem
Boundary Value Problems
• In contrast, we may also have conditions a
different value of the independent variable.
• These are often specified at the extreme point or
boundaries of as system and customarily referred
to as boundary value problems
• To approaches to the solution
- shooting method
- finite difference approach
General Methods for
Boundary Value Problems
The conservation of heat can be used to develop a heat
balance for a long, thin rod. If the rod is not insulated
along its length and the system is at steady state. The equation
that results is:
d 2T
 h '  Ta  T   0
2
dx
Ta
T1
T2
Ta
Ta
T1
T2
Ta
d 2T
 h '  Ta  T   0
2
dx
Clearly this second order
ODE needs 2 conditions.
This can be satisfied by
knowing the temperature
at the boundaries,
i.e. T1 and T2
T(0) = T1
T(L) = T2
d 2T
 h '  Ta  T   0
2
dx
T(0) = T1
T(L) = T2
Use these conditions to solve
the equation analytically.
For a 10 m rod with
Ta = 20
T(0) = 40
T(10) = 200
h’ = 0.01
T  73.45e0.1x  53.45e0.1x  20
Now that we have an analytical solution, lets evaluate our
two proposed numerical methods.
Shooting Method
Given:
d 2T
 h '  Ta  T   0
2
dx
dT
z
dx
dz
 h '  Ta  T 
dx
We need an initial value
of z.
For the shooting method, guess
an initial value.
Guessing z(0) = 10
dz
 h '  Ta  T 
dx
Guessing z(0) = 10
Using a fourth-order RK method with a step size
of 2, T(10) = 168.38
This differs from the BC T(10) = 200
Making another guess, z(0) = 20
T(10) = 285.90
Because the original ODE is linear, the estimates
of z(0) are linearly related.
Using a linear interpolation formula between the values
of z(0), determine a new value of z(0)
Recall:
first estimate z(0) = 10 T(20) = 168.38
second estimate z(0)=20 T(20) = 285.90
What is z(0) that would give us T(20)=200?
T(20)
300
250
200
150
0
5
10
15
z(0)
20
25
T(20)
300
250
200
150
0
5
10
15
20
25
z(0)
20  10
z 0  10 
 200  168.38  12.69
285.90  168.38
T(20)
300
250
200
150
0
5
10
15
20
25
z(0)
20  10
z 0  10 
 200  168.38  12.69
285.90  168.38
We can now use this
to solve the first order
ODE
d 2T
 h '  Ta  T   0
2
dx
dT
z
dx
dz
 h '  Ta  T 
dx
250
200
Analytical
Solution
T
150
Shooting
Method
100
50
0
0
5
10
distance (m)
For nonlinear boundary value problems, linear interpolation
will not necessarily result in an accurate estimation.
One alternative is to apply three applications of the shooting
method and use quadratic interpolation..
Finite Difference Methods
The finite divided difference approximation for
the 2nd derivative can be substituted into the
governing equation.
d 2 T Ti 1  2 Ti  Ti 1

2
dx
x 2
d 2T
 h ' Ta  T   0
2
dx
Ti 1  2 Ti  Ti 1
 h ' Ta  Ti   0
2
x
Collect terms
Ti 1  2 Ti  Ti 1
 h ' Ta  Ti   0
2
x
 Ti 1   2  h ' x 2  Ti  Ti 1  h ' x 2 Ta
We can now apply this equation to each interior node
on the rod.
Divide the rod into a grid, and consider a “node” to be
at each division. i.e..  x = 2m
x=2m
T1
T2
L = 10 m
 Ti 1  2  h' x 2 Ti  Ti 1  h' x 2Ta
x=2m
T(0)
T(10)
L = 10 m
Consider the previous problem:
L = 10 m
We need to solve for the
Ta = 20
temperature at the interior
T(0) = 40
nodes (4 unknowns).
T(10) = 200
h’ = 0.01
Apply the governing
equation at these nodes (4
equations).
What is the matrix?
For the four interior nodes, we get the following
4 x 4 matrix
0
0  T1   40.8 
2.04  1
  1 2.04  1
0  T2   0.8 

   

T
0

1
2
.
04

1
0
.
8

 3  

 0

0
 1 2.04 T4  200.8

T   65.97 93.78 124.54 159.48
T
250
200
T
150
100
50
0
0
2
4
6
distance (m)
8
10
Analytical So lutio n
Sho o ting M etho d
Finite Difference
Example
Consider the previous example, but with
x=1. What is the matrix?