Euler’s and Heun’s Methods
Douglas Wilhelm Harder, M.Math. LEL
Department of Electrical and Computer Engineering
University of Waterloo
Waterloo, Ontario, Canada
ece.uwaterloo.ca
dwharder@alumni.uwaterloo.ca
© 2012 by Douglas Wilhelm Harder. Some rights reserved.
Euler's and Heun's Methods
Outline
This topic discusses numerical differentiation:
–
–
–
–
Initial-value problems
Euler’s method
Heun’s method
Multi-step methods
2
Euler's and Heun's Methods
Outcomes Based Learning Objectives
By the end of this laboratory, you will:
– Understand how to approximate a solution to a 1st-order IVP
using Euler’s method
– Understand the limitations of Euler’s method
– Be able to apply the same ideas from the trapezoidal rule to
improve Euler’s method, i.e., Heun’s method
3
Euler's and Heun's Methods
Initial-value Problems
Given the initial value problem
y (1)  t   f  t , y  t  
y  t0   y0
Invariably, initial-value problems deal with time:
– We know the state y0 of a system at time t0
– We understand how the system evolves (through the ODE)
– We want to approximate the state in the future
4
Euler's and Heun's Methods
Ordinary Differential Equations
Your first question should be:
Can we always write a 1st-order ODE in the form:
y (1)  t   f  t , y  t   ?
For example, the ODE could be implicitly defined as:
3
(1)
(1)
F  t , y  t  , y  t     t  y  t   1  sin  y  t    0
Fortunately, the implicit function theorem says that, in
almost all cases, “yes”
– We may end up using a truncated approximation similar to Taylor
series
5
Euler's and Heun's Methods
Ordinary Differential Equations
What does the formula
y (1)  t   f  t , y  t  
mean?
Given any point (t*, y*), if a solution y(t) to the ODE
passes through that point, the derivative of the solution
must be: y (1)  t *   f  t * , y* 
6
Euler's and Heun's Methods
Ordinary Differential Equations
For example, the ODE
y(1)  t   t  y  t   y  t   t  cos  y  t  
suggests, for example, at the point (1, 2), the slope is
approximately 1 2  2  1  cos  2  1.416146836
We could pick a few hundred points, determine the
slopes at each of these lines, and plot that slope
7
Euler's and Heun's Methods
Ordinary Differential Equations
Doing this with the ODE
y(1)  t   t  y  t   y  t   t  cos  y  t  
yields
8
Euler's and Heun's Methods
Ordinary Differential Equations
The following are three solutions that satisfy these initial
conditions
y  0  1
y  0  0
y  0   1
9
Euler's and Heun's Methods
Ordinary Differential Equations
The ODE
y(1)  t   t  y  t   y  t   t  cos  y  t  
was chosen because there is no explicit solution
The next example does have explicit solutions
10
Euler's and Heun's Methods
Ordinary Differential Equations
Consider the ODE
y (1)  t    y  t   1  t  1
2
This has the following field plot:
2
11
Euler's and Heun's Methods
Ordinary Differential Equations
This clearly has y(t) = 1 as one
solution; however,
3
2
another solution is y  t   3t  32t  3t
t  3t  3t  3
y (1)  t    y  t   1  t  1
2
2
12
Euler's and Heun's Methods
Ordinary Differential Equations
This clearly has y(t) = 1 as one
solution; however,
3
2
another solution is y  t   3t  32t  3t
t  3t  3t  3
We can confirm this by substitution
y (1)  t    y  t   1  t  1
2
2
13
Euler's and Heun's Methods
14
Ordinary Differential Equations
Calculating the derivative:
9  t  1
d  t 3  3t 2  3t 
 3
 3
2
dt  t  3t  3t  3   t  3t 2  3t  32
2
Substituting the function into the equation  y  t   1  t  1
2
2
2
 t 3  3t 2  3t

2

1
t

1



 3

2
t

3
t

3
t

3


 t 3  3t 2  3t   2  t 3  3t 2  3t t 3  3t 2  3t  3  t 3  3t 2  3t  3
2
t
3
 3t  3t  3
2
2
2
 t  1
2
Everything cancels in the numerator except the one 32  9
Euler's and Heun's Methods
Ordinary Differential Equations
Now, we see that y(1) = 1:
2t 3  6t 2  6t  11
2  6  6  11
3
y 1   3




1
2
2t  6t  6t  17 t 1
2  6  6  17
3
The slope at this point should be:
y (1) 1  f 1,1  1  1 1  1  16
2
2
If we evaluate the calculated derivative at t = 1, we get:
2
36 1  1
36  4

 16
2
9
 2  6  6  17 
15
Euler's and Heun's Methods
Euler’s Method
Now, suppose we have an initial condition:
y(t0) = y0
We want to approximate the solution at t0 + h; therefore,
we can look at the Taylor series:
1
y  t0  h   y  t0   y 1  t0  h  y  2   h 2
2
where    t0 , t0  h 
16
Euler's and Heun's Methods
17
Euler’s Method
We can replace the initial condition
y(t0) = y0
into the Taylor series
y  t0  h   y0  y 1  t0  h 
1  2
y   h 2
2
Next, we also know what the derivative is from the ODE:
y 1  t   f  t , y  t  
Thus,
y  t0  h   y0  f  t0 , y0  h 
1  2
y   h 2
2
Euler's and Heun's Methods
Euler’s Method
Thus, we have a formula for approximating the next point
y  t0  h   y0  h  f t0 , y0 
together with an error term
1  2
y   h 2
2
.
18
Euler's and Heun's Methods
Euler’s Method
Using our example:
y (1)  t    y  t   1  t  1
2
2
y (0)  0
we can implement both the right-hand side of the ODE
and the solution:
function [dy] = f2a(t, y)
dy = (y - 1).^2 .* (t - 1).^2;
end
function [y] = y2a( t )
y = (t.^3 - 3*t.^2 + 3*t)./(t.^3 - 3*t.^2 + 3*t + 3);
end
19
Euler's and Heun's Methods
Euler’s Method
Using our example:
y (1)  t    y  t   1  t  1
2
2
y (0)  0
we can therefore approximate y(0.1):
>> approx = 0 + 0.1*f2a(0,0)
actual =
0.100000000000000
>> actual = y2a(0.1)
actual =
0.082849281565271
>> abs( actual - approx )
ans =
0.017150718434729
20
Euler's and Heun's Methods
Euler’s Method
Now, if we halve h, the error should drop by a factor of 4
We will therefore approximate y(0.05):
>> approx = 0 + 0.05*f2a(0,0)
approx =
0.050000000000000
>> actual = y2a(0.05)
actual =
0.045384034047969
>> abs( actual - approx )
ans =
0.004615965952031
Previous error when h = 0.1:
0.017150718434729
21
Euler's and Heun's Methods
Euler’s Method
Lets consider what we are doing:
– The actual solution is in red
– The two approximations are shown as circles
• We are following the same slope out from (0, 0)
22
Euler's and Heun's Methods
Euler’s Method
The problem is, the second approximation does not
approximate y(0.1)—it approximates the solution at the
closer point t = 0.05
– How can we proceed to approximate y(0.1)?
23
Euler's and Heun's Methods
Euler’s Method
How about finding the slope at (0.05, 0.05) and following
that out for another h = 0.05?
24
Euler's and Heun's Methods
Euler’s Method
How about finding the slope at (0.05, 0.05) and following
that out for another h = 0.05?
>> 0.05 + 0.05*f2a(0.05, 0.05)
ans =
0.090725312500000
25
Euler's and Heun's Methods
Euler’s Method
We could repeat this process again, and approximate
the solution at t = 0.15?
>> 0.090725312500000 + 0.05*f2a( 0.1, 0.090725312500000 )
ans =
0.124209921021793
26
Euler's and Heun's Methods
Euler’s Method
As you can see, the three points are shadowing the
actual solution
27
Euler's and Heun's Methods
Euler’s Method
Note that we require more work if we reduce h:
– Dividing h by 2 requires twice the work, and
– Dividing h by 10 requires ten times the work
to approximate the same final point
28
Euler's and Heun's Methods
Euler’s Method
In addition, we are using an approximation to
approximate the next approximation, and so on…
– The error for approximating one point is O(h2)
– In the laboratory, you will attempt to determine how this affects
the error
29
Euler's and Heun's Methods
Euler’s Method
Thus, given an IVP
(1)
t   f t, y t 
y  t0   y0
y
and suppose we want to
approximate y(tfinal)
We could simply use
h = tfinal – t0
and find y0 + h f(t0, y0)
Problem: we have no control
over the accuracy
30
Euler's and Heun's Methods
Euler’s Method
Thus, given an IVP
(1)
t   f t, y t 
y  t0   y0
y
and suppose we want to
approximate y(tfinal)
Instead, divide the interval [t0, tfinal] into n points and now
repeat Euler’s method n – 1 times
31
Euler's and Heun's Methods
Euler’s Method
For example, if we chose n = 11, we would find
approximations at
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 ?
?
?
?
?
?
?
?
?
?
where y(0) = 0 and we want to approximate y(1)
32
Euler's and Heun's Methods
Euler’s Method
Use the initial points to approximate y(0.1):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 ?
?
?
?
?
?
?
?
?
33
Euler's and Heun's Methods
Euler’s Method
Use the next two points to approximate y(0.2):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 ?
?
?
?
?
?
?
?
34
Euler's and Heun's Methods
Euler’s Method
Use the next two points to approximate y(0.3):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 ?
?
?
?
?
?
?
35
Euler's and Heun's Methods
Euler’s Method
Use these two points to approximate y(0.4):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 ?
?
?
?
?
?
36
Euler's and Heun's Methods
Euler’s Method
Use these two points to approximate y(0.5):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 0.2615 ?
?
?
?
?
37
Euler's and Heun's Methods
Euler’s Method
Use these two points to approximate y(0.6):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 ?
?
?
?
38
Euler's and Heun's Methods
Euler’s Method
Use these two points to approximate y(0.7):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 ?
?
?
39
Euler's and Heun's Methods
Euler’s Method
Use these two points to approximate y(0.8):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 ?
?
40
Euler's and Heun's Methods
Euler’s Method
Use these two points to approximate y(0.9):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 0.2902 ?
41
Euler's and Heun's Methods
Euler’s Method
Finally, use these two to approximate y(1.0):
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 0.2902 0.2907
Our approximation is
y(1.0) ≈ 0.290681404577720
42
Euler's and Heun's Methods
Euler’s Method
You will implement Euler’s method:
function [t_out, y_out] = euler( f, t_rng, y0, n )
where
f
t_rng
y0
n
a function handle to the bivariate function f(t, y)
a row vector of two values [t0, tfinal]
the initial condition
the number of points that we will break the interval
[t0, tfinal] into
You will return two vectors:
t_out
a row vector of n equally spaced values from t0 to tfinal
y_out
a row vector of n values where
y_out(1) equals y0
y_out(k) approximates y(t) at t_out(k) for k from 2 to n
43
Euler's and Heun's Methods
Euler’s Method
This function will:
1. Determine h 
tfinal  t0
n 1
2. Assign to
a.
b.
tout a vector of n equally spaced points going from t0 to tfinal, and
yout a vector of n zeros where yout, 1 is assigned the initial value y0,
3. For k going from 1 to n – 1, repeat the following:
a.
b.
Using f, calculate the slope K1 at the point tout,k and yout,k, and
Set yout,k 1  yout,k  h  K1 .
44
Euler's and Heun's Methods
Euler’s Method
45
For example, consider our initial-value problem
2
2
y (1)  t    y  t   1  t  1
y  0  0
Approximating the solution on [0, 1] with n = 11 points yields:
>> [t2a, y2a] = euler( @f2a, [0, 1], 0, 11 )
t2a =
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
y2a =
0 0.1000 0.1656 0.2102 0.2407 0.2615 0.2751 0.2835 0.2882 0.2902 0.2907
Euler's and Heun's Methods
Euler’s Method
The function ode45 is Matlab’s built-in ODE solver:
[t2a,
plot(
[t2a,
plot(
y2a]
t2a,
y2a]
t2a,
= euler( @f2a, [0, 1], 0, 11 );
y2a, 'or' ); hold on
= ode45( @f2a, [0, 1], 0 );
y2a, 'b' )
46
Euler's and Heun's Methods
Euler’s Method
The function ode45 is Matlab’s built-in ODE solver:
[t2a,
plot(
[t2a,
plot(
y2a]
t2a,
y2a]
t2a,
= euler( @f2a, [0, 1], 0, 21 );
y2a, 'or' ); hold on
= ode45( @f2a, [0, 1], 0 );
y2a, 'b' )
47
Euler's and Heun's Methods
Euler’s Method
For example, consider our initial-value problem
y (1)  t   t  y  t   y  t   t  cos  y  t  
y  0  1
Approximating the solution on [0, 1] with n = 11 points
yields:
>> [t2b, y2b] = euler( @f2b, [0, 1], 1, 11 )
t2b =
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
y2b =
1 1.0460 1.1000 1.1626 1.2343 1.3154 1.4059 1.5057 1.6144 1.7310 1.8543
48
Euler's and Heun's Methods
Euler’s Method
In this case, Euler’s method does not fare so well:
hold on
[t2b, y2b]
plot( t2b,
[t2b, y2b]
plot( t2b,
= euler( @f2b, [0, 1], 1, 11 );
y2b, 'or' )
= ode45( @f2b, [0, 1], 1 );
y2b, 'b' )
49
Euler's and Heun's Methods
Euler’s Method
We can increase the number of points by a factor of 10:
hold on
[t2b, y2b]
plot( t2b,
[t2b, y2b]
plot( t2b,
= euler( @f2b, [0, 1], 1, 101 );
y2b, '.r' )
= ode45( @f2b, [0, 1], 1 );
y2b, 'b' )
50
Euler's and Heun's Methods
51
Error Analysis
Now, we saw the error for Euler’s method was O(h2)
– However, except with the first point, we are using an
approximation to find an approximation
n 1
n 1
1  2
y  k  h 2
k 1 2
E
 k   tk , tk 1 
h n 1  2
  y  k  h
2 k 1
h

2
tfinal

y  2   d
t0
– Thus, repeatedly applying Euler
results in an error of O(h)
tfinal
 y    h  
k 1
2
k
t0
y  2   d
Euler's and Heun's Methods
Improving on Euler’s Method
In the lab, you will find that, for Euler’s method:
– Reducing the error by half requires twice as much effort and
memory
– Reducing the error by a factor of 10 requires ten times the time
and memory
This is exceptionally inefficient and we will therefore take
this lab and the next lab to see how we can improve on
Euler’s method
52
Euler's and Heun's Methods
Improving on Euler’s Method
Suppose you are approximating the integral of a function
over an interval:
b
 g  x dx
a
53
Euler's and Heun's Methods
Improving on Euler’s Method
One of the worst approximations would be to simply use
the value of the function at one end-point:
b
 g  x dx  g  a  b  a 
a
54
Euler's and Heun's Methods
Improving on Euler’s Method
At the very least, it would be better to approximate the
integral by taking the average of the two end-points:
g  a   g b 
b  a 
a g  x dx 
2
b
This is the trapezoidal rule of integration
55
Euler's and Heun's Methods
Improving on Euler’s Method
When we are essentially integrating using information
only at the initial value:
y 1  t   f  t , y  t  
t0  h

y 1  t  dt 
t0
t0  h
 f t , y t   dt
t0
y  t0  h   y  t0  
t0  h
 f t , y  t   dt
t0
y  t0  h   y  t 0  
t0  h
 f t , y t   dt
t0
 y  t0   h  f  t0 , y0 
56
Euler's and Heun's Methods
Improving on Euler’s Method
The problem is, we would have to know the slope at t0 +
h in order to approximate mimic the trapezoidal rule
Note, however, that Euler’s method gives us
an approximation K1  f  t0 , y0 
of y(t0 + h)
y(t0 + h) ≈ y0 + hK1
Therefore, we can approximate the
the slope at t0 + h with
K2  f  t0  h, y0  h  K1 
57
Euler's and Heun's Methods
Improving on Euler’s Method
Thus, we have one slope and one approximation of a
slope:
K1  f  t0 , y0 
K 2  f  t0  h, y0  h  K1 
Applying the same principle as the
trapezoidal rule, we would then
approximate
y  t 0  h   y0  h 
K1  K 2
2
58
Euler's and Heun's Methods
Heun’s Method
Graphically, Euler’s method follows the initial slope out a
distance h
– We calculate only one slope: K1  f  t0 , y0 
K1
59
Euler's and Heun's Methods
Heun’s Method
Heun’s method states that we determine the slope at the
second point, too
K2  f  t0  h, y0  h  K1 
K2
K1
60
Euler's and Heun's Methods
Heun’s Method
Take the average of the two slopes and follow that new
slope out a distance h:
K1  K 2
y0  h
2
K2
K1
K1  K 2
2
61
Euler's and Heun's Methods
Heun’s Method
Thus, you will write a second function, heun(), that has
the same signature as euler(), where you will
1. Determine h  tfinal  t0
n 1
2. Assign to
a.
b.
tout a vector of n equally spaced points going from t0 to tfinal, and
yout a vector of n zeros where yout, 1 is assigned the initial value y0,
3. For k going from 1 to n – 1, repeat the following:
a.
b.
c.
Using f, calculate the slope K1 at the point tout,k and yout,k,
Use K1 to find K2, and K  K
1
2
y

y

h

Set out,k 1
.
out,k
2
62
Euler's and Heun's Methods
Heun’s Method
For example, consider our initial-value problem
2
2
y (1)  t    y  t   1  t  1
y  0  0
Approximating the solution on [0, 1] with n = 11 points
yields:
[t2a, y2a] = heun( @f2a, [0, 1], 0, 11 )
t2a =
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
y2a =
0 0.0828 0.1399 0.1798 0.2074 0.2262 0.2382 0.2454 0.2491 0.2505 0.2508
63
Euler's and Heun's Methods
Heun’s Method
The function ode45 is Matlab’s built-in ODE solver:
[t2a,
plot(
[t2a,
plot(
y2a]
t2a,
y2a]
t2a,
= heun( @f2a, [0, 1], 0, 11 );
y2a, 'or' ); hold on
= ode45( @f2a, [0, 1], 0 );
y2a, 'b' )
64
Euler's and Heun's Methods
Heun’s Method
For example, consider our initial-value problem
y (1)  t   t  y  t   y  t   t  cos  y  t  
y  0  1
Approximating the solution on [0, 1] with n = 11 points
yields:
>> [t2b, y2b] = heun( @f2b, [0, 1], 1, 11 )
t2b =
0 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000
y2b =
1 1.0500 1.1091 1.1779 1.2569 1.3463 1.4462 1.5562 1.6756 1.8029 1.9362
65
Euler's and Heun's Methods
Heun’s Method
Heun’s method is significant better than Euler:
[t2b,
plot(
[t2b,
plot(
y2b]
t2b,
y2b]
t2b,
Euler’s Method
= heun( @f2b, [0, 1], 1, 11 );
y2b, 'or' ); hold on
= ode45( @f2b, [0, 1], 1 );
y2b, 'b' )
66
Euler's and Heun's Methods
Heun’s Method
Comparing the accuracy of
– Euler’s method (11 and 41 points in magenta) , and
– Heun’s method (11 points in red)
We see that Heun is significantly better
67
Euler's and Heun's Methods
68
Heun’s Method
The absolute errors are also revealing:
– A reduction by a factor of three
0.00705
0.0239
Euler's and Heun's Methods
69
Heun’s Method
To be fair, we should count function evaluations:
– Euler’s method with n points has n – 1 function evaluations
– Heun’s method with n points has 2(n – 1) function evaluations
Still, Heun’s method comes out ahead...
0.00705
0.0239
Euler's and Heun's Methods
Error Analysis
Without proof, the error for Heun’s method is O(h3)
– However, again, except with the first point, we are using an
approximation to find an approximation
– As with Euler’s method, repeatedly applying Heun’s method will
results in an error of O(h2)
70
Euler's and Heun's Methods
Summary
We have looked at Euler’s and Heun’s methods for
approximating
1st-order IVPs:
– Euler’s method is a direct application of Taylor’s series
– Heun’s method uses the ideas from the trapezoidal rule to
improve on Euler’s method
– Heun’s method requires twice as many function evaluations as
does Euler’s method and yet it is significantly more accurate
71
Euler's and Heun's Methods
References
[1]
Glyn James, Modern Engineering Mathematics, 4th Ed., Prentice Hall, 2007.
[2]
Glyn James, Advanced Modern Engineering Mathematics, 4th Ed., Prentice
Hall, 2011.
72