Numerical Methods for Eng [ENGR 391]
[Lyes KADEM 2008]
CHAPTER II
Truncation Errors and the Taylor series
I. Definition
Truncation errors are those that result from using an approximation in place of an exact
mathematical procedure.
Example
Approximation of the derivative
df f
f ti 1   f ti 


dt t
ti 1  ti
One of the most important methods used in numerical methods to approximate mathematical
functions is: Taylor series
Brook Taylor, born at Edmonton on August 18, 1685, and died in London on
December 29, 1731, was educated at St. John's College, Cambridge, and was
among the most enthusiastic of Newton's admirers. From the year 1712 onwards he
wrote numerous papers in the Philosophical Transactions, in which, among other
things, he discussed the motion of projectiles, the centre of oscillation, and the forms
taken by liquids when raised by capillarity. In 1719 he resigned the secretaryship of
the Royal Society and abandoned the study of mathematics. His earliest work, and
that by which he is generally known, is his Methodus Incrementorum Directa et
Inversa, published in London in 1715. This contains a proof of the well-known
theorem by which a function of a single variable can be expanded in powers of it. He
does not consider the convergency of the series, and the proof which involves
numerous assumptions is not worth reproducing.
II. Taylor series
The Taylor series provides a means to predict a function at one point in terms of the function
value and its derivative at another point.
Therefore, any smooth function can be approximated as a polynomial.
Figure.2.1. Taylor series.
Truncation Errors and the Taylor Series
8
Numerical Methods for Eng [ENGR 391]
[Lyes KADEM 2008]
The expression of Taylor series for any function is:
f xi 1   f xi   f ' xi xi 1  xi  
n
f ' ' xi 
xi 1  xi 2  f ' ' ' xi  xi 1  xi 3  ...  f xi  xi 1  xi n  Rn
2!
3!
n!
A remainder term is included to account for high order terms neglected (from (n+1) to infinity):
f n 1  
xi 1  xi n1
Rn 
n  1!
This represents the remainder for the nth-order approximation.  is a value that lies somewhere
between xi and xi+1.
It is very convenient to write the Taylor series under a compact form by replacing (x i+1- xi) by the
step (h):
f xi1   f xi   f ' xi h 
f ' ' xi  2 f ' ' ' xi  3
f n xi  n
h 
h  ... 
h  Rn
2!
3!
n!
and
f n1   n1
Rn 
h
n  1!
Although, theoretically an infinite number of terms are needed to yield to an exact estimate, in
practice only few terms are sufficient.
The number of terms needed is dependent on the application, the precision needed and it is
determined using the remainder term of the expansion.
However, the determination of the remainder term Rn is not straightforward, since we have:
-
To know 
To differentiate the function f (n+1) times, and for that we need to know the function f !!!!
The only term that we can control in this expression is (h). Therefore, it is very convenient to
express Rn as:
 
Rn  0 h n 1
Which must be interpreted as a truncation error of order n+1. This means that the error is
proportional to hn+1.
Therefore, if h is sufficiently small, an accurate estimate for
only few terms.
Truncation Errors and the Taylor Series
9
f xi 1  can be reached using
Numerical Methods for Eng [ENGR 391]
[Lyes KADEM 2008]
II.1. Using Taylor series to estimate truncation errors
Let us see how Taylor series may be used to estimate truncation errors. If we consider an
approximation of the derivative:
f ' xi  
f  xi 1   f xi 
xi 1  xi
How to determine the error introduced by using this formulation to compute the derivative instead
of using the real mathematical definition?
Using a Taylor series truncated at the first-order:
f xi 1   f xi   f ' xi xi 1  xi   R1
Therefore, f '  xi  
f  xi 1   f  xi 
R1

xi 1  xi 
xi 1  xi 
We have now an estimate of the truncation error:
Truncation error =
R1
f ' '  
xi 1  xi   0xi 1  xi   0h 

xi 1  xi 
2!
So the order of the error due to the formulation used to compute the derivative is h.
III. Numerical differentiation
Another interesting point using the Taylor series is that we are able to compute the derivative of a
function using the values of the function at x i+1 and xi and we know how to compute the error due
to this approximation.
The formulation:
f '  xi  
f xi 1   f  xi 
f
  xi 1  xi   i  0h 
xi 1  xi 
h
Is called first forward difference. Forward because we are using (i) and (i+1)
- First-backward difference approximation
f xi 1   f xi   f ' xi h 
gives:
f '  xi  
f '  xi  
f ' ' xi  2
h  ....
2!
f xi   f xi 1 
f
 xi  xi 1   i  0h 
xi  xi 1 
h
f  xi   f  xi 1 
 0h 
h
Truncation Errors and the Taylor Series
10
Numerical Methods for Eng [ENGR 391]
[Lyes KADEM 2008]
- Centered difference approximation
f xi 1   f xi   f ' xi h 
f ' ' xi  2
h  ....
2!
and
f xi 1   f xi 1   2 f ' xi h  2
f ' ' ' xi  3
h  ....
3!
Will give
f ' xi  
f xi 1   f xi 1 
 0 h2
2h
 
Note that the truncation error is of the order of h2, so it is more accurate than the forward and
backward formulation.
Example
Use forward and backward difference approximations of 0(h) and a centered
difference approximation of 0(h2) to estimate the first derivative of:
f x   0.1x4  0.15x3  0.5x 2  0.25x  1.2
For x=0.5 using h=0.5;0.25.
III.1. Finite difference approximations of higher derivatives
When you look at the Taylor series, you could notice that we can use it to derive any derivative of
the function f [second; third; …]
Example
Second order derivative:
Forward
Backward
Centered
f  xi  2   2 f xi 1   f  x 
 0h 
h2
f xi   2 f xi 1   f xi  2 
f ' ' x  
 0h 
h2
f xi 1   2 f xi   f xi 1 
f ' ' x  
 0 h2
h2
f ' ' x  
 
Note that for the centered formulation, it is a derivation of a derivative:
f xi 1   f xi  f xi   f xi 1 

h
h
f ' ' x  
 0 h2
h
 
Truncation Errors and the Taylor Series
11
Numerical Methods for Eng [ENGR 391]
[Lyes KADEM 2008]
III.2. Functions of more than one variable
Taylor series can be applied to functions of more than one variable.
For example for a function of two variables:
f ui 1 , vi 1   f ui , vi  
 2

f
ui 1  ui   f vi 1  vi   1   f2 ui 1  ui 2  ....
u i
v i
2!  u i

IV. Error propagation
IV.1. Functions of single variable
In this part, the objective is to estimate the effect of a discrepancy between
of a function f , or mathematically speaking, to estimate:
x and ~
x on the value
f ~
x   f x  f ~
x
f x  because we do not know x . But, thanks to Taylor
~
series we can estimate f x  using x (if it is close to x ) and f ~
x  is continuous and
But the problem is that we do not know
differentiable:
f ' ' ~
x
x  ~x 2  ....
f  x   f ~
x   f ' ~
x x  ~
x 
2!
keeping only the zero and first order:
f x  f ~
x   f ' ~
x x  ~
x   0x  ~
x
thus,
x   f ' ~
x x  ~
x
f x  f ~
x   f ' ~
x x  ~
x  gives f ~
This allows us to approximate the error in
of the error in the independent variable.
f x  given the derivative of a function and an estimate
Figure.2.2. Estimation of the error in f x  .
Truncation Errors and the Taylor Series
12
Numerical Methods for Eng [ENGR 391]
[Lyes KADEM 2008]
V. Stability and Condition
The condition of a mathematical problem relates to its sensitivity to changes in its input values.
We say than a computation is numerically unstable if the uncertainty of the input values is grossly
magnified by the numerical method.
We have:
f x  f ~
x   f ' ~
x x  ~
x
We can compute the relative error on
The error on x is:
f x  by:
f  x   f ~
x  f ' ~
x
x  ~x 

f x 
f x 
x~
x
~
x
The butterfly effect is a phrase that encapsulates the more technical notion of
sensitive dependence on initial conditions in chaos theory. Small variations of the initial
condition of a dynamical system may produce large variations in the long term behavior of
the system. This is sometimes presented as esoteric behavior, but can be exhibited by
very simple systems: for example, a ball placed at the crest of a hill might roll into any of
several valleys depending on slight differences in initial position. The phrase refers to the
idea that a butterfly's wings might create tiny changes in the atmosphere that ultimately
cause a tornado to appear (or, for that matter, prevent a tornado from appearing). The
flapping wing represents a small change in the initial condition of the system, which causes
a chain of events leading to large-scale phenomena. Had the butterfly not flapped its
wings, the trajectory of the system might have been vastly different.Sensitive dependence
on initial conditions was first described in the literature by Jacques Hadamard in 1890 and
popularized by Pierre Duhem's 1906 book. The term "butterfly effect" itself is related to the
work of Edward Lorenz, who in a 1963 paper for the New York Academy of Sciences
noted that "One meteorologist remarked that if the theory were correct, one flap of a
seagull's wings could change the course of weather forever." Later speeches and papers
by Lorenz used the more poetic butterfly. According to Lorenz, upon failing to provide a
title for a talk he was to present at the 139th meeting of the American Association for the
Advancement of Science in 1972, Philip Merilees concocted Does the flap of a butterfly’s
wings in Brazil set off a tornado in Texas? as a title.
Example
Given a value of ~
x =2.5 with an error of x~  0.01, estimate the resulting error in the
function: f x   x3 .
V.1. Condition number
It can be defined as the ratio between the error on
f x  and the error on x [from above]:
f ' ~
x
~
Condition number (CN) = x
f x 
The condition number provides a measure of the extent to which an uncertainty in x is
magnified by f x 
Truncation Errors and the Taylor Series
13
Numerical Methods for Eng [ENGR 391]
CN = 1
CN > 1
CN < 1
CN >> 1
[Lyes KADEM 2008]
The relative error on f x  is identical to the relative error on x
The relative error is amplified
The relative error is attenuated
Ill-conditioned
VI. Total numerical error
The total error is the sum of the truncation and round-off errors, however:
Round-off 
Round-off 
Truncation errors 
 the number of figures
With subtractive cancellation and the increase in the number of
computations.
 h [step], however this leads to subtractive cancellation or to the
increase in computations.
Therefore, we are facing a dilemma: decreasing one component of the total errors increases the
other.
However, with actual computers, the round-off errors can be minimized and therefore we will be
able to decrease the truncation error by reducing (h).
Figure.2.3. Variation of total error as a function of the step size.
VII. Some advice to control numerical errors
-
Avoid subtracting two nearly equal numbers.
Avoid subtractive cancellation by reformulating your problem.
When you add or subtract numbers sort them and start with the smallest one.
Truncation Errors and the Taylor Series
14