Part 3
Truncation Errors
1
Key Concepts
• Truncation errors
• Taylor's Series
– To approximate functions
– To estimate truncation errors
• Estimating truncation errors using other
methods
– Alternating Series, Geometry series,
Integration
2
Introduction
How do we calculate
sin( x), cos( x), e x , x y ,
x , log( x), ...
on a computer using only +, -, x, ÷?
One possible way is via summation of infinite
series. e.g.,
2
3
n
n 1
x
x
x
x
ex  1  x 

 ... 

 ...
2! 3!
n! (n  1)!
3
Introduction
2
3
n
n 1
x
x
x
x
ex  1  x 

 ... 

 ...
2! 3!
n! (n  1)!
• How to derive the series for a given function?
• How many terms should we add?
or
• How good is our approximation if we only sum
up the first N terms?
4
A general form of approximation is in
terms of Taylor Series.
5
Taylor's Theorem
Taylor's Theorem: If the function f and its first n+1
derivatives are continuous on an interval containing a
and x, then the value of the function at x is given by
( 3)
f " (a)
f
(a)
2
f ( x)  f (a)  f ' (a)( x  a) 
( x  a) 
( x  a)3  ...
2!
3!
f ( n ) (a)

( x  a) n  Rn
n!
where the remainder Rn is defined as
Rn  
x
a
( x  t ) n ( n1)
f
(t )dt
n!
(the integral form)
6
Derivative or Lagrange Form of the remainder
The remainder Rn can also be expressed as
( n 1)
f
(c )
Rn 
( x  a) n1
(n  1)!
(the Lagrange form)
for some c between a and x
The Lagrange form of the remainder makes
analysis of truncation errors easier.
7
Taylor Series
( 3)
f " (a )
f
(a )
2
f ( x )  f (a )  f ' (a )( x  a ) 
( x  a) 
( x  a ) 3  ...
2!
3!
f ( n ) (a )

( x  a ) n  Rn
n!
• Taylor series provides a mean to approximate any
smooth function as a polynomial.
• Taylor series provides a mean to predict a function
value at one point x in terms of the function and its
derivatives at another point a.
• We call the series "Taylor series of f at a" or "Taylor
series of f about a".
8
Example – Taylor Series of ex at 0
f ( x)  e x  f ' ( x)  e x  f " ( x)  e x  f ( k ) ( x)  e x for any k  0
Thus f ( k ) (0)  1 for any k  0.
With a  0, the Taylor series of f at 0 becomes
(n)
f " ( 0)
f
(0)
2
f (0)  f ' (0)( x  0) 
( x  0)  ... 
( x  0) n  ...
2!
n!
x 2 x3
xn
 1  x    ... 
 ...
2! 3!
n!
Note:
Taylor series of a function f at 0 is also known as the
Maclaurin series of f.
9
Exercise – Taylor Series of cos(x) at 0
f ( x )  cos( x )  f (0)  1
f ' ( x )   sin( x )  f ' (0)  0
f " ( x )   cos( x )  f " (0)  1
f ( 4 ) ( x )  cos( x )  f ( 4 ) (0)  1
f ( 3) ( x )  sin( x )  f ( 3) (0)  0
f ( 5) ( x )   sin( x )  f ( 5) (0)  0

With a  0, the Taylor series of f at 0 becomes
f " ( 0)
f ( n ) ( 0)
2
f (0)  f ' (0)( x  0) 
( x  0)  ... 
( x  0) n  ...
2!
n!
x2
x4
x6
 1 0 
0
0
 ...
2!
4!
6!
2n

n x
  ( 1)
( 2n )!
n 0
10
Question
2
3
n
n 1
x
x
x
x
ex  1  x 

 ... 

 ...
2! 3!
n! (n  1)!
What will happen if we sum up only the first n+1
terms?
11
Truncation Errors
Truncation errors are the errors that result from
using an approximation in place of an exact
mathematical procedure.
Approximation
Truncation Errors
2
3
n
n 1
x
x
x
x
ex  1  x 

 ... 

 ...
2! 3!
n! (n  1)!
Exact mathematical formulation
12
How good is our approximation?
2
3
n
n 1
x
x
x
x
ex  1  x 

 ... 

 ...
2! 3!
n! (n  1)!
How big is the truncation error if we only sum up
the first n+1 terms?
To answer the question, we can analyze the
remainder term of the Taylor series expansion.
( 3)
f " (a)
f
(a)
f ( x)  f (a)  f ' (a)( x  a) 
( x  a) 2 
( x  a)3  ...
2!
3!
f ( n ) (a)

( x  a) n  Rn
n!
13
Analyzing the remainder term of the
Taylor series expansion of f(x)=ex at 0
The remainder Rn in the Lagrange form is
( n 1)
f
(c )
n 1
Rn 
( x  a)
for some c between a and x
(n  1)!
For f(x) = ex and a = 0, we have f(n+1)(x) = ex. Thus
ec
Rn 
x n 1 for some c in [0, x]
(n  1)!
x
e
n 1

x
(n  1)!
We can estimate the largest possible
truncation error through analyzing Rn.
14
Example
Estimate the truncation error if we calculate e as
1 1 1
1
e  1     ... 
1! 2! 3!
7!
This is the Maclaurin series of f(x)=ex with x = 1 and
n = 7. Thus the bound of the truncation error is
1
ex
e
e
71
8
4
R7 
x  (1)   0.6742  10
(7  1)!
8!
8!
The actual truncation error is about 0.2786 x 10-4.
15
Observation
For the same problem, with n = 8, the bound of the truncation
error is
e
R8 
9!
 0.7491  105
With n = 10, the bound of the truncation error is
e
R10 
 0.6810  107
11!
More terms used implies better approximation.
16
Example (Backward Analysis)
This is the Maclaurin series expansion for ex
2
3
n
x
x
x
e x  1  x    ...   ...
2! 3!
n!
If we want to approximate e0.01 with an error
less than 10-12, at least how many terms are
needed?
17
With x  0.01, 0  c  0.01, f ( x)  e x  f ( n1) ( x)  e x
0.01
ec
e
1.1
n 1
n 1
Rn 
x 
(0.01) 
(0.01)n1
(n  1)!
(n  1)!
(n  1)!
Note:1.1100 is about 13781 > e
To find the smallest n such that Rn < 10-12, we can find
the smallest n that satisfies
1.1
n 1
12
(0.01)  10
(n  1)!
With the help of a computer:
n=0 Rn=1.100000e-02
n=1 Rn=5.500000e-05
n=2 Rn=1.833333e-07
n=3 Rn=4.583333e-10
So we need at least 5 terms
n=4 Rn=9.166667e-13
18
Same problem with larger step size
With x  0.5, 0  c  0.5, f ( x)  e x  f ( n1) ( x)  e x
0.5
ec
e
1.7
n 1
n 1
Rn 
x 
(0.5) 
(0.5)n1
(n  1)!
(n  1)!
(n  1)!
Note:1.72 is 2.89 > e
With the help of a computer:
n=5 Rn=3.689236e-05
n=0 Rn=8.500000e-01
n=6 Rn=2.635169e-06
n=1 Rn=2.125000e-01
n=7 Rn=1.646980e-07
n=2 Rn=3.541667e-02
n=8 Rn=9.149891e-09
n=3 Rn=4.427083e-03
n=9 Rn=4.574946e-10
n=4 Rn=4.427083e-04
n=10 Rn=2.079521e-11
n=11 Rn=8.664670e-13
So we need at least 12 terms
19
To approximate e10.5 with an error less than 10-12,
we will need at least 55 terms. (Not very efficient)
How can we speed up the calculation?
20
Exercise
If we want to approximate e10.5 with an error less
than 10-12 using the Taylor series for f(x)=ex at 10,
at least how many terms are needed?
The Taylor series expansion of f ( x ) at 10 is
f ' (10)
f " (10)
f ( n ) (10)
2
f ( x )  f (10) 
( x  10) 
( x  10)  ... 
( x  10) n  Rn
1!
2!
n!
2
n
(
x

10
)
(
x

10
)
 e10 (1  ( x  10) 
 ... 
)  Rn
2!
n!
f ( n 1) ( c )
Rn 
( x  10) n 1 for some c between 10 and x
( n  1)!
The smallest n that satisfy Rn < 10-12 is n = 18. So we need
at least 19 terms.
21
Observation
• A Taylor series converges rapidly near the
point of expansion and slowly (or not at all)
at more remote points.
22
Taylor Series Approximation Example:
More terms used implies better approximation
f(x) = 0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2
23
Taylor Series Approximation Example:
Smaller step size implies smaller error
Errors
Reduced step size
f(x) = 0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2
24
Taylor Series (Another Form)
If we let h = x – a, we can rewrite the Taylor series
and the remainder as
(n)
f " (a) 2
f (a) n
f ( x)  f (a )  f ' (a )h 
h  ... 
h  Rn
2!
n!
f ( n1) (c) n1
Rn 
h
(n  1)!
When h is small, hn+1 is much
smaller.
h is called the step size.
h can be +ve or –ve.
25
The Remainder of the Taylor Series Expansion
f ( n1) (c) n1
Rn 
h  O(h n1 )
(n  1)!
Summary
To reduce truncation errors, we can reduce h or/and increase
n.
If we reduce h, the error will get smaller quicker (with less n).
This relationship has no implication on the magnitude of the
errors because the constant term can be huge! It only give
us an estimation on how much the truncation error would
reduce when we reduce h or increase n.
26
Other methods for estimating
truncation errors of a series
S  t0  t1  t2  t3  ...  tn  tn1  tn 2  tn3  ...
 


Sn
Rn
1. By Geometry Series
2. By Integration
3. Alternating Convergent Series Theorem
Note: Some Taylor series expansions may exhibit certain
characteristics which would allow us to use different methods
to approximate the truncation errors.
27
Estimation of Truncation Errors
By Geometry Series
If |tj+1| ≤ k|tj| where 0 ≤ k < 1 for all j ≥ n, then
Rn
 tn 1  tn  2  tn 3  ...
 tn 1  k tn 1  k 2 tn 1  ...
 tn 1 (1  k  k 2  k 3  ...)
tn 1

1 k
k tn
Rn 
1 k
28
Example (Estimation of Truncation Errors by Geometry Series)
What is |R6| for the following series expansion?
S  1   2 
tj 
2 4 
j 2 j  ...
j 2 j
Solution:
Is there a k (0 ≤ k < 1) s.t.
|tj+1| ≤ k|tj| or |tj+1|/|tj| ≤ k
for all j ≤ n (n=6)?
If you can find this k, then
k tn
Rn 
1 k
3 6  ... 
t j 1
tj
t j 1
tj
j  1 2 j  2

2 j
j

1 2
 1 
6
 0.11
k  0.11,
1
1 2

j
j  6
t6  3  106
k tn
0.11
R6 

 3  106
1 k
1  0.11
29
Estimation of Truncation Errors
By Integration
If we can find a function f(x) s.t. |tj| ≤ f(j) j ≥ n
and f(x) is a decreasing function x ≥ n, then
Rn  tn 1  t n  2  t n 3  ... 

t
j  n 1
j


 f ( j)
j  n 1

Rn   f ( x )dx
n
30
Example (Estimation of Truncation Errors by Integration)
Estimate |Rn| for the following series expansion.
S  t j
where
j 1
t j  ( j 3  1) 1
Solution:
We can pick f(x) = x–3 because it would provide a
tight bound for |tj|. That is
1
1

3
j
1 j3
So
Rn 
j  1


n
1
1
dx 
3
x
2n 2
31
Alternating Convergent Series Theorem
(Leibnitz Theorem)
If an infinite series satisfies the conditions
– It is strictly alternating.
– Each term is smaller in magnitude than that
term before it.
– The terms approach to 0 as a limit.
Then the series has a finite sum (i.e., converge)
and moreover if we stop adding the terms after the
nth term, the error thus produced is between 0 and
the 1st non-zero neglected term not taken.
32
Alternating Convergent Series Theorem
Example 1:
Maclaurin series of ln(1  x )
n

x2 x3 x4
x
S  x     ...   ( 1) n 1
2
3 4
n
n 1
With n  5,
1 1 1 1
S  1      0.7833333340
2 3 4 5
ln 2  0.693
1
R  S  ln 2  0.09   0.16666
6
Actual error
( 1  x  1)
Eerror
estimated
using the
althernating
convergent
series
theorem
33
Alternating Convergent Series Theorem
Example 2:
Maclaurin series of cos( x )
2 n 1

x2 x4 x6
x
S  1     ...   ( 1)n
2! 4! 6!
(2n  1)!
n 0
With n  5,
12 14 16 18
S  1      0.5403025793
2! 4! 6! 8!
cos(1)  0.5403023059
1
7
S  cos(1)  2.73  10 
 2.76  107
10!
Actual error
Eerror
estimated
using the
althernating
convergent
series
theorem
34
Exercise
If the sine series is to be used to compute sin(1) with an
error less than 0.5x10-14, how many terms are needed?
13 15 17 19 111 113 115 117
sin( 1)  1     



 ...
3! 5! 7! 9! 11! 13! 15! 17!
R0
R1 R2
R3
R4
R5
R6
R7
Solution:
This series satisfies the conditions of the Alternating
Convergent Series Theorem.
Solving
1
1
Rn 
  1014
( 2n  3)!
2
for the smallest n yield n = 7 (We need 8 terms)
35
Exercise
4
1
1
1
 1  4  4  4  ...
90
2
3
4
How many terms should be taken in order to compute
π4/90 with an error of at most 0.5x10-8?
Rn  tn 1  tn  2

1
 ...  

4
j  n 1 ( j  1)


n
( x  1)  4 dx
Solution (by
integration):
3 
3
( x  1)

3
n
( n  1)

3
1
1
8


10
 ( n  1)  406  n  405
3
3( n  1)
2
Note: If we use f(x) = x-3 (which is easier to analyze) instead
of f(x) = (x+1)-3 to bound the error, we will get n >= 406
(just one more term).
36
Summary
• Understand what truncation errors are
• Taylor's Series
– Derive Taylor's series for a "smooth" function
– Understand the characteristics of Taylor's Series
approximation
– Estimate truncation errors using the remainder term
• Estimating truncation errors using other methods
– Alternating Series, Geometry series, Integration
37