NUMERICAL DIFFERENTIATION
The derivative of f (x) at x0 is:
f  x0  
Limit
h 0
f  x0  h  f ( x0 )
h
An approximation to this is:
f  x0  h  f ( x0 )
f  x0  
h
for small values of h.
Forward Difference
Formula
Let f ( x )  ln x and x0  1.8
Find an approximate value for f 1.8
h
f (1.8)
f (1.8  h)
0.1 0.5877867 0.6418539
f (1.8  h)  f (1.8)
h
0.5406720
0.01 0.5877867 0.5933268
0.5540100
0.001 0.5877867 0.5883421
0.5554000
The exact value of
f 1.8  0.55 5
Assume that a function goes through three points:
 x0 , f  x0 ,  x1 , f  x1  and  x2 , f  x2 .
f ( x)  P( x)
P  x   L0  x  f  x0   L1  x  f  x1   L2  x  f  x2 
Lagrange Interpolating Polynomial
P  x   L0  x  f  x0   L1  x  f  x1   L2  x  f  x2 
( x  x1 )( x  x2 )
P x 
f  x0 
( x0  x1 )( x0  x2 )
( x  x0 )( x  x2 )

f  x1 
( x1  x0 )( x1  x2 )
( x  x0 )( x  x1 )

f  x2 
( x2  x0 )( x2  x1 )
f ( x )  P ( x )
2 x  x1  x2
P  x  
f  x0 
( x0  x1 )( x0  x2 )
2 x  x0  x2

f  x1 
( x1  x0 )( x1  x2 )
2 x  x0  x1

f  x2 
( x2  x0 )( x2  x1 )
If the points are equally spaced, i.e.,
x1  x0  h and x2  x0  2h
2 x0  ( x0  h)  ( x0  2h)
P  x0  
f  x0 
x0  ( x0  h)x0  ( x0  2h)
2 x0  x0  ( x0  2h)

f  x1 
( x0  h)  x0 ( x0  h)  ( x0  2h)
2 x0  x0  ( x0  h)

f  x2 
( x0  2h)  x0 ( x0  2h)  ( x0  h)
 3h
 2h
h
P  x0  
f  x0  
f  x1   2 f  x2 
2
2
2h
h
2h
1
P  x0    3 f  x0   4 f  x1   f  x2 
2h
Three-point formula:
1
f  x0    3 f  x0   4 f  x0  h  f  x0  2h
2h
If the points are equally spaced with x0 in the middle:
x1  x0  h and x2  x0  h
2 x0  ( x0  h)  ( x0  h)
P  x0  
f  x0 
x0  ( x0  h)( x0  ( x0  h)
2 x0  x0  ( x0  h)

f  x1 
( x0  h)  x0 ( x0  h)  ( x0  h)
2 x0  x0  ( x0  h)

f  x2 
( x0  h)  x0 ( x0  h)  ( x0  h)
0
h
h
P  x0  
f  x0   2 f  x1   2 f  x2 
2
h
2h
2h
Another Three-point formula:
1
f  x0    f  x0  h  f  x0  h
2h
Alternate approach (Error estimate)
Take Taylor series expansion of f(x+h) about x:
2
3
h 2 
h 3 
f  x  h  f  x   hf  x  
f x 
f x  
2
3!
h2 2 
h3 3 
f  x  h  f  x   hf  x  
f x 
f x  
2
3!
f  x  h  f  x 
h 2 
h2 3 
 f  x   f  x  
f x  
h
2
3!
.............. (1)
f  x  h  f  x 
 f  x   O h
h
f  x  h  f  x 
f  x  
 O h
h
f  x  h  f  x 
f  x  
h
Forward Difference
Formula
h 2 
h2 3 
Oh  f  x  
f x  
2
3!
4h 2  2 
8h 3  3 
f  x  2h  f  x   2hf  x  
f x 
f x  
2
3!
4h 2  2 
8h 3  3 
f  x  2h  f  x   2hf  x  
f x 
f x  
2
3!
f  x  2h   f  x 
2h  2 
4h 2  3 
 f  x  
f x 
f x  
2h
2
3!
................. (2)
f  x  h  f  x 
h 2 
h 3 
 f  x   f  x  
f x  
h
2
3!
.............. (1)
2
f  x  2h   f  x 
2h  2 
4h 2  3 
 f  x  
f x 
f x  
2h
2
3!
................. (2)
2 X Eqn. (1) – Eqn. (2)
f  x  h  f  x  f  x  2 h  f  x 
2

h
2h
2h2 3 
6h3 4 
 f  x  
f x  
f x   
3!
4!
 f  x  2 h  4 f  x  h  3 f  x 
2h
2
3
2h  3 
6 h 4 

 f x  
f x  
f x   
3!
4!
 f   x   O h2
 
 f  x  2h   4 f  x  h   3 f  x 
 f  x   O h2
2h
 
 f  x  2h   4 f  x  h   3 f  x 
f  x  
 O h2
2h
 
 f  x  2h   4 f  x  h   3 f  x 
f  x  
2h
Three-point Formula
 
2
3
2
h
6
h
O h2  
f 3   x  
f 4   x   
3!
4!
The Second Three-point Formula
Take Taylor series expansion of f(x+h) about x:
h2 2 
h3 3 
f  x  h  f  x   hf  x  
f x 
f x  
2
3!
Take Taylor series expansion of f(x-h) about x:
2
3
h 2 
h 3 
f  x  h  f  x   hf  x  
f x 
f x   
2
3!
Subtract one expression from another
2h 3  3 
2h 6  6 
f  x  h  f  x  h  2hf  x  
f x 
f x  
3!
6!
2h 3  3 
2h 6  6 
f  x  h  f  x  h  2hf  x  
f x 
f x  
3!
6!
f  x  h  f  x  h
h2 3 
h5 6 
 f  x  
f x  
f x  
2h
3!
6!
f  x  h  f  x  h h2 3 
h5 6 
f  x  

f x  
f x  
2h
3!
6!
f  x  
f  x  h  f  x  h
 O h2
2h
 
 
2
5
h
h
O h2  
f 3   x  
f 6   x   
3!
6!
f  x  h  f  x  h 
f  x  
2h
Second Three-point Formula
Summary of Errors
f  x  h  f  x 
f  x  
h
Error term
Forward Difference
Formula
h 2 
h2 3 
Oh  f  x  
f x  
2
3!
Summary of Errors continued
First Three-point Formula
 f  x  2h   4 f  x  h   3 f  x 
f  x  
2h
 
2
3
2
h
6
h
Error term O h2  
f 3   x  
f 4   x   
3!
4!
Summary of Errors continued
Second Three-point Formula
f  x  h  f  x  h 
f  x  
2h
 
2
5
h
h
f 3   x  
f 6   x   
Error term O h2  
3!
6!
Example:
f  x   xe
x
Find the approximate value of
x
f x
1.9
12.703199
2.0
14.778112
2.1
17.148957
2.2
19.855030
f 2
with
h  0.1
Using the Forward Difference formula:
1
f   x0    f  x0  h   f  x0 
h
1
f   2 
f  2.1  f  2 

0. 1
1

17.148957  14.778112
0. 1
 23.708450
Using the 1st Three-point formula:
1
 3 f  x0   4 f  x0  h  f  x0  2h
f  x0  
2h
1
 3 f ( 2)  4 f ( 2.1)  f ( 2.2)
f 2 
2  0.1
1
  3  14.778112  4  17.148957

0.2
 19.855030 
 22.032310
Using the 2nd Three-point formula:
1
 f  x0  h  f  x0  h
f  x0  
2h
1
 f ( 2.1)  f (1.9)
f 2 
2  0.1
1
 17.148957  12.703199

0.2
 22.228790
The exact value of
f 2 is : 22.167168

Comparison of the results with h = 0.1
The exact value of f   2 is 22.167168
Formula
f   2
Error
Forward Difference
23.708450
1.541282
1st Three-point
22.032310
0.134858
2nd Three-point
22.228790
0.061622
Second-order Derivative
h2 2 
h3 3 
f  x  h  f  x   hf  x  
f x 
f x  
2
3!
2
3
h 2 
h 3 
f  x  h  f  x   hf  x  
f x 
f x  
2
3!
Add these two equations.
2h 2  2 
2h4  4 
f  x  h  f  x  h  2 f  x  
f x 
f x  
2
4!
2 h2  2 
2h4  4
f  x  h  2 f  x   f  x  h 
f  x 
f  x 
2
4!
f  x  h  2 f  x   f  x  h
2h  4 
2 
 f x 
f x  
2
h
4!
2
2






f
x

h

2
f
x

f
x

h
2
h
4 
x  
f 2   x  

f
2
h
4!
f
2 
f  x  h  2 f  x   f  x  h
x 
h2
NUMERICAL INTEGRATION
b
 f ( x )dx 
a
area under the curve f(x) between
x  a to x  b.
In many cases a mathematical expression for f(x) is
unknown and in some cases even if f(x) is known its
complex form makes it difficult to perform the integration.
y
f(b)
f(x)
f(a)
x0 = a
x0 = b
x
y
f(b)
f(x)
f(a)
x0 = a
x0 = b
x
Area of the trapezoid
The length of the two parallel sides of the trapezoid
are: f(a) and f(b)
The height is b-a
b

a
ba
 f ( a )  f ( b )
f ( x )dx 
2
h
  f ( a )  f ( b )
2
Simpson’s Rule:

x2
x0
x2
f ( x )dx   P ( x )dx
x0
x1  x0  h and x2  x0  2h

x2
P  x  dx 
x0

x2
x0
( x  x1 )( x  x2 )
f  x0 dx
( x0  x1 )( x0  x2 )
( x  x0 )( x  x2 )
f  x1 dx
( x1  x0 )( x1  x2 )

( x  x )( x  x )

f  x  dx
 ( x  x )( x  x )

x2
x0
x2
0
1
2
x0
2
0
2
1

x2
x0
x2
f ( x )dx   P ( x )dx
x0
h
  f  x0   4 f ( x1 )  f ( x 2 )
3
y
f(b)
f(x)
f(a)
x0 = a
x0 = b
x
y
f(b)
f(x)
f(a)
x0 = a
x0 = b
x
y
f(b)
f(x)
f(a)
x0 = a
x0 = b
x
y
f(b)
f(x)
f(a)
x0 = a
x0 = b
x
Composite Numerical Integration
Riemann Sum
The area under the curve is subdivided into n
subintervals. Each subinterval is treated as a
rectangle. The area of all subintervals are added
to determine the area under the curve.
There are several variations of Riemann sum as
applied to composite integration.
 xIn Left
a / n
 b Riemann
Left Riemann Sum
y
x1 sum,
a the leftx2
x3
a
b
x
n
 f  x  dx   f  x   x
b
a
i 1
i
x
side sample of

  x is
theafunction
used as the
height
a  2ofthe
x
individual
rectangle.
xi  a   i  1  x

y
a
b
x
n
 f  x  dx   f  x  x
b
a
i 1

 xRight
 bRiemann
a / n
In
sum,
x1  the
a right-side
x
sample of the
x2  a is2used
 x as
function
the height of the
x3  a  3rectangle.
x
individual
Right Riemann Sum
i
x
xi  a  i x
Midpoint Rule
y
a

b
a
 x In bthe
 aMidpoint
/ n
Rule, the sample at
x1  the
a  middle
2  1  1of
x / 2
 the
x2 subinterval
a   2  2  1is
 x / 2
used
as the height of the
x3  individual
a   2  3  1  x / 2 
rectangle.
b
x
n
f  x  dx   f  xi  x
i 1
x
xi  a   2  i  1  x / 2 
Composite Trapezoidal Rule:
Divide the interval into n subintervals and apply
Trapezoidal Rule in each subinterval.
b

a

h
f ( x )dx   f a   2 f ( xk )  f (b)
2
k 1

n 1
where
ba
h
and x k  a  kh for k  0, 1, 2, ... , n
n
π
Find
 sin (x)dx
0
by dividing the interval into 20 subintervals.
n  20
ba π
h

n
20
kπ
x k  a  kh 
, k  0, 1, 2, ....., 20
20
π

h
f ( x k )  f ( b )
0 sin( x )dx  2  f a   2
k 1

n 1

π 
 kπ 

sin0   2 sin   sinπ 

40 
 20 
k 1

19
 1.995886
Composite Simpson’s Rule:
Divide the interval into n subintervals and apply
Simpson’s Rule on each consecutive pair of subinterval.
Note that n must be even.
b

a
h
f ( x )dx   f a   2
3
n/ 2
 4
k 1
( n / 2 ) 1
 f (x
k 1
2k
)

f ( x2 k 1 )  f (b)

where
ba
h
and x k  a  kh for k  0, 1, 2, ... , n
n
π
Find
 sin (x)dx
0
by dividing the interval into 20 subintervals.
ba π
n  20
h

n
20
kπ
x k  a  kh 
, k  0, 1, 2, ....., 20
20
π
9
π 
 2kπ 
sin

0 sin( x )dx  60 sin0  2
 20 
k 1

 ( 2k  1)π 
 4 sin
  sin( π)
20 

k 1

 2.000006
10