College of Natural and Computational Science
Department of Physics
Introduction to Computational Physics (Phys 402)
Chapter – 4:
Methods of Differentiation and Integration
February 2023
HU, CNCS, Department of Physics
1
Outline Contents
Methods of differentiation and integration
4.1 Numerical differentiation
• Forward Difference
• Backward Difference
• Central Difference
4.2. Numerical Integration
 Trapezoid Rule
 Simpson's Rule
 Mid-Point Rule
HU, CNCS, Department of Physics
2
Taylor’s Expansion
1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  O(h 3 )
2
1
f ( x)h  f ( x  h)  f ( x)  f ( x)h 2  O(h 3 )
2
f ( x  h)  f ( x ) 1
f ( x) 
 f ( x)h  O(h 2 )
h
2
3
Forward Difference Formula for f (x)
f ( x  h)  f ( x )
f ( x) 
h
error  O (h)
Geometrically
f (x)
f (x)
f ( x  h)  f ( x)
h
x
xh
4
Backward Difference Formula for f (x)
Similarly
1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  O (h 3 )
2
Geometrically
f ( x )  f ( x  h)

f ( x) 
h
error  O(h)
f (x)
f ( x)  f ( x  h)
h
xh x
x
5
Central Difference Formula for
–)
f (x)
f ( x  h)  f ( x)  f ( x)h 
1
1
1
f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O ( h 5 )
2
6
4!
f ( x  h)  f ( x)  f ( x)h 
1
1
1
f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O (h 5 )
2
6
4!
f ( x  h)  f ( x  h)  2hf ( x)

1
f ( x ) h 3
3
 O( h 5 )
1
2hf ( x)  f ( x  h)  f ( x  h)  f ( x)h 3  O (h 5 )
3
6
Central Difference Formula for f (x)
f ( x) 
f ( x  h)  f ( x  h)
2h
Geometrically
f (x)
f (x)
error  O(h 2 )
f ( x  h)  f ( x  h)
2h
xh
x
xh
x
7
Example
FD: h=0.1
h=0.05
BD: h=0.1
h=0.05
CD:
h=0.1
h=0.05
f (1) 
f (1.1)  f (1)
 3.31
0 .1
error  0.31
f (1) 
f (1.05)  f (1)
 3.1525
0.05
error  0.1525
f (1)  f (0.9)
 2.71
0.1
f (1)  f (0.95)
f (1) 
 2.8453
0.05
f (1) 
 error  h
error  0.29
error  0.1547
f (1.1)  f (0.9)
error  0.01
 error  h2
 3.01
0 .2
f (1.05)  f (0.95)
f (1) 
 3.00250 error  0.00250
0.1
f (1) 
8
Forward Difference Formula for f (x)
1
1
f ( x)(2h) 2  f ( x)(2h) 3  O (h 4 )
2
6
1
1
2
3
4






f
(
x

h
)

f
(
x
)

f
(
x
)
h

f
(
x
)
h

f
(
x
)
h

O
(
h
)
–2）
2
6
1
4 1
f ( x  2h)  2 f ( x  h)   f ( x)  f ( x)h 2 [2  2  ]  f ( x)h 3 [  ]  O (h 4 )
2
3 3
f ( x  2h)  f ( x)  f ( x)2h 
f ( x)h 2  f ( x  2h)  2 f ( x  h)  f ( x)  f ( x)h3  O(h 4 )
f ( x  2h)  2 f ( x  h)  f ( x )
f ( x) 
h2
error  O(h)
9
Backward Difference Formula for f (x)
1
1
f ( x)(2h) 2  f ( x)(2h) 3  O (h 4 )
2
6
1
1
2
3
4






f
(
x

h
)

f
(
x
)

f
(
x
)
h

f
(
x
)
h

f
(
x
)
h

O
(
h
)
–2）
2
6
1
4 1
f ( x  2h)  2 f ( x  h)   f ( x)  f ( x)h 2 [2  2  ]  f ( x)h 3 [  ]  O (h 4 )
2
3 3
f ( x  2h)  f ( x)  f ( x)2h 
f ( x)h 2  f ( x  2h)  2 f ( x  h)  f ( x)  f ( x)h3  O(h 4 )
f ( x )  2 f ( x  h)  f ( x  2h)
f ( x) 
h2
error  O(h)
10
Central Difference Formula for
+)
f (x)
f ( x  h)  f ( x)  f ( x)h 
1
1
1
f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O ( h 5 )
2
6
4!
f ( x  h)  f ( x)  f ( x)h 
1
1
1
f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O (h 5 )
2
6
4!
f ( x  h)  f ( x  h)  2 f ( x )
 f ( x)h 2
1 ( 4)

f ( x)h 4  O(h 6 )
12
f ( x  h)  f ( x  h)  2 f ( x )
2
f ( x) 

O
(
h
)
2
h
Similar remark on the selection of FD|BD|CD applies for f”(x)
11
More Accurate FD Formula for f (x)
f ( x  h)  f ( x)  f ( x) h 
1
f ( x)h 2  O (h 3 )
2
1
f ( x  2h)  2 f ( x  h)  f ( x )
f ( x  h)  f ( x)  f ( x)h  h 2 [
 O (h)]  O (h 3 )
2
2
h
 f ( x)  f ( x)h 
f ( x) 
f ( x) 

1
1
f ( x  2h)  f ( x  h)  f ( x )  O ( h 3 )
2
2
1
3
f ( x  2h)  2 f ( x  h)  f ( x )
2
2
 O(h 2 )
h
 f ( x  2h)  4 f ( x  h)  3 f ( x)
 O(h 2 )
2h
12
Numerical Integration
Outline
Definite Integrals
Lower and Upper Sums
 Reimann Integration or Reimann Sums
Uniformly-spaced samples




Trapezoid Rules
Romberg Integration
Simpson’s Rules
Adaptive Simpson’s Scheme
Non-uniformly spaced samples
 Gaussian Quadrature Formulas
13
Motivation
What does an integral represent?
Basic definition of an integral:

b
a
f ( x)dx  area
d
b
c
a

f ( x)dxdy  volume
f(x)

b
a
n
f ( x )dx  lim  f ( xk )x
n 
k 1
where x  b  a
n
sum of height  width
x
14
Reimann Integral Theorem
• Integration is a summing process. Thus virtually
all numerical approximations can be represented
by
n
I   f ( x )dx   wi f ( xi )  Et
b
a
i 1
in which wi are the weights, xi are the sampling
points, and Et is the truncation error
• Valid for any function that is continuous on the
closed and bounded interval of integration.
February 20, 2023
15
Partitioning the Integral
• The most common numerical integration formula
is based on equally spaced data points.

xn
x0
f ( x)dx
• Divide [x0 , xn] into n intervals (n1)

xn
x0
x1
x2
f ( x)dx   f ( x)   f ( x) 
February 20, 2023
x0
x1

xn

f ( x)
xn1
16
Newton-Cotes Formulas
• The m’s (order of the polynomials) may be the
same or different.

xn
x0
f ( x)dx  
x0m1
x0
pm1 ( x)dx  
x0m1m2
x0m1
pm2 ( x)dx  ...  
xn
xnmn
• Different choices for m’s lead to different
formulas: m Polynomial
Formula
February 20, 2023
Trapezoid
pmn ( x)dx
Error
O(h 2 )
1
linear
2
quadratic
Simpson' s 1/3 O(h 4 )
3
cubic
Simpson' s 3/8 O(h 4 )
17
Trapezoid Method
Multiple Application Rule
The interval [a, b] is
f(x)
f ( x2 )  f ( x1 )
x2  x1 
Area 
2
partitione d into n segments
a  x0  x1  x2  ...  xn  b

b
a
f ( x)dx  sum of the areas
of the trapezoid s
x
x0
a
x1
x2
x3
b
18
Trapezoid Method
General Formula and Special Case
If the interval is divided into n segments (not necessarily equal)
a  x0  x1  x2  ...  xn  b

b
f ( x)dx 
a
n 1
1
xi 1  xi  f ( xi 1 )  f ( xi ) 

i 0 2
Special Case ( Equaliy spaced base points)
xi 1  xi  h for all i

b
a
n 1
1

f ( x)dx  h   f ( x0 )  f ( xn )   f ( xi ) 
i 1
2

19
Example
Given a tabulated
values of the velocity of
an object.
Time (s)
0.0
1.0
2.0
3.0
Velocity (m/s) 0.0
10
12
14
Obtain an estimate of
the distance traveled in
the interval [0,3].
Distance = integral of the velocity
Distance 

3
0
V (t ) dt
20
Example 1
The interval is divided
into 3 subintervals
Base points are0,1,2,3
Time (s)
0.0
1.0
2.0
3.0
Velocity
(m/s)
0.0
10
12
14
Trapezoid Method
h  xi 1  xi  1
1
 n 1

T  h  f ( xi )   f ( x0 )  f ( xn ) 
2
 i 1

1


Distance  1(10  12)  (0  14)  29
2


21
Error in estimating the integral
Theorem
Assumption :
Theorem :
approximat e
f ' ' ( x) is continuous on [a,b]
Equal intervals (width  h)
If Trapezoid Method is used to

b
a
f ( x)dx
then
b  a 2 ''
Error  
h f ( ) where   [a,b]
12
ba 2
Error 
h max f ' ' ( x)
x[ a ,b ]
12
22
Example
Integrate from f ( x)  e
 x2
a = 0 to b = 2
Use trapezoidal rule:
2
I   e dx
 x2
0

b  a
(2  0)
 f a   f b 
 f (2)  f (0)

2
 1  (e 4  e0 )  1.0183
February 20, 2023
2
23
Example
Estimate error: Et   1 f  h 3
12
Where h = b - a and a <  < b
Don’t know  - use average value
f ( x)  (2  4 x )e
2
h  20 2
 x2
f (0)  2
f ( 2)  0.2564
23  f 0  f 2
Et  Ea  
 0.58
12
2
February 20, 2023
24
Composite Trapezoid Rule
• If we do multiple intervals, we can avoid duplicate
function evaluations and operations:
• Use n+1 equally spaced points.
• Each interval has: h  b n a
• Break up the limits of integration and expand.
I 
a h
a
February 20, 2023
f x dx  
a 2 h
a h
f x dx  ...  
b
b h
f x dx
25
Composite Trapezoid Rule
• Substituting the trapezoid rule for each integral.
ah
a2h
b
I   f  x  dx 
ah f  x  dx  ...  bh f  x  dx
a

a  h  a
2
.... 
 f  a   f  a  h  
b  b  h
2
 a  2h  a  h 
2
 f  a  h   f  a  2h 
 f  b  h   f  b 
• Results in the Composite Trapezoid Formula:
n 1
h

I   f a   2 f a  ih   f b 
2
i 1

February 20, 2023
26
Composite Trapezoid Rule
• Think of this as the width times the average
height.
n 1
h

I   f a   2 f a  ih   f b 
2
i 1

n 1
 b  a 
f a   2 f a  ih   f b 
i 1
2n
width
Average height
February 20, 2023
27
Error
• The error can be estimated as:
Ea

b  a h 2

12
3

b  a
f  
12n
2
f 
2
O(h )
• Where, f  is the average second derivative.
• If n is doubled, h  h/2 and Ea  Ea/4
• Note, that the error is dependent upon the
width of the area being integrated.
February 20, 2023
28
Example
• Integrate:
• from
a=0.2
to
b=0.8
f x   0.3  20 x  140 x 2  730 x 3  810 x 4  200 x 5
40
35
30
25
20
15
10
5
0
0
February 20, 2023
0.2
0.4
0.6
0.8
1
1.2
29
Example
• A single application of the Trapezoid rule.
f a   f b 
I  b  a 
2
34.22  3.81
 0.8  0.2 
2
 11.26
• Error:
February 20, 2023
1
3
Et  
f  b  a 
12
30
Example
• We don’t know  so approximate with
average f
f x   20  280 x  2190 x 2  3240 x 3  1000 x 4
f x   280  4380 x  9720 x 2  4000 x 3

f x  
0.8
0.2
f  dx
0.8  0.2
f (0.8)  f (0.2)

 131.6
0.8  0.2
February 20, 2023
31
Example
• The error can thus be estimated as:
Et
b  a h


2
b  a

f  
2
3
f 
12
12n
1
3
   131.6  0.8  0.2   2.37
12
February 20, 2023
32
Using Three Intervals
• Use intervals (0.2,0.4),(0.4,0.6),(0.6,0.8):
– (n = 3, h = 0.2)
n 1
I  b  a 
f a   2 f a  ih   f b 
i 1
2n
f 0.2   2 f 0.4   f 0.6   f 0.8
 0.8  0.2 
(2)(3)
3.31  213.93  30.16   34.22
 0 .6
6
 12.57
True value of integral is 12.82
February 20, 2023
33
Using Six Intervals
• Use intervals (0.2,0.3),(0.3,0,4), etc.
– (n = 6, h = 0.1)
f 0.2   2 f 0.3  f 0.4   f 0.5  f 0.6  f 0.7   f 0.8
( 2)(6)
3.31  27.34  13.93  22.18  30.16  35.22   34.22
 0.6
12
 12.76
I  0.8  0.2 
True value of integral is 12.82
February 20, 2023
34
THE MIDPOINT RULE

b
a
f ( x) dx  M n
 x [ f ( x1 )  f ( x 2 )  ...  f ( x n )]
ba
where x 
n
and xi  12 ( xi 1  xi )  midpoint of [ xi 1, xi ]
Example
As a = 0, b = 1, and n = 10, the Midpoint Rule
gives:
1
e
0
x2
dx
 x [ f (0.05)  f (0.15)  ...  f (0.85)  f (0.95)]
e

e
e
e
e
 0.1  0.3025 0.4225 0.5625 0.7225 0.9025 
e
e
e
e
 e

 1.460393
0.0025
0.0225
0.0625
0.1225
0.2025
Simpson’s 1/3 Rule
• If we use a 2nd order polynomial (need 3
points or 2 intervals):
– Lagrange form.
x  x2 

 x1  0

2 

 x  x1 x  x2 
x  x0 x  x2  f x 
I  
f x0  
1
x0  x  x  x  x 



x

x
x

x
1
0
1
2
 0 1 0 2
x2


x  x0 x  x1 

f x2 dx
x2  x0 x2  x1 

February 20, 2023
37
Simpson’s 1/3 Rule
• Requiring equally-spaced intervals:
I 
x2
x0
  x  x0  h  x  x0  2h 
x  x0  x  x0  2h 

f  x0  
f  x1 

h  2h 
 h  h 


x  x0  x  x0  h 


f  x2   dx
 2h  h 

February 20, 2023
38
Simpson’s 1/3 Rule
• Integrate and simplify:
ba
h
2
h
I   f  x0   4 f  x1   f  x2 
3
12
10
Quadratic
Polynomial
8
6
4
2
0
3
February 20, 2023
5
7
9
11
13
15
39
Simpson’s 1/3 Rule
• If we use a = x0 and b = x2, and
x1 = (b+a)/2
f  x0   4 f  x1   f  x2 
I  b  a 
6
width
February 20, 2023
average height
40
Simpson’s 1/3 Rule
• Error for Simpson’s 1/3 rule

h 5 4 
b  a  4 
Et  
f    
f  
90
2880
5
4
O(h )
ba
h
2
4 
   0
f
Integrates a cubic exactly:
February 20, 2023
41
Composite Simpson’s 1/3 Rule
• As in composite trapezoid, break integral up
into n/2 sub-integrals:
x
x
x
I   f x dx   f x dx  ...   f x dx
x
x
x
2
0
4
n
n 2
2
• Substitute Simpson’s 1/3 rule for each
integral and collect terms.
n 1
I  b  a 
f  x0   4  f  xi   2
i 1,3,5
n2
 f x  f x 
j  2,4,6
j
n
3n
n+1 data points, an odd number
February 20, 2023
42
Error Estimate
• The error can be estimated by:
nh 5 ( 4 ) b  a h 4 ( 4 )
4
O(h )
Ea 
f 
f
180
180
• If n is doubled, h  h/2 and Ea  Ea/16
f ( 4 ) is the average 4th derivative
February 20, 2023
43
Example
 x2
• Integrate f ( x)  e from a = 0 to b = 2.
• Use Simpson’s 1/3 rule:
ba
h
1
2
2
I  e
0
 x2
x0  a  0
ab
x1 
1
2
x2  b  2
1
dx  h  f  x0   4 f ( x1 )  f  x2  
3
1
 f  0   4 f (1)  f  2  
3
1 0
 (e  4e 1  e 4 )  0.82994
3

February 20, 2023
44
Example
• Error estimate:
h 5 4 
Et  
f  
90
• Where h = b - a and a <  < b
• Don’t know 
– use average value
Et  Ea  
February 20, 2023
5
1  4
f
90
 4
 4
 4

f
x

f
x

f




 x2 
0
1
1 

90
3
5
45
Another Example
• Let’s look at the polynomial again:
f ( x )  0.2  25 x  200 x 2  675 x 3  900 x 4  400 x 5
– From a = 0 to b = 0.8
h
ba
 0.4
2
x0  a  0
x1 
ab
 0.4
2
x 2  b  0 .8
1
I   f ( x )dx  h f  x0   4 f ( x1 )  f  x2 
0
3
(0.4)
 f 0  4 f (0.4)  f 0.8

3
 1.36746667
Exact integral is 1.64053334
2
February 20, 2023
46
Error
• Actual Error: (using the known exact value)
E  1.64053334 - 1.36746667  0.27306666 16%
• Estimate error: (if the exact value is not
available)
h 5 4 
Et  
f  
90
• Where a <  < b.
February 20, 2023
47
Error
• Compute the fourth-derivative
f
( 4)
( x )  21600  48000 x
0.45
Et  Ea  
f
90
4 
0.45
x1   
f
90
4 
0.4  0.27306667
middle point
• Matches actual error pretty well.
February 20, 2023
48
Example Continued
• If we use 4 segments instead of 1:
– x = [0.0 0.2 0.4 0.6 0.8]
f (0)  0.2
f (0.6)  3.464
I  b  a 
f  x0   4
f (0.2)  1.288
f (0.8)  0.232
n 1
h
ba
 0.2
n
f (0.4)  2.456
n 2
 f x   2  f x   f x 
i 1, 3, 5
i
j 2, 4,6
j
n
3n
f (0)  4 f (0.2)  2 f (0.4)  4 f (0.6)  f (0.8)
 0.8  0 
(3)(4)
0.2  4(1.288  3.464)  2( 2.456)  0.232
 0 .8
12
 1.6234667
Exact integral is 1.64053334
February 20, 2023
49
Error
• Actual Error: (using the known exact value)
E  1.64053334 - 1.6234667  0.01706667
1%
• Estimate error: (if the exact value is not
available)
0.25 4 
0.25 4 
Et  Ea  
f x2   
f 0.4  0.0085
90
90
middle point
February 20, 2023
50
Error
• Actual is twice the estimated, why?
• Recall:
f
( 4)
max
( x )  21600  48000 x


x 0,0.8
f
( 4)
f
( 4)
( x)

f
( 4)
(0)  21600
(0.4)  2400
February 20, 2023
51
Error
• Rather than estimate, we can bound the
absolute value of the error:
0.25  4
0.25  4
Ea  
f   
f  0   0.0768
90
90
• Five times the actual, but provides a safer
error metric.
February 20, 2023
52
Simpson’s 3/8 Rule
• Determine a’s with Lagrange polynomial
• For evenly spaced points
3
I  h f  x0   3 x1   3 f  x2   f  x3 
8
ba
h
3
February 20, 2023
53
Error
• Same order as 1/3 Rule.
– More function evaluations.
– Interval width, h, is smaller.
Et  
3 5  4
h f  
80
O(h 4 )
• Integrates a cubic exactly:

February 20, 2023
f 4     0
54
Comparison
• Simpson’s 1/3 rule and Simpson’s 3/8 rule have
the same order of error
– O(h4)
– trapezoidal rule has an error of O(h2)
• Simpson’s 1/3 rule requires even number of
segments.
• Simpson’s 3/8 rule requires multiples of three
segments.
• Both Simpson’s methods require evenly spaced
data points
February 20, 2023
55