'
$
IN229/Simulation: Lecture 2
Knut–Andreas Lie
Department of Informatics
University of Oslo
February 2003
&
%
'
$
IN229/Simulation: Lecture 2
Computing derivatives
Question:
How do we compute the derivative of a
given function f (x) on a computer?
One possibility: by finite differences
Consider the definition of the derivative:
f (x + h) − f (x)
f 0(x) = lim
.
h→0
h
Idea: use a finite h to estimate f 0(x):
f (x + h) − f (x)
f (x) ≈
h
This is a forward difference. Clearly h must be
small for this to be a good approximation
0
&
1
%
'
$
IN229/Simulation: Lecture 2
Finite differences
‘Rigorous’ derivation from MacLoren series:
h2 00
f (x + h) = f (x) + hf (x) + f (ξ)
2
for x ≤ ξ ≤ x + h. We can rearrange to
0
f (x + h) − f (x) h 00
f (x) =
− f (ξ)
h
2
0
Thus, the error we make by using forward
differences is
f (x + h) − f (x)
0
−
f
(x)
≤ Mh
h
where M depends on f 00.
We call the approximation first order since the
error is of power one in h.
&
2
%
'
$
IN229/Simulation: Lecture 2
Finite differences contd.
Similarly, we derive backward differences
f (x) − f (x − h)
f (x) ≈
h
which also are first order
f (x) − f (x − h)
− f 0(x) ≤ M h.
h
0
We can also derive central differences
f (x + h) − f (x − h)
f 0(x) ≈
2h
which are second order
f (x + h) − f (x − h)
0
−
f
(x)
≤ M h2.
2h
Here M depends on f 000.
Playing with Taylor series, one can define a host of approximations....
Exercise: Verify the above formulaes.
&
3
%
&
backward
forward
central
exact
backward
forward
central
Graphical interpretation
'
$
IN229/Simulation: Lecture 2
4
%
&
N
8
16
32
64
128
256
Fb0(x) =
Fc0(x) =
f (x + h) − f (x − h)
2h
Fb0(x) Fb0(x) − f 0(x) Fc0(x) Fc0(x) − f 0(x)
2.4664
-7.39e-02 2.5389
-1.41e-03
2.5037
-3.66e-02 2.5400
-3.52e-04
2.5221
-1.82e-02 2.5402
-8.79e-05
2.5312
-9.07e-03 2.5403
-2.20e-05
2.5358
-4.53e-03 2.5403
-5.50e-06
2.5380
-2.26e-03 2.5403
-1.37e-06
f (x + h) − f (x)
,
h
Ff0 (x) Ff0 (x) − f 0(x)
2.6114
7.11e-02
2.5762
3.59e-02
2.5583
1.80e-02
2.5493
9.03e-03
2.5448
4.52e-03
2.5426
2.26e-03
f (x + h) − f (x)
,
h
f 0(x)
2.5403
2.5403
2.5403
2.5403
2.5403
2.5403
Ff0 (x) =
Example: f 0(1.0) for f (x) = x2 + sin(x)
'
$
IN229/Simulation: Lecture 2
5
%
&
where the error is bounded by |eh(x)| ≤ supx |f (4)(x)|h2/12.
f (x + h) − 2f (x) + f (x − h)
00
h
=
f
(x)
+
e
(x)
h2
0
h2 00
h3 (3)
h4 (4)
f (x + h) = f (x) + hf (x) + f (x) + f (x) + f (ξ)
2
6
24
2
3
4
h
h
h
f (x − h) = f (x) − hf 0(x) + f 00(x) − f (3)(x) + f (4)(η)
2
6
24
Adding and rearranging terms we obtain
Consider once more the Taylor series
Second order derivatives
'
$
IN229/Simulation: Lecture 2
6
%
&
− f (x)−fh(x−h)
h
bwd.
− f (x)−fh(x−h)
h
f (x+h)−f (x)
f 0(x) − f 0(x − h) fwd.
00
h
f (x)−→
−→
h
f (x + h) − 2f (x) + f (x − h)
=
h2
0
0
fwd. f (x + h) − f (x) bwd.
00
h
f (x)−→
−→
h
f (x + h) − 2f (x) + f (x − h)
=
h2
or the other way around
f (x+h)−f (x)
Alternatively, we can use forward and backward approximations:
Second order derivatives contd.
'
$
IN229/Simulation: Lecture 2
7
%
&
0
0
ctr. f (x + h/2) − f (x − h/2) ctr.
00
f (x)−→
−→
h
f (x + h) − 2f (x) + f (x − h)
=
h2
f (x+h)−f (x)
h
− f (x)−fh(x−h)
h
And to make the confusion complete; we can apply central
differences twice
Second order derivatives contd.
'
$
IN229/Simulation: Lecture 2
8
%