NEWTON's Method in Comparison with the Fixed Point Iteration
Univ.-Prof. Dr.-Ing. habil. Josef BETTEN
RWTH Aachen University
Mathematical Models in Materials Science and Continuum Mechanics
Augustinerbach 4-20
D-52056 A a c h e n , Germany
<betten@mmw.rwth-aachen.de>
Abstract
This worksheet is concerned with finding numerical solutions of non-linear equations
in a single unknown. Using MAPLE 12 NEWTON's method has been compared with
the fixed-point iteration. Some examples have been discussed in more detail.
Keywords: NEWTON's method; zero form and fixed point form; BANACH's fixed-point
theorem; convergence order
Convergence Order
A sequence with a high order of convergence converges more rapidly than a sequence with a lower
order. In this worksheet we will see that NEWTON's method is quadratically convergent, while
the fixed point iteration converges linearly to a fixed point. Some examples illustrade the
convergence of both iterations in the following. Before several examples are discussed in more
detail, let us list some definitions in the following.
A value x = p is called a fixed point for a given function g(x) if g(p) = p. In finding the
solution x = p for f(x) = 0 one can define functions g(x) with a fixed point at x = p in several
ways, for example, as g(x) = x - f(x) or as g(x) = x - h(x)*f(x) , where h(x) is a continuous
function not equal to zero within an interval [a, b] considered. The iteration process is expressed
by
> restart:
> x[n+1]:=g(x[n]);
#
n = 0,1,2,...
xn + 1 := g( xn )
with a selected starting value for n = 0 in the neighbourhood of the expected fixed point x = p.
An unique solution f(x) = 0 exists, if BANACH's fixed-point theorem is fulfilled:
Let g(x) be a continuous function in [a, b]. Assume, in addition, that g'(x) exists on (a, b)
and that a constant L = [0, 1) exists with
> restart:
1
> abs(diff(g(x),x))<=L;
d
g( x ) ≤ L
dx
for all x in [a,b]. Then, for any selected initial value in [a, b] the sequence defined by
> x[n+1]:=g(x[n]);
# n = 0,1,2,...
xn + 1 := g( xn )
converges to the unique fixed-point x = p in [a, b]. The constant L is known as LIPPSCHITZ
constant. Based upon the mean value theorem we arrive from the above assumption at
> abs(g(x)-g(xi))<=L*abs(x-xi);
g( x ) − g( ξ ) ≤ L x − ξ
for all x and xi in [a, b]. The BANACH fixed-point theorem is sometimes called the
contraction mapping principle.
A sequence converges to p of order alpha, if
> restart:
> Limit(abs(x[n+1]-p)/abs(x[n]-p)^alpha,n=infinity)=epsilon;
−xn + 1 + p
=ε
α
−xn + p
with an asymptotic error constant epsilon. Another definition of the convergence order
is given by:
> abs(x[n+1]-p)<=C*abs(x[n]-p)^alpha;
lim
n→∞
−xn + 1 + p ≤ C −xn + p
α
where C is a constant. The fixed-point iteration converges linearly (alpha = 1) with
a constant C = (0, 1). This can be shown, for example, as follows:
> g(x)=g(x[n])+(x-x[n])*Diff(g(x),x)[x=x[n]]+
(x-x[n])^2*Diff(g(x),x$2)[x=xi]/2;
# TAYLOR
⎛d
⎞
g( x ) = g( xn ) + ( x − xn ) ⎜⎜ g( x ) ⎟⎟
⎝ dx
⎠
x=x
n
2
⎞
1
2⎛ d
⎜
+ ( x − xn ) ⎜ 2 g( x ) ⎟⎟
2
⎝ dx
⎠
x=ξ
> g(x):=g(x[n])+(x-x[n])*`g'`(x[n])+(x-x[n])^2*`g''`(xi)/2;
1
2
( x − xn ) g''( ξ )
2
where xi in the remainder term lies between x and x[n]. For x = p we arrive at
> g(p):=g(x[n])+(p-x[n])*`g'`(x[n])+(p-x[n])^2*`g''`(xi)/2;
g( x ) := g( xn ) + ( x − xn ) g'( xn ) +
1
2
( −xn + p ) g''( ξ )
2
The 3rd term on the right hand side may be neglected, because x[n] is an approximation
near to the fixed-point, (p - x[n])^2 << (p - x[n]) , hence:
> g(p):=g(x[n])+(p-x[n])*`g'`(x[n]);
g( p ) := g( xn ) + ( −xn + p ) g'( xn ) +
g( p ) := g( xn ) + ( −xn + p ) g'( xn )
Corresponding with the fixed-point iteration g(p) = p and g(x[n]) = x[n+1] we arrive from
2
this equation at:
> abs(x[n+1]-p)=abs(`g'`(x[n]))*abs(x[n]-p);
−xn + 1 + p = g'( xn ) −xn + p
where
> abs(`g'`(x[n]))<=L;
g'( xn ) ≤ L
> abs(x[n+1]-p)<=L*abs(x[n]-p);
−x n + 1 + p ≤ L −xn + p
Thus, the fixed-point iteration converges linearly to the fixed-point p. In contrast, NEWTON's
method is quadratically convergent. This can be shown, for example, as follows.
> G(x)=G(p)+(x-p)*Diff(G(x),x)[x=p]+
(x-p)^2*Diff(G(x),x$2)[x=xi]/2;
⎛d
⎞
G( x ) = G( p ) + ( x − p ) ⎜⎜ G( x ) ⎟⎟
⎝ dx
⎠
x=p
⎛ 2
⎞
1
2⎜ d
+ ( x − p ) ⎜ 2 G( x ) ⎟⎟
2
⎝ dx
⎠
x=ξ
> G(x):=G(p)+(x-p)*`G'`(p)+(x-p)^2*`G''`(xi)/2;
G( x ) := G( p ) + ( x − p ) G'( p ) +
1
( x − p )2 G''( ξ )
2
where xi lies between x and p. For x = x[n] we have:
> G(x[n])=G(p)+(x[n]-p)*`G'`(p)+ (x[n]-p)^2*`G''`(zeta)/2;
1
2
( xn − p ) G''( ζ )
2
where zeta lies between x[n] and p. Because of G(x[n]) = x[n+1] and G(p) = p we get:
> x[n+1]-p=(x[n]-p)*`G'`(p)+(x[n]-p)^2*`G''`(zeta)/2;
G( xn ) = G( p ) + ( xn − p ) G'( p ) +
1
2
( xn − p ) G''( ζ )
2
The first term on the right hand side must be equal to zero, if the iteration x[n+1] = G(x[n])
should converge quadratically to the fixed-point p :
> `G'`(p):=0;
xn + 1 − p = ( xn − p ) G'( p ) +
G'( p ) := 0
> x[n+1]-p=`G''`(zeta)*(x[n]-p)^2;
2
xn + 1 − p = ( xn − p ) G''( ζ )
NEWTON's method has the convergence order alpha = 2 ---> G'(p) = 0. Hence:
> restart:
> G(x):=x-h(x)*f(x); # iteration function
G( x ) := x − h( x ) f( x )
> `G'`(x):=diff(G(x),x);
⎛d
⎞
⎛d
⎞
G'( x ) := 1 − ⎜⎜ h( x ) ⎟⎟ f( x ) − h( x ) ⎜⎜ f( x ) ⎟⎟
⎝ dx
⎠
⎝ dx
⎠
> `G'`(x):=1-`h'`(x)*f(x)-h(x)*`f '`(x);
G'( x ) := 1 − h'( x ) f( x ) − h( x ) f '( x )
3
> `G'`(p):=subs(x=p,%);
G'( p ) := 1 − h'( p ) f( p ) − h( p ) f '( p )
At the fixed-point p we have G'(p) = 0 and f(p) = 0. Hence:
> h(p):=1/`f '`(p);
h(x):=1/`f '`(x);
h( p ) :=
1
f '( p )
h( x ) :=
1
f '( x )
> G(x):=x-f(x)/`f '`(x);
f( x )
f '( x )
Because of x[n+1] = G(x[n] we find NEWTON's iteration method:
> x[n+1]:=x[n]-f(x[n])/`f '`(x[n]);
G( x ) := x −
xn + 1 := xn −
f( xn )
f '( xn )
>
NEWTON's iteration method has the disadvantage that it cannot be continued, if f '(x[n]) is
equal to zero for any step x[n]. However, the method is most effective, if f '(x[n]) is bounded
away from zero near the fixed-point p.
Note, in cases when there is a point of inflection or a horizontal tangent to the function f(x)
in the vicinity of the fixed-point, the sequence x[n+1] = G(x[n]) need not converge to the
fixed-point. Thus, before applying NEWTON 's method, one should investigate the behaviour
of the derivatives f '(x) and f ''(x) in the neighbourhood of the expected fixed-point.
Instead of the analytical derivation of NEWTON's method one can find the approximations
x[n] to the fixed-point p by using tangents to the graph of the given function f(x) with f(p) = 0.
Beginning with x[0] we obtain the first approximation x[1] as the x-intersept of the tangent line
to the graph of f(x) at (x[0], f(x[0])). The next approximation x[2] is the x-intercept of the
tangent line to the graph of f(x) at (x[1], f(x[1])) and so on. Following this procedure we arrive
at NEWTON's iteration characterized by the iteration function G(x) defined before. The derivative
of this function is given by:
> restart:
> Diff(G(x),x)=
simplify(1-((Diff(f(x),x))^2-f(x)*Diff(f(x),x$2))/
Diff(f(x),x)^2);
d
G( x ) =
dx
⎛ d2
⎞
f( x ) ⎜⎜ 2 f( x ) ⎟⎟
⎝ dx
⎠
⎞
⎛d
⎜⎜ f( x ) ⎟⎟
⎝ dx
⎠
> `G'`(x):=f(x)*`f ''`(x)/(`f '`(x))^2;
G'( x ) :=
2
f( x ) f ''( x )
f '( x )2
At the fixed-point the given function has a zero, f(p) = 0. Hence:
4
> `G'`(p):=Diff(G(x),x)[x=p]=0;
⎛d
⎞
G'( p ) := ⎜⎜ G( x ) ⎟⎟
⎝ dx
⎠
=0
x=p
>
Résumé: NEWTON's method converges optimal ( G'(p) = 0 ) to the fixed-point p unless
f '(x) = 0 for some x[n]. The derivative G'(p) = 0 implies quadratic convergence.
>
Examples
The first example is concerned with the root-finding problem f(x) = x - cos(x) = 0 by using
the iteration functions g(x) und G(x) characterized by linear and quadratic convergence,
respectively.
> restart:
> f(x):=x-cos(x);
> p:=fsolve(f(x)=0,x);
f( x ) := x − cos( x )
# MAPLE solution by the command "fsolve"
p := 0.7390851332
> `f '`(x):=diff(f(x),x);
f '( x ) := 1 + sin( x )
> `f ''`(x):=diff(f(x),x$2);
f ''( x ) := cos( x )
>
> alias(H=Heaviside,th=thickness,co=color):
> p[1]:=plot({f(x),1+sin(x),cos(x)},x=0..Pi/2,-1..2,
th=3,co=black):
> p[2]:=plot({2*H(x-1.571),-H(x-1.571)},x=1.57..1.572,co=black,
title="f(x), f'(x), f''(x),
Fixed Point p = 0.7391"):
> p[3]:=plot({2,-1},x=0..Pi/2,co=black):
> p[4]:=plot(1.674*H(x-0.74),x=0.739..0.741,
linestyle=4,co=black):
> p[5]:=plot([[0.74,1.674]],style=point,
symbol=circle,symbolsize=30,co=black):
> p[6]:=plots[textplot]({[1.15,1,`f(x)`],[0.2,1.4,`f'(x)`],
[0.2,0.8,`f''(x)`]},co=black):
> plots[display](seq(p[k],k=1..6));
5
>
In this Figure we see that the first derivative f '(x) is not equal to zero in the entire range
considered, which is an essential condition for the application of NEWTON's method.
The iteration functions are given by
> g(x):=x-f(x);
G(x):=x-f(x)/diff(f(x),x);
g( x ) := cos( x )
G( x ) := x −
x − cos( x )
1 + sin( x )
>
> alias(H=Heaviside,sc=scaling,th=thickness,co=color):
> p[1]:=plot({x,g(x),G(x)},x=0..1,0..1,
sc=constrained,th=3,co=black):
> p[2]:=plot(H(x-1),x=0.99..1.001,co=black):
> p[3]:=plot(1,x=0..1,co=black,
title="Iteration Functions g(x) and G(x)"):
> p[4]:=plot([[0.7391,0.7391]],style=point,
symbol=circle,symbolsize=30,co=black):
> p[5]:=plots[textplot]({[0.15,0.8,`G(x)`],
[0.6,0.92,`g(x)`]},co=black):
> p[6]:=plot(0.7391*H(x-0.7391),x=0.739..0.7392,
linestyle=4,co=black):
> plots[display](seq(p[k],k=1..6));
6
>
Both operators, G and g, mapp the interval x = [0, 1] to itself. The iteration function G(x) has
a horizontal tangent in x = p because of G'(p) = 0, id est: quadratic convergence.
Now, let's discuss the absolute derivatives of the iteration functions.
> abs(`g'`(x))=abs(diff(g(x),x));
g'( x ) = sin( x )
> abs(`G'`(x))=abs(diff(G(x),x));
G'( x ) =
>
>
>
>
>
>
>
( x − cos( x ) ) cos( x )
( 1 + sin( x ) )2
p[1]:=plot({rhs(%%),rhs(%)},x=0..1,0..1,
sc=constrained,th=3,co=black):
p[2]:=plot(0.67*H(x-0.74),x=0.735..0.745,linestyle=4,co=black,
title="Absolute Derivatives | G'(x)| and | g'(x) |"):
p[3]:=plot([[0.74,0.67]],style=point,
symbol=circle,symbolsize=30,co=black):
p[4]:=plot(H(x-1),x=0.99..1.001,co=black):
p[5]:=plot(1,x=0..1,co=black):
p[6]:=plots[textplot]({[0.16,0.9,`| G'(x) |`],
[0.85,0.9,`| g'(x) |`]},co=black):
plots[display](seq(p[k],k=1..6));
7
>
This Figure illustrades that both derivatives exist on (0, 1) with | g'(x) | < L and
| G'(x) | < K for all x = [0, 1], where K < 1 and L < 1. Considering the last two
Figures, we establish that both iteration functions are compatible with BANACH's
fixed-point theorem.
The iterations generated by g(x) and G(x) are given as follows:
> x[0]:=0.7; x[1]:=evalf(subs(x=0.7,g(x)));
x0 := 0.7
x1 := 0.7648421873
Fixed Point Iteration:
> for i from 2 to 25 do x[i]:=evalf(subs(x=%,g(x))) od;
x2 := 0.7214916396
x3 := 0.7508213288
x4 := 0.7311287726
x5 := 0.7444211836
x6 := 0.7354802004
x7 := 0.7415086517
x8 := 0.7374504531
x9 := 0.7401852854
x10 := 0.7383436103
x11 := 0.7395844287
x12 := 0.7387487097
x13 := 0.7393117103
x14 := 0.7389324892
x15 := 0.7391879474
x16 := 0.7390158724
x17 := 0.7391317864
8
x18 := 0.7390537063
x19 := 0.7391063024
x20 := 0.7390708732
x21 := 0.7390947389
x22 := 0.7390786627
x23 := 0.7390894918
x24 := 0.7390821972
x25 := 0.7390871109
>
The 25th iteration x[25] is nearly identical to the MAPLE solution p = 0.7390851332.
NEWTON's Method:
> X[0]:=0.7; X[1]:=evalf(subs(x=%,G(x)));
X0 := 0.7
X1 := 0.7394364979
> for i from 2 to 5 do X[i]:=evalf(subs(x=%,G(x))) od;
X2 := 0.7390851605
X3 := 0.7390851332
X4 := 0.7390851332
X5 := 0.7390851332
>
The 3rd iteration is already identical to the MAPLE solution. In contrast, the fixed-point
method needs about 25 iterations. The NEWTON method converges quadratically, while
the fixed point sequence is linear convergent.
>
Similar to the first example, the next one is concerned with solving the problem f(x) = 0,
where
> restart:
> f(x):=x-(sin(x)+cos(x))/2;
1
1
sin( x ) − cos( x )
2
2
# fixed point by MAPLE
f( x ) := x −
> p:=fsolve(f(x)=0,x);
> `f '`(x):=diff(f(x),x);
p := 0.7048120020
`f ''`(x):=diff(f(x),x$2);
1
1
cos( x ) + sin( x )
2
2
1
1
f ''( x ) := sin( x ) + cos( x )
2
2
f '( x ) := 1 −
>
> alias(H=Heaviside,th=thickness,co=color):
9
> p[1]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)},
x=0..Pi/2,th=3,co=black,scaling=constrained):
> p[2]:=plot({1.571*H(x-1.571),-0.5*H(x-1.571)},
x=1.57..1.572,co=black):
> p[3]:=plot(0.943*H(x-0.705),x=0.704..0.706,
linestyle=4,co=black):
> p[4]:=plot({-0.5,Pi/2},x=0..Pi/2,co=black,
title="f(x), f'(x), f''(x), Fixed Point p = 0.7048"):
> p[5]:=plot([[0.705,0.943]],style=point,
symbol=circle,symbolsize=30,co=black):
> p[6]:=plots[textplot]({[0.85,0.3,`f(x)`],[0.85,1.25,`f'(x)`],
[0.85,0.6,`f''(x)`]},co=black):
> plots[display](seq(p[k],k=1..6));
>
In this Figure we see that the first derivative f '(x) is not equal to zero in the entire range
considered, which is an essential condition for the application of NEWTON 's method.
The iteration functions are given by
> g(x):=x-f(x);
G(x):=x-f(x)/diff(f(x),x);
g( x ) :=
1
1
sin( x ) + cos( x )
2
2
x−
1
1
sin( x ) − cos( x )
2
2
1−
1
1
cos( x ) + sin( x )
2
2
G( x ) := x −
>
> alias(H=Heaviside,sc=scaling,th=thickness,co=color):
> p[1]:=plot({x,g(x),G(x)},x=0..1,0..1,th=3,
sc=constrained,co=black):
> p[2]:=plot({1,H(x-1)},x=0..1.001,co=black,
title="Iteration Functions g(x) and G(x)"):
> p[3]:=plot(0.7048*H(x-0.7048),x=0.7047..0.7049,
10
linestyle=4,co=black):
> p[4]:=plot([[0.7048,0.7048]],style=point,
symbol=circle,symbolsize=30,co=black):
> p[5]:=plots[textplot]({[0.15,0.52,`g(x)`],
[0.15,0.8,`G(x)`]},co=black):
> plots[display](seq(p[k],k=1..5));
>
Both operators, g and G , mapp the interval x = [0, 1] to itself. The iteration function G(x)
has a horizontal tangent in the fixed-point p = 0.7048120020 because of G'(p) = 0, id est:
quadratic convergence. In contrast, the iteration function g(x) has a horizontal tangent in
x = Pi/4 = 0.785398163, id est: in the neighbourhood of the fixed-point.
Now let's discuss the absolute derivatives of the iteration functions.
> abs(`g'`(x))=abs(diff(g(x),x));
1
1
cos( x ) − sin( x )
2
2
> abs(`G'`(x))=abs(diff(G(x),x));
g'( x ) =
G'( x ) =
⎛
1
1
⎞ ⎛1
1
⎞
⎜⎜ x − sin( x ) − cos( x ) ⎟⎟ ⎜⎜ sin( x ) + cos( x ) ⎟⎟
⎝
2
2
⎠ ⎝2
2
⎠
⎛
1
1
⎞
⎜⎜ 1 − cos( x ) + sin( x ) ⎟⎟
⎝
2
2
⎠
2
>
> p[1]:=plot({abs(diff(g(x),x)),abs(diff(G(x),x))},x=0..1,0..1,
scaling=constrained,th=3,co=black):
> p[2]:=plot({1,H(x-1)},x=0..1.001,co=black,
title="Absolute Derivatives | g'(x) | and | G'(x) |"):
> p[3]:=plots[textplot]({[0.2,0.3,`| g'(x) |`],
[0.2,0.9,`| G'(x) |`]},co=black):
> plots[display](seq(p[k],k=1..3));
11
>
This Figure illustrades that both derivatives exist on (0, 1) with | g'(x) | < L and
| G'(x) | < K for all x = [0, 1], where K < 1 and L < 1. Considering the last two
Figures, we establish that both iteration functions are compatible with BANACH 's
fixed-point theorem.
The iterations generated by g(x) and G(x) are given as follows:
> x[0]:=0.5; x[1]:=evalf(subs(x=0.5,g(x)));
x0 := 0.5
x1 := 0.6785040503
Fixed Point Iteration:
> for i from 2 to 10 do x[i]:=evalf(subs(x=%,g(x))) od;
x2 := 0.7030708012
x3 := 0.7047118221
x4 := 0.7048062961
x5 := 0.7048116773
x6 := 0.7048119834
x7 := 0.7048120009
x8 := 0.7048120019
x9 := 0.7048120020
x10 := 0.7048120020
>
The 9th iteration x[9] is identical to the MAPLE solution p = 0.7048120020 based
upon the command fsolve.
NEWTON 's Method:
> X[0]:=0.5;
X[1]:=evalf(subs(x=0.5,G(x)));
X0 := 0.5
X1 := 0.7228733439
> for i from 2 to 6 do X[i]:=evalf(subs(x=%,G(x))) od;
12
X2 := 0.7049323822
X3 := 0.7048120074
X4 := 0.7048120021
X5 := 0.7048120020
X6 := 0.7048120020
>
The 5th iteration X[5] is already identical to the MAPLE solution. In contrast, the fixed-point
method needs 9 iterations.
>
The next example is concerned with the zero form f(x) = x - exp(x^2 -2) = 0.
> restart:
> f(x):=x-exp(x^2-2);
2
(x − 2)
> p:=fsolve(f(x)=0,x);
f( x ) := x − e
# fixed-point by MAPLE
p := 0.1379348256
> `f '`(x):=diff(f(x),x);
f '( x ) := 1 − 2 x e
> `f ''`(x):=diff(f(x),x$2);
2
(x − 2)
>
>
>
>
>
>
>
>
2
(x − 2)
2
(x − 2)
f ''( x ) := −2 e
− 4 x2 e
alias(H=Heaviside,sc=scaling,th=thickness,co=color):
p[1]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)},
x=0..0.5,-1..1,th=3,co=black):
p[2]:=plot({H(x-0.5),-H(x-0.5)},x=0.499..0.501,co=black):
p[3]:=plot(0.96*H(x-0.14),x=0.139..0.141,linestyle=4,co=black):
p[4]:=plot({-1,1},x=0..0.5,co=black,
title="f(x) ,
f'(x) ,
f''(x)"):
p[5]:=plot([[0.14,0.96]],style=point,symbol=circle,
symbolsize=30,ytickmarks=4,co=black):
p[6]:=plots[textplot]({[0.3,0.3,`f(x)`],[0.3,0.8,`f'(x)`],
[0.3,-0.5,`f''(x)`]},co=black):
plots[display](seq(p[k],k=1..6));
13
>
In this Figure we see that the first derivative f'(x) is not equal to zero in the entire range
considered, which is a necessary condition for convergence of the NEWTON method.
The iteration functions are given by
> g(x):=x-f(x);
G(x):=x-f(x)/diff(f(x),x);
g( x ) := e
G( x ) := x −
2
(x − 2)
x−e
2
(x − 2)
1−2xe
2
(x − 2)
>
> p[1]:=plot({x,g(x),G(x)},x=0..0.5,
sc=constrained,th=3,co=black):
> p[2]:=plot({0.5,0.5*H(x-0.5)},x=0..0.5001,co=black,
title="Iteration Functions g(x) and G(x)"):
> p[3]:=plot(0.14*H(x-0.14),x=0.139..0.141,
linestyle=4,co=black):
> p[4]:=plot([[0.14,0.14]],style=point,symbol=circle,
symbolsize=30,co=black):
> p[5]:=plots[textplot]({[0.3,0.18,`g(x)`],
[0.3,0.11,`G(x)`]},co=black):
> plots[display](seq(p[k],k=1..5));
14
>
Both operators, g and G , mapp the interval x = [0, 0.5] to itself. The iteration function G(x) has
a horizontal tangent in the fixed-point p = 0.1379348256 because of G'(p) = 0, id est quadratic
convergence. In contrast, the iteration function g(x) has a horizontal tangent in x = 0.
Now let's discuss the absolute derivatives of the iteration functions.
> abs(`g'`(x))=abs(diff(g(x),x));
2
( −2 + ℜ( x ) )
g'( x ) = 2 e
> abs(`G'`(x))=abs(diff(G(x),x));
G'( x ) =
(x − e
2
(x − 2)
) ( −2 e
2
(x − 2)
(1 − 2 x e
x
− 4 x2 e
2
(x − 2)
2
(x − 2)
)
2
)
>
> p[1]:=plot({abs(diff(g(x),x)),abs(diff(G(x),x))},
x=0..0.5,th=3,co=black,
title="Absolute Derivatives | g'(x) | and
G'(x) |"):
> p[2]:=plot({0.25,0.25*H(x-0.5)},x=0..0.5001,co=black):
> p[3]:=plots[textplot]({[0.25,0.1,`| g'(x) |`],
[0.33,0.05,`| G'(x) |`]},co=black):
> plots[display](seq(p[k],k=1..3));
15
>
This Figure illustrades that both derivatives exist on (0, 0.5) with | g'(x) | < L and
| G'(x) | < K for all x = [0, 0.5], where K < 1 and L < 1. Considering the last two
Figures, we find that both iteration functions are compatible with BANACH's
fixed-point theorm.
The iterations generated by g(x) and G(x) are listed in the following:
> x[0]:=0.1;
x[1]:=evalf(subs(x=0.1,g(x)));
x0 := 0.1
x1 := 0.1366954254
Fixed Point Iteration:
> for i from 2 to 8 do x[i]:=evalf(subs(x=%,g(x))) od;
x2 := 0.1378878837
x3 := 0.1379330396
x4 := 0.1379347575
x5 := 0.1379348229
x6 := 0.1379348254
x7 := 0.1379348256
x8 := 0.1379348256
>
The 7th iteration x[7] is identical to the MAPLE solution p = 0.1379348256 based
upon the command fsolve.
NEWTON's Method:
> X[0]:=0.1;
X[1]:=evalf(subs(x=0.1,G(x)));
X0 := 0.1
X1 := 0.1377268428
> for i from 2 to 5 do X[i]:=evalf(subs(x=%,G(x))) od;
X2 := 0.1379348191
X3 := 0.1379348255
16
X4 := 0.1379348256
X5 := 0.1379348256
>
The 4th iteration X[4] is already identical to the MAPLE solution. In contrast, the fixed-point
method needs 7 iterations.
Another example is concerned with the root-finding problem f(x) = 0, where
> restart:
> f(x):=1+cosh(x)*cos(x);
> p:=fsolve(f(x)=0,x);
f( x ) := 1 + cosh( x ) cos( x )
# fixed-point by MAPLE
p := 1.875104069
> `f '`(x):=diff(f(x),x);
f '( x ) := sinh( x ) cos( x ) − cosh( x ) sin( x )
> `f ''`(x):=diff(f(x),x$2);
>
>
>
>
>
>
>
f ''( x ) := −2 sinh( x ) sin( x )
alias(H=Heaviside,sc=scaling,th=thickness,co=color):
p[1]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)},
x=1.5..2,-7..1.5,th=3,co=black):
p[2]:=plot({-7,-7*H(x-2),1.5,1.5*H(x-2)},x=1.5..2.001,co=black,
title="f(x) , f'(x) , f''(x) , Fixed-Point p =
1.875104069"):
p[3]:=plot(-4.17*H(x-1.88),x=1.87..1.89,linestyle=4,co=black):
p[4]:=plot([[1.88,-4.17]],style=point,symbol=circle,
symbolsize=30,co=black):
p[5]:=plots[textplot]({[2.03,-0.57,`f(x)`],[2.03,-5,`f'(x)`],
[2.03,-6.6,`f''(x)`]},co=black):
plots[display](seq(p[k],k=1..5));
>
In this Figure we see that the first derivative f '(x) is not equal to zero in the vicinity of the
fixed-point, which is a necessary condition for the application of NEWTON 's method. Its
iteration function is given by
> G(xi):=xi-f(xi)/diff(f(xi),xi); G(x):=x-f(x)/diff(f(x),x);
17
G( ξ ) := ξ −
G( x ) := x −
f( ξ )
d
f( ξ )
dξ
1 + cosh( x ) cos( x )
sinh( x ) cos( x ) − cosh( x ) sin( x )
> x[n+1]:=G(x[n]);
> x[0]:=2;
xn + 1 := G( xn )
x[1]:=evalf(subs(x=2,G(x)));
x0 := 2
x1 := 1.885274675
> for i from 2 to 5 do x[i]:=evalf(subs(x=%,G(x))) od;
x2 := 1.875179254
x3 := 1.875104073
x4 := 1.875104069
x5 := 1.875104069
>
The 4th iteration x[4], beginning with a starting point x[0] = 2, is already identical to the
MAPLE
solution. Selecting the starting point x[0] = 1.2, then the 6th iteration
x[6] is identical to the fixed
point. However, the starting point x[0] = 1 does not lead to convergence.
Improving the convergence or obtaining convergence, necessary in cases of small derivatives
f '(x) << 1, one can extend the classical NEWTON method in the following way:
> restart:
> X[n+1]:=G(X[n]); G(X):=X-lambda*h(X)*f(X);
Xn + 1 := G( Xn )
G( X ) := X − λ h( X ) f( X )
> `G'(X)`:=diff(G(X),X);
⎛ d
⎞
⎛ d
⎞
G'(X) := 1 − λ ⎜⎜ h( X ) ⎟⎟ f( X ) − λ h( X ) ⎜⎜ f( X ) ⎟⎟
⎝ dX
⎠
⎝ dX
⎠
> lambda[LAGRANGE]:=solve(diff(G(X),X)=0,lambda);
λLAGRANGE :=
1
G( X ) := X −
h( X ) f( X )
⎛ d
⎞
⎛ d
⎞
⎜⎜ h( X ) ⎟⎟ f( X ) + h( X ) ⎜⎜ f( X ) ⎟⎟
⎝ dX
⎠
⎝ dX
⎠
> G(X):=subs(lambda=%,G(X));
⎛ d
⎞
⎛ d
⎞
⎜⎜ h( X ) ⎟⎟ f( X ) + h( X ) ⎜⎜ f( X ) ⎟⎟
⎝ dX
⎠
⎝ dX
⎠
>
Assuming h(X) = -exp(-X), we arrive at
> h(x):=-exp(-x);
h( x ) := −e
( −x )
18
> G(x):=subs({X=x,h(X)=h(x)},G(X));
e
G( x ) := x +
( −x )
f( x )
⎛d
⎞
( −x ) ⎞
( −x ) ⎛ d
⎜⎜ ( −e ) ⎟⎟ f( x ) − e
⎜⎜ f( x ) ⎟⎟
⎝ dx
⎠
⎝ dx
⎠
> G(xi):=xi-f(xi)/(`f '`(xi)-f(xi));
G( ξ ) := ξ −
f( ξ )
f '( ξ ) − f( ξ )
> f(x):=1+cosh(x)*cos(x);
f( x ) := 1 + cosh( x ) cos( x )
> G(x):=x-f(x)/(diff(f(x),x)-f(x));
1 + cosh( x ) cos( x )
sinh( x ) cos( x ) − cosh( x ) sin( x ) − 1 − cosh( x ) cos( x )
x[1]:=evalf(subs(x=2,G(x)));
G( x ) := x −
> x[0]:=2;
x0 := 2
x1 := 1.870407086
> for i from 2 to 5 do x[i]:=evalf(subs(x=%,G(x))) od;
x2 := 1.875098228
x3 := 1.875104069
x4 := 1.875104069
x5 := 1.875104069
>
We see, the iteration has been improved from 4 to 3 iterations. With a starting point x[0] = 1,
the 4th iteration leads to the fixed-point instead of divergence by using the classical NEWTON
method.
The fixed-point iteration
> x[n+1]:=g(x[n])=x[n]-h(x[n])*f(x[n]);
xn + 1 := g( xn ) = xn − h( xn ) f( xn )
does not fulfill BANACH 's theorem for h(x) = 1. A compatible function is given by the
function h(x) = -exp(-x) introduced before. Thus, we arrive at the following iteration function:
> g(x):=x+exp(-x)*f(x);
g( x ) := x + e
>
> x[0]:=2;
( −x )
( 1 + cosh( x ) cos( x ) )
x[1]:=evalf(subs(x=%,g(x)));
x0 := 2
x1 := 1.923450867
> for i from 2 to 22 do x[i]:=evalf(subs(x=%,g(x))) od;
x2 := 1.893171371
x3 := 1.881762275
x4 := 1.877544885
19
x5 := 1.875997102
x6 := 1.875430574
x7 := 1.875223412
x8 := 1.875147687
x9 := 1.875120010
x10 := 1.875109895
x11 := 1.875106198
x12 := 1.875104847
x13 := 1.875104353
x14 := 1.875104173
x15 := 1.875104107
x16 := 1.875104083
x17 := 1.875104074
x18 := 1.875104071
x19 := 1.875104070
x20 := 1.875104069
x21 := 1.875104069
x22 := 1.875104069
>
The 20th iteration x[20] leads to the fixed-point. In contrast to only 4 or 3 iterations based upon
NEWTON 's classical or extended method, respectively.
The following two Figures should illustrade that both iteration functions, g(x) and G(x) , are
compatible with BANACH 's fixed-point theorem.
>
> alias(H=Heaviside,sc=scaling,th=thickness,co=color):
> p[1]:=plot({x,g(x),G(x)},x=1.5..2,1.5..2,
sc=constrained,th=3,co=black):
> p[2]:=plot(2*H(x-2),x=1.99..2.001,co=black):
> p[3]:=plot(2,x=1.5..2,co=black,
title="G(x) in comparison with g(x)"):
> p[4]:=plot([[1.8751,1.8751]],style=point,
symbol=circle,symbolsize=30,co=black):
> p[5]:=plots[textplot]({[1.57,1.9,`G(x)`],
[1.57,1.74,`g(x)`]},co=black):
> p[6]:=plot(1.8751*H(x-1.8751),x=1.875..1.8752,
linestyle=4,co=black):
> plots[display](seq(p[k],k=1..6));
20
>
Both operators, G and g, mapp the interval x = [1.5, 2] to itself. The function G(x) has
a horizontal tangent in the fixed-point p = 1.875104069 within the interval considered.
This means quadratic convergence of the extended NEWTON method.
Corresponding to BANACH 's theorem the absolute derivatives | g'(x) | and | G'(x) |
should be less than one as shown in the next Figure.
> abs(`g'`(x))=abs(diff(g(x),x));
( −x )
g'( x ) = −1 + e
( 1 + cosh( x ) cos( x ) ) − e
> abs(`G'`(x))=abs(diff(G(x),x));
G'( x ) = 1 −
+
( −x )
( sinh( x ) cos( x ) − cosh( x ) sin( x ) )
sinh( x ) cos( x ) − cosh( x ) sin( x )
sinh( x ) cos( x ) − cosh( x ) sin( x ) − 1 − cosh( x ) cos( x )
( 1 + cosh( x ) cos( x ) ) ( −2 sinh( x ) sin( x ) − sinh( x ) cos( x ) + cosh( x ) sin( x ) )
( sinh( x ) cos( x ) − cosh( x ) sin( x ) − 1 − cosh( x ) cos( x ) )2
>
> p[1]:=plot({abs(diff(g(x),x)),abs(diff(G(x),x))},
x=1.5..2,0..0.5,th=3,co=black):
> p[2]:=plot(0.5*H(x-2),x=1.99..2.001,co=black):
> p[3]:=plot(0.5,x=1.5..2,co=black,
title="Absolute Derivatives | g'(x) | and | G'(x) |"):
> p[4]:=plot(0.3655*H(x-1.8751),x=1.875..1.8752,
linestyle=4,co=black):
> p[5]:=plot([[1.8751,0.3655]],style=point,symbol=circle,
symbolsize=30,co=black):
> p[6]:=plots[textplot]({[1.6,0.32,`| g'(x) |`],
[1.6,0.15,`| G'(x) |`]},co=black):
> plots[display](seq(p[k],k=1..6));
21
>
The last two Figures illustrade that both iterations, extended NEWTON and fixed-point, are
compatible with BANACH 's theorem. Both operators, g and G, mapp the interval x = [1.5, 2]
to itself. In addition, both derivatives exist on (1.5, 2) with | g'(x) | < L and | G'(x) | < K for
all x = [1.5, 2] , where K < L < 1. The number L is the LIPPSCHITZ constant.
NEWTON 's method converges quadratically because of | G'(p) | = 0.
>
Another example illustrades as before that the extended NEWTON method is most effective
in cases when the first derivative f '(x[n]) is equal to zero or very small for any step x[n] if
the classical NEWTON method does not work.
> restart:
> f(x):=x-2*sin(x);
f( x ) := x − 2 sin( x )
> p:=fsolve(f(x)=0,x,1..2);
# fixed-point immediately found by MAPLE command "fsolve"
p := 1.895494267
> `f '`(x):=diff(f(x),x);
f '( x ) := 1 − 2 cos( x )
> `f '`(P):=evalf(subs(x=p,%));
f '( P ) := 1.638045048
> `f ''`(x):=diff(f(x),x$2);
f ''( x ) := 2 sin( x )
> `f ''`(P):=evalf(subs(x=p,%));
f ''( P ) := 1.895494267
> alias(H=Heaviside,th=thickness,sc=scaling,co=color):
> p[1]:=plot({f(x),diff(f(x),x),diff(f(x),x$2)},
x=0..Pi,-1..Pi,sc=constrained,th=3,co=black):
> p[2]:=plot({-1,Pi,Pi*H(x-Pi),-H(x-Pi)},
x=0..1.001*Pi,co=black,
title="f(x), f'(x), f''(x), Fixed-Point p"):
> p[3]:=plot(1.638*H(x-1.8955),x=1.8954..1.8956,
linestyle=4,co=black):
> p[4]:=plot([[1.8955,1.638]],style=point,
symbol=circle,symbolsize=30,co=black):
22
> p[5]:=plots[textplot]({[2.8,1.5,`f(x)`],
[2.0,2.5,`f'(x)`],[0.5,1.5,`f''(x)`]},co=black):
> plots[display](seq(p[k],k=1..5));
>
In the vicinity of the fixed-point the first derivative f '(x) is not very small so that the classical
NEWTON method can work, if the starting-point x[0] is close enough to the expected
fixed-point.
However, in order to test the extended NEWTON formular, the starting-point should be selected,
for instance, at x = 1 close to the zero of f '(x):
> X[ZERO]:=fsolve(diff(f(x)=0,x));
XZERO := 1.047197551
>
Classical NEWTON Method
> G(xi):=xi-f(xi)/`f '`(xi);
G( ξ ) := ξ −
f( ξ )
f '( ξ )
> G(x):=x-f(x)/diff(f(x),x);
x − 2 sin( x )
1 − 2 cos( x )
> x[0]:=1; x[1]:=evalf(subs(x=1,G(x))); # starting-point
G( x ) := x −
x0 := 1
x1 := -7.472740617
> for i from 2 to 7 do x[i]:=evalf(subs(x=%,G(x))) od;
x2 := 14.47852462
x3 := 6.935146381
x4 := 16.63630229
23
x5 := 8.340204744
x6 := 4.943086934
x7 := -7.753209046
>
In the vicinity of the selected starting-point x[0] = 1 the first derivative f '(x) is very small.
Thus, the classical NEWTON method does not converge. With the same starting-point we will
obtain convergence by applying the extended NEWTON formular:
> restart:
> f(x):=x-2*sin(x);
f( x ) := x − 2 sin( x )
> p:=fsolve(f(x)=0,x,1..2);
p := 1.895494267
> G(xi):=xi-f(xi)/(`f '`(xi)-f(xi));
f( ξ )
f '( ξ ) − f( ξ )
> G(x):=x-f(x)/(diff(f(x),x)-f(x));
G( ξ ) := ξ −
x − 2 sin( x )
1 − 2 cos( x ) − x + 2 sin( x )
x[1]:=evalf(subs(x=1,G(x))); # starting-point
G( x ) := x −
> x[0]:=1;
x0 := 1
x1 := 2.133819712
> for i from 2 to 7 do x[i]:=evalf(subs(x=%,G(x))) od;
x2 := 1.861487604
x3 := 1.895031787
x4 := 1.895494177
x5 := 1.895494267
x6 := 1.895494267
x7 := 1.895494267
>
We see, the 5th iteration x[5] leads already to convergence although the first
derivative f '(x) is very small in the neighbourhood of the selected starting-point.
The extended NEWTON method is most effective in cases of small derivatives f '(x).
This worksheet is concerned with finding numerical solutions of non-linear equations
in a single unknown. A generalization to systems of non-linear equations has been
discussed in more detail, for instance, by BETTEN , J. in: Finite Elemente für Ingenieure 2,
zweite Auflage, 2004, Springer-Verlag, Berlin / Heidelberg / New York.
>
24