1.2 Fixed-Point Iteration
Consider a sequence of point which is computed by a formula of the form
xn+1 = F ( xn )
( n ³ 0) .
(1)
The algorithm defined by such an equation is called functional iteration. Formula (1) can be
used to generate sequences that do not converge. For example, the sequence 1, 3,9,27,
which arises if x0 = 1 and F ( x ) = 3x . However, our interest lies mainly in those cases
where lim xn = s exists.
n®¥
Suppose that
lim xn = s .
n®¥
What is the relation between s and
(
F ? If F is continuous, then
)
F ( s ) = F lim xn = lim F ( xn ) = lim xn+1 = s
n®¥
n®¥
n®¥
Thus, F ( s ) = s , and we will call s a fixed point of the function
F.
Definition 1.4 (Fixed Point) The real number s is a fixed point of the function
F(s) = s
F if
We could think of a fixed point as a value that the function “locks onto” in the iterative
process.
Based on formula (1), we have the following algorithm
Fixed-Point Iteration (FPI)
x0 = initial guess
For i = 0,1,2,…
xi+1 = F(xi )
end for
1
A mathematical problem often can be reduced to the problem of finding a fixed point of a
function. Very interesting applications occur in differential equations, optimization theory, and
other areas. Usually the function F whose fixed points are sought will be a mapping from
one vector space into another. We intend to analyze the simplest case when F maps closed
set C Í R into itself.
Definition (Contractive Mapping) A mapping (or function)
there exists a number l less than 1 such that
()
F is said to be contractive if
( )
F x - F y £ l | x - y |.
For all points x and y in the domain of
(2)
F.
As illustrates in following figure, the distance between a and b is mapped into a small
distance between F ( a ) and F ( b ) by the contractive function F .
y
F (b)
F (a)
x
a
b
Theorem (Contractive mapping Theorem)
Let C be a closed subset of the real numbers. If F is a contractive mapping of C into
C , then F has a unique fixed point. Moreover, this fixed point is the limit of every sequence
obtained from equation (1) with a starting point x0 ÎC .
Proof We use the contractive property (2) together with equation (1) to write
xn - xn-1 = F ( xn-1 ) - F ( xn- 2 ) £ l xn-1 - xn-2 .
(3)
This argument can be repeated to get
xn - xn-1 £ l xn-1 - xn-2 £ l 2 xn-2 - xn-3 £
£ l n-1 x1 - x0 .
Since xn can be written as the form
xn - x0 = ( x1 - x0 ) + ( x2 - x1 ) +
+ ( xn - xn-1 ) .
2
¥
We see that the sequence {xn } converges if and only if the series
å( x
n =1
n
- xn -1 )
converges. To prove that this series converges, it suffices to prove that the series
¥
å( x
n =1
n
- xn -1 ) converges. This is easy because we can use the comparison test and the
previous work:
¥
¥
n=1
n=1
å xn - xn-1 £ å l n-1 x1 - x0 =
1
x1 - x0 .
1- l
Since the sequence converges, let s = lim xn , then F ( s ) = s , as noted previously. (Observe
n®¥
that the contractive property implies continuity of
point, if x and y are fixed points, then
F ). As for the uniqueness of the fixed
x - y = F ( x ) - F ( y) £ l x - y .
since
l < 1, x - y = 0 . Finally, note that the point s that is obtained belongs to C since
s is the limit of a sequence lying in C .
Example: Show that the sequence {xn } in the contractive Mapping Theorem satisfies the
Cauchy criterion for convergence.
Solution. Recall the Cauchy criterion for the sequence {xn }: Given any q , there exists an
xn - xm < e whenever n, m ³ N . If n ³ m ³ N , then by the
integer N such that
triangle inequality and the equation following (3),
xn - xm £ xn - xn-1 + xn-1 - xn-2 +
+ xn+1 - xn
£ l n-1 x1 - x0 + l n-2 x1 - x0 + +l m x1 - x0
(
+ l n-1- m
(
)= l
£ l m x1 - x0 1 + l + l 2 +
£ l N x1 - x0 1 + l + l 2 +
For any q > 0 , there exists an
increase
N
)
x1 - x0 (1 - l ) .
-1
N such that xn - xm < e whenever n, m ³ N . We just
N until l N x1 - x0 (1 - l ) < e .
-1
3
Example: Prove that the sequence defined recursively as follows is convergent
ì x0 = -15
ï
1
í
ïî xn +1 = 3 - 2 xn
Solution the function F ( x ) = 3 -
( n ³ 0 ).
1
x is a contraction because
2
F ( x ) - F ( y) = 3 -
1
1
1
1
x - 3+ y = y - x £ y - x
2
2
2
2
by the triangle inequality. By Theorem, the sequence described must converge to the unique
fixed point of F , which is readily seen to be. In the theorem, we can take C to be R .
Example: Use the Contractive Mapping Theorem to compute a fixed point of the function
1
F ( x ) = 4 + sin 2x .
3
Solution By the mean-value Theorem, we have
F ( x ) - F ( y) =
1
2
2
sin 2x - sin 2y = cos 2s x - y £ x - y .
3
3
3
for some s between x and y . This shows that
F is contractive, with l =
F has a fixed point.
Theorem 1,
2
.. By
3
Error analysis (Linear convergence of Fixed-Point Iteration)
Let us start with the definition of linear convergence:
Definition (Linear Convergence) Let en denote the error at step n of an iterative method.
If
lim
n®¥
en+1
= S <1
en
the method is said to obey linear convergence with S .
Now let us analyze the errors in the process of functional iteration. We suppose that F has a
fixed point
s
and that a sequence
{xn } has been defined by the formula x n+1 = F ( x n ) . Let
en = xn - s ,
4
If F ¢ exists and is continuous, then by the mean-value theorem,
xn+1 - s = F ( xn ) - F ( s) = F ¢ (xn ) ( xn - s) ,
en+1 = F ¢ (xn ) en
Or
where
xn is a point between xn and s . The condition F ¢ ( x ) < 1 for all x ensures that
the error decrease in magnitude. If en is small, then
xn is near s and F ' (xn ) » F ' ( s) .
One would expect rapid convergence if F ¢ ( s ) is small. An ideal situation would be
F ¢ ( s) = 0 .
F is
Theorem 1.6 (Linear Convergence of Fixed-Point iteration) Assume that
continuously differentiable, that F(s) = s , and that S =| F '(s) |< 1 . Then Fixed-Point
Iteration converges linearly with rate S to the fixed point
close to s .
s for initial guesses sufficiently
Definition 1.7 (Locally Convergent) An iterative method is called locally convergent to
method converges to s for initial guesses sufficiently close to s .
Example: Explain why the Fixed-Point Iteration
s
if the
F(x) = cos x converges.
Example: Find the fixed points of F(x) = 2.8x - x 2 .
Example: Calculate
2 by using FPI.
5