Using Iterative Methods to Solve Equations
Newton’s method is a iterative scheme for solving equations by finding zeros of a function. The
f  xn 
f  x
Newton iteration xn 1  xn 
finds the zeros of f by finding the fixed point of g  x   x 
.
f  x
f   xn 
Recall that a fixed point of a function is one for which the input value is equal to the output value, that is,
 is a fixed point of function g if g     . In Newton’s method, notice that for any value of x for
which f  x   0 , we have xn 1  xn 
0
and xn1  xn . Since the input and output are the same, we
f   xn 
f  x
.
f  x
There are many other iterative methods that one can use to solve equations. Let’s take a very
simple example. Suppose we want to solve the equation x 2  5 using iteration. There are many ways we
can use algebra to write equivalent equations and then use iteration to approximate a solution. By
approximating a solution to x 2  5 , we are really finding a decimal approximation to 5 .
have found the fixed point of g  x   x 
For example, if we want to find the solution to x 2  5 , we can rewrite the equation as
5  3x  x 2
. Algebraically, the solution to the equation
3
5  3x  x 2
. Geometrically, the graph of y  x 2 intersects the
x 2  5 is the same as the solution to x 
3
x2  3x  5  3x . “Solving for x”, we have x 
graph of y  5 at the same x-coordinate as the graph of y  x is crossed by y 
this form of the equation to create an iterative equation, xn1 
5  3x  x 2
.
3
We can use
5  3xn  xn2
, whose fixed points the
3
solution we seek. If we begin with x0  1.5 and iterate, the values will converge to
Spiraling in on xn  5
5.
Solutions to x 
5  x  x2
and x 2  5  0 are the same
3
If, instead, we had rewritten x 2  5 as x2  x  5  x , so  x  5  x  x2 and x  5  x  x2 ,
both the algebraic and geometric interpretations remain the same, but the iteration xn  5  xn  xn2 does
not converge to the fixed point.
Or, suppose we decide that x 2  5 implies that x3  x 2  x3  5 an so x 3  5  x 2  x 3 . This means
that x  3 5  x3  x2 has the same solution as x 2  5 and the iterative equation xn1  3 5  xn3  xn2 has
5 . If we start with x0  1.5 , will the iteration converge to the solution we want. Or, if
a fixed point at
we say x2  x  5  x , so x  x  1  5  x so x 
5 x
, will iterating this function find our solution?
x 1
The central question is, “Under what conditions will the iterations converge to the desired solution?”
How could we know if our revision of the equation will work or not?
1.
For each of the iterations below, show that x  5 is a solution to the equation and a fixed point
for the iteration. Which will give an iterative approximation for the square root of 5 if we start with
x0  1.5 ?
5 x
.
x 1
a)
x2  x  5  x , so x  x  1  5  x so x 
b)
x  x  1  5  x so x 
c)
x 2  5 , so x 
d)
Make up your own algebraic revision x 2  5 and turn it into an iterative equation.
5 x
1.
x
5
.
x
Another way to create an iterative equation with the desired fixed point is to write an equation that will
give you x  x when the desired solution is substituted for x.


2
For example, x  x  x  5 which is another way to generate xn 1  xn  xn2  5 . Does this converge
when x0  1.5 ?
2.
For each of the iterations below, show that x  5 is a solution to the equation and a fixed point for
the iteration. Which will give an iterative approximation for the square root of 5 if we start with x0  1.5 ?
a)
x2  5
x  x
x
b)
x  x
c)
x  x  x3  x 2  5
x2  5
2x
d)
Make up your own algebraic revision x  x  f  x   x 2  5  and turn it into an iterative equation.
Which of the iterations defined above converge to xn  2.2361 ? For an iteration x  g  x  , look at the
graphs of y  x and y  g  x  around x  5 . What do they have in common? What characteristics
must f have for the iteration to succeed in finding x  5 ?
There are two important theorems which rely on basic calculus governing this iterative process. The two
theorems are known as The Fixed Point Theorem and the Attracting Fixed Point Theorem.
Theorems About Fixed Points
Definitions: Any point  which satisfies G     is a fixed point for the iterative equation
xn1  G  xn  .
Fixed Point Theorem: If the continuous function G maps values from the interval [a, b] onto itself,
G :  a, b   a, b , then there must be a fixed point for G in  a, b . This means the graph of y  G  x  must
intersect the graph of y  x at least once in the interval  a, b . Use the diagram below to see why this is
true.
1.
Draw the line y  x ..
2. Draw in the square representing the
domain and range G :  a, b   a, b
3.
In what range must the value of
G  a  fall?
4.
In what range must the value of
G  b  fall?
5.
Explain why you can’t go from
G  a  to G  b  continuously without
crossing y  x .
Formal Proof of the Fixed Point Theorem:
Theorem: If G is continuous on  a, b  and for all x  a, b , G  x    a, b , then there is a fixed point for
G in  a, b  .
Proof:
Define H  x   G  x   x , and we see that H is continuous also. Moreover, H  a   G  a   a
and H  b   G  b   b . Consider the graph you drew above. We know that H  a   0 and H  b   0 .
Since H is continuous, by the Intermediate Value Theorem, there must be at least one point c   a, b at
which H  c   0 or, equivalently, c  G  c  . Then x  c is the fixed point guaranteed by the theorem.
Attracting Fixed Points
Under what conditions will iterations converge to a fixed point? If a function is differentiable, we
can answer this question. If a function is differentiable, then it is locally linear, that is, the graph of the
function looks like a like over some small interval. So, there are only 4 ways a function can cross the line
y  x (in red). If the slope is positive, it can move from above to below, as shown at left in the figures
below. Or, it can move from below to above, as shown at right.
If the slope is negative, it must cross from above to below, but it can cross less or more steeply than a
perpendicular to the line.
For each figure above, begin with x0  2.8 and follow several iterations from the graph of the function
(blue) to y  x (red) and back. In which figures do the iterations move towards or away from the fixed
point? Can you see why in one case they move towards (attracting) and the other away (repelling)?
Definition: Suppose  is a fixed point for G. Then  is an attracting fixed point if G    1 and a
repelling fixed point if G    1 . If G    1 , then  is called neutral or indifferent. Go back to the
iterations in the first section and look at the graphs close to the intersection at x  5 . Do they follow
the pattern you notice here?
Attracting Fixed Point Theorem: Suppose  is an attracting fixed point for the continuously
differentiable map G. Then there is an interval I that contains  in its interior and in which, if x  I , then
the nth composition, G  n   x  , has the following properties: G  n   x   I for all n, and G  n   x    as
n  .
What you should have noticed is the iterations approach the fixed point if the slope of the function G in
xn1  G  xn  is not too steep. This theorem gives a boundary on that steepness. The slope must smaller
than 1 in absolute value.
Proof: Since we are given that  is an attracting fixed point we know from the definition that G    1
. Then there exists a value  so that G      1. We can always fit a value between G   and 1.
Since G    1 , there is some interval    ,     in which, for any x  I , G  x   I (see diagram at
right).
Now, let p     ,     be any point in I. By the Mean Value Theorem (recall that G is continuously
differentiable),
G  p   G  
 G  some c    and G  p   G     p   . Since   G   , we
p 
have G  p      p   . This means that G  p  is closer to  than p was. Also, since G  p   I , we
can repeat the process and get the same result.
Taking G  p  as our initial value, we have
G  G  p    G  G      G  p   G   .
But, recall that  is a fixed point, so G     .
This means that G  G  p       G  p      2 p   . Again, we have G  G  p    I and repeating
the process gives the inequality G 
n
 p   n
p   . Each iteration is closer to the fixed point and,
moreover, since  n  0 as n   , the iterates converge to the fixed point.
The big theorem is that the an iterative equation will converge to the fixed point if the slope of the
function is less than 1 in absolute value at the fixed point and the initial value is in the basin of attraction.
3.
Prove that Newton’s Method, xn 1  xn 
f  xn 
, will converge when the initial condition is in
f   xn 
the interval of convergence. Note that G  x   x 
f  x
.
f  x
4.
The equation 3x3  5 x 2  4 x  4  0 has a solution near x  0.7 . Show how each of the following
iterative equations was derived from the equation above and determine (without actually doing the
iteration) if they will converge to the solution.
a)
5 4
4
xn 1  
 2
3 3 xn 3 xn
c)
xn 1 
3xn3  4 xn  4
5
Which will converge to the solution?
b)
xn 1  1 
d)
xn 1 
3
3xn3  5 xn2
4
5 xn2  4 xn  4
3