1.4 Newton’s Method
Newton’s method is a general procedure that can be applied in many diverse situations. When
specialized to the problem of locating a zero of real-valued function of a real variable, it is
often called Newton-Raphson iteration. In general, Newton’s method is faster than the
bisection method and Fixed-Point iteration since its convergence is quadratic rather than
linear. Once the quadratic becomes effective, that is, the values of Newton’s method sequence
are sufficiently close to the root, the convergence is so rapid that only a few more values are
needed. Unfortunately, the method is not guaranteed always to convergence. Newton’s
method is often combined with other slower method in a hybrid method that is numerically
globally convergence.
Suppose that we have a function f whose zeros are to be determined numerically. Let r be a
zero of f (x) and let x be an approximation to r . If f  exists and is continuous, then by
Taylor’s Theorem,
 
0  f r   f x  h  f x   hf x   o h 2 ,
where h  r  x . If h is small (that is, x is near r ), then it is reasonable to ignore the
 
o h2
term and solve the remaining equation for h . If we do this, the result is
h   f x f x. If x is an approximation to r , then x  f x f x should be r .
Newton’s method begins with an estimate x 0 of r and then defines inductively
xn+1 = xn -
f (xn )
f ¢ ( xn )
( n ³ 0) .
Newton’s Algorithm
x0 = initial guess
xn+1 = xn -
f (xn )
f ¢ ( xn )
for
n = 0,1, 2,
Before examining the theoretical basis for Newton’s method, let’s give a graphical
interpretation of it. From the description already given, we can say that Newton’s method
involves linearizing the function. That is, f was replaced by a linear function. The usual
way of doing this is to replace f by the first two terms in the Taylor series. Thus, if
1
f x   f c   f c x  c  
1
2
f c x  c    .
2!
Then the linearization (at c) produces the linear function
l x  f c  f cx  c .
(1)
Note that l is a good approximation to f in the vicinity of c , and in fact we have
l c   f c  and l c   f c . Thus, the linear function has the same value and the same
slope as fit the point c . So in Newton’s method we are constructing the target line to f at a
point near r , and finding where the target line intersects the x -axis.
Example: Find the Newton’s method for x 3 + x -1 = 0 .
Error Analysis (Quadratic convergence of Newton’s method)
Now we analyze the errors in Newton’s method. By errors, we mean the quantities
en  x n  x . (We are not considering round-off errors)
Let us assume that
f  is continuous and r is a simple zero of
f
so that
f r   0  f r  . From the definition of the Newton iteration, we have
en 1  xn 1  r  xn 
f  xn 
f  xn  en f ( xn )  f ( xn ) en f ( xn )  ( f ( xn )  f (r ))
 r  en 


.
f '  xn 
f '  xn 
f ( xn )
f ( xn )
(2)
By Taylor’s Theorem, we have
( ) ()
( )(
)
( )
1
f x n - f r = f ' x n x n - r + e n2f '' xn ,
2
2
1
1
f (xn ) en2
f '' ( r )
2
2
en+1 »
»
en2 = cen2 .
f ¢ (r)
f ¢ (r)
(3)
Suppose that c  1 and e n  10 4 . Then by Equation (3), we have e n 1  10 8 and
l n 1  10 16 . We are impressed that only a few additional iterations are needed to obtain more
than machine precision.
2
This equation tells us that e n 1 is roughly a constant times e n . This desirable state of
affairs is called quadratic convergence. It accounts for the apparent doubling of precision with
each iteration of Newton’s method in many applications.
Definition (Quadratically convergent): A iterative method is quadratically convergent if
M = lim
n®¥
en+1
e 2n
<¥
We have yet to establish the convergence of the method. By Equation (3), the idea of the
proof is simple: If e n is small and if
1
f  n  f x n  is not too large, then e n 1 will be
2
smaller than e n . Define a quantity c  dependent on  by
c  
1
m a x f x  m i n f  x 
x  r 
2 x  r 
  0 .
(4)
We select small enough  to be sure that the denominator in Equation (4) is positive, and
then if necessary we decrease  so that  c    1 . This is possible because as 
converges to 0, c  converges to
1
f r  f r  , and so c  converges to 0 .
2
Having fixed  , set   c  . Suppose that we start the Newton iteration with a point x 0
satisfying x0  r   . Then e0   and  0  r   . Hence, by the definition of c ,
we have
1
f  0  f  x 0   c  .
2
Therefore, Equation (3) yields
x1  r  e1  e20 c   0e 0e c  0e  c
3
 e0   e0   .
This shows that the next point x1 , also lies within  units of r . Hence, the argument can
be repeated, with the results
e1   e0 ,
e2   e1   2 e0 ,
e3   e2   3 e0 .
In general, we have
e n   n e0 .
Since 0    1 , we have lim  n  0 , and so lim e n  0 .
n
n 
Summarizing, we obtain the following theorem on Newton’s method.
Theorem (Theorem on Newton’s method)
Let f be twice continuously differentiable and f (r) = 0 . If f '(r) ¹ 0 , then Newton’s
method is locally and quadratically convergent to r and satisfies
xn+1 - r £ c xn - r
2
( n ³ 0) .
In some situations Newton’s method can be guaranteed to converge from an arbitrary starting
point. We give one such theorem as a sample.
Theorem (Theorem on Newton’s method for a convex function)
If f belongs to C 2 R  , is increasing, convex and has a zero, then the zero is unique, and
the Newton’s method will converge to it from any starting point.
Proof. Recall that a function f is convex if f x   0 for all x . Since f is increasing,
f   0 on R . By Equation (3), en 1  0 . Thus, x n  r for n  1 . Since f
is
increasing, f x n   f r   0 . Hence, by Equation (2), en 1  en . Thus, the sequences
{en }
the
and
limit
{ xn }
are decreasing and bounded below (by 0 and r , respectively). Therefore,
e* = lim en
n®¥
and
x *  lim x n
n 
exists.
From
Equation
(2),
we
have
e* = e* - f ( x *) f ' ( x *) , where f x *  0 and x*  r .
4
Example: Find an efficient method for computing square roots using Newton’s method.
Theorem above does not say that Newton’s method always converges quagratically. See
following example,
Example: Use Newton’s method to find a root of f (x) = x 2
Theorem (Linear convergence of Newton’s Method)
Assume that the (m +1) -time continuously differentiable function
multiplicity
f on
[ a, b] has
a
m root at r . Then Newton’s Method is locally and linearly convergent to r
Newton’s Method, like Fixed-Point Method, may not converge to a root.
Example: Use Newton’s method to find a root of f (x) = -x 4 +3x 2 + 2 with x0 = 1
5