Newton’s Method
x
k +1
f (xk )
=x −
f '(x k )
k
Pros:
In general, Netwon’s method has quadratic convergence: the error is squared at each
iteration (the number of accurate digits doubles).
Cons:
1.
Newton's method requires calculating the derivative. In many cases, the function is
given by a complex formula and an analytical expression for the derivative may not
be easy to calculate. Solution: We can use Secant method, which approximates the
derivative by using the slope of a line through two points on the function.
2.
If the initial value is too far from the true zero, Newton's method may fail to
converge (has only local convergence). Solutions: Practical implementation of
Newton's method should put an upper limit on the size of the iterates. Instead, we can
use bisection to obtain a better estimate for the zero to use as an initial point.
Example: For some functions and some
starting points, it may enter an infinite cycle,
preventing convergence. Let
f ( x) = x 3 − 2 x + 2 and take 0 as the starting
point. The first iteration produces 1 and the
second iteration returns back to 0, so the
sequence will oscillate between the two
without converging to a root.
3.
If the function is not continuously differentiable in a neighborhood of the root, it is
possible that Newton's method will always diverge or fail. Solution: Try another
initial point.
Example: Newton's method diverges for the cube root, which is continuous and
infinitely differentiable, except for x = 0, where its derivative is undefined:
f (x k )
f ( x) = 3 x → x k +1 = x k −
= −2 x k
k
f '(x )
4.
Newton's method will fail in cases where the derivative is zero. When the derivative
is close to zero, the tangent line is nearly horizontal and hence may overshoot the
desired root (numerical difficulties). Solution: Try another initial point.
5.
In some cases the iterates converge, but do not converge quadratically. Solution: Use
simpler methods, like bisection, which in that case converge as quickly (linearly) as
Newton's method.
Example: If the first derivative is zero at the root, then convergence will not be
quadratic:
f (x k ) x k
=
f ( x) = x 2 → x k +1 = x k −
2
f '(x k )