1.5 Root-Finding without derivatives

Secant Method
Recall that the Newton iteration is defined by the equation:
xn 1  xn 
f ( xn )
.
f   xn 
(1)
One of the drawbacks of Newton’s method is that it involves the derivative of the function whose
zero is sought. To overcome this disadvantage, a number of methods have been proposed. For
example, Steffen’s iteration
 f  xn  
xn 1  xn 
f  xn  f  xn    f  xn 
2
gives one approach to this problem. Another is to replace f   x  in Equation (1) by a difference
quotient, such as
f   xn  
f  xn   f  xn 1 
.
xn  xn 1
(2)
The approximation given in (2) comes directly from the definition of f  as a limit, namely
f   x   lim
ux
f  x   f u 
.
x u
When this replacement is made, the resulting algorithm is called Secant method. It formula is


xn  xn 1
xn 1  xn  f  xn  

 f  xn   f  xn 1  
 n  1 .
Since the calculation of xn 1 requires xn and xn 1 , two initial points must be prescribed at the
beginning.
Secant Method
x0, x1 = initial guesses
xn+1 = xn -
f (xn )(xn - xn-1 )
for n = 1, 2,…
f (xn ) - f (xn-1 )
1
Example: Apply the Secant Method to find the root of f (x) = x 3 + x -1
From the definition of the secant method we have, with en  xn  r ,
en 1  xn 1  r  en  f n

xn  xn 1 xn  xn 1 en ( f n  f n 1 )

(
 fn )
f n  f n 1 f n  f n 1
xn  xn 1
xn  xn 1 f n  f n 1 f n
x x
f  f n 1 ( xn  xn 1 ) f n
(
 )en  n n 1 ( n

)en
f n  f n 1 xn  xn 1 en
f n  f n 1 xn  xn 1
xn  xn 1 en
( f n / en 1  f n 1 / en 1 
xn  xn 1
f n / en )
en 1

xn  xn 1
f n  f n 1

xn  xn 1 f n / en  f n 1 / en 1
en en 1 ,
f n  f n 1
xn  xn 1
xn  xn 1
en en 1
(4)
f n  f ( xn )  f ( xn  r  r )  f (en  r )  f (r )  en f '(r ) 
f n 1  f ( xn 1 )  f (en 1  r )  f (r )  en 1 f '(r ) 
en 2
f ''(r )  o(en 3 ) ,
2
en21 f ''(r )
 o(en31 ) .
2
Since en  en 1  xn  xn 1 , we arrive at the equation
f n / en  f n 1 / en 1 1
 f ''(r ) .
xn  xn 1
2
The first bracketed expression can be written as
xn  xn 1
1

.
f n  f n 1 f '(r )
Hence, we have shown that
en 1 
f ''(r )
en en 1  cen en 1 .
2 f '(r )
(5)
This equation is similar to the one encountered in the analysis of Newton’s method. To discover
the order of convergence of the secant method, we postulate the following asymptotic relationship:
2
|e n+1 | A |e n |a ,
(6)

where A is a positive constant. This means that ration | en 1 | /( A | en | ) tends to 1 as
and implies
n 
 -order convergence. Hence
|e n | A |e n-1 |a ,
and
|e n-1 | (A -1 |e n |)1/a .
(7)
In equation (5), we substitute the asymptotic values of | en 1 | and | en 1 | from equations (6)
and (7) .The result is
A |e n |a |c ||e n | A -1/a |e n |1/a .
This can be written as
A 1+1/a |c |-1 |e n |1-a +1/a .
(8)
Since the left side of this relation is a nonzero constant while en  0 , we conclude that
1    1/   0 ,
  (1  5) / 2  1.62 ,
taking the positive root. Hence, the secant method’s rate of convergence is super-linear (that is,
better than linear). Now we can determine A since the right had side of relation (8) is 1. Thus,
using the equation 1  1/    , we have
A | c |1/(11/ ) | c |1/ | c | 1 | c |0.62 |
f ''(r ) 0.62
| .
2 f '(r )
With A as just given, we finally have for the secant method
| en1 | A | en |(1
5 )/2
.
Since (1  5) / 2  1.62  2 , the rapidity of convergence of the secant method is not as
good as Newton’s method but is better than the bisection method. However, each step of the
secant method requires only one new function evaluation, whereas each step of the Newton
method requires two function evaluation: f(x) and f’(x). Since function evaluations constitute the
principal burden in these algorithms, a pair of steps in the secant method is comparable to one step
in the Newton method. For two steps of the secant method, we have
| en 2 |~ A | en1 | ~ A1 | en |  A1 | en |(3
2
5 ) /2
.
This is considerably better than the quadratic convergence of the Newton method since
(3  5) / 2  2.62 . Of course, two steps of the secant method would require more work per
iteration.
3

Method of False Position
Method of False Position, or Regula Falsi, is similar to the Bisection method, but where the
midpoint is replaced by a Secent Method-like approximation. Given an interval
[ a, b]
that
brackets a root, define the next point
c=
bf (a) - af (b)
f (a) - f (b)
as in the Secant Method.

Muller’s Method
Instead of intersecting the line through two previous points with the x-axis, Muller’s Method uses
three previous points to draw a parabola
y = p(x) through them, and intersect the parabola with
the x-axis.

Inverse Quadratic Interpolation
IQI is similar with Muller’s Method. However, the parabola is of the form
x = p(y) instead of
y = p(x) .
 Brent’s Method
Brent’s Method is a hybrid method. Roughly speaking, the Inverse Quadratic Interpolation method
is attempted, and the results is updated if (1)
f (xn ) gets smaller and (2) the bracketing interval
is cut at least half. If not, the Secant Method is attempted with the same goal. If it fails as well, a
Bisection Method step is taken , guaranteeing that the uncertainty is cut at least in half.
4