6.006 Intro to Algorithms Recitation 12 October 26, 2011
Find root of f ( x ) = 0 through successive approximation e.g., f ( x ) = x 2 − a x i x i+1 y = f(x)
Figure 1 : Newton’s Method.
Tangent at ( x i
, f ( x i
)) is line y = f ( x i
) + f
0
( x i
) · ( x − x i
) where f
0
( x i
) is the derivative.
x i +1
= intercept on x-axis x i +1
= x i
− f ( x i
) f 0 ( x i
)
We want to find the square root of a number a through successive approximation using
Newton’s Method, so that our answer x will converge on a
1 / 2
. It looks like f ( x ) = a
1 / 2 − x has a root at x = a 1 / 2
. Let’s try to use it with Newton’s Method.
f ( x ) = a
1 / 2 − x f
0
( x ) = − 1
χ i +1
= χ i
− f ( χ i
) f 0 ( χ i
) a
1 / 2 − χ
= χ i
−
= χ i
+ a
1 / 2
− 1
− χ i
= a
1 / 2
The choice of this function was unsuccessful, since we’re unable to compute x i +1 using x i
Let’s try squaring both sides of x = a 1 / 2 . This time, it looks like we get something usable:
.
1

6.006 Intro to Algorithms Recitation 12 October 26, 2011 f ( x ) = x
2
− a
χ i +1
= χ i
−
( χ i
2 − a )
2 χ i
= a
χ i
+
χ i
2
Suppose X n
=
√ a · (1 + n
) n may be + or -
Then,
X n +1
=
=
=
=
=
X n
+ a/X n
√
2 a (1 + n
) + √ a a (1+ n
)
2 p
( a )
(1 + n
) +
1
(1+ n
) p p
(
( a a
)
)
2
2 + 2 n
+ n
2
2(1 + n
2 n
)
1 +
2(1 + n
)
Therefore, n +1
= n
2
2(1 + n
)
Quadratic convergence, as ] correct digits doubles each step.
We apply a first level of Newton’s method to solve f ( x ) = x 2 − a . Each iteration of this first level requires a division. If we set the precision to d digits right from the beginning, then convergence at the first level will require lg d iterations. This means the complexity of computing a square root will be Θ( d α lg d ) if the complexity of multiplication is Θ( d α ) , given that we have shown that the complexity of division is the same as the complexity of multiplication.
However, we can do better, if we recognize that the number of digits of precision we need at beginning of the first level of Newton’s method starts out small and then grows. If the complexity of a d -digit division is Θ( d α ) , then a similar summation to the one above tells us that the complexity of computing square roots is Θ( d α ) .
2

6.006 Intro to Algorithms Recitation 12 October 26, 2011
Iteration: χ i +1
= b
χ i
+ b a/χ i c c
2
Do floors hurt? Does program terminate? ( α and β are the fractional parts below.)
Iteration is
χ i +1
=
=
χ i
+
χ i
+ a
χ i
2 a
χ i
2
− α
− β
− γ where γ =
α
2
+ β and 0 ≤ γ < 1
Since a + b
≥
√ ab,
χ i
+ a
χ i ≥
√ a , so subtracting γ always leaves us
2 2 won’t stay stuck above if i
< 1 (good initial guess).
≥ b
√ a c . This
Now that we’ve seen square roots, let’s calculate cube roots. This time, x will converge on a
1 / 3
, instead. We can use f ( x ) = a − x
3
: f ( x ) = x
3
− a f
0
( x ) = 3 x
2
χ i +1
= χ i
− f ( χ i
) f 0 ( χ i
)
= χ i
1
− χ
3 i
−
2
= χ
3 i
+
1 a
3 χ 2 i a
χ 2 i
3

6.006 Intro to Algorithms Recitation 12 October 26, 2011
Suppose x i +1
Then,
= a 1 / 3 · (1 + i
) i may be + or -
χ i +1
2
= χ
3 i
+
1 a
3 χ 2 i
2
= a
3
1 / 3
(1 + i
) +
1 a
3 ( a 1 / 3 (1 + n
)) 2
=
=
=
=
= a 1 / 3
3
2(1 +
1 i
) +
(1 + i
) 2 a
1 / 3
1
2(1 + 3
3 (1 + i
) 2 a 1 / 3
3
3 + 6 i
+ 6
1 + 2 i
2 i
+ 2
+ 2 i a 1 / 3
3 +
3 2 i
+ 2 3 i
3 1 + 2 i
+ 2 i
3 i a
1 / 3
1 +
2 i
+
2 3
3 i
(1 + i
) 2 i
+ 3
2 i
+
3 i
) + 1
Notice that the cubic term gets dominated by the quadratic term in the numerator, and that the denominator is very close to 1. Therefore, we have quadratic convergence here, just like in the square root case. The running time analysis is very similar to the square root case as well: the only difference is one additional multiplication in the denominator, so the running time stays the same as the running time of a multiplication.
4

MIT OpenCourseWare http://ocw.mit.edu
Fall 201 1
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .