# Newton’s Method 6.006 Intro to Algorithms Recitation 12 October 26, 2011

## Newton’s Method

Find root of f ( x ) = 0 through successive approximation e.g., f ( x ) = x 2 − a x i x i+1 y = f(x)

Figure 1 : Newton’s Method.

Tangent at ( x i

, f ( x i

)) is line y = f ( x i

) + f

0

( x i

) · ( x − x i

) where f

0

( x i

) is the derivative.

x i +1

= intercept on x-axis x i +1

= x i

− f ( x i

) f 0 ( x i

)

### Square Roots

We want to find the square root of a number a through successive approximation using

Newton’s Method, so that our answer x will converge on a

1 / 2

. It looks like f ( x ) = a

1 / 2 − x has a root at x = a 1 / 2

. Let’s try to use it with Newton’s Method.

f ( x ) = a

1 / 2 − x f

0

( x ) = − 1

χ i +1

= χ i

− f ( χ i

) f 0 ( χ i

) a

1 / 2 − χ

= χ i

= χ i

+ a

1 / 2

− 1

− χ i

= a

1 / 2

The choice of this function was unsuccessful, since we’re unable to compute x i +1 using x i

Let’s try squaring both sides of x = a 1 / 2 . This time, it looks like we get something usable:

.

1

6.006 Intro to Algorithms Recitation 12 October 26, 2011 f ( x ) = x

2

− a

χ i +1

= χ i

( χ i

2 − a )

2 χ i

= a

χ i

+

χ i

2

### Error Analysis of Newton’s Method

Suppose X n

=

√ a · (1 + n

) n may be + or -

Then,

X n +1

=

=

=

=

=

X n

+ a/X n

2 a (1 + n

) + √ a a (1+ n

)

2 p

( a )

(1 + n

) +

1

(1+ n

) p p

(

( a a

)

)

2

2 + 2 n

+ n

2

2(1 + n

2 n

)

1 +

2(1 + n

)

Therefore, n +1

= n

2

2(1 + n

)

Quadratic convergence, as ] correct digits doubles each step.

### Complexity of Computing Square Roots

We apply a first level of Newton’s method to solve f ( x ) = x 2 − a . Each iteration of this first level requires a division. If we set the precision to d digits right from the beginning, then convergence at the first level will require lg d iterations. This means the complexity of computing a square root will be Θ( d α lg d ) if the complexity of multiplication is Θ( d α ) , given that we have shown that the complexity of division is the same as the complexity of multiplication.

However, we can do better, if we recognize that the number of digits of precision we need at beginning of the first level of Newton’s method starts out small and then grows. If the complexity of a d -digit division is Θ( d α ) , then a similar summation to the one above tells us that the complexity of computing square roots is Θ( d α ) .

2

6.006 Intro to Algorithms Recitation 12 October 26, 2011

### Termination

Iteration: χ i +1

= b

χ i

+ b a/χ i c c

2

Do floors hurt? Does program terminate? ( α and β are the fractional parts below.)

Iteration is

χ i +1

=

=

χ i

+

χ i

+ a

χ i

2 a

χ i

2

− α

− β

− γ where γ =

α

2

+ β and 0 ≤ γ < 1

Since a + b

√ ab,

χ i

+ a

χ i ≥

√ a , so subtracting γ always leaves us

2 2 won’t stay stuck above if i

< 1 (good initial guess).

≥ b

√ a c . This

### Cube Roots

Now that we’ve seen square roots, let’s calculate cube roots. This time, x will converge on a

1 / 3

, instead. We can use f ( x ) = a − x

3

: f ( x ) = x

3

− a f

0

( x ) = 3 x

2

χ i +1

= χ i

− f ( χ i

) f 0 ( χ i

)

= χ i

1

− χ

3 i

2

= χ

3 i

+

1 a

3 χ 2 i a

χ 2 i

3

6.006 Intro to Algorithms Recitation 12 October 26, 2011

Suppose x i +1

Then,

= a 1 / 3 · (1 + i

) i may be + or -

χ i +1

2

= χ

3 i

+

1 a

3 χ 2 i

2

= a

3

1 / 3

(1 + i

) +

1 a

3 ( a 1 / 3 (1 + n

)) 2

=

=

=

=

= a 1 / 3

3

2(1 +

1 i

) +

(1 + i

) 2 a

1 / 3

1

2(1 + 3

3 (1 + i

) 2 a 1 / 3

3

3 + 6 i

+ 6

1 + 2 i

2 i

+ 2

+ 2 i a 1 / 3

3 +

3 2 i

+ 2 3 i

3 1 + 2 i

+ 2 i

3 i a

1 / 3

1 +

2 i

+

2 3

3 i

(1 + i

) 2 i

+ 3

2 i

+

3 i

) + 1

Notice that the cubic term gets dominated by the quadratic term in the numerator, and that the denominator is very close to 1. Therefore, we have quadratic convergence here, just like in the square root case. The running time analysis is very similar to the square root case as well: the only difference is one additional multiplication in the denominator, so the running time stays the same as the running time of a multiplication.

4

MIT OpenCourseWare http://ocw.mit.edu

### 6.006 Introduction to Algorithms

Fall 201 1

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms .