Iteration Methods
“Mini-Lecture” on a method to solve problems by iteration
• Ch. 4: (Nonlinear Oscillations & Chaos). Some
nonlinear problems are solved by iteration (Sect. 4.7
& also Homework Ch. 4!).
• Direct (linear) iteration might or might not be
efficient! In some cases it might not even converge,
especially if you make a poor first guess!
• A method which is often more efficient (has
convergence in fewer iterations) than direct iteration
is
NEWTON’S METHOD
Linear Iteration
• A typical problem (as in Sect. 4.7 or in Ch. 4
homework) may result in having to solve (find the
roots x) of a nonlinear algebraic equation:
f(x) = 0
(1)
where f(x) = some nonlinear function of x
• Linear (direct) Iteration Method
– First, rewrite f(x) = 0 in the form: x = g(x) (2)
– Often there is no one, unique way to do this!
However, one can usually do this. (2) is equivalent
to (1)! Requiring this equivalence defines g(x).
– Goal: Find x’s which satisfy (2)
f(x) = 0 (1)

x = g(x)
(2)
• Choose an initial approximation for x which
(approximately) satisfies (2). Call this x0
– A judicious choice of x0 (first guess) is needed or the
following scheme might not converge!
– How to choose x0? As an educated guess, DRAW
A GRAPH to get an approximate root, ...
– Put x0 in the right side of (2) to generate a new
approximation  x1.
That is: x1 = g(x0)
– Repeat to generate a new approximation  x2
That is:
x2 = g(x1)
f(x) = 0 (1) 
x = g(x)
(2)
• At the nth iteration, we have: xn+1 = g(xn)
• Goal: Want the ratio |xn+1 - xn|/|xn+1| < ε
where ε = some specified small number
• Mathematicians have developed criteria (& proved
theorems!) for
– When this linear iteration process will converge
– How large the error is on the nth iteration
– etc., etc.
– We aren’t concerned with these here! See any numerical
analysis book! Take Physics 4301!
Newton’s Method
• Even if the linear iteration method converges,
depending on the function f(x) (or g(x)) it might
converge very slowly & thus be very inefficient!
• A better method (more efficient, usually faster
convergence!) is
NEWTON’S METHOD
• We still want to solve f(x) = 0 (1)
• We still do this by first rewriting (1) as
x = g(x)
(2)
• Newton’s Method really just makes a very special &
judicious choice for the (up to now arbitrary) function
g(x)! (Back to this point soon!)
• We want to solve:
f(x) = 0
(1)
• Suppose we have very good insight & can make a
very good initial approximation (first guess) for x
satisfying (1)  x0
• Even though x0 is a good first guess, it still won’t be
an exact zero of (1)!
 Let x be the “Exact Root” of (1) & do a
Taylor’s series expansion of (1) about the first
guess x0! Assume that x0 is near enough to the true
root x that we can stop at the linear term.
• That is: f(x)  f(x0) + (x - x0)(df/dx)0 +..
(3)
– Could take the expansion to higher orders if
desired. Usually this is not necessary!
• We still want to solve:
• (1) & (3) together

• Solve (4) for x:
f(x) = 0
(1)
f(x0) + (x - x0)(df/dx)0  0 (4)
x = x0 - f(x0)/(df/dx)0
(5)
– Assumption! (df/dx)0  0
• Use (5) to iterate & get a new approximation
for x  x1 as:
x1 = x0 - f(x0)/(df/dx)0
• So we have:
1st iteration:
x1 = x0 - f(x0)/(df/dx)0
2nd iteration: x2 = x1 - f(x1)/(df/dx)1
– etc., etc. for 3rd, 4th, ….
• At the nth iteration, we have:
xn+1 = xn - f(xn)/(df/dx)n
• Note: This is of the form
x = g(x)
(2)
with a special choice of the function g(x) being
g(x)  x - f(x)/(df/dx)
• Newton’s Method of iteration:
xn+1 = xn - f(xn)/(df/dx)n
• Goal (as in linear iteration):
Want the ratio |xn+1 - xn|/|xn+1| < ε
where ε = some specified small number
• Mathematicians have developed criteria (& proved theorems!)
for
– When this iteration process will converge (usually faster
than linear iteration!)
– How large the error is on the nth iteration (assuming a good 1st
guess, usually faster than linear iteration!)
– etc., etc.
– We aren’t concerned with these here! See any numerical analysis book!
Iteration Example
Hand held calculator!
• Ch. 4, Prob. # 5 a: Find the root (4 significant figures) of:
x + x2 + 1 = tan(x), [0  x  (½)π]. First, ALWAYS make a
rough graph 
The root we want
is where the curve
x + x2 + 1 crosses
the curve tan(x)!
From the graph, a
reasonable first
guess is x0  (3π/8)
Try Linear Iteration!
x + x2 + 1 = tan(x), [0  x  (½)π].
x0 = (3π/8) = 1.1781
(radians!)
• Write: x = tan-1[x + x2 + 1]. This is in the form: (the
Choice of g(x) not unique!): x = g(x) = tan-1[x + x2 + 1]
• The iteration procedure (& results):
xn+1 = g(xn)
x1= g(x0) = tan-1[x0 + x02 + 1] = 1.2974
x2 = g(x1) = tan-1[x1 + x12 + 1] = 1.3247
x3 = g(x2) = tan-1[x2 + x22 + 1] = 1.3304
x4 = g(x3) = tan-1[x3 + x32 + 1] = 1.3316
x5 = g(x4) = tan-1[x4 + x42 + 1] = 1.3318
x6 = g(x5) = tan-1[x5 + x52 + 1] = 1.3319

x = 1.3319
Try Newton’s Method!
x + x2 + 1 = tan(x), [0  x  (½)π].
x0 = (3π/8) = 1.1781
(radians!)
• Write: tan(x) - (x + x2 + 1) = 0. This is in the form:
f(x) = tan(x) - (x + x2 + 1) = 0
 (df/dx) = [cos(x)]-2 - (1 +2x)
• Iteration procedure (& results): xn+1 = xn - f(xn)/(df/dx)n
x1 = x0 - f(x0)/(df/dx)0 = 1.5098, x2= x1 - f(x1)/(df/dx)1 = 1.4646
x3 = x2 - f(x2)/(df/dx)2 = 1.4085, x4= x3 - f(x3)/(df/dx)3 = 1.3588
x5 = x4 - f(x4)/(df/dx)4 = 1.3354, x6= x5 - f(x5)/(df/dx)5 = 1.3320
x7 = x6 - f(x6)/(df/dx)6 = 1.3319
 x = 1.3319
In this case, Newton’s Method takes more iterations to
converge than linear iteration!!