1
APM2613 TUTORIAL 2:
Solutions of Nonlinear Equations
Reading: Chapter 2 and Chapter 10.2 of textbook; Lesson 2
Other resources:
https://sites.google.com/site/numericalanalysis1burden/
1.1
Objective
To go over (an) example(s) and give exercises to practise on the use of various methods to
solve a non-linear equation.
1.1.1
The methods
1. Bisection method;
2. Fixed point method;
3. Newton’s method and its extensions: Secant method; Regula falsi method;
4. Horner’s method and Muller’s method
5. Newton’s method for systems of two equations in two unknowns.
As these methods are applied, the following will be highlighted where appropriate:
ˆ Key aspects of investigating convergence and error of these methods;
ˆ Key points of other methods used for specific purposes in the study of solutions of
nonlinear equations;
Nature of the problem
The methods are used to solve a nonlinear equation in two forms, namely,
f (x) = 0 referred to as the root (or root-finding) problem
(1)
x = g(x) referred to as thefixed − pointproblem
(2)
and the
The root problem involves rearranging the given equation so that one side of the equation
is 0. Whereas a solution of this problem is the usual x-intercept of the graph of f (x), the
solution of the fixed-point problem is x-value at the intersection of y = x and y = g(x).
It is also worth noting that the use of the terms root or, zero and fixed-point in the
context of these two equations.
1
Example 1
Consider the problem of finding the zeros of the function f (x) = x3 − 9x2 + 12 using the
various methods.
Before we solve this problem, let us analyse the function itself.
First we note that the function is smooth; i.e. it is defined everywhere, continuous and
differentiable. This is typical of polynomial functions.
The function has three roots: one in the interval (−2, 0) another one in (0, 2) and the other
in (4, 10).
The function has two extrema (critical points/turning points): one maximum and one minimum, from which it follows that there is a point of inflection. (It is left as an exercise to
verify these facts)
f(x)=x 3-9x+12
150
100
x=8.8467
50
x=1.2438
f(x)
0
x=-1.0951
-50
-100
-150
-200
-4
-2
0
2
4
6
8
10
x
Figure 1: The function f (x) of the root equation
A quick Octave check for the roots using the code
f=inline(’x.^3-9*x.^2+12’);
c=[1 -9 0 12]; % Coefficients of the polynomial
r=roots(c)
gives the roots as x = −1.0905; 1.2438; 8.8467. Let us then go through the various methods
to find the root in 4 < x < 10.
2
Nflat
1.1.2
Bisection Method
Note that the iterative formula for the bisection method for a root in [a, b] is
xk+1 =
xk − xk−1
2
Let us, for simplicity, use endpoints a and b and new iterate c. so we start with x0 = a = 4
and x1 = b = 10.
A quick check using the Intermediate Value Theorem with f (x) = x3 − 9x2 + 12 yields
f (a) = −68;
f (b) = 112,
from which we see that f (a) · f (b) < 0
which confirms that there is a root between 4 and 10; i.e. f (4) and f (10) have opposite
signs).
First iteration:
The new iterate is c = (a + b)/2 = (4 + 10)/2 = 7. So x2 = c = 7
For this, f (7) = −86 < 0, so f (c) · f (b) < 0 and hence we discard x0 = a = 4 and replace
it with c = 7. The new interval that contains the root is thus 7 < x < 10, and now
a = 7, b = 10.
Second iteration:
The new iterate is c = (7 + 10)/2 = 8.5, and
f (a) · f (c) = (−86)(−24.125) > 0 and f (b) · f (c) = (112)(−24.125) < 0.
We observe that, still the function has the same sign at the left endpoint, a as at c. So we
replace x2 = 7 with c = x3 = 8.5, and repeat the bisection.
If we do this bisection and the intermediate value test we get
i
0
1
2
3
4
5
6
7
8
..
.
a
b
c
f (a)
f (b)
f (c)
4.0000 10.0000 7.0000 −68.0000 112.0000 −86.0000
7.0000 10.0000 8.5000 −86.0000 112.0000 −24.1250
8.5000 10.0000 9.2500 −24.1250 112.0000 33.3906
8.5000 9.2500 8.8750 −24.1250 33.3906
2.1543
8.5000 8.8750 8.6875 −24.1250 2.1543 −11.5852
8.6875 8.8750 8.7812 −11.5852 2.1543
−4.8679
8.7812 8.8750 8.8281 −4.8679
2.1543
−1.3952
8.8281 8.8750 8.8516 −1.3952
2.1543
0.3699
8.8281 8.8516 8.8398 −1.3952
0.3699
−0.5151
..
..
..
..
..
.
.
.
.
.
···
We can observe that the method converges to a value between 8 and 9, the last bracket above
is [8.8281, 8.8398]tobeprecise, at which point the value of f is f (c) = f (8.8398) = 0.5151
in absolute value.
Convergence The nice thing about the Bisection Method is that the repeated application of the intermediate point test after each calculation of the midpoint ensures that
the iterations converge to the root because each time it is bracketed by the endpoints.
Furthermore, the convergence rate of the method is simple and given in Theorem 2.1.
Termination criterion
3
Since the next iterate is obtained simply as the midpoint of the current interval, it is
possible to determine the number of iterations n required to attain a specified tolerance ϵ
by solving the inequality
b−a
< ϵ, b > a
(3)
2n
for n.
Example 2 on p 52 illustrates this calculation.
Note that this calculation does not involve the function f (x).
1.1.3
Fixed-point Iteration
Fixed-point iteration is the core of the discussion of the methods presented in Chapter
2. In simple terms, a fixed-point iteration formula involves rearranging the root equation
f (x) = 0 in various ways to obtain the fixed-point equation x = g(x) associated with an
iteration formula xk+1 = g(xk ). The different ways of rearranging f (x) = 0 lead to different
expressions of g(x) in x = g(x).
Example
For our example, the root equation f (x) = x3 −9x2 +12 = 0 has solutions x = −1.0444; 1.3165; 8.7279;
i.e. can be rearranged in various expressions of x = g(x), to name one,
x=
x3 + 12
,
9x
3
+12
, which is a hyperbola which is asymptotic to x = 0.
where g(x) = x 9x
Graphically, the line y = x crosses the hyperbola at the values of x corresponding to
the roots of f (x). (You can verify this for yourself.) The iterative formula is
xk+1 =
x3 + 12
9x
Various iterations using different initial values x0 lead to different results as shown in the
table:
x0
-2
2
5
8
10
Result
converges to 1.2438
converges to 1.2438
converges to 1.2438
converges to 1.2438
diverges
If you apply the Fixed-point Theorem to check the intervals from which x0 can be chosen
and obtain convergence, you would solve the inequality
|g ′ (x)| < 1
2
8
or − 1 < x + x−2 < 1
9
3
4
x=(x3+12)/(9x)
15
x=8.8467
10
5
g(x)
x=-1.0905
0
x=1.2438
-5
-10
y=x
y=g(x)
-15
-4
-2
0
2
4
6
8
10
x
Figure 2: The graph of a fixed-point equation x = g(x)
with g(x) =
x3 + 12
.
9x
You can think of other iteration formulae x = g(x) and try them using different initial
values.
In a sense, the rest of the methods discussed for solving nonlinear equations entails finding
different expressions of g(x).
1.1.4
Newton’s Method
The Newton’s (also known simply as Newton-Raphson’s) Method is popular for it’s fast
convergence.
It makes use of the function f (x) from the root equation f (x) = 0 and the iteration equation
f (x)
g(x) = x − ′ , leading to the iterative formula
f (x)
xi+1 = xi −
f (xi )
.
f ′ (xi )
So with our example, the root equation is f (x) = x3 − 9x2 + 12 = 0, from which we get
f ′ (x) = 3x2 − 18x. The iterative formula then becomes
xk+1 = xk −
x3 − 9x2 + 12
3x2 − 18x
5
fxpoint
Having determined that there are three roots between −2 and 10, we run the algorithm for
different initial values, x0 and get the following results:
x0 = −2
−2.0000
−1.3333
−1.1162
−1.0909
−1.0905
−1.0905
x0 = 2 x0 = 7 x0 = 10
2.0000 7.0000 10.0000
1.3333 11.0952 9.0667
1.2460 9.5036 8.8571
1.2438 8.9281 8.8467
1.2438 8.8482 8.8467
. . . 8.8467
...
We can see how fast the method converges for the various initial values. Note, in particular
that the convergence is not always monotonic; see the case for x0 = 7.
So Newton’s method is straight forward and normally much faster. We use the example to demonstrate the variants of Newton’s method: the secant method and the regula
falsi methods. For these methods the derivative is approximated by the slope
f ′ (xk ) ≈
f (xk ) − f (x + k − 1)
(x + k − xk−1 )
to get the iterative algorithm
xk+1 = xk − f (xk ) ·
xk − xk−1
f (xk ) − f (xk−1 )
NB: These method requires two initial values, x0 and x1 .
1.1.5
The Secant and Regula Falsi Methods
i
0
1
2
3
4
5
6
7
8
9
10
1.2
Secant
4.0000
10.0000
6.2667
7.9834
10.1136
8.6127
8.7887
8.8500
8.8466
8.8467
8.8467
Regula Falsi
4.0000
10.0000
6.2667
7.9834
10.1136
8.6127
8.7887
8.8500
8.8466
8.8467
8.8467
Methods for Polynomial Functions
Horner’s method and Muller’s method have been presented as useful techniques for handling polynomial functions, with specific mention of possible existence of complex roots for
Muller’s method.
1.2.1
Horner’s Method
See Example 2 on p93 for an illustration of this method.
6
1.2.2
Muller’s Method
An illustration for this method is given on p97.
1.3
Newton’s Method for Multi-variable Functions
Recall the similarities and differences in Newton’s formula in the two cases, using a function
F (x, y) or F (x1 , x2 ) or F (x),
where
x=
x1
x2
While Newton’s method for f (x) = 0 uses the iteration (fixed-point) formula
with ϕ(x) = 1/f ′ (x)
x = g(x) = x − ϕ(x)f (x),
Newton’s method for a function, F (x1 , x2 ) = 0 of two variables,
F1 (x1 , x2 )
F (x1 , x2 ) =
,
F2 (x1 , x2 )
uses
x1
x2
(k+1)
=
x1
x2
(k)
− J −1 (x1 , x2 )F (x1 , x2 )
(4)
where J(x1 , x2 ) is now the Jacobian of F (x1 , x2 ), defined by


∂F1 (x1 , x2 ) ∂F1 (x1 , x2 )


∂x2
1
JF (x1 , x2 ) =  ∂F ∂x

∂F
(x
,
x
)
(x
,
x
)
2 1
2
2 1
2
∂x1
∂x2
which now replaces f ′ (x) in the case of one variable function.
Example 2
Consider the problem of approximating the solution of the nonlinear system:
5x21 = x22
0.25(sin x1 + cos x2 ) = x2
Here
F (x1 , x2 ) =
F1 (x1 , x2 )
F2 (x1 , x2 )
∂F1
∂x2
∂F2
∂x2
10x1
−2x2
−0.25(cos x1 )1 + 0.25(sin x2 )
=
5x21 − x22
x2 − 0.25(sin x1 + cos x2 )
The Jacobian is
∂F1
J(x1 , x2 ) =
∂x1
∂F2
∂x1
=
So that, using an initial guess of (x1 , x2 ) = (0, 0) we have
0
0
4
(0)
(0)
(0)
(0)
F (x1 , x2 ) = F (0, 0) =
J(x1 , x2 ) =
0.2500
−4 −12
7
and the next iterate is
Iteration 1
"
(1)
x1
(1)
x2
#
0
0
0
4
=
−
−4 −12
0.062500
=
0
−1 ∗
0
0.2500
Iteration 2
"
#
−1 (2)
x1
0.062500
−044274
4.0
0.019531
=
−
∗
(2)
0
−3.8750 −12.0
0.234385
x2
0.13560
=
−0.0040737
Carrying out a few more iterations yields
k
0
1
2
3
4
5
6
7
8
9
10
11
12
13
x1
0
0.0625
0.1356
0.2570
0.5430
1.6178
7.2729
9.0181
9.8137
10.3153
10.7114
11.0374
11.3143
11.5550
We can observe that there is some convergence.
8
x2
0
0
−0.0041
−0.0241
−0.0925
−0.3362
−0.6543
0.3750
1.6723
−5.2785
−5.2483
−5.2335
−5.2308
−5.2380