'In104
Nonlinear Algebraic Equations
$
Nonlinear Algebraic Equations
Let f = f (x) be a nonlinear function
6f
-x
then the problem of nding x = x∗ such that
f (x∗) = 0,
is referred to as a nonlinear algebraic equation.
&
1
%
'In104
Nonlinear Algebraic Equations
$
The linear case
( ) = αx + β,
where α and β are constants, is very simple:
f x
6f
( ) = 0,
implies that
f x∗
-x
r
−β/α
= −β/α.
The case of a second order polynomial is also
quite simple. Given
f (x) = αx + βx + γ,
for constants α, β and γ , we have that
x∗
2
6f
implies
( ) = 0,
f x∗
x∗
&
=
r
−β ±
2
q
r
4
β 2 − αγ
2α
-x
.
%
'In104
Nonlinear Algebraic Equations
$
For more complicated functions, it may be
very hard to derive analytical formulas for
the solution. Also, it may be hard to know
whether there is a point x∗ such that
f (x∗) = 0
6
-
&
3
%
'In104
Nonlinear Algebraic Equations
$
Basic observation
If f is a continuous function and there are
two points a < b such that
f (a) · f (b) < 0,
then there is a point x∗ ∈ (a, b) such that
f (x∗) = 0
6f
s
a
t
x∗
b
-x
s
&
4
%
'In104
Nonlinear Algebraic Equations
$
How can we compute x∗ numerically?
&
•
The bisection method
•
The secant method
•
Newton's method
5
%
'In104
Nonlinear Algebraic Equations
$
The Bisection Method
Given a smooth function f = f (x) and two
values a < b such that
f (a) · f (b) < 0.
We assume that there is only one point x∗ in
(a, b) such that
f (x∗) = 0
Dene
1
c = (a + b).
2
6f
s
a
cs
s
b
-x
s
If f (a) · f (c) < 0 then put b = c.
If f (a) · f (c) > 0 then put a = c.
6
&
%
'In104
Nonlinear Algebraic Equations
$
Repeat the process until |f (c)| is suciently
small.
Algorithm
Given a, b such that f (a) · f (b) < 0 and
given a tolerance .
Dene c = (a + b).
1
2
while |f (c)| > do
if f (a) · f (c) < 0 then b = c
else a = c
c
end
&
= (a + b)
1
2
7
%
'In104
$
Nonlinear Algebraic Equations
Example
( ) = 2 + x − ex,
f x
n
1
2
3
4
5.
.
10.
.
15
&
c
1.5000
0.7500
1.1250
1.3125
1.2188
..
1.1455
..
1.1461
a
()
−0.9817
0.6330
0.0448
−0.4030
−0.1642
..
0.0015
..
1.0 · 10−
= 0,
b
=3
f c
f
6
s
-x
4
8
%
'In104
Nonlinear Algebraic Equations
$
Newton's method
6
f
,
,
,
,
,
,
x
x
s
-x
s
s
1
0
Given a smooth function f (x) and an initial
guess x . We want to nd a value h such that
f (x + h) ≈ 0.
The Taylor series of f around x = x is given
by
f (x + h) = f (x ) + hf 0 (x ) + O(h ).
0
0
0
0
&
0
0
9
2
%
'In104
Nonlinear Algebraic Equations
$
Since we have
f (x + h) ≈ f (x ) + hf 0(x ),
we can nd h such that
f (x + h) ≈ 0
by solving
f (x ) + hf 0(x ) = 0,
thus
h = −f (x )/f 0(x ).
We dene
f (x )
x =x +h=x − 0
.
f (x )
And similarly
f (xk )
xk
= xk − f 0(x ) , k = 0, 1, . . .
k
0
0
0
0
0
0
0
1
0
0
0
0
0
+1
&
10
%
'In104
Nonlinear Algebraic Equations
$
Algorithm
Given an initial approximation x and
a tolerance .
0
k
=0
while |f (xk)| > do
= xk − ff0 xxkk
= +1
xk+1
k
k
end
&
(
)
(
)
11
%
'In104
Nonlinear Algebraic Equations
$
Example
( ) = 2 + x − ex,
f x
x0
= 2,
= 10−
8
( )
0 2.0
−3.4
1 1.4696 −8.8 · 10−
2 1.2073 −1.4 · 10−
3 1.1488 −5.6 · 10−
4 1.146198 −1.1 · 10−
5 1.146193 −3.9 · 10−
k
xk
f xk
01
01
03
05
11
&
12
%
'In104
Nonlinear Algebraic Equations
$
The Secant Method
6f
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
-x
x x
x
s
s
2
()
1
0
f x
Given x
, dene the linear function
f (x ) − f (x )
v (x) = f (x ) −
(x − x)
0
, x1
1
1
x1 − x0
Note that
and
0
1
( ) = f (x )
v x0
0
( ) = f (x ).
v x1
&
1
13
%
'In104
Nonlinear Algebraic Equations
$
Since v approximates f , we get an approximate solution of
f (x∗) = 0,
by solving
v (x) = 0.
The solution of this equation is given by
f (x )
x=x −
.
1
f (x1)−f (x0 )
x1−x0
1
Hence we dene
x =x
1 −
2
or
x2
&
=x
( )
f x1
f (x1)−f (x0 )
x1−x0
.
x
( ) f (xx ) −
− f (x )
1 − f x1
14
1
1
0
0
%
'In104
Nonlinear Algebraic Equations
$
Algorithm
Given two initial approximations x and x
and a tolerance .
0
k
1
=1
while |f (xk)| > do
= xk − f (xk ) f
k =k+1
xk+1
xk −xk−1
(xk )−f (xk−1 )
end
&
15
%
'In104
Nonlinear Algebraic Equations
$
Example
( ) = 2 + x − ex,
f x
x1
= 2,
= 10−
8
( )
1 2.0
−3.38907
2 1.076750
1.41632 · 10−
3 1.113782
6.79253 · 10−
4 1.147913 −3.69598 · 10−
5 1.146152
8.87754 · 10−
6 1.14619317 1.11843 · 10−
7 1.14619322 −3.39107 · 10−
k
xk
f xk
01
02
03
05
07
12
&
16
%
'In104
Nonlinear Algebraic Equations
$
Application I
Recall the method of least squares for computing a constant approximation to a set of data
(ti, pi), i = 0, 1, . . . , n − 1.
Minimize
( )=
R α
n−
X1
i=0
(α − pi)
2
A minimum is computed by dierentiating R,
( )=2
R0 α
n−
X1
i=0
(α − pi),
and solving the equation
R0(α) = 0.
The solution is given by
n−
X
1
α=
p.
(1)
1
&
n i=0
17
i
%
'In104
Nonlinear Algebraic Equations
$
Since (1) is linear, the equation is very easy
to solve. But suppose that we want to minimize
( )=
R α
n−
X1
i=0
(α − pi)
4
.
Then dierentiation gives
( )=4
R0 α
n−
X1
i=0
(α − pi)
3
.
And we want to solve
R0(α) = 0.
This is a nonlinear equation.
&
18
%
'In104
Nonlinear Algebraic Equations
$
Example
In this example, we use the Norsk Hydro stock
prizes in the time period 1.8.97 to 14.8.97 as
data points. We need a termination criteria
for the dierent methods. We stop the iterations when |f (xk )| ≤ , and we choose
= 5.0 · 10−
8
R(α)
4
6
x 10
5.5
5
4.5
4
3.5
404
&
404.5
405
405.5
406
406.5
Plot of R(α) =
407
Pn−1
19
i=0
407.5
408
408.5
(α − pi)4.
409
%
'In104
Nonlinear Algebraic Equations
R’(α)
4
2
$
x 10
1.5
1
0.5
0
−0.5
−1
−1.5
−2
404
404.5
405
405.5
406
406.5
407
407.5
408
408.5
409
P
1
Plot of R0(α) = 4 n−
(α − pi)3. We see from the gure that
i=0
we have only one solution to the equation R0(α) = 0.
&
n
The Bisection Method
c
f (c)
1
405.0
2
407.5
3
406.25
4
406.875
5
..
.
406.5625
..
.
20
..
.
406.342516
..
.
36
406.342525
20
−8.88 · 103
7.02 · 103
−5.81 · 102
3
3.28 · 10
3
1.37 · 10
..
.
−5.93 · 10−2
..
.
−3.188 · 10−9
%
'In104
Nonlinear Algebraic Equations
$
The Secant Method
k
xk
1
415.0267
2
411.1726
3
407.8734
4
406.39415
5
406.340587
6
406.342528
7
406.3425254307
8
406.3425254305
f (xk )
4
6.46 · 10
2.98 · 104
9.23 · 103
2
3.22 · 10
−1.21 · 101
−2
2.10 · 10
−6
1.37 · 10
−11
1.05 · 10
Newton's Method
k
&
xk
1
413.53
2
408.34
3
406.31
4
406.3424
5
406.34252540304
6
406.34252540305
21
f (xk )
4
4.89 · 10
1.20 · 104
−2.17 · 102
−2.55 · 10−1
−3.52 · 10−7
−11
1.06 · 10
%
'In104
Nonlinear Algebraic Equations
$
P
425
420
415
410
405
400
395
390
0
1
2
3
4
5
6
7
8
9
t
P
n−1
The constant approximations
R(α) =
(
α − pi )2 (upper
i
=0
P 1
line) and R(α) = n−
(α − pi)4 (lower line)
i=0
&
22
%
'In104
Nonlinear Algebraic Equations
$
Example II
We now compare minimizing
( )=
R1 α
n−
X1
i=0
(α − pi)
2
,
which is a constant approximation using the
method of least squares, to minimizing the
function
( )=
R2 α
n−
X1
i=0
(α − pi)
4
,
for a set of data points (ti, pi).
The sum in the rst case involves the square
of the dierence between the unknown constant α and the data points pi.
In the second case, this power is increased to
four.
&
23
%
'In104
Nonlinear Algebraic Equations
$
In order to minimize R (α), the second method
will place α such that the big dierences decrease compared to the rst method.
2
445
440
435
430
P
425
420
415
410
405
400
395
390
0
1
2
3
4
5
6
7
8
9
t
P
n−1
The constant approximations
R2 (α) =
(α − pi)4 (upper
i=0
Pn−1
line) and R1(α) = i=0 (α − pi)2 (lower line)
&
24
%
'In104
Nonlinear Algebraic Equations
$
Application II
When is a stock prize at its highest?
p
425
420
r
415
410
405
400
395
390
0
1
2
3
4
5
6
t∗
7
8
9
7
8
9
t
And when does it turn?
p
425
420
415
410
r
405
400
395
390
&
0
1
2
3
t∗
4
25
5
6
t
%
'In104
Nonlinear Algebraic Equations
$
We rst compute an approximation by a fourth
order polynomial, u(t), to the discrete data
points (ti, pi), i = 0, . . . , n − 1:
Find α, β, γ, δ and φ such that
(
)=
R α, β, γ, δ, φ
=
n−
X1
i=0
n−
X1
i=0
(u(ti) − pi)
2
(α + βti + γti + δti + φti − pi)
2
3
4
2
,
is minimized.
After the above approximation u(t) is calculated, we dierentiate it once and solve
u0(t∗) = 0.
(2)
Since (2) is a nonlinear equation, we use the
methods discussed in this lesson to solve it.
&
26
%
'In104
Nonlinear Algebraic Equations
$
When we compute the turning point for the
function u, we dierentiate once more.
Again, the desired time t∗ is found by solving
u00(t∗) = 0,
with either the bisection method, the secant
method or Newton's method.
&
27
%