CHAPTER 3: NON-LINEAR ALGEBRAIC EQUATIONS
3.1 INTRODUCTION
Reactors are often described as being the heart of chemical processes. The
continuous stirred tank reactor (CSTR) is the basic type of chemical reactors. Consider
the CSTR shown in Fig. 3.1 where a liquid phase chemical reaction is taking place:
(3-1)
k
A⎯
⎯→ B
The reaction is assumed to be irreversible. As shown in Figure 3-1, the feed enters the
reactor with volumetric rate Ff, density ρf and concentration CAf. The product comes out
of the reactor at volumetric rate Fo, density ρ0 and concentration CAo and CBo. We also
assume isothermal conditions.
Figure 3-1: Isothermal CSTR
Our objective is to develop a model for the variation of the concentration of species A in
the reactor at steady state conditions (i.e. no variations of either volume or concentrations
with time). If we assume that the volume is constant then the overall mass balance yields
F f = Fo
(3-2)
Since the reactor is a CSTR then the conditions inside the reactor are the same as those
of the effluent. We have then:
ρ o = ρ,
C Ao = C A ,
C Bo = C B
63
(3-3)
The component balance on A yields
F f C Af − rV = Fo C A
(3-4)
where r is the rate of reaction.
Let us assume that the reaction is of third order, then the reaction rate is given by:
(3-5)
r = kC A3
Substituting Eq. (3.5) in Eq.(3.4) yields
(3-6)
F f C Af = F f C A + kC A3
The feed conditions Ff, CAf and the rate constant k are assumed to be given.
This
equation is therefore an algebraic equation with the single variable CA as the unknown.
This equation is also nonlinear in CA because of the cubic term that appears in it. The
students know from their math courses that they can solve, analytically, quadratic
equations. Cubic equations such as Eq.(3.6) also have an analytical solution but it is long
and complicated.
But generally we can not solve nonlinear algebraic equations
analytically and we need the help of numerical techniques.
Ff , CAf , Tf
Qe
V
Fo, CA, T
Figure 3-2: Non-Isothermal CSTR
Now let us reconsider the same CSTR
but we assume that reaction A Æ B
taking place is exothermic. The heat generated in the reactor is removed via a cooling
system as shown in figure 3.2. The effluent temperature is different from the inlet
temperature due to heat generation by the exothermic reaction. We would like to write
the balance equations that describe the reactor. We need to write a mass balance
64
However we know that the reaction rate constant k is dependent on the
temperature through the Arrhenius law
k = koe-E/RT
(3-7)
For the energy balance we will assume constant density and average heat capacity. The
energy balance yields:
~
~
ρF f C p (T f − Tref ) + Qr = ρFo C p (T − Tref ) + Qe
(3-8)
where Qr (J/s) is the heat generated by the reaction, and Qe (J/s) the rate of heat removed
by the cooling system and Tref is a reference temperature. We can assume
Tref = 0 for
simplicity.
The rate of heat exchanged Qr due to reaction is given by:
Qr = -λVr
(3-9)
where λ (J/mole) is the heat of reaction (has negative value for exothermic reaction and
positive value for endothermic reaction). The equations describing the non-isothermal
CSTR are therefore:
F f (C Af − C A ) = k 0 e − E / RT C A3V
~
0 = ρF f C p (T f − T ) + (−λ )Vk 0 e − E / RT C 3 A − Qe
(3-10)
(3-11)
Note that we have two algebraic equations with the unknown variables being CA and T.
These two equations are also nonlinear CA and T. These variables also appear in both
equations. These two equations are therefore coupled. Sometimes it is possible to use
one equation to solve one variable as function of the other variable and then substitute
in the other equation. Thus the two equations could be reduced to one. But this reduction
is not always possible.
65
Nonlinear algebraic equations also arise
in many other aspects of chemical
engineering applications. In this chapter we deal with the following types of problems:
•
Finding the roots of a single variable equation .
•
Finding the roots of a system of multi-variable equations.
•
Finding the roots of a polynomial.
3.2 SINGLE VARIABLE EQUATION
Consider the following single nonlinear equation
f(x) = 0
(3-12)
where the real variable x belongs to an interval [a,b]. This type of equation is
encountered when finding the steady state solution of a dynamic model describing a
chemical process.
This problem is also faced when finding the minimum or the
maximum of a function in which f(x) represents the first derivative of the objective
function.We shall consider in the following sections various numerical techniques for
solving this problem.
3.2-1 Fixed-Point Iteration
The simplest method to solve the single nonlinear algebraic equation is to use the fixedpoint iteration (also called successive substitution) method. The idea is as follows. Let
consider the nonlinear algebraic equation,
f(x) = 0
(3-13)
The equation can be always rearranged in the following way
x = g(x)
(3-14)
The arrangement can be made through some algebraic manipulations, but it can also
simply made by adding x to each side of Eq.(3-14), to yield
66
x + f(x) = x
(3-15)
and g(x) is equal to f(x)+x.
The idea of using Eq.(3-15) is that this equation can serve as a formula to obtain
successive approximations to the solution in the following way:
xi+1 = g(xi)
(3-16)
Starting with initial guess x0, the application of this equation yields x1=g(x0) then x2=g(x1)
and so on. If the method converges then the successive points xi and xi+1 should become
close to each other.
Figure 3-3: Fixed point iteration method, (a) convergence, (b) divergence
Graphically, the solution of Eq. (3.16) amounts to finding the crossing of the
curve y = x and the curve y = g(x). Depending on the
choice of initial guess and the
properties of g(x), the method could converge or diverge. Figure 3-3 shows two cases: a
converging case and a diverging case. A sufficient case for the convergence of the
scheme is that
|g'(x) |<1
Organigram 3.1: Fixed Point Iteration Method
67
(3-17)
The fixed-point iteration algorithm is depicted in Figure 3-4 which solves f(x) = 0,
given an initial guess x0. The stopping criteria for the algorithm are either one of these
conditions:
• A given accuracy |xi+1 − xi|/|xi| ≤ε is reached.
• The maximum number of iterations is reached.
Figure 3-4: Fixed point iteration method
Example 3.1: Simple Quadratic Function
Consider the simple problem of solving the following quadratic equation :
x2 = a
(3-18)
68
We know that the solution is x = ± a for a ≥ 0 . But let us develop a fixed-point iteration
scheme to solve the problem. Eq (3-18) can be written as
x=
Therefore the first choice is g(x) =
(3-19)
a
x
a
a
. However the derivative is g'(x) = − 2 and
x
x
substituting Eq. (3-19) yields g'(x) = −1 which does not satisfy the condition of Eq.(317). A clever way to manipulate Eq. (3-18) is to add the term αx, where α is an arbitrary
parameter, to each side of Eq. (3-19) to yield:
x + αx =
a
+ αx
x
(3-20)
This is equivalent to
a
+ αx
x
x=
1+α
(3-21)
a
+ αx
So in this case we have g(x) = x
.
1+α
Now let us choose α so as to satisfy the condition of Eq (3-17) One way is to try to
simply guarantee that g'(x) = 0. This will automatically satisfy the condition of Eq.(3-17).
Taking the derivative of g(x) yields
g ' ( x) =
The condition g'(x) = 0 yields α =
The function g(x) is therefore
−
a
+α
x2
1+α
(3.22)
a
. Substituting in Eq. (3-17) yields simply α = 1 .
x2
1
a
( x + ) and the fixed-point iteration is
2
x
69
a
1
xi +1 = ( xi + )
xi
2
(3.23)
3.2.2 Bisection Method
Assume the function f to be continuous in the interval a ≤ x ≤ b and satisfies
f(a)f(b) < 0 then by the intermediate value theorem, Equation (3.12) must have at least
one real root r, i.e.
f(r) = 0
(3-24)
The bisection method attempts to bracket the root by repeatedly halving the subintervals
of [a,b] and in each step locating the half containing the root.
To begin, set a0 = a and b0 = b and evaluate the function at the interval's mid point
c=
1
( a o + bo )
2
(3-25)
Then the sign of f(c) is tested against that of f(a0) and f(b0). If f(a0)f(c) < 0 then the root
must lie in the interval [a0, c]. Set a1 = a0 and b1 = c and the procedure is repeated for the
interval [a1,b1]. If on the other hand f(b0)f(c) < 0 the root must lie in the interval [c,b0].
We set a1 = c and b1 = b0 and the procedure is repeated in a similar way.
At each step the interval is then divided in half hence the other name for the procedure is
Interval Halving Method. Figure 3-5 illustrates graphically the steps of the procedure.
70
Figure 3-5: Graphical Interpretation of Bisection method
Error analysis
After a number of iterations n the interval where the root lies is [an, bn] and at iteration
(n+1) the interval becomes [an+1,bn+1]. Since at each iteration the interval is halved we
have
| bn +1 − a n +1 |=
1
| bn − a n |
2
(3-26)
Applying this relation repeatedly leads to
|bn − an|= 2-n |b0 − a0|
(3-27)
Taking the limits of both sides of the equality, as the iterations number becomes large,
leads to
lim bn − lim an = lim 2− n (bo − ao ) = 0
n →∞
n →∞
n →∞
71
(3.28)
Hence
lim bn = lim a n
n→∞
n →∞
(3.29)
Therefore after a 'large' number of steps the bisection method always converges Equation
(3.26) can also be used to estimate the number of steps required to compute the root
within a desired accuracy .Suppose that at stage (n) the interval [an, bn] has been reached.
The best estimate of the root r is the midpoint of the interval cn = (an + bn)/2 . The error
en = r − cn can be bounded as follows
|r − cn| ≤ |bn − an|/2 = 2-(n+1) |b0 − a0|
(3.30)
Let ε be the desired accuracy i.e.
r − cn
r
≤ε
(3.31)
Using inequality (Eq. 3.29) and noting that |r| > a0, in order to satisfy Eq.(3.30), it
suffices that
2 − ( n +1) | bo − a o |
≤ε
| ao |
(3.32)
or equivalently
⎛ | ao |
⎞
ε ⎟⎟
ln⎜⎜
⎝ | bo − a o | ⎠
n≥
−1
ln(2)
72
(3.33)
Choosing the initial interval
Several methods can be used to scan the space of the variable (x) and to determine
an initial and acceptable range for search for the root. A variable grid spacing is
probably the fastest method for the initial bracketing of the root . Consider the following
iteration known as Shwan Iteration:
xn+1 = xn + δ2n-1,
(3.34)
n≥ 1
By choosing an appropriate value of δ and starting with an initial guess x0, this equation
allows the scanning a wide range of values of x. This procedure is illustrated in the
following example.
Example 3.2: Shawan iteration
It is desired to bracket the roots of the following function:
f(x) = x2 − 36
(3.35)
The two roots are evidently –6 and 6. Starting with x0 = 0 and choosing δ = 1, Table 3-1
shows the values of x obtained from Equation (3.33) as well as those of f(x ) (Eq.
3.34).
n
Table 3-1: choosing an initial bracket for Example 3.2
0
1
2
3
4
5
6
x
0
1
3
7
15
31
63
f(x)
-6
-35
-27
13
189
925
3933
It can be noticed that six iterations are enough to reach a value of 63 for the
variable x. Since the values of f(3) and f(7) are opposite in sign we can choose [a0,b0] =
[3,7] as the initial interval. An even tighter interval can be obtained by starting the
procedure at x1 = 3 and using a smaller value of δ. This bracketing method, although
useful, is not appropriate when faced with the existence of several roots.
73
Remarks
• The bisection method always converges. However the technique finds only one
root. To compute all the roots a graph of the function can be used to find the initial
intervals for each root.
• Equation (3.26) indicates that the procedure converges linearly. The bisection
method can be therefore slow in converging.
Organigram 3.2: Bisection Method
The bisection algorithm is depicted in Figure 3-6 which solves f(x) = 0 given a and b
such that f(a)f(b) < 0.
The stopping criteria for the algorithm are either one of these conditions:
• The accuracy |bn − an| ≤ε is reached
• The accuracy for the value of the function |f(cn)| ≤ δ is reached.
• The maximum number of iterations is reached.
Example 3.3: Exothermic Catalytic Reactor
Consider the exothermic catalytic hydrogenation of ethylene which has the form:
A+BÆP
(3.36)
where A represents hydrogen, B represents ethylene and P is the product (ethane). The
reaction takes place in the CSTR shown in Figure 3-7. Two streams are feeding the
reactor. One concentrated feed with flow rate F1 (m3/s) and concentration CB1 (mole/m3),
and another dilute stream with flow rate F2 (m3/s) and concentration CB2 (mole/m3). The
effluent has flow rate Fo (m3/s) and concentration CB (mole/m3). Hydrogen is assumed to
be in excess.
74
Figure 3-6: Bisection Method
Figure 3-7: Reaction in CSTR
75
The reaction rate is assumed to be:
r=
k1C B
(1 + k 2C B ) 2
( mole / m 3 .s )
(3.37)
where k1 is the reaction rate constant and k2 is the adsorption equilibrium constant.
Assuming the operation is isothermal and the density is constant, the reactor model is
obtained by applying total mass and component balances:
Total mass balance:
dL
= F1 + F2 − Fo
dt
(3.38)
d (C B )
= F1 ( C B1 − C B ) + F2 ( C B 2 − C B ) − rV
dt
(3.39)
A
Component B balance:
V
The effluent flow rate is related to the height (L) by :
F0 = α L
(3.40)
where (α) is the discharge coefficient. Substituting for the volume V=AL where (A) is
the area of the reactor yields the following equations of the reactor model:
A
dL
= F1 + F2 − α L
dt
d (C B ) F1
F
k1CB
=
( C B1 − C B ) + 2 ( C B 2 − C B ) −
(1 + k2CB ) 2
dt
AL
AL
(3.41)
(3.42)
The steady state behavior of the reactor is obtained by dropping the accumulation term
of the left hand side of each equations, to yield:
76
(3.43)
0 = F1 + F2 − α L
0=
F1
F
k1C B
( C B1 − C B ) + 2 ( C B 2 − C B ) −
AL
AL
(1 + k2CB ) 2
(3.44)
These equations are a pair of coupled nonlinear algebraic equations. Suppose that the
level is maintained constant by some mechanism in the reactor. Then the static behavior
of the reactor model is given by the nonlinear equation Eq.(3.43) where (CB) is the
unknown variable:
0=
F1
F
k1C B
( C B1 − C B ) + 2 ( C B 2 − C B ) −
:=f( C B )
AL
AL
(1 + k 2 C B ) 2
(3.45)
The values of the model parameters are given in Table 3-2
Parameter
value
Table 3-2: Parameters values for example 3.3
k1
k2
Cb1
Cb2
F1
F2
A
L
α
1
100.
0.2
1
1
24.9
0.10
1.0
1.0
0.3
f(Cb)
0.2
0.1
f(Cb)=k1 Cb/(1 + k2 Cb)2
f(Cb)=[(Cb1 - Cb)W1+ (Cb2 - Cb)W2]/L
0.0
0
2
4
6
Cb
Figure 3-8: Roots of f(Cb)
77
8
10
A rough graph of the function f(Cb) (Figure 3-8) shows that the equation admits
three real roots. This means that for the value of the height (L = 100 cm) the reactor may
be operated at three values of the concentration. This phenomenon is called steady state
multiplicity. Let us compute for instance the root in the middle .From the graph it can
be seen that the desired root lies in the range Cb ∈ [1,5]. Table (3.3) summarizes the
iterations of the bisection method
Table 3-3: Iterations for the bisection method (Example 3.3 )
Iteration, n
a
B
C=(a+b)/2
|f(c)|
|b-a|
0
1.000000
5.000000
3.000000
0.250000E-02
0.400000E+01
1
1.000000
3.000000
2.000000
0.122222E-01
0.200000E+01
2
2.000000
3.000000
2.500000
0.408164E-02
0.100000E+01
3
2.500000
3.000000
2.750000
0.555559E-03
0.500000E+00
4
2.750000
3.000000
2.875000
0.103277E-02
0.250000E+00
10
2.789062
2.792969
2.791016
0.211630E-04
0.390625E-02
15
2.792603
2.792725
2.792664
0.398481E-07
0.122070E-03
20
2.792660
2.792664
2.792662
0.153198E-07
0.381470E-05
25
2.792660
2.792661
2.792661
0.756167E-09
0.119209E-06
It can seen from the value of the errors that the bisection method converges slowly to the
root 2.792660
3.2.3 Newton-Raphson Method
Netwon-Raphson method is perhaps the most famous method among the single
variables root-finding methods. To develop the method we start with an initial guess x0 of
the root r and we write a Taylor series expansion around x0 . Let r= x0 +h we have then
0 = f (r ) = f ( xo + h) = f ( x ) + hf ′( xo ) +
h2
hn n
f ′′( xo ) + L +
f ( xo ) + L
2!
n!
(3.46)
For small values of the step size h the terms of order 2 and higher can be discarded to
yield
78
0 = f(x0) + hf '(x0)
(3.47)
We can solve for h to yield
h=−
f ( xo )
f ′( xo )
(3.48)
Therefore, the point
x1 := xo + h = xo −
f ( xo )
f ′( xo )
(3.49)
should be a better approximation of the root r. This iteration scheme can be generalized.
The Newton-Raphson method consists of the following steps:
•
An initial guess of the root, x0
•
and the iteration
xn+1 = xn −
f ( xn )
f ′( xn )
(3.50)
Graphically the method consists as, shown in Figure 3-9, in linearizing the function
around the point xn and then extending the tangent line until it crosses the zero axis.
The next approximate xn+1 corresponds to the abscissa of that zero-crossing.
Error analysis
Let the error at iteration (n) be en = xn-r. Using Equation (3.49) the error at iteration
(n+1) is
e n +1 = x n +1 − r = x n − r −
f (xn )
f ( x n ) e n f ′( x n ) − f ( x n )
= en −
=
f ′( x n )
f ′( x n )
f ′( x n )
(3.51)
Using Taylor's theorem we can write
0 = f(r) = f(xn − en) = f(xn) − enf '(xn) + (1/2)en2f ''(ξ)
79
(3.52)
•
•
•
•
Figure 3-9: Graphical Interpretation of Newton Method
where ξ ∈ [xn, r]. From this equation, the following relation is obtained
enf '(xn) − f(xn) = 1/2f ''(ξ)en2
(3.53)
Substituting in equation (3.50) yields
en +1 1 f ′′(ξ )
=
en
2 f ′(ξ )
(3.54)
Assuming that f '(r) ≠ 0 i.e. r is a simple zero then the right hand side can be
approximated by
1 f ′′( r )
which is a constant (C). The error sequence obeys then the law
2 f ′(rx )
en+1 = Cen2
(3.55)
This mode of convergence is called quadratic convergence since the numerical precision
is practically doubled at each iteration.
Remarks
Compared to the bisection method the Newton-Raphson method is obviously faster but
unfortunately it can fail. Since the method requires the evaluation of the derivative, the
selection of the initial guess is crucial for the success of the iteration scheme. Figure 3-10
80
shows a case where the search encounters a local minimum, i.e. f '(x) = 0 and the method
fails, since the slope became parallel to the x-axis and never crosses it. Figure 3-11 shows
another case where the search encounters a non-convergence cycle. The only remedy to
these situations is to change the initial guess in hope that it is close to the root. Plotting
graphs of the function can help in choosing a good initial guess.
y
y = f(x)
•
xn+1
•
x•n
x
•
Figure 3-10: Newton Method diverges at extremum
Figure 3-11: Newton Method facing a non-converging cycle
81
Organigram 3.3: Newton Method
The algorithm of Newton method is depicted in Figure 3-12 which solves f(x) = 0 given
an initial guess xo.
Figure 3-12: Newton-Raphson Method
The stopping criteria for the method are one of the following:
• The accuracy for the value of the function |f(x)| ≤ δ is reached.
• The accuracy for the error |xn+1 − xn|/| xn | < ε is reached.
• Exceeding a given maximum number of iterations.
82
Example 3.4: Solution of Example 3.3 using Newton-Raphson method
We apply the Newton-Raphson method to solve the same problem in example 3.3. The
iteration sequence is as follows:
Cb ,n+1 = Cb ,n −
(3.56)
f (Cb ,n )
f ′(Cb ,n )
With
f ′(Cb ) = −
(1 − k2Cb )
w1 w2
−
− k1
L L
(1 + k2Cb )3
(3.57)
Starting with an initial guess Cb0 = 1.5, Table (3.4) shows the iterations for this method.
Table 3-4: Newton-Raphson method iterations for example 3.4
Iteration n
|f(Cb,n)|
|Cb,n−Cb,n-1|/|Cb,n-1|
Cb,n
0
0.150000E+01
0.200000E-01
-
1
0.316667E+01
0.426667E-02
0.111111E+01
2
0.273794E+01
0.715092E-03
0.135387E+00
3
0.279180E+01
0.110183E-04
0.196723E-01
4
0.279266E+01
0.280914E-08
0.306741E-03
5
0.279266E+01
0.184599E-15
0.782201E-07
It can be seen that the error is reduced much faster than in the bisection method.
3.2.4 Secant Method
To circumvent the need of the derivative evaluation in Newton-Raphson procedure, a
number of methods have been proposed. By replacing the derivative f '(x) by a difference
quotient such as
f ′( xn ) =
f ( xn ) − f ( xn −1 )
xn − xn−1
The resulting iteration scheme is:
83
(3.58)
xn+1 = xn − f ( xn )
xn − xn−1
f ( xn ) − f ( xn−1 )
(3.59)
This method is called the secant method .
Graphically as shown in Figure 3-13, starting from the two guesses xn-1 and xn the next
guess xn+1 is obtained as a result of the intersection of the line joining the two points with
the zero axis.
•
•
•
•
•
•
Figure 3-13: Geometric interpretation of Secant method
Remarks
It can be shown through a lengthy analysis that the error en = xn -r for the secant method
obeys the following formula
|en+1| = C|en| (1+ 5 )/2
(3.60)
where C is a constant. Similar to the Newton-Raphson method the secant method
converges also geometrically. However since the exponent value (1 + 5 )/2 = 1.618 is
less than 2 the secant method is slower than the Newton-Raphson procedure. The secant
method has the advantage of requiring only one function evaluation instead of two (f(x)
and f '(x)) used in Newton-Raphson method but it requires two initial guesses instead of
one.
84
The secant method can also fail as it can be seen from the example of Figure
3-14. A good choice of initial guesses is the remedy for this problem.
•
•
•
Figure 3-14: Example for failure of Secant method
Organigram 3.4: Secant Method
The algorithm finds the solution of f(x) = 0 given two initial guesses (xo, x1). The
organigram is shown in Figure 3-15. The stopping criteria for the method are the same as
those for the Newton-Raphson method.
Example 3.5: Solution of example 3.3 using secant method
Starting with the initial conditions Cb0 = 0.1 and Cb1 = 4.0 the iterations for the secant
method for example 3.3 are summarized in Table (3.5).
Table 3-5: Iterations for Secant method for example 3.5
Iteration
Cb,0
Cb,1
Cb,2
|f(Cb,2)|
|Cb,2−Cb,1 |/| Cb,1|
1
0.100000E+00
0.400000E+01
0.425104E+01
0.108075E-01
0.627593E-01
2
0.400000E+01
0.425104E+01
0.891323E+00
0.170010E-01
0.790328E+00
3
0.425104E+01
0.891323E+00
0.294532E+01
0.187312E-02
0.230443E+01
4
0.891323E+00
0.294532E+01
0.274147E+01
0.668277E-03
0.692094E-01
5
0.294532E+01
0.274147E+01
0.279507E+01
0.310170E-04
0.195523E-01
6
0.274147E+01
0.279507E+01
0.279270E+01
0.464869E-06
0.850605E-03
7
0.279507E+01
0.279270E+01
0.279266E+01
0.334560E-09
0.129535E-04
8
0.279270E+01
0.279266E+01
0.279266E+01
0.360176E-14
0.931586E-08
9
0.279266E+01
0.279266E+01
0.279266E+01
0.185670E-17
0.100342E-12
85
Start
Input K,
, x0 ,x1
k=0
k>K
Yes
No
x2 = x1 −
f ( x1 )( x1 − x0 )
f ( x1 ) − f ( x0 )
|f(x2)| < or
|x2 - x1|/|x1|<
No
Set: k=k+1
x0=x1
x1=x2
Yes
Print x2 , f(x2)
maximum iteration
exceeded
Stop
Figure 3-15: Secant Method
From Table 3.5 it can be seen that the secant method works almost as fast as the Newton
method for this example.
3.3 SYSTEM OF NONLINEAR EQUATIONS
The problems considered so far are of single variable algebraic equation.
However, in many cases, a system of coupled multi-variable equations is encountered in
typical chemical engineering problems. In the following we consider tackling such
problem using Newton Method .
3.3.1 Newton's Method for System of Non-linear Algebraic Equations
The solution of a system of nonlinear algebraic equations using Newton's method
follows in a straight forward manner the strategy used for one single equation.
Consider for example a system of two equations:
86
⎧ f1 ( x1 , x2 ) = 0
⎨
⎩ f 2 ( x1 , x2 ) = 0
(3.61)
Let (x1,n,x2,n) be an approximation of the root (r1,r2) at iteration n. Let us determine the
increments h1 and h2 so that (x1,n+1 = x1,n + h1, x2,n+1 = x2,n + h2) can be a better
approximation to the solution.
Using a first order Taylor expansion for the function of two variables yields:
∂f 1
∂f 1
⎧
⎪0 = f 1 ( x1, n + h1 , x 2, n + h2 ) = f 1 ( x1, n , x 2, n ) + h1 ∂x ( x1, n , x 2, n ) + h2 ∂x ( x1, n , x 2, n )
⎪
1
2
⎨
⎪0 = f ( x + h , x + h ) = f ( x , x ) + h ∂f 2 ( x , x ) + h ∂f 2 ( x , x ) (3.62)
2
1, n
1
2, n
2
2
1, n
2, n
1
1, n
2, n
2
1, n
2, n
⎪⎩
∂x1
∂x 2
The two equations can be put in matricial form as follows:
f ( x1, n , x 2, n ) + J h = 0
(3.63)
⎡ f1 ( x1,n , x2,n ) ⎤
f ( x1,n , x2,n ) = ⎢
⎥
⎣ f 2 ( x1,n , x2,n )⎦
(3.64)
⎡h ⎤
h=⎢ 1⎥
⎣h2 ⎦
(3.65)
where
and
87
⎡ ∂f 1
⎢ ∂x ( x1n , x 2 n )
J ( x1n , x 2 n ) = ⎢ 1
⎢ ∂f 2
⎢ ∂x ( x1n , x 2 n )
⎣ 1
∂f1
⎤
( x1n , x 2 n )⎥
∂x 2
⎥
⎥
∂f 2
( x1n , x 2 n )⎥
∂x 2
⎦
(3.66)
The matrix J is called the Jacobian matrix of f1 and f2. If the matrix J is not singular the
vector of increaments h can be readily calculated from Eq. (3.62) as follows:
⎡ f1 ( x1,n , x2,n ) ⎤
⎡h1 ⎤
−1
=
−
J
⎢
⎥
⎢h ⎥
⎣ 2⎦
⎣ f 2 ( x1,n , x2,n )⎦
(3.67)
The Newton's iteration for the system of coupled equations is then:
⎡ x1,n +1 ⎤ ⎡ x1,n ⎤ ⎡h1,n ⎤
⎢
⎥=⎢ ⎥+⎢ ⎥
⎣ x 2,n +1 ⎦ ⎣ x 2,n ⎦ ⎣h2,n ⎦
(3.68)
with the vector h=(h1, h2) given by (Eq. 3.66). The same strategy can be extended for a
general system of (n) equations i.e.:
⎧ f 1 ( x1 , x 2 ,L , x n ) = 0
⎪ f ( x , x ,L, x ) = 0
⎪ 2 1 2
n
⎨
⎪M
⎪⎩ f n ( x1 , x 2 , L, x n ) = 0
(3.69)
The system of equation can be written using the vectorial notation
f ( x) = 0
(3.70)
with
⎡ f1 ⎤
⎢f ⎥
f = ⎢ 2⎥
⎢M ⎥
⎢ ⎥
⎣ fn ⎦
88
(3.71)
and
⎡ x1 ⎤
⎢x ⎥
x = ⎢ 2⎥
⎢M ⎥
⎢ ⎥
⎣ xn ⎦
(3.72)
is the vector of unknowns.
Let J be the n × n Jacobian matrix whose elements are
J (i, j ) =
∂f i
∂x j
(3.73)
The Newton's iteration is therefore
x n +1 = x n + h
(3.74)
⎡h1 ⎤
⎢h ⎥
Where h = ⎢ 2 ⎥ is the solution of the linear system
⎢M ⎥
⎢ ⎥
⎣hn ⎦
J ( x n )h = − f ( x n )
(3.75)
Remarks
The implementation of the Newton's method for the solution of system of nonlinear
algebraic equations includes the tedious task of evaluation of the derivative and the
solution of the linear system .This can be sometimes not a pleasant task . Many
commercial software, introduce changes for the implementation of this method .
89
Organigram 3.5: Newton Method For Systems of Nonlinear Algebraic Equations
The algorithm is shown in Figure 3-16. The algorithm solves f ( x) = 0 given an initial
guess x o . The stopping criteria for the method are one of the following:
• The accuracy for the norm of the set of functions f (x) ≤ δ is reached.
• The accuracy for the norm of error
x n +1 − x n
xn
is reached.
• Exceeding a given maximum number of iterations.
Example 3.6: CSTR model
Consider the steady-state model of the reactor (Eqs. 3.43-3.44 )
0 = w1 + w2 − 0.2 L0.5 := f1 (Cb )
0 = (C b1 − C b )
k 1C b
w1
w
+ (C b 2 − C b ) 2 −
:= f 2 ( L, C b )
L
L (1 + k 2 C b ) 2
(3.76)
(3.77)
The problem could be reduced to one algebraic equation by noting that first equation
depends on (L) alone. However we will carry out the Newton iteration as an exercise.
The derivatives forming the Jacobian are given by
∂f1
= −0.01L−0.5
∂L
(3.78)
∂f1
=0
∂Cb
(3.79)
w
w
∂f 2
= −(Cb1 − Cb ) 21 − (Cb 2 − Cb ) 22
∂L
L
L
(3.80)
(1 − k 2 Cb )
w w
∂f 2
= − 1 − 2 − k1
∂Cb
Ls Ls
(1 + k 2 Cb ) 3
(3.81)
90
Figure 3-16: Newton method for system of nonlinear algebraic equations
In the following, the first hand iteration is presented:
First iteration :
x0 = [ L0 , Cb 0 ] = (85,1.6)
91
(3.82)
f ( x0 ) = [ f1 ( L, Cb ), f 2 ( L, Cb )] = (0.15609,0.01978)
(3.83)
0
⎡ − 0.010846
⎤
J=⎢
⎥
⎣− 0.003017 0.010608⎦
(3.84)
0 ⎤ ⎡0.156091⎤
⎡ − 92.2000
h = − J −1 f = ⎢
⎥⎢
⎥
⎣− 26.2224 94.2685⎦ ⎣0.019784⎦
(3.85)
⎡+ 14.39150⎤
h=⎢
⎥
⎣+ 2.22820 ⎦
(3.86)
⎡ L1 ⎤ ⎡99.39150⎤
⎢C ⎥ = ⎢3.82820 ⎥
⎦
⎣ b1 ⎦ ⎣
(3.87)
and the new approximation is
The rest of the iterations are shown in Table (3.6). As it can be seen the method
converges quite fast to the solution L = 100 and Cb = 2.79266.
Table 3-6: Iterations for Example 3.6
(|f1|,|f2|)
(Ln,Cbn)
Iteration
2
(99.99907, 1.98761)
(0.92781E-05, 0.12410E-01)
3
(100.0000, 2.71748)
(0.29802E-07, 0.98833E-03)
4
(100.0000, 2.79106)
(0.29802E-07,0.20588E-04)
5
(100.0000, 2.79265)
(0.29802E-07,0.13111E-06)
6
(100.0000, 2.79266)
(0.29802E-07,0.23436E-08)
3.4 ZEROS OF POLYNOMIALS
A function of the form
P(x) = anxn + an-1xn-1 + … + a1x + a0
92
(3.88)
where the ai's are constants and an ≠ 0 is a polynomial of degree n .The ai are called the
coefficients of P . A polynomial of degree n has always n roots. These roots can be real
or complex, distinct or multiple. In general we are interested in finding all the roots of
the polynomial. Therefore the bisection, or Newton-Raphson method are not adequate for
this purpose. A number of techniques have been developed specifically for finding the
roots of polynomials. We will describe in this section the Müller's technique.
3.4.1 Müller's method
Müller's method generalizes the secant method previously discussed for a single
nonlinear equation. It uses a quadratic interpolation among three points, instead of
linear interpolation, to find the next approximation point. As shown by Figure 3-17,
starting with three initial approximations xn-2, xn-1 and xn, of the root, a parabola that
passes through the three points (xn-2, f(xn-2)) , (xn-1, f(xn-1)) and (xn, f(xn)) can be formed.
The next guess xn+1 is found by the intersection of the parabola with the zero axis.
The equation of the parabola is:
P(x) = a(x – xn)2 + b(x − xn) + c
•
•
(3.89)
•
•
Figure 3-17: Geometric Interpretation of Muller Method
The constants a, b and c are determined from the conditions:
f(xn-2) = a(xn-2 − xn) 2 + b(xn-2 − xn) + c
93
(3.90)
f(xn-1) = a(xn-1 − xn) 2 + b(xn-1 − xn) + c
(3.91)
f(xn) = a.02 + b.0 + c
(3.92)
Which results in the following formulas:
c = f(xn)
b=
(3.93)
( f ( x n −1 ) − f ( x n ))( x n − 2 − x n ) ( f ( x n − 2 ) − f ( x n ))( x n −1 − x n )
−
( x n −1 − x n )( x n −2 − x n −1 )
( x n − 2 − x n −1 )( x n − 2 − x n )
(3.94)
f ( x n−2 ) − f ( xn )
f ( x n −1 ) − f ( x n )
−
( x n −2 − x n −1 )( x n − 2 − x n ) ( x n −1 − x n )( x n − 2 − x n −1 )
(3.95)
a=
The new guess xn+1 is obtained by solving the following equation:
P(x) = 0
(3.96)
which leads to:
xn+1 = xn −
2c
b ± b 2 − 4ac
(3.97)
This gives two possible values for xn+1. In Müller's method the sign is chosen to be the
sign of (b). This will result in the denominator being large in magnitude and a root xn+1
the closest to xn. Once xn+1 is determined the procedure is reinitialized using xn-1,xn and
xn+1 in place of xn-2,xn-1 and xn. It should be noted that since the method involves the
radical
b 2 − 4ac , it can also find complex roots as well .
Organigram 3.6: Muller Method
The algorithm is shown in Figure 3-18. The algorithm finds a real solution to f(x) = 0
given the three approximations x0, x1, x2.
94
Example 3.7: Third Order Polynomial
Consider the following polynomial
p(x) = (x − 5) 2(x + 1)
(3.98)
The roots are evidently 5 (multiplicity 2) and -1 (simple root). Table (3.7) shows the
iterations starting with initial conditions x0 = − 0.5, x1 = 0 and x1 = 0.5. Table (3.8) shows
the iterations starting with initial conditions x0 = − 0.5, x1 = 0 and x1 = 2.5 .
Table 3-7: Iterations for Müller's method starting with initial conditions x0 = − 0.5,
x1 = 0 and x1 = 0.5
Iter
x0
x1
x2
|P(x2)|
x 2 − x1
x1
0
-0.500000E+00
0.000000E+00
0.500000E+00
0.303750E+02
-
1
0.000000E+00
0.500000E+00
-0.102242E+01
0.813175E+00
0.304484E+01
2
0.500000E+00
-0.102242E+01
-0.999027E+00
0.350338E-01
0.228808E-01
3
-0.102242E+01
-0.999027E+00
-0.999999E+00
0.327093E-04
0.973516E-03
4
-0.999027E+00
-0.999999E+00
-0.100000E+01
0.198321E-10
0.908591E-06
5
-0.999999E+00
-0.100000E+01
-0.100000E+01
0.000000E+00
0.550893E-12
Table 3-8: Iterations for Müller's method starting with initial conditions x0 = − 0.5,
x1 = 0 and x1 = 2.5
x 2 − x1
Iter
x0
x1
x2
|P(x2)|
x1
0
-0.500000E+00
0.000000E+00
0.250000E+01
0.218750E+02
-
1
0.000000E+00
0.250000E+01
0.337853E+01
0.115119E+02
0.351410E+00
2
0.250000E+01
0.337853E+01
0.406823E+01
0.440023E+01
0.204143E+00
3
0.337853E+01
0.406823E+01
0.454622E+01
0.114208E+01
0.117493E+00
4
0.406823E+01
0.454622E+01
0.479178E+01
0.251106E+00
0.540152E-01
5
0.454622E+01
0.479178E+01
0.491794E+01
0.398493E-01
0.263286E-01
10
0.499861E+01
0.499981E+01
0.499998E+01
0.168136E-08
0.348289E-04
13
0.500000E+01
0.500000E+01
0.500000E+01
0.288492E-18
0.412166E-08
95
Start
Input x0, x1, x2, , N
i=2
h1 = x1 - x0
h2 = x2 - x1
1 = f(x1) - f(x2)
2 = f(x2) - f(x1)
a=−
b=
δ1 + δ 2
δ
+ 2
h1 (h1 + h2 ) h1h2
(h1 + h2 )δ 2 (δ 2 + δ1 )h2
+
h1h2
h1 (h1 + h2 )
c = f ( x2 )
i<N
Yes
"no solution found"
Yes
"no real solution"
No
= (b2 - 4ac)
<0
No
x3 = x 2 −
2c
b + sign(b) Δ
x3 − x2
<
x2
Yes
Print x3
No
x0 = x1 , x1 = x2
i=i+1
Stop
Figure 3-18: Müller Method
96
3.5 OTHER SOLUTION METHODS
It was shown that Newton-Raphson method is of quadratic convergence. The
following iteration, known as Household iteration [52], provides a general scheme that
provides arbitrary higher order of convergence. The scheme is as follows:
⎛ (1 / f ) ( p )
x n +1 = x n + ( p + 1)⎜⎜
( p +1)
⎝ (1 / f )
⎞
⎟⎟
⎠
(3.99)
where p is the derivative order.
The Household iteration is of convergence order (p+2). When p = 0 the formula
gives back the Newton-Raphson method (of quadratic convergence). When
p = 1, the
convergence is cubic and the formula yields
x n +1 = x n −
2 f ( xn ) f ' ( xn )
2( f ' ( x n )) 2 − f ( x n ) f ' ' ( x n )
(3.100)
Which is known as the Halley’s method
For the solution of the system of nonlinear algebraic equations we have seen that
the implementation of Newton method requires the use of the Jacobean matrix and
therefore the evaluation of n2 partial derivatives.
Some commercial software uses
Newton method but with modifications to approximate these derivatives, such us using
finite differences (to be described later in Chapter 5)
with special features.
The
discussion of these modifications is very much beyond the scope of this book. The
most common of these methods are Broyden update and Power hybrid method. (The very
much interested reader could consult references [45,62-63] for details).
3.6 IMSL ROUTINES
Some IMSL routine to solve nonlinear algebraic equations are as follows:
Routine
Features
ZREAL
Find the real zeros of single function using Müller's method.
NEQNF
Solve a system of nonlinear equations based on Newton method but
using a variation called modified Powell hybrid algorithm and a finitedifference approximation to the Jacobean.
LSLXG
It is the same algorithm as NEQNF but the user has to provide an
analytical Jacobean.
97
98