10.34: Numerical Methods
Applied to
Chemical Engineering
Lecture 7:
Solutions of nonlinear equations
Newton-Raphson method
1
Recap
•
•
Singular value decomposition
Iterative solutions to linear equations
2
Recap
•
•
Iterative solutions to linear equations
•
•
•
Given: x0
Iterate on: xi+1 = Cxi + c
Until converged to solution of: Ax = b
Assume the iterations converge. When should I stop?
3
Systems of Nonlinear Eqns.
•
•
Formally: f (x) = 0
•
•
•
•
where: x 2 RN
where: f : RN ! RN
x are called the roots of f (x)
linear equations are represented as f (x) = Ax - b
Common chemical engineering examples include:
•
•
•
Equations of state
Energy balances
Mass balances with nonlinear reactions
4
Systems of Nonlinear Eqns.
•
Example: van der Waals equation of state
✓
◆✓
◆
3
1
8
Pˆ + 2
v̂ = Tˆ
3
3
v̂
•
Pˆ , Tˆ, vˆ are reduced pressure, temperature, and
molar volume
2
T̂ = 1.1
P̂ 1
T̂ = 0.9
1
•
v̂L
v̂
3
2
v̂G
Given pressure and temperature, there are 1-3 molar
volumes that satisfy the equation of state.
5
Systems of Nonlinear Eqns.
•
Example: van der Waals equation of state
✓
◆✓
◆
3
1
8
Pˆ + 2
v̂ = Tˆ
3
3
v̂
•
•
•
Given pressure and temperature, 1, 2 or 3 solutions
for molar volume possible.
✓
◆✓
◆
3
1
8
f (v̂; Pˆ , Tˆ) = P̂ + 2
vˆ - T̂ = 0
3
3
v̂
In general, nonlinear equations can have any number of
solutions. It is impossible to predict beforehand.
For gas-liquid coexistence, can the pressure and
temperature be specified independently?
6
Systems of Nonlinear Eqns.
•
Example: van der Waals equation of state
•
For gas-liquid coexistence, can the pressure and
temperature be specified independently?
•
•
No!
Thermal equil. – same temperature in gas/liquid
TˆG = TˆL = Tˆ
•
Mechanical equil. – same pressure in gas/liquid
PˆG = PˆL = Pˆsat
•
Chemical equil. – same chemical potential in gas/liquid
Z
v̂L
(P̂ (v̂)
P̂sat ) dv̂ = 0
v̂G
7
Systems of Nonlinear Eqns.
•
Example: van der Waals equation of state
•
For gas-liquid coexistence, can the pressure and
temperature be specified independently?
•
•
•
Given the temperature, there are 3 unknowns
•
•
The saturation pressure
The molar volumes of the gas and liquid
There are three nonlinear equations to solve:
•
•
Equation of state in gas/liquid
Maxwell equal area construction
Must solve: f (P̂sat , v̂G , v̂L ) = 0
8
Systems of Nonlinear Eqns.
•
Example: van der Waals equation of state
•
Must solve: f (P̂sat , v̂G , v̂L ) = 0
✓
◆✓
◆
3
1
8
f1 (P̂sat , v̂G , v̂L ) = P̂sat + 2
v̂G - T̂ = 0
v̂G
3
3
✓
◆✓
◆
3
1
8
f2 (P̂sat , v̂G , v̂L ) = P̂sat + 2
v̂L - T̂ = 0
v̂L
3
3
f3 (P̂sat , v̂G , v̂L ) =
Z
v̂L
v̂G
(P̂ (v̂) - P̂sat ) dv̂ = 0
9
Systems of Nonlinear Eqns.
•
Example: van der Waals equation
Z of state
•
Use Pˆsat =
from:
v̂L
1
v̂L
v̂G
Pˆ (ˆ
v)dv̂ to eliminate Pˆsat
v̂G
f1 (P̂sat , v̂G , v̂L ), f2 (P̂sat , v̂G , v̂L ),
3
f1 (v̂G , v̂L ) = 0
2
v̂G
1
f2 (v̂G , v̂L ) = 0
1
v̂L
2
3
10
Systems of Nonlinear Eqns.
•
•
•
•
N
N
f
:
R
!
R
Given:
⇤
Find: x 2 R
•
•
•
N
⇤
: f (x ) = 0
There could be no solutions
There could be 1 < n < 1 locally unique solutions
There could be 1 solutions
⇤
x
A solution, , is locally unique if there exists a ball of finite
⇤
x
radius such that
is the only solution within the ball.
Consider the simple function:
✓
f1 (x1 , x2 )
f2 (x1 , x2 )
◆
=0
11
✓
Systems of Nonlinear Eqns.
f1 (x1 , x2 )
f2 (x1 , x2 )
◆
=0
x2
f 1 ( x 1 , x2 ) = 0
x1
f2 (x1 , x2 ) = 0
12
✓
Systems of Nonlinear Eqns.
f1 (x1 , x2 )
f2 (x1 , x2 )
◆
=0
f1 (x1 , x2 ) = 0
x2
tangent curves, potentially
not locally unique
x1
locally unique solution
f2 (x1 , x2 ) = 0
13
Systems of Nonlinear Eqns.
•
•
•
•
Inverse function theorem:
•
•
•
⇤
⇤
6 0,
If f (x ) = 0 and det J(x ) =
⇤
then x is a locally unique solution,
where the Jacobian is:
0
B
B
J(x) = B
B
@
@f1
@x1
@f2
@x1
@f1
@x2
@f2
@x2
@fN
@x1
@fN
@x2
..
.
..
.
...
...
..
.
...
@f1
@xN
@f2
@xN
..
.
@fN
@xN
The Jacobian describes the rate of change of a vector
function with respect to all of its independent variables.
1
C
C
C
C
A
⇤
If det J(x ) = 0 , solution may/may not be locally unique
Most numerical methods can only find one locally unique
solution at a time.
14
Systems of Nonlinear Eqns.
•
Example:
•
Compute the Jacobian of:
f (x) =
✓
x21 + x22
x21 x22
◆
15
✓
Systems of Nonlinear Eqns.
f1 (x1 , x2 )
f2 (x1 , x2 )
◆
=0
f1 (x1 , x2 ) = 0
x2
tangent curves, potentially
not locally unique
x1
locally unique solution
f2 (x1 , x2 ) = 0
17
Systems of Nonlinear Eqns.
•
Inverse function theorem:
•
•
Consider a linear equation: f (x) = Ax
b
The Jacobian of the function is:
J(x) = A
•
•
•
•
The equation: f (x) = 0 , has a locally unique solution
when det J(x) = det A 6= 0
There is a locally unique solution when A is invertible
The inverse function theorem is just a generalization of what
we learned in our study of linear algebra.
In fact, in a neighborhood close to a root of f (x), we can
often treat the function as linear!
18
Linearization
•
•
•
Linearizing 1-D nonlinear functions:
•
•
f (x +
x) = f (x) + f 0 (x) x + O( x2 )
typically valid as
x!0
Linearizing generalized nonlinear functions:
•
•
f (x +
x) = f (x) + J(x) x + O(k xk22 )
typically valid as k xk2 ! 0
Part of a Taylor expansion for each component of f (x) :
fi ( x +
x) = fi (x) +
N
X
@fi (x)
j=1
+
N X
N
X
1
@ 2 fi (x)
2
j=1 k=1
@xj @xk
@xj
xj
x j xk + . . .
19
Iterative Solutions to NLEs
•
•
•
•
⇤
Nonlinear equations, f (x ) = 0 , are solved iteratively
The algorithmic map: xi+1 = g(xi ) , is designed so that:
•
•
⇤
⇤
x = g(x )
equivalently, x⇤ is a fixed point of the map, g(x)
Iterations stop when the map is sufficiently converged.
Two common criterion for stopping are:
•
•
Function norm criterion:
kf (xi+1 )kp  ✏
Step norm criterion:
kxi+1
xi kp  ✏R kxi+1 kp + ✏A
20
Iterative Solutions to NLEs
•
Failure of function norm criterion:
xi
1
xi xi+1
x
f (x)
<✏
•
Failure of step norm criterion:
xi
< ✏A
1
xi
xi+1
f (x)
x
21
Convergence Rate
•
The rate of convergence is addressed by examining:
•
kxi+1 x⇤ kp
lim
=
C
q
k!1 kxi
x⇤ kp
when the limit exists and is not zero:
•
•
•
•
q = 1, C < 1 , convergence is linear
•
If C = 10 1 each iteration is 1 digit more
accurate than the previous
q > 1 , convergence is super-linear
q = 2 , convergence is quadratic
•
The number of accurate digits doubles with each
iteration.
Jacobi and Gauss-Seidel showed linear convergence rates
22
•
Newton-Raphson Method
Utilize linear approximations of the function to find a root
iteratively:
f (x)
xi+1
0
xi
f (xi+1 ) ⇡ 0 = f (xi ) + f (xi )(xi+1
x
xi )
xi+1 = xi
f (xi )
f 0 (xi )
23
Newton-Raphson Method
•
When the iterate is sufficiently close to the root,
convergence is guaranteed (local convergence)!
•
Extending this idea to systems nonlinear equations is easy:
•
Approximate the function as linear:
f (xi+1 ) ⇡ 0 = f (xi ) + J(xi )(xi+1 - xi )
•
f (xi+1 ) ⇡ 0 = f (xi ) + J(xi )di
Solve for the displacement:
J(xi )di =
•
f (xi ) ) di =
[J(xi )]
1
f (xi )
Update the iterate:
xi+1 = xi + di
xi+1 = xi - [J(xi )]
1
f (xi )
24
ric example
Newton-Raphson Method
e intersection of two circles of radius 3 with centers at
3, 1). Equations for the points (x1 , x2 ) residing on
Example:
the
intersection
of
circles
of each circle are
x2
0 = f 1 (x1 , x2 ) = (x1 2)2 + (x2 2)2 9,
•
0 = f 2 (x1 , x2 ) = (x1 + 3)2 + (x2 + 1)2
9.
tions between the circles will be the points that satisfy
se equations simultaneously. Since the circles are
ric, they either do not intersect and these equations
ution. Or, the circles just touch and there is a unique
r, the circles overlap and there are two solutions.
n of the function f (x) = ( f 1 (x1 , x2 ), f 2 (x1 , x2 )) is
!
2(x1 2) 2(x2 2)
.
(3.36)
J (x) =
2(x1 + 3) 2(x2 + 1)
inant of the Jacobian is
f2 (x1 , x2 ) = 0
(x)) = 4(x1 2)(x2 + 1) 4(x2 2)(x1 + 3),
(3.37)
f 1 ( x 1 , x2 ) = 0
x1
25
0iterations
= f 1 (x1 , x2of) =the
(x1Newton-Raphson
2) + (x2 2) method
9,
first
four
which are
A geometric example
2
2
,
x
)
=
(x
+
3)
+
(x
+
1)
9.
= fa2 (x
converging0 on
point
near
(
0.86,
1.1).
2
1 2
1
Consider the intersection of two circles of radius 3 with centers at
figure
The
convergence
(2,
2)In
and
( 3,??. 1).
Equations
for of
themany
points
(x1points
, x2 )initial
residing
on to
The
intersections
between
the circles
will
bedifferent
the
that iterates
satisfy
the solutions
of circle
the system
of
depicted. Initial iterates
Example:
the
intersection
ofequations
circles isSince
the
surface
of each
are
both
of these
equations
simultaneously.
the circles are
in spaces colored blue converged to the solution in the blue
not concentric, they either do not intersect
and
2
2 these equations
0 = f (x
= (x1 in
2) spaces
+ (x2 colored
2)
9,red converged to
region. Those
initial
1 , x2 )iterates
have no solution.1 Or,
the circles
just touch
and there is a unique
2 color is darkened
2
the solution
inf 2the
red
region.
The
in proportion
(x
,
x
)
=
(x
+
3)
+
(x
+
1)
0
=
2
1 overlap
1 and there2 are two9.
solution. Or, the circles
solutions.
to the number of iterations it took to reach and function norm
Thetolerance
Jacobian of
function
(x)an= initial
( f 1 (xbe
xthe
f 2 (x
10 . Clearly,
2 ), points
2 )) issatisfy
1 , guess
1 , xthat
The
intersections
between
the fcircles
will
of the
10
that
resides
near a
! the circles
bothsolution
of thesetoequations
simultaneously.
the equations
is desirable. Since
However,
such aare
guess
2(x1 2) 2(x2 2)
not must
concentric,
not intersect
and these
equations
Jnear
(x) either
=placesdowhere
.is nearly
(3.36) –
not bethey
the Jacobian
singular
2(x1 + 3) 2(x2 + 1)
havethe
noline,
solution.
touch and there is a unique
3x1 Or,
5x2the
+ 5,circles
in thisjust
case.
solution.
Or, the circles
The determinant
of the overlap
Jacobianand
is there are two solutions.
Newton-Raphson Method
•
The Jacobian of the function f (x) = ( f 1 (x1 , x2 ), f 2 (x1 , x2 )) is
det (J (x)) = 4(x1 2)(x2 + 1) 4(x2 2)(x
(3.37)
!1 + 3),
(k)
(k 1) k
(k) )k
k
x(k)
f2(x
(x(k)
)
k
x
x
k
f
(x
2)
2(x
2)
2
2
2
1
.
(3.36)
J (x) =
2(x11.0)
2(x2 + 1)
1 + 3)
0
( 1.00, 3.00)
(1.00,
11.1
1
( 1.25, 1.75)
(1.63, 1.63)
0.556
2.30
The determinant of the Jacobian is
2 ( 0.963, 1.27) (0.310, 0.310)
0.173
0.439
3 ( det
0.875,
1.124)
0.042
2)(x0.030)
4(x2 0.020
2)(x1 + 3),
(3.37)
(J (x)
) = 4(x1(0.030,
2 + 1)
0.003
0.006
4 ( 0.864, 1.101) (0.004, 0.004)
8
Table 3.1
Newton-R
tersection
26
Figure 3.
•
2
3
4
( 0.963, 1.27) (0.310, 0.310)
0 = f(0.030,
( 0.875, 1.124)
0.030)
2 (x1 , x
2) =
( 0.864, 1.101) (0.004, 0.004)
0.173
(x1 + 0.020
3)2 + (x2
0.003
0.439
2
+ 1)0.042
9.
0.006
Newton-Raphson Method
the intersection
of circles
The Example:
intersections
between the
circles will be the points that satisfy
both of these8 equations simultaneously. Since the circles
Figureare
3.4
circles: (x
not concentric, they either do not intersect and these equations
(x1 + 3)2 +
the stars. R
have no solution. Or, the circles just touch and there is a unique
guesses fo
verged to
solution. Or, the circles overlap and there are two solutions.
in orange
colors ind
x
The Jacobian2 of the function f (x) = ( f 1 (x1 , x2 ), f 2 (x1 , x2 )) isquired dur
J (x) =
2(x1 2) 2(x2 2)
2(x1 + 3) 2(x2 + 1)
!
.
(3.36)
2)(x1 + 3),
(3.37)
8
The determinant
8 of the Jacobian is
8
x1
det (J (x)) = 4(x1
•
2)(x2 + 1)
4(x2
Notice that convergence is slowest near where det J(x) = 0
28
MIT OpenCourseWare
https://ocw.mit.edu
10.34 Numerical Methods Applied to Chemical Engineering
Fall 2015
For information about citing these materials or our Terms of Use, visit: https://ocw.mit.edu/terms.