Solutions for Nonlinear Equations
Lecture 8
Alessandra Nardi
Thanks to Prof. Newton, Prof. Sangiovanni, Prof. White, Jaime Peraire,
Deepak Ramaswamy, Michal Rewienski, and Karen Veroy
Outline
• Nonlinear problems
• Iterative Methods
• Newton’s Method
–
–
–
–
Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing
• Multidimensonal Newton Method
– Basic Algorithm
– Quadratic convergence
– Application to circuits
Nonlinear Problems - Example
1
Vd
I1
Id
Ir
I d  I s (e
0
Need to Solve
I r  I d  I1  0
e1
Vt
1
e1  I s (e  1)  I1  0
R
g (e1 )  I1
Vt
 1)  0
Nonlinear Equations
• Given g(V)=I
• It can be expressed as: f(V)=g(V)-I
 Solve g(V)=I equivalent to solve f(V)=0
Hard to find analytical solution for f(x)=0
Solve iteratively
Nonlinear Equations – Iterative Methods
• Start from an initial value x0
• Generate a sequence of iterate xn-1, xn, xn+1
which hopefully converges to the solution x*
• Iterates are generated according to an iteration
function F: xn+1=F(xn)
Ask
• When does it converge to correct solution ?
• What is the convergence rate ?
Newton-Raphson (NR) Method
Consists of linearizing the system.
Want to solve f(x)=0  Replace f(x) with its linearized
version and solve.
df *
f ( x)  f ( x )  ( x )( x  x* )
dx
df k k 1 k
k 1
k
f ( x )  f ( x )  ( x )( x  x )
dx
*
Taylor Ser ies
1
x
k 1
 df k 
k
 x   ( x ) f ( x )
 dx

k
Iteration function
Note: at each step need to evaluate f and f’
Newton-Raphson Method – Graphical View
Newton-Raphson Method – Algorithm
Define iteration
Do k = 0 to ….
1
x
k 1
 df k 
 x   ( x ) f ( x k )
 dx

k
until convergence
• How about convergence?
• An iteration {x(k)} is said to converge with order q if there
exists a vector norm such that for each k  N:
x
k 1
 xˆ  x  xˆ
k
q
Newton-Raphson Method – Convergence
2
df
d
f
*
k
k
*
k
0  f ( x )  f ( x )  ( x )( x  x )  2 ( x)( x*  x k ) 2
dx
dx
some x  [ x k , x* ]
Mean Value theorem
truncates Taylor series
But
df k k 1 k
0  f ( x )  ( x )( x  x )
dx
k
by Newton
definition
Newton-Raphson Method – Convergence
df k k 1 * d 2 f
k
* 2
( x )( x  x )  2 ( x)( x  x )
Subtracting
dx
d x
Dividing through
2
df
d
f
k 1
*
k 1
k
* 2
( x  x )  [ ( x )]
(
x
)(
x

x
)
2
dx
d x
df k 1 d 2 f
k
Let [ ( x )]
(
x
)

K
2
dx
d x
then x
k 1
x K x x
*
k
k
* 2
Convergence is quadratic
Newton-Raphson Method – Convergence
Local Convergence Theorem
If
df
a)
dx
d2 f
b)
2
dx

bounded away from zero 

 K is bounded

bounded

Then Newton’s method converges given a
sufficiently close initial guess (and
convergence is quadratic)
Newton-Raphson Method – Convergence
Example 1
f ( x)  x 2  1  0,
df k
( x )  2 xk
dx
find x ( x*  1)


2

k
k 1
x ) x
k
k 1
 x )  2x (x  x )   x
2x (x
2x (x
or ( x
k 1
k
*
k
k
*
1
k
1
 x )  k ( x k  x* ) 2
2x

k
 x  
2
*
2
*
Convergence is quadratic
Newton-Raphson Method – Convergence
Example 2
f ( x)  x  0, x  0
df k
( x )  2 xk
dx
k
k 1
k
2
 2 x ( x  0)  ( x  0)
2
*

1
 df 
 not bounded
 dx 
Note : 
away from zero

1 k
x 0  x 0
for x k  x*  0
2
1
*
*
or ( xk 1  x )  ( xk  x )
2
k 1
Convergence is linear
Newton-Raphson Method – Convergence
Example 1, 2
Newton-Raphson Method – Convergence
x = Initial Guess, k  0
0
Repeat {
 
f x k
x


 
x k 1  x k   f x k
k  k 1
} Until ?
x
k 1
 x  threshold ?
k

f x
k 1
  threshold ?
Newton-Raphson Method – Convergence
Convergence Checks
Need a "delta-x" check to avoid false convergence
f(x)
x
x
k 1
x
k 1
 x   xa   xr x
k
k
x


f x k 1   fa
k 1
*
X
Newton-Raphson Method – Convergence
Convergence Checks
Also need an "f  x  " check to avoid false convergence
f(x)


f x k 1   fa
x
x
k 1
*
X
x k 1 x k
 x   xa   xr x
k
k 1
Newton-Raphson Method – Convergence
demo2
Newton-Raphson Method – Convergence
Local Convergence
Convergence Depends on a Good Initial Guess
f(x)
1
x
1
x
x
2
x
0
x
0
X
Newton-Raphson Method – Convergence
Local Convergence
Convergence Depends on a Good Initial Guess
Nonlinear Problems – Multidimensional Example
v1
i2 + v2b -
v2
Nodal Analysis
i3
i1
+
+
b
1
b
3
v
v
-
Nonlinear
Resistors
i  g v
At Node 1: i1  i2  0
 g  v1   g  v1  v2   0
At Node 2: i3  i2  0
 g  v3   g  v1  v2   0
Two coupled
nonlinear equations
in two unknowns
Multidimensional Newton Method
 
Problem: Find x such that F x  0
*
x 
*
N
*
and F :
F ( x)  F ( x )  J ( x )( x  x )
*
*
*
F1 ( x) 
 F1 ( x)

 x


x
1
N


J ( x)   

 
 FN ( x)  FN ( x) 
 x1
x N 


k 1
k
k 1
k
 x  x  J (x ) F (x )
N

N
Taylor Ser ies
Jacobian Matrix
Iteration function
Multidimensional Newton Method
Computational Aspects
Iteration : x k 1  x k  J ( x k ) 1 F ( x k )
k 1
Do not compute J ( x ) (it is not sparse).
Instead solve :
J ( x k )( x k 1  x k )   F ( x k )
Each iteration requires:
1. Evaluation of F(xk)
2. Computation of J(xk)
3. Solution of a linear system of algebraic
equations whose coefficient matrix is J(xk)
and whose RHS is -F(xk)
Multidimensional Newton Method
Algorithm
x = Initial Guess, k  0
0
Repeat {
   
 x  x  x    F  x  for x
Compute F x k , J F x k
Solve J F
k
k 1
k
k
k 1
k  k 1
} Until
x k 1  x k ,

f x k 1

small enough
Multidimensional Newton Method
Convergence
Local Convergence Theorem
If
 
a) J F1 x k  
 Inverse is bounded 
b) J F  x   J F  y  
x y
 Derivative is Lipschitz Cont 
Then Newton’s method converges given a
sufficiently close initial guess (and
convergence is quadratic)
Application of NR to Circuit Equations
Companion Network
• Applying NR to the system of equations we
find that at iteration k+1:
– all the coefficients of KCL, KVL and of BCE of
the linear elements remain unchanged with respect
to iteration k
– Nonlinear elements are represented by a
linearization of BCE around iteration k
 This system of equations can be interpreted as
the STA of a linear circuit (companion
network) whose elements are specified by the
linearized BCE.
Application of NR to Circuit Equations
Companion Network
• General procedure: the NR method applied to a
nonlinear circuit whose eqns are formulated in
the STA form produces at each iteration the
STA eqns of a linear resistive circuit obtained
by linearizing the BCE of the nonlinear
elements and leaving all the other BCE
unmodified
• After the linear circuit is produced, there is no
need to stick to STA, but other methods (such
as MNA) may be used to assemble the circuit
eqns
Application of NR to Circuit Equations
Companion Network – MNA templates
Note: G0 and Id depend on the iteration count k
 G0=G0(k) and Id=Id(k)
Application of NR to Circuit Equations
Companion Network – MNA templates
Modeling a MOSFET
(MOS Level 1, linear regime)
d
Modeling a MOSFET
(MOS Level 1, linear regime)
DC Analysis Flow Diagram
For each state variable in the system
Implications
• Device model equations must be continuous with
continuous derivatives and derivative calculation must
be accurate derivative of function (not all models do
this - Poor diode models and breakdown models don’t be sure models are decent - beware of user-supplied
models)
• Watch out for floating nodes (If a node becomes
disconnected, then J(x) is singular)
• Give good initial guess for x(0)
• Most model computations produce errors in function
values and derivatives. Want to have convergence
criteria || x(k+1) - x(k) || <  such that  > than model
errors.
Summary
• Nonlinear problems
• Iterative Methods
• Newton’s Method
–
–
–
–
Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing
• Multidimensonal Newton Method
– Basic Algorithm
– Quadratic convergence
– Application to circuits