EE 616
Computer Aided Analysis of Electronic Networks
Lecture 5
Instructor: Dr. J. A. Starzyk, Professor
School of EECS
Ohio University
Athens, OH, 45701
09/19/2005
Note: materials in this lecture are from
the notes of EE219A UC-berkeley
1
http://wwwcad.eecs.berkeley.edu/~nardi/EE219A/contents.html
Outline



Nonlinear problems
Iterative Methods
Newton’s Method
–
–
–
–

Multidimensonal Newton Method
–
–
–
2
Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing
Basic Algorithm
Quadratic convergence
Application to circuits
DC Analysis of Nonlinear Circuits - Example
1
Vd
I1
I
Ir
I d  I s (e
Vt
 1)  0
d
0
Need to Solve
I r  I d  I1  0
e1
Vt
1
e1  I s (e  1)  I1  0
R
3
g (e1 )  I1
Nonlinear Equations


Given g(V)=I
It can be expressed as: f(V)=g(V)-I
 Solve g(V)=I equivalent to solve f(V)=0
Hard to find analytical solution for f(x)=0
Solve iteratively
4
Nonlinear Equations – Iterative Methods



Start from an initial value x0
Generate a sequence of iterate xn-1, xn, xn+1
which hopefully converges to the solution x*
Iterates are generated according to an
iteration function F: xn+1=F(xn)
Ask
• When does it converge to correct solution ?
• What is the convergence rate ?
5
Newton-Raphson (NR) Method
Consists of linearizing the system.
Want to solve f(x)=0  Replace f(x) with its linearized version
and solve.
df *
f ( x)  f ( x )  ( x )( x  x* )
dx
df k k 1 k
k 1
k
f ( x )  f ( x )  ( x )( x  x )
dx
*
Taylor Ser ies
1
x
k 1
 df k 
 x   ( x ) f ( x k )
 dx

k
Iteration function
Note: at each step need to evaluate f and f’
6
Newton-Raphson Method – Graphical View
7
Newton-Raphson Method – Algorithm
Define iteration
Do k = 0 to ….?
1
x
k 1
 df k 
 x   ( x ) f ( x k )
 dx

k
until convergence


8
How about convergence?
An iteration {x(k)} is said to converge with order q if there exists
a vector norm such that for each k  N:
x
k 1
 xˆ  x  xˆ
k
q
Newton-Raphson Method – Convergence
2
df
d
f
*
k
k
*
k
*
k 2
0  f ( x )  f ( x )  ( x )( x  x )  2 ( x)( x  x )
dx
dx
some x  [ x k , x* ]
Mean Value theorem
truncates Taylor series
But
df k k 1 k
0  f ( x )  ( x )( x  x )
dx
k
9
by Newton
definition
Newton-Raphson Method – Convergence
Subtracting
df k k 1 * d 2 f
( x )( x  x )  2 ( x)( x k  x* )2
dx
d x
2
df
d
f
k 1
*
k 1
k
* 2
( x  x )  [ ( x )]
(
x
)(
x

x
)
2
dx
d x
Dividing
df k 1 d 2 f
k
Let [ ( x )]
(
x
)

K
2
dx
d x
then x
10
k 1
x K x x
*
k
k
* 2
Convergence is quadratic
Newton-Raphson Method – Convergence
Local Convergence Theorem
If
df
a)
dx
d2 f
b)
dx 2

bounded away from zero 

 K is bounded

bounded

Then Newton’s method converges given a
sufficiently close initial guess
(and convergence is quadratic)
11
Newton-Raphson Method – Convergence
x = Initial Guess, k  0
0
Repeat {
 
f x k
x


 
x k 1  x k   f x k
k  k 1
} Until ?
x k 1  x k  threshold ?
12


f x k 1  threshold ?
Newton-Raphson Method – Convergence Check
False convergence since increment in f small but with large increment in X
Need a "delta-x" check to avoid false convergence
f(x)
x
x
k 1
x
k 1
 x   xa   xr x
k
k
x


f x k 1   fa
13
k 1
*
X
Newton-Raphson Method – Convergence Check
False convergence since increment in x small but with large increment in f
Also need an "f  x  " check to avoid false convergence
f(x)


f x k 1   fa
x*
14
x k 1 x k
x k 1  x k   xa   xr x k 1
X
Newton-Raphson Method – Convergence
15
Newton-Raphson Method – Local Convergence
Convergence Depends on a Good Initial Guess
f(x)
1
x
1
x
x
16
2
x
0
x
0
X
Newton-Raphson Method – Local Convergence
Convergence Depends on a Good Initial Guess
17
Nonlinear Problems – Multidimensional Example
v1
i2 + v2b -
i3
i1
Nodal Analysis
+ At Node 1: i1  i2  0
+
v1b
v3b
-
Nonlinear
Resistors
i  g v
18
v2
 g  v1   g  v1  v2   0
At Node 2: i3  i2  0
 g  v3   g  v1  v2   0
Two coupled
nonlinear equations
in two unknowns
Multidimensional Newton Method
 
Problem: Find x such that F x  0
*
F ( x)  F ( x* )  J ( x* )( x  x* )
19
F1 ( x) 
 F1 ( x)

 x


x
1
N


J ( x)   

 
 FN ( x)  FN ( x) 
 x1
x N 


 x k 1  x k  J ( x k ) 1 F ( x k )
*
Taylor Ser ies
Jacobian Matrix
Iteration function
Multidimensional Newton Method – Computational Aspects
Iteration : x
k 1
k 1
 x  J (x ) F (x )
k
k
k 1
Do not compute J ( x ) (it is not sparse).
Instead solve :
J ( x k )( x k 1  x k )   F ( x k )
Each iteration requires:
1. Evaluation of F(xk)
2. Computation of J(xk)
3. Solution of a linear system of algebraic
equations whose coefficient matrix is J(xk) and
whose RHS is -F(xk)
20
Multidimensional Newton Method – Algorithm
x = Initial Guess, k  0
0
Repeat {
   
 x  x  x    F  x  for x
Compute F x k , J F x k
Solve J F
k
k 1
k
k
k 1
k  k 1
} Until
21
x k 1  x k ,

f x k 1

small enough
Multidimensional Newton Method – Convergence
Local Convergence Theorem
If
 
a) J F1 x k  
 Inverse is bounded 
b) J F  x   J F  y  
x y
 Derivative is Lipschitz Cont 
Then Newton’s method converges given a
sufficiently close initial guess
(and convergence is quadratic)
22
Application of NR to Circuit Equations
Companion Network

Applying NR to the system of equations we
find that at iteration k+1:
–
–
all the coefficients of KCL, KVL and of BCE of the
linear elements remain unchanged with respect to
iteration k
Nonlinear elements are represented by a
linearization of BCE around iteration k
 This system of equations can be interpreted
as the STA of a linear circuit (companion
network) whose elements are specified by
the linearized BCE.
23
Application of NR to Circuit Equations
Companion Network


24
General procedure: the NR method applied to
a nonlinear circuit (whose eqns are
formulated in the STA form) produces at each
iteration the STA eqns of a linear resistive
circuit obtained by linearizing the BCE of the
nonlinear elements and leaving all the other
BCE unmodified
After the linear circuit is produced, there is no
need to stick to STA, but other methods (such
as MNA) may be used to assemble the circuit
eqns
Application of NR to Circuit Equations
Companion Network – MNA templates
25
Note: G0 and Id depend on the iteration count k
 G0=G0(k) and Id=Id(k)
Application of NR to Circuit Equations
Companion Network – MNA templates
26
Modeling a MOSFET
(MOS Level 1, linear regime)
d
27
Modeling a MOSFET
(MOS Level 1, linear regime)
gds
28
DC Analysis Flow Diagram
For each state variable in the system
29
Implications




Device model equations must be continuous with
continuous derivatives and derivative calculation must be
accurate derivative of function
Watch out for floating nodes (If a node becomes
disconnected, then J(x) is singular)
Give good initial guess for x(0)
Most model computations produce errors in function
values and derivatives.
–
30
Want to have convergence criteria || x(k+1) - x(k) || <  such that  >
than model errors.
Summary



Nonlinear problems
Iterative Methods
Newton’s Method
–
–
–
–

Multidimensional Newton Method
–
–
–
31
Derivation of Newton
Quadratic Convergence
Examples
Convergence Testing
Basic Algorithm
Quadratic convergence
Application to circuits
Improving convergence


Improve Models (80% of problems)
Improve Algorithms (20% of problems)
Focus on new algorithms:
Limiting Schemes
Continuations Schemes
32
Outline

Limiting Schemes
–
–
Direction Corrupting
Non corrupting (Damped Newton)



Continuation Schemes
–
–
–
Source stepping
More General Continuation Scheme
Improving Efficiency

33
Globally Convergent if Jacobian is Nonsingular
Difficulty with Singular Jacobians
Better first guess for each continuation step
Multidimensional Newton Method
Convergence Problems – Local Minimum
Local
Minimum
34
f
At a local minimum,
0
x
Multidimensional Case: J F  x  is singular
Multidimensional Newton Method
Convergence Problems – Nearly singular
f(x)
1
x
x
35
Must Somehow Limit the changes in X
0
X
Multidimensional Newton Method
Convergence Problems - Overflow
f(x)
x
36
0
Must Somehow Limit the changes in f(x)
X
1
x
Newton Method with Limiting
Newton Algorithm for Solving F  x   0
x = Initial Guess, k  0
Repeat {
0
   
 x  x  F  x  for x
 limited  x 
Compute F x k , J F x k
Solve J F
x k 1  x k
}
37
k
k 1
k
k 1
k 1
k  k 1
k 1
k 1

x
,
F
x

 small enough
Until
Newton Method with Limiting
Limiting Methods
• Direction Corrupting


limited x k 1 
i
xik 1 if xik 1  
 sign  xik 1  otherwise
• NonCorrupting


limited x k 1  x k 1


  min 1, k 1
 x
38

limited x k 1




x k 1

Scales differently
different dimensions
Scales proportionally
In all dimensions

limited x
k 1

Heuristics, No Guarantee of Global Convergence

x k 1
Newton Method with Limiting
Damped Newton Scheme
General Damping Scheme
 
 
Solve J F x k x k 1   F x k for x k 1
x k 1  x k   k x k 1
Key Idea: Line Search

Pick  to minimize F x   x
k

F x   x
39
k
k
k 1

2
2

k
k
 F x   x
k
k
k 1
k 1

2
2
 
T
F x k   k x k 1
Method Performs a one-dimensional search in
Newton Direction

Newton Method with Limiting
Damped Newton – Convergence Theorem
If
 
 Inverse is bounded 
a) J F1 x k  
b) J F  x   J F  y  
x y
 Derivative is Lipschitz Cont 
Then
There exists a set of  k ' s   0,1 such that



F x k 1  F x k   k x k 1
40

 
  F xk
with  <1
Every Step reduces F-- Global Convergence!
Newton Method with Limiting
Damped Newton – Singular Jacobian Problem
x2
1
x
1
D
x
x
0
X
Damped Newton Methods “push” iterates to local minimums
41
Finds the points where Jacobian is Singular
Newton with Continuation schemes
Basic Concepts - General setting
Newton converges given a close initial guess
 Idea: Generate a sequence of problems, such that a
problem is a good initial guess for the following one
F (x)  0  Solve F  x    ,    0 where:
a) F  x  0  , 0   0 is easy to solve  Starts the continuation
b) F  x 1 ,1  F  x 
Ends the continuation
c) x    is sufficiently smooth Hard to insure!
42
Newton with Continuation schemes
Basic Concepts – Template Algorithm
Solve F  x  0  , 0  , x   prev   x  0 
While   1 {
x0     x   prev 
Try to Solve F  x    ,    0 with Newton
}
43
If Newton Converged
x   prev   x    ,      ,   2
Else
1
   ,    prev  
2
Newton with Continuation schemes
Basic Concepts – Source Stepping Example
44
Newton with Continuation schemes
Basic Concepts – Source Stepping Example
R
Vs
+
-
v
1
f  v    ,    idiode  v    v  Vs   0
R
Diode
f  v,  
v
45

idiode  v 
v

1
 Not  dependent!
R
Source Stepping Does Not Alter Jacobian
Newton with Continuation schemes
Jacobian Altering Scheme
F  x    ,     F  x      1    x   
Observations
 =0 F  x  0  , 0   x  0   0
F  x  0  , 0 
x
I
Problem is easy to solve and
Jacobian definitely
nonsingular.
 =1 F  x 1 ,1  F  x 1 
F  x (1),1)
0 , 0
46
x

F  x 1 
x
Back to the original problem
and original Jacobian
Summary

Newton’s Method works fine:
–

given a close enough initial guess
In case Newton does not converge:
–
Limiting Schemes


Direction Corrupting
Non corrupting (Damped Newton)
–
Globally Convergent if Jacobian is Nonsingular
– Difficulty with Singular Jacobians
–
Continuation Schemes



47
Source stepping
More General Continuation Scheme
Improving Efficiency
–
Better first guess for each continuation step