Computer Aided Verification of Electronic Circuits and Systems
EE219A: Fall 2002
Professor: Alberto Sangiovanni-Vincentelli
Instructor: Alessandra Nardi
LECTURE NOTES WRITTEN BY THE CLASS STUDENTS
Please be aware that these notes have not been reviewed.
Please e-mail any comments, bugs, typos to: nardi@eecs.berkeley.edu
Circuit Equation Formulation and Gaussian Elimination
(Lectures 2-3-4)
Students: Guang Yang, Yanmei Li
QR Factorization and Iterative Methods for solving linear systems
(Lectures 5-6-7)
Students: Gerald Wang, Haibo Zeng
Solution of Nonlinear Equations
(Lectures 8-9)
Students: Farhana Sheikh, Joshua Garrett
Solution of Ordinary Differential Equations
(Lectures 10-11-12)
Students: Douglas Densmore, Rui Xu, Jian Yang
Analog Circuit Simulation
(Lectures 17-18-19-20)
Students: Cong Liu, Frank Gennari, Shiying Xiong
2
Contents
I Fundamentals of Circuit Simulation
7
1
Formulation of Circuit Equations
1.1 Sparse Tableau Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Modified Nodal Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . .
8
9
10
2
Solving Systems of Linear Equations
2.1 Direct Methods . . . . . . . . . . . . . . . . . . . . . .
2.1.1 Gaussian Elimination . . . . . . . . . . . . . . .
2.1.2 QR Factorization . . . . . . . . . . . . . . . . .
2.2 Iterative Methods . . . . . . . . . . . . . . . . . . . . .
2.2.1 Stationary Methods . . . . . . . . . . . . . . . .
2.2.2 Non-Stationary Methods . . . . . . . . . . . . .
2.3 Prelimenaries . . . . . . . . . . . . . . . . . . . . . . .
2.4 Congugate Gradient Method . . . . . . . . . . . . . . .
2.4.1 Arbitrary Subspace Methods . . . . . . . . . . .
2.4.2 Krylov Subspace Methods . . . . . . . . . . . .
2.4.3 Krylov Subspace Methods Convergence Analysis
.
.
.
.
.
.
.
.
.
.
.
13
13
13
21
22
22
24
24
25
25
26
26
.
.
.
.
.
.
.
.
.
.
.
29
29
30
31
32
33
33
34
35
36
36
37
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Solving Systems of Non-Linear Equations
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . .
3.3 Newton-Raphson (NR) Method . . . . . . . . . . . . . . . .
3.3.1 Convergence . . . . . . . . . . . . . . . . . . . . .
3.3.2 Generalization of NR to Multidimensional Case . . .
3.3.3 Application of NR to Circuits . . . . . . . . . . . .
3.3.4 Example 1 . . . . . . . . . . . . . . . . . . . . . .
3.3.5 Example 2 . . . . . . . . . . . . . . . . . . . . . .
3.4 Modified Newton Methods . . . . . . . . . . . . . . . . . .
3.4.1 Limiting Schemes and the Damped Newton Method
3.4.2 Continuation Schemes . . . . . . . . . . . . . . . .
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4
Solving Ordinary Differential Equations
4.1 Lecture 12: Large Timestep Issues . . . . . . . . . . . . . . . . . .
4.1.1 Last Lecture Review . . . . . . . . . . . . . . . . . . . . .
4.1.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.3 Multistep Methods-Common Algorithms . . . . . . . . . .
4.1.4 Multistep Methods-Convergence Analysis . . . . . . . . . .
4.1.5 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.6 Aside on Solving Difference Equations . . . . . . . . . . .
4.1.7 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.8 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.9 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.10 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.11 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.12 Multistep Methods - Stability . . . . . . . . . . . . . . . .
4.1.13 Dahlquists Stability Barriers . . . . . . . . . . . . . . . . .
4.1.14 Multistep Methods - Convergence Analysis . . . . . . . . .
4.1.15 FE region of Stability . . . . . . . . . . . . . . . . . . . . .
4.1.16 BE region of Stability . . . . . . . . . . . . . . . . . . . .
4.1.17 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1.18 Stiff Problems - Large Timestep Issues . . . . . . . . . . .
4.1.19 Stiff Problems - Large Timesetp Issues . . . . . . . . . . .
4.1.20 Application Problem . . . . . . . . . . . . . . . . . . . . .
4.1.21 FE on two time-constant circuit . . . . . . . . . . . . . . .
4.1.22 BE on two time-constant circuit . . . . . . . . . . . . . . .
4.1.23 Multistep Methods . . . . . . . . . . . . . . . . . . . . . .
4.1.24 Regions of Stability and Stiff Problems . . . . . . . . . . .
4.1.25 Stiff Problems . . . . . . . . . . . . . . . . . . . . . . . .
4.1.26 Backward Differentiation Methods - BDF . . . . . . . . . .
4.1.27 Gear’s region of stability . . . . . . . . . . . . . . . . . . .
4.1.28 Variable Step Size . . . . . . . . . . . . . . . . . . . . . .
4.1.29 Variable Step Size . . . . . . . . . . . . . . . . . . . . . .
4.1.30 More Observations . . . . . . . . . . . . . . . . . . . . . .
4.1.31 Summary on Stiff Stability . . . . . . . . . . . . . . . . . .
4.1.32 Multistep Methods - More on Implicit Methods . . . . . . .
4.1.33 Multistep Methods - Solution with Newton . . . . . . . . .
4.1.34 Application of linear multistep methods to circuit equations
4.1.35 Transcient Analysis Flow . . . . . . . . . . . . . . . . . . .
4.1.36 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
39
52
52
52
53
53
54
54
54
54
55
55
55
55
56
56
56
56
56
57
57
57
57
58
58
58
58
58
59
59
59
59
60
60
60
60
60
61
II Analog Simulation
62
5
.
.
.
.
.
.
.
.
.
63
63
63
64
64
64
65
66
71
73
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
75
75
76
76
78
78
78
78
79
79
80
80
81
82
82
83
83
84
85
85
85
86
86
87
87
87
6
Mixed-signal Behavioral Modeling with Verilog-AMS
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.1.1 Top-down Design Methodology for Mixed-Signal Design
5.1.2 Mixed-Signal Behavioral Modeling . . . . . . . . . . . .
5.2 Behavioral Modeling with Verilog . . . . . . . . . . . . . . . . .
5.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . .
5.2.2 Digital Signal Domain . . . . . . . . . . . . . . . . . . .
5.2.3 Analog Signal Domain . . . . . . . . . . . . . . . . . . .
5.2.4 Mixed Signal Domain . . . . . . . . . . . . . . . . . . .
5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Steady State Analysis
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Steady State . . . . . . . . . . . . . . . . . . . . . . . . .
6.3 Steady State Methods . . . . . . . . . . . . . . . . . . . .
6.4 Steady State Analysis . . . . . . . . . . . . . . . . . . . .
6.4.1 AC Linear Analysis . . . . . . . . . . . . . . . . .
6.4.2 AC Small Signal Analysis for Non-Linear Circuits
6.4.3 Finding the Poles and Zeros of Linear Circuits . .
6.4.4 Noise Analysis . . . . . . . . . . . . . . . . . . .
6.4.5 Distortion Analysis . . . . . . . . . . . . . . . . .
6.4.6 Frequency Translation Analysis . . . . . . . . . .
6.5 Periodic Steady State Analysis . . . . . . . . . . . . . . .
6.5.1 PSS Algorithm #1: Harmonic Balance . . . . . . .
6.6 Periodic Steady State: Finite Differences . . . . . . . . . .
6.6.1 The Algorithm . . . . . . . . . . . . . . . . . . .
6.6.2 GMRES . . . . . . . . . . . . . . . . . . . . . . .
6.6.3 Preconditioning . . . . . . . . . . . . . . . . . . .
6.6.4 PSS-FD Algorithm Outline . . . . . . . . . . . . .
6.7 The Shooting Method . . . . . . . . . . . . . . . . . . . .
6.7.1 Transition Function . . . . . . . . . . . . . . . . .
6.7.2 Sensitivity Analysis . . . . . . . . . . . . . . . .
6.7.3 Shooting Algorithm Outline . . . . . . . . . . . .
6.8 Algorithm Comparison . . . . . . . . . . . . . . . . . . .
6.8.1 AC vs. PSS . . . . . . . . . . . . . . . . . . . . .
6.8.2 Shooting vs. PSS-FD . . . . . . . . . . . . . . . .
6.8.3 GMRES Convergence Analysis . . . . . . . . . .
5
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6.9
Other Topics . . . . . . . . . . . . . . . . . . .
6.9.1 Multi-Interval Chebyshev Discretization
6.9.2 Small Signal Analysis . . . . . . . . .
6.9.3 Cyclostationary Noise . . . . . . . . .
6.9.4 Other Analyses . . . . . . . . . . . . .
6.10 Acronyms . . . . . . . . . . . . . . . . . . . .
6.11 References . . . . . . . . . . . . . . . . . . . .
6
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
88
88
88
88
88
89
89
Part I
Fundamentals of Circuit Simulation
7
Chapter 1
Formulation of Circuit Equations
When applying a simulator to a circuit, the circuit description, inputs and setup conditions
are fed into the simulator. Then, these information are transformed into a set of equations
according to analysis requirements. Typically, the circuit analysis includes DC analysis,
transient analysis, AC analysis, as well as noise, distorsion and sensitivity analysis. Based
on different circuits, the circuit equations may be a set of mixed linear equations, nonlinear
equations, and ordinary differential equations. Usually, numerical techniques are applied to
solve these circuit equations. This chapter will introduce the formulation method of circuit
equations.
For a circuit with branches and
nodes(one node is the reference node), the set of
unknowns will include branch currents( ), node voltages( ), and branch voltages( ).
And the set of equations will include
Conservation Laws, and Branch Constitutive Equations.
The Branch Constitutive Equations(BCEs) are determined by the mathematical model
of the electrical behavior of a component, e.g. for a linear resistor,
. In most of circuit simulators, this mathematical model is expressed in terms of ideal elements. Generally,
the ideal elements are divided into two types: two-terminal ideal elements like resistors,
capacitors, inductors, and independent sources, and two-port ideal elements like dependent
sources. The branch voltages and currents are measured according to the associated reference directions(refer to slide 9 of lecture 2). Also, a reference node should be defined.
The BCE for different two-terminal and two-port ideal elements are shown in Table 1.1.
Generally, any mathematical model of an electronic device can be represented by an interconnection of these ideal elements.
The Conservation Laws are determined by the topology of the circuit, including Kirchoff Current Law (KCL) and Kirchoff Voltage Law (KVL). KVL describes the relation
between the branch voltages and the node voltages. Every circuit node has a unique voltage
with respect to the reference node. The voltage across a branch is equal to the difference
between the positive and negative referenced voltages of the nodes on which it is incident.
8
Element
Resistor
Capacitor
Inductor
Voltage Source
Current Source
VCVS
VCCS
CCVS
CCCS
BCE
!
"$#%&
')($*+ & ,)-*. & )"-/'%0& !
Table 1.1: Branch Constitutive Equations for Ideal Elements
KCL specifies the relations of the branch currents at the node to be analyzed. The algebraic
sum of all the currents flowing out of (or into) any circuit node is zero. An example is given
in the slides 12,13,14 of lecture 2.
For a circuit with branches and
nodes, there will be KCL equations, and
KVL equations. KCL and KVL can be written by using of the reduced incidence matrix 1
as follows:
1 32
45176 82
where is the vector of branch currents, is the vector of branch voltages, and
vector of node voltages. There will be branch equations:
9;:
9 :
?>
=<
is the
where
is a
submatrics containing the coefficients of the branch equations and <
is the vector of independent current sources.
KCL, KVL and BCE completely characterize the electrical behavior of a circuit. There
are different ways to combine KCL,KVL and BCE. In the following section, we will introduce two popular methods for formulating circuit equations: Sparse Tableau Analysis
(STA) and Modified Nodal Analysis (MNA).
1.1 Sparse Tableau Analysis
The Sparse Tableau Analysis method is a fundamental method of formulating circuit equations. Other methods, such as Modified Nodal Analysis, can be derived from it. In STA
method, KCL, KVL and BCE are written as follows.
32
451 6 82
1
9
9
9 :
>
The coefficient matrix of the STA equations has dimension since
there are equations( KCL equations, KVL equations,and branch equations)
and unknowns( branch currents, branch voltages and node voltages).
1
9 :
9
>
9
2
2
9;:
2
471 6
2
2
2
where
and
are
submatrices containing the coefficients of the branch equations,
is the vector of independent current and voltage sources.
The main advantages of STA are:
1. STA can be applied to any circuit containing all the basic ideal elements.
2. STA can be assembled directly from the input data.
3. The coefficient matrix is very sparse. It is a very important structural property when
solving the equations.
One of the main disadvantages is that in order to fully exploit the sparsity of the coefficient matrix, sophisticated programming techniques and data structures are required for
time and memory efficiency.
1.2 Modified Nodal Analysis
Nodal Analysis
Modified Nodal Analysis(MNA) comes from the Nodal Analysis method. In Nodal Analysis method, KCL at the nodes of the circuit are written first: 1
2 . Then, the BCE are
used to express the branch currents in terms of the branch voltages: . After that,
KVL are used to relate the branch voltages to node voltages: . Finally, we obtain
the equations like:
<
> where
is the nodal matrix, and < is the right hand side vector(RHS). Totally,
we have equations and node voltages as unknowns. An example is shown in slide 20
of lecture 2.
Actually, the NA matrix can be derived from STA. The main advantage of NA is that
can be obtained by direct inspection. To do that, stamping procedure can be applied, where
stamps are used
as templates to place data into the matrix. Each element has an associated
is the composition of all the elements’ stamps. Here, we employ the resistor
stamp, and
10
stamp as an example to explain how to derive the stamp of an element. Suppose the SPICE
input format is:
4
Its branch current will appear in only two node equations, i.e. KCL at node
at node 4 . After BCE and KVL substitutions, the KCL equations at node
are:
< 4 < 4 4 We can find that
4
4
=<
contributes to only four entries in
and KCL
and 4
=<
as shown below.
Obviously, a resistor will only contribute to the diagonal if it is connected to ground.
Similarly, the stamps of independent current source and VCCS can be obtained(refer to
slide 23,24).
The main advantages of NA are:
1. The circuit equations can be assembled directly from input data by inspection.
2.
has non-zero diagonal entries.
3.
4.
!#"%$ &(') &+'* ! ,& ! ) and symmetric.
is often diagonally dominant(i.e. !
is sparse even though it is not as sparse as the coefficient matrix of STA. And it is
>
> more compact (
) than STA ( ).
Unfortunately, NA can not handle the circuits containing independent voltage sources,
VCVS, CCCS, and CCVS, because their branch equations do not provide a branch current
versus branch voltage relation, and consequently the substitution of branch currents with
branch voltages can not be accomplished.
Modified Nodal Analysis
MNA was introduced to deal with these problems associated with NA. MNA equations can
be derived as follows. First, KCL equations are built in the same way as in NA. And branch
equations are used to subsitute as many branch currents as possible with branch voltages.
Note that there may be some branch currents which can not be substituted with branch
voltages using BCE. Then, the branch equations which are not used in the previous steps
are appended. Finally, all the branch voltages are substituted with the node voltages using
11
KVL. An example shows this process clearly in slides 31˜34. The augmented nodal matrix
for MNA is shown below.
2
where is some branch currents, and
represents voltage sources.
When discussing NA method, a stamping procedure can be employed to obtain the coefficient matrix directly from the input data. Similarly, the coefficient matrix and the right
hand side of MNA can also be derived directly from the circuit by composing all elements’
stamps. From the above matrix, we can find the MNA matrix consists of two parts. One
part is same as the conventional NA, which contains the elements like resistors, independent current sources, and VCCS, while the other part is generated by the elements that do
not have admittance representations, e.g. independent voltage source, VCVS, CCCS, and
CCVS. How to deal with this kind of elements? Suppose the SPICE input for an independent voltage source
between the node
and the node :
Its current can not be explicitly expressed in terms of node voltages. So, the branch current
independent voltage source has to be added as an unknown. That means, a new column in
of
is added. Consequently, node voltages and are not independent
variables anymore,
and a new constraint has to be added. That causes a new row in . The resulting stamp for
the independent voltage source is shown below. Note that it also contributes to the RHS.
4
4
2
Similiarly, we can obtain the stamps for CCCS and CCVS(refer to slide 30). In general,
a branch current is always be introduced as an additional variable for a voltage source or
an inductor. For current sources, resistors, conductors, and capacitors, the branch current is
introduced only if any circuit element depends on that branch current or that branch current
is requested as output.
The main advantages of MNA are:
1. MNA can be applied to any circuits with the ideal elements.
2. The MNA coefficient matrix and the right hand side can be assembled directly from
input data by inspection.
3. The MNA coefficient matrix has a zero-nonzero stucture similar to
is sparse.
of NA, which
But one possible problem is sometimes we still have zeros on the main diagonal.
12
Chapter 2
Solving Systems of Linear Equations
2.1 Direct Methods
2.1.1 Gaussian Elimination
Systems of Linear Equations
After formulating the circuit equations with either STA or MNA method, the next step would
. Generally, given a linear system, we are
be solving the system of linear equations,
interested in the existence and uniqueness of the solution. To answer these two questions,
let’s look at the linear equations
from another point of view.
.
Matrix could be regarded as the composition of a set of column vectors,
such that
So, solving the linear equations is same as finding a set of weights the weighted sum of the column vectors is equal to the RHS vector :
Suppose that the matrix
is square. We consider the following two cases.
is full rank. Suppose the set of column vectors are ,
, ...,
having full
.
rank means that all column vectors are linearly independent. Such a matrix is called
nonsingular. Since the matrix
is square, each of its column vector
has
elements. In other words,
is in a -dimension ( -D) space. The same thing is
with . Because the set of vectors
are linearly independent, they form a spanning
set of -D space. This means that any -D vector can be represented by a unique
linear combination of
. Coming back to solving
, if
is full rank, must
be in the span of the column vectors
. This ensures the existence of a solution. In
.
addition, the solution must be unique as well. Actually, the solution is
is less than full rank. In this case,
can not span the whole -D space, but a subspace. It means that only vectors in that subspace can be represented by the set of
. For example, in -D space, the spanning set is
, . If in ,
column vectors
13
, if we have = 4 ,
= , they are linearly dependent and unable to represent a vector
.
having factor. While for a vector , it can be represented as When solving
is less than full rank, there are either infinite number of
solutions when is in the spanning set of
, or there is no solution when is outside
the spanning set of
.
Following above discussion, we can gereralize
matrix
has columns and rows.
"
to non-square matrices. Suppose
The column vectors must be linearly dependent. And the column rank must be less
than or equal to . If it is equal to , that means the column vectors span the whole
-D space. There will be infinite number of solutions. If the column rank is less than
, and is in the spanned subspace, there are infinite number of solutions, otherwise,
there is no solution if is outside the span of
.
In this case, it requires at least vectors to span the whole -D space. Therefore,
column vectors can span only a subspace. So, if is in the subspace spanned by
, there will be solutions. The number of solutions depends on whether column
vectors are linearly independent or not. If column vectors are linearly independent,
there will be a unique solution, otherwise, the number of solutions is infinite.
In conclusion, for linear equations
, a solution exists when is in the span of
are
the column vectors of . And a solution is unique only if the column vectors of
linearly independent.
For most applications in circuit simulation, it is often the case that
is square, symmetric, diagonally dominant and most importantly nonsingular. This means that we can
almost always get a unique solution for a linear system.
Gaussian Elimination Basics
Basically, there are two sorts of methods for solving the system of linear equations. One
is direct methods, the other is iterative methods. Direct methods can find the exact
solutions in finite number of steps. Iterative methods produce a sequence of approximate
solutions, hopefully they converge to the exact solution. Note that iterative methods can
converge only if some conditions of the coefficient matrix are satisfied.
We will introduce one efficient direct method for solving linear system, Gaussian Elimination. It can produce accurate results for a broad range of matrices even though it is computationally expensive. Its key idea is to recursively eliminate the unknowns in a systematic
way and convert a system of equations to a smaller one. Consequently, we get an equivalent
system of the original equations, where the coefficient matrix is triangular. It is easy to
solve the triangular system because only one unknown is needed to be solved from one of
14
the equations. In this way, the solutions for all the unknowns can be obtained by forward
substitution or backward substitution, i.e. recursively substitute the solved unknowns into
equations so that one unknown can be solved from each equation.
Let’s look at the example shown at slide 13 of lecture 3. The 1st equation is used to
eliminate from the 2nd and 3rd equations, i.e. subtract the 1st equation multiplied by a
multiplier
from the 2nd equation, and similiarly, subtract the 1st equation multiplied by
from the 3rd equation. Then, the 2nd and the 3rd equations contain only two unknowns
and . The original system of three equations in three unknowns is now reduced to a
system of two equations in two unknowns. Then, we can eliminate
from the 3rd equation
in a similar way. Identical operations are repeated until we obtain a system of one equation
with only one remained unknown. In this process, at step k, the coefficients of the reduced
system of equations are further reduced using the updated diagonal element
, which is
called as pivot. Now, an upper triangular system is obtained. To solve it, the unknown ( )
can be first calculated. After that, substitute the value into the 2nd equation, and get the
and into the 1st equation, can be solved.
solution for . Similarly, substitute both
This process of solving the upper triangular system is called backward substitution.
Actually, the GE method can be performed with matrix. Subtracting two equations is
equavilent to substracting the corresponding two rows of the coefficient matrix . In the
final
matrix, the lower triangular part has all 2 -element. The equation becomes
. Be aware that we need to apply the same subtraction operation to the corresponding
elements of RHS also. That is why the RHS is other than . The update operations of
, where
can be transformed to the matrix operations shown in slide 20, i.e.
is a unit lower triangular matrix filled with all the multipliers. Put the above two equations
, or
as shown below.
together, we get
2
2
2
2
2
2
> In summary, 4
steps are needed when applying GE to a
matrix. The basic
operation at th step is to eliminate
from the following 4
equations by subtracting
.
) the th equation multiplied by a multiplier
This process is repeated until is transformed
into or , where is a upper triangular, and comes from , where is a unit lower triangular.
from the th equation (
LU Factorization Basics
Based on the basic operations of GE, we can find an efficient way to implement GE, LU
into two matrices and
factorization. First, we need to factorize the coefficient matrix
15
. Then, the solution for
can be found by solving
for with forward
for with backward substitution.
elimination and solving
The details of LU factorization can be explained as follows(refer to slide 25). First, the
1st row is choosen as the source row. We try to eliminate from all the following 4
is selected as the pivot. For the 2nd row, the pivot is used to calculate the
equations.
. Then, for each element
, which are stored as the updated
multiplier
> is substracted and the result is used as the updated
,
. This is
basically eliminating
from the equation 2. Keep applying the identical operations on the
other two target rows, i.e. 3rd row and 4th row. Up to now, we obtain the 1st row in matrix
and the 1st column in matrix . Then, we move to the reduced submatrix whose upper
. Then, the
is choosen as the pivot. The above operations
left corner element is
are performed
on
the
submatrix.
This
process
is repeated until the submatrix is only the
element .
The above example shows the basic idea of source-row oriented approach of LU factorization. Similiarly, we have a target-row oriented approach. In this approach, the operations
are performed based on the target row (
). For each target row, take one row
4
) as source row, and apply the operations same as those operations of
(
the source-row oriented approach to the th row sequentially. These two approaches have
different active sets, which reflect the current computation regions(See slide 28). This is
caused by the different computation orders. However, they need exactly the same amount
of computation. And their computational complexity are both .
In contrast to the basic GE method introduced in the previous section, LU factorization
is more desirable when solving a system of equations with more than one RHS. With the
basic GE method, the operations on the coefficient matrix are needed for each RHS. But
with LU factorization, the matrix
need not to be decomposed with respect to different
RHS. Only the backward substitution and forward substitution are required for different
RHS.
Pivoting Strategies
& In the above GE method, we simply choose the diagonal elements as pivots. But at step ,
if
happens to be zero, the GE procedure would fail because
can not
be produced. More than that, if a pivot is different from zero, but is too small compared
with other elements in the reduced coefficient matrix, there are still problems. The small
pivots will result in a large round-off error, which leads to non-accurate or even wrong
solution(refer to the example in slides 33,34). To avoid these problems, pivoting strategies
can be used to reorder the rows(or rows and columns) of matrix . In partial pivoting,
only rows are interchanged.
At step , when we are going to choose a pivot, other than simply choose the diagonal
16
, we search all elements below it, i.e. & . If partial pivoting is just
used for zero pivot problem, any element & different from zero can be choosen as a
pivot. Note
that partial pivoting always succeeds if is non-singular. Otherwise, it means
&
all 32 . This contradicts with non-singularity of . Note that zero-pivots
never happen when LU factorization is applied to a diagonally dominant matrix . In this
case, the absolute values of all multipliers are smaller
than . If pivot
is zero, it means
>
'
)
)
'
that ! ( ! ! $
! $ ! ! , which contradicts with
pivots at step ,
the diagonal dominance condition of matrix . If we aim to avoid
small
&
which cause a large round-off error, we should choose the that has the
maximum absolute value. Having selected a desired pivot & , we swap row and row .
one
Note that the corresponding two elements in RHS also need to be swapped. After that, the
normal elimination operations can be continued. This process is called partial pivoting,
which helps to complete LU factorization and avoid large round-off error.
Besides partial pivoting, there are two other pivoting strategies, complete pivoting and
threshold pivoting. Complete pivoting allows both row and column swapping to move an
element to the pivot location. Note that the corresponding unknowns should be interchanged
according to column swapping. When combined with threshold pivoting, a partial pivoting
or complete pivoting is applied only when the prospective pivot is found to be smaller than
a certain threshold.
As we will introduce in late sections, these pivoting strategies for accuracy may be
conflict with the requirement to maintain sparsity of a sparse matrix.
Error Mechanisms
There are three cases in which there will be errors during solving the equations. The first
one is the round-off error. Round-off can become excessive as the decomposition proceeds.
But this growth of error may be controlled by good strategy of pivoting, which makes the
multipliers small. The second mechanism is ill-conditioning in the system, which occurs
when the system is almost singular. A set of equations are ill-conditioned if the solution is
very sensitive to disturbance in the inputs. It is caused by the circuit, and the uncertainty
in the solution is unavoidable. Even good pivoting strategy can not help. Another possible
error comes from the numerical stability of the solving method.
When discussing errors in numerical problems, norms are often useful. A norm is simply a scalar measure of the size of a vector or a matrix. For a matrix, the vector induced
norm is defined as:
1
1
')
1
The common norms for vectors and matrices include norm,
norm, and norm as
shown in Table 2.1. All vector norms and matrix norms respectively have useful properties
shown in Table 2.2.
17
$ $ " $ " !"#$%&"'
" $( " )&*,+-&"'
" /.0*1)2%3241)25$ 6)"728:9" * "<; > = Norm
Vector Norm
Matrix Norm
Definition
Table 2.1: Vector norms and Matrix norms
Norm
Vector Norm
Property
BA @ DCFE
if
&G#
G
GCIH DCIE
if
KJML3ON P 8JQL3
LCFE
if
<?R@
A @
if
S
S
SBCDH
if
<?-@
"
Matrix Norm
"
" "
"
JUTV<NP
T%<NP
" "
WJXTV
" (T%
YZ \[
Table 2.2: Properties for vector norms and matrix norms
. It is easy to prove
Condition number for a matrix 1 is defined as 1 1
1
that 1 ^]
. Condition number is a measure of ill-conditioning in a system. And large
1 means matrix is almost singular(ill-conditioned).
Using the properties of norms, we can find what happens if
or is perturbed (refer
to slide 9 and slide 10 of lecture 4).
`_
ba
dc
_
ba
:
_ a
ea
ea
dc
_
ea
:
a
a
a
From the results of these perturbation analysis, we can reach a conclusion, i.e. large is not desirable, which means matrix
is almost singular and the solution will be very
sensitive to any disturbance in the inputs.
In summary, the round-off error is caused by machine finite precision, and there will be
an error in the solution even if the algorithm is perfect. Pivoting helps to reduce it. Another
possible error source is the matrix conditioning. If a matrix is good(not ill-conditioned),
then complete pivoting solves any round-off problem. But if a matrix is almost singular,
nothing can help avoiding the uncertainty of the solution. From the view of numerical
18
stability, rounding errors may accumulate and propagate unstably in a bad algorithm. But it
can be proven that for Gaussian Elimination, the accumulated error is bounded.
Sparse Matrices
We have shown in the previous sections that, if we apply LU decomposition to solve a
, the number of multiplications is , where is the
linear equation system
dimension of matrix . We can not afford this computational complexity when the circuit
is large. In order to reduce the complexity, we will exploit the natural sparsity that occurs in
circuit equations. Here, sparsity means that there are many zero elements in the matrix. A
matrix is said to be sparse when it is advantageous to exploit its sparsity. The computational
complexity can be reduced to to by exploiting sparsity. It will dramatically
reduced the computation time.
There are three main goals when exploiting the sparsity of a matrix. The first one is to
avoid storing zero entries. This is desirable because the required memory will be reduced,
and decomposition is faster since you do not need to access the zero entries. The second
goal is to avoid the trivial operations, i.e. multiplication by zero and addition with zero.
A considerable amount of operations may be saved if trivial operations are not performed.
The third goal is to avoid losing the sparsity during the decomposition. Because the number
of nonzero elements may grow during LU decomposition, some strategy should be applied
to prevent this.
While an initial zero element of
which remains zero during GE need not be stored
or operated upon, a new nonzero created during the decomposition must be stored and
operated. Such a nonzero is called a fill-in. At step in GE:
&
,& 4
>
&
&
& are different from
A fill-in is created in position
only when both
and
zero. Note that fill-ins may propagate, that is, fill-ins generated from step result in other
fill-ins in step . An example of fill-ins is shown below.
2
2 dc
To reduce the number of fill-ins, node reordering in the circuit may provide helps. This
operation will not change properties of the original matrix like symmetry and diagonal dominance. Actually, node reordering in circuits is equivalent to swapping rows and columns in
matrix.
19
In order to exploit and maintain sparsity of matrix, we should find good permutation for
so that GE decomposition needs the minimum number of operations, and
sparse matrix
the minimum number of fill-ins is created during GE. However, it is a very hard problem.
Pivoting to maintain sparsity has been proven to be a NP-complete problem. So, some
heuristics should be used to find a permutation. There are several heuristic algorithms,
such as Markowitz reordering, Hsnieh and Ghausi algorithm. We will introduce Markowitz
reordering, which may produce
more fill-ins than the best methods but it is fast.
as a pivot, a fill-in is created in position
During
the GE process, when choosing
,&
& are different from zero. In Markowitz reorderonly when both
and
ing, the maximum number of fill-ins generated by choosing ,& as pivot are estimated by
Markowitz product:
4 > & 4 where is the number of nonzeros in the unfactored part of row , and & is the number of
nonzeros in the unfactored part of column . Markowitz reordering is a greedy algorithm,
which tries to minimize the maximum number of fill-ins at each step. At each step , the
pivot element is selected as the diagonal element && ] which is structurally nonzero
with the minimum Markowitz product. If
, row and column are swapped with row
and column . After that, the factorization operations are performed on the new row . In
this algorithm, only diagonal elements are considered as the pivot candidates such that the
matrix properties like diagonal dominance and symmetry are preserved. Also, the search
cost is considerably reduced. This method is also called Diagonal pivoting.
So far, different pivoting strategies have been introduced to realize two different goals.
As discussed in the previous section, when considering round-off errors, we need pivoting
for accuracy. But that may conflict with pivoting to maintain sparsity. For example, if
complete or partial pivoting is used to maintain accuracy, a large number of fill-ins may be
generated. The sparsity of the matrix is destroyed. On the other hand, if we choose pivots
only considering the Markowitz product, we may obtain a solution which has large errors
because the pivot selection depends only on the zero-nonzero structure, without considering
the values of pivots and possible exact cancellation during GE. Therefore, pivoting strategies
should be considered together to avoid these problems.
There are several ways of storing a sparse matrix in a compact form. When selecting
a data structure, we should consider the trade-off between the needed amount of storage
and the cost of data accessing and update procedures. Here, we introduce an efficient data
structure, linked list. One simple and efficient method is orthogonal linked list, where each
nonzero element is represented by a node. Each node has three data and two pointers.
The data are the value of element, its row index and its column index. The two pointers
respectively point to the location of the next nonzero element in the same column and in
the same row. So, each node is linked to the one below it and to the right one. A set of
pointers are provided to locate the first element in each row and in each column. Usually,
20
there is another set of pointers to locate the diagonal elements. This data structure allows
easy access to any element.
It is also possible to implement the linked list structure using array(refer to slide 31).
There is a vector of row pointers, which point to the first non-zero elements in each row.
And an array structure is built for each row, where the array element is a pair of data, i.e.
the value and the column index of matrix entry. But it is expensive for GE operations to
access the array elements. When eliminating a source row from a target row , you need
to find the entries in row which match row . To do that, you have to read all the entries in
row since there is no direct relation between the column index and the array index. One
method to avoid the unnecessary memory reference is to read all the elements in row and
scatter them in a -length vector. By doing this, you can access only the needed elements
using array indexing(shown in slide 33).
Finally, we introduce a graph based approach associated with the sparse matrices. A
direct relation can be built between a structurally symmetric matrix and a graph, where one
node should be added corresponding to a row, and one edge between two nodes corresponds
to an off-diagonal pair in matrix(refer to slide 34 in lecture 4). In a graph, node degree
represents the number of nodes connected to it. Based on a graph, Markowitz product can
be represented as:
The operations in LU factorization can be directly mapped to the operations on corresponding nodes and edges in the graph. Basically, if a row is factorized, one node associated with
it will be deleted from the graph. And if fill-ins are produced, new edges should be added,
and the graph edges are tied together(refer to slide 36). This procedure is shown with an
example in slides 37,38.
Summary
In this section, we first discuss the existence and uniqueness of the solution to a system of
linear equations. Then, basic Gaussian Elimination method and LU Factorization method
are introduced. Following that, pivoting strategies for accuracy are presented. Also, the
error mechanisms during solving the linear equations are briefly introduced. Finally, we
discuss how sparsity can be exploited to obtain significant reduction in computational complexity and how a sparse matrix can be stored efficiently.
2.1.2 QR Factorization
Even if the matrix M is singular, sometimes the solution still exists. In this case, we can not
get a solution by LU factorization. But QR factorization is still nice.
By weighted sum of columns view of systems of equations, we can get the QR method:
21
Step1. By projection and normalization, we get the QR factorization of matrix M;
Step2. Backsolve
6 (2.1)
When matrix M is singular, Q will have zero columns, and R will have corresponding
zero rows. That is, if column i of Q is zero, then row i of R is zero, too. We can get a
solution by the above two steps, the remaining question is: Is there infinite solutions or
no solution? Is that ”solution” a real solution? Can this be judged by the method itself?
Theoretically we can’t get the solution because the inverse of Q doesn’t exist. But we still
can get a solution by the above two steps, because in step two, the zero rows of left and
right side are corresponding. So from the QR method itself, we can’t know there exist some
solutions or no solutions.
If there is no exact solution, we still can get a ”solution” by QR factorization. What can
we say about that solution? Is that the ”best” one we can get?
To evaluate the solution, we use residual:
4 (2.2)
It is reasonable that the smaller the residual r is, the better the ”solution” x is.
In a nonstationary iterative method:
(2.3)
we should decide the search direction p and coefficient a. To eliminate coupling term,
we change search directions. This decoupling thought is the same as in QR factorization,
that is, the orthonomalized search directions are corresponding to columns of Q. So when
there is no exact solution, by QR factorization we get one of the best solutions that minimizes the residual.
2.2 Iterative Methods
2.2.1 Stationary Methods
The stationary methods include Jacobi, Gauss-seidel and successive Overrelaxation. Think
about the re-ordering of Jacobi and Gauss-seidel. For Jacobi:
4
$ & '*
,&
& (2.4)
So either we change the order of the equation (that is, exchange the rows of the matrix)
or change the order of the variables (that is, exchange the columns of the matrix), we may
can change the diagonal of matrix m, then the convergency will be changed. But if we
22
exchange row i, j and column i, j at the same time, that is, the diagonal of the matrix m
remains the same, we will not change the convergency of the Jacobi method.
For Gauss-seidel:
4
$ & ')
,&
&
4
,&
' $ &
& (2.5)
So either we change the order of the equation (that is, exchange the rows of the matrix)
or change the order of the variables (that is, exchange the columns of the matrix), we may
change the diagonal of matrix m and the order of the variables, then the convergency will
be changed.
For example, consider the linear equation:
4
4
4
(2.6)
Use zero as initial guess, let’s check the results after each possible re-ordering:
Jacobi
m
ex row 1,2 ex column 1,2 ex row and column 1,2
Convergency
Iteration Times
Solution
Yes
68
0.2000
-0.2667
No
N/A
N/A
Gauss-seidel
m
ex row 1,2
Convergency
Iteration Times
Solution
Yes
No
28
N/A
0.2002 N/A
-0.2665
The successive overrelaxation method is
seidel in the form of weighted average:
No
N/A
N/A
Yes
68
-0.2667
0.2000
ex column 1,2
ex row and column 1,2
No
N/A
N/A
Yes
27
-0.2669
0.1998
devised by extrapolation applied to Gauss-
4
(2.7)
So Gauss-seidel method is a special example (while w equal to 1) of successive overrelaxation.
If matrix M is diagonally dominant, that is,
!
!"
&+'*
!
,& ! (2.8)
we can prove that the sequence produced by the Jacobi iteration converges to the solution for any initial vector. This theorem is useful in circuit analysis, because normally the
nodal matrix of the circuit is diagonally dominant.
23
2.2.2 Non-Stationary Methods
2.3 Prelimenaries
The span of a set of vectors:
belongs to , the set of all linear
Given a collection of vectors com
, that is
binations of these vectors is a subspace referred to as the span of &
&
&
(
&
)
'
.
$
Eigenvalues and Eigenvectors:
> matrix and , then are called the eigenvalues and If
is an
are the corresponding eigenvectors.
is defined as the set of all eigenvalues of .
The spectrum of
The Spectral Radius of
is the radius of the smallest circle that is centered at the
origin and encloses all of eigenvalues.
Vector norms:
norm, denoted by , is defined as $ ') ! ! . norm(also known as Euclidean
norm), denoted by , is defined as $ ') ! ! .
norm, denoted by , is defined as ! ! .
Matrix norm:
Vector induced norm: 1
1 The intuition of the induced norm 1
and 1
of 1 is the maximum magnifica
&
,&
)
'
tion of by 1 . For example, 1
$
! 1 ! , which is the maximum absolute
value of the column sums.
$ &(') ! 1 ,& ! , which is the maximum absolute value of the row sums.
1 1 & ! eigenvalue of 1
6 1
!.
Theorem:
bound on the
spectral
Any induced norm is an upper
radius.
then,
Proof:
First pick ')
! !
" ! !
For example,
2
2 2 2 .
and
Clearly both hold. Whether it is a loose or tight upper bound depends on the matrix itself.
Spectral Mapping Theorem:
If a matrix
is applied to a polynomial , then the spectrum
of the matrix’s polynomial is the matrix’s spectum’s polynomial.
proof:
24
, where is the vector of all the eigenvectors of (or the matrix formed
by the eigenvectors of ) and is the diagonal matrix formed by the eigenvalues of .
, where By mathematical induction, it can been shown that
.
Substitute with
and obtain that 2.4 Congugate Gradient Method
It proceeds by generating vector sequences of iterates, i.e. successive approximations to the
solution, residuals corresponding to the interates, and search directions used in updating the
iterates and residuals.
where
is the step along search direction and is the search direction. Iterates
are updated in each iteration by a multiple
of the search direction vectors.
6 6 .
The choices of minimizes 6 Using residual to update the search directions.
6 6 4
ensures that and
, or equivalently 2.4.1 Arbitrary Subspace Methods
are orthogonal.
and Approximate as a weighted
Reesidual Minimization Approximately solves sum of that is $ ' by the definition of the residual, 4 , obtain 4 . Take the zero vector as the initial guess of , get
pick to minimize
the residual, that is to minimize
4 $ ' . Now
6 4
$ '
If is orthogonal to & , i.e. easy. Therefore we can use Gram-Schmidt on
such that Applying Gram-Schmidt approach, obtain,
6
&
2
, the minimization will be
to create a set of vectors &
and 6 32 .
4 by 6 32 obtain 6 6 4 4 by 6 32 , obtain 6 6 by 6 82 , obtain 6 6 6 6
In general, 4 $ ' , where the projection coefficients 25
.
Here is the algorithm of arbitrary subspace method to and a set of search directions: 1. Given
2. Make 6 orthogonal and get new search directions:
'
4 $ , where 6 6 3. Minimize the residual to get the coefficients,
$ ' for 6 6
2.4.2 Krylov Subspace Methods
9
to
1 Krylove subspace, denoted by
to be the span 1 1 1 ,9 is defined
is chosen to be the initial set of search
In the Arbitrary Subspace Method, if
, this method is called Krylov Subspace Methods.
derections where is a order polynomial.
'
$
Hence, 4 A nice observation with the subspace generation: for any is not zero, 4 4 4 That is to
4
$ '
say that the set of residuals can be used as a representation of the Krylov-Subspace. This
leads to the Generalized Conjugate Residual Algorithm which uses the residual to generate
next search directions.
Here is the summarry for the
step in the Generalized Conjugate Residual Method:
- I: to determine the optimal stepsize in the
search direction:
6 - II: update the solution while trying to minimize residual: - III: update the residual: 4
- IV: compute the new orthogonalized search direction by using the most recently residual:
& & & 4 $ &(' & & , where &
6
6
Time complexity analysis for the
step in the Generalized Conjugate Residual Methods: - in I: vector inner product: ; matrix and vector product: if the matrix is
sparse. - in II: vector addition: - in IV: inner products: ; total cost If the matrix is sparse the running time is ap . As the number of iterations
proaches , total cost = .
When the coefficient matrix
is symmetric, i.e.
, the following simplifica6
tion is obtained:
is orthogonal to & for is symmetric, then . Therefore if
.
4 & , where 6
6
Following the above analysis on time complexity of the generalized Conjugate Gradient
Methods, now we know that the running time is when M is semmetric .
2.4.3 Krylov Subspace Methods Convergence Analysis
Basic properties:
26
6
in the Generalized Conjugate Residual Methods, then:
.
, where is the order polynomial which minimizes 4 4 4 , where
<
is the order polynomial minimizing and subject to 2 If & 32 for all
(1) (2)
(3) .
The optimality proprety of the Generalized Conjugate Residual Methods polynomial:
< order polynomial
The Generalized Conjugate
Residual
Methods picks the best subject
minimizing
and
to 2 .
Hence polynomial
sub , where is any < order
ject to 2 . Rearranging the equation gives is
, that
any polynomial satisfying the constraints can be used to get an upper bound on ! .
The norm of the matrix polynomials:
, where is the ..
.
eigenvector
space.
! ! .
Taking
norm gives that Two important observations:
(1) The generalized Conjugate Residual Methods Algorithm converges to the exact so
lution in at most steps.
Proof:
Let 4 4 4 , where belongs to . Then
! !
2 . Therefore 2 and hence 2 , that is the above
observation holds.
Following a similar proof, the following obsearvation is proved.
(2) If M has only distinct eigenvalues, the Generalized Conjugate Residual Methods
Algorithm converges to the exact solution in at most steps.
Convergence for the case where
is symmetric:
If the coefficient matrix
is symmetric, i.e.
6
(1)
has ortho-normal eigenvectors. Therefore Hence ! !
(2) has real eigenvalues. If is positive definite, then " 2 .
Polynomial min-max problem:
Consider belongs to " 2 . Then a good polynomial(by saying
good, we mean that
1
is small) can be found by solving the min-max problem:
order polynomial s.t. ') ! ! .
The Chebyshev Polynormials give the solution of the min-max problem.
The chebyshev Polynomials are:
where 4
27
The solution of the above min-max
problem is:
! 4 4 ! 4 4 ! 4
" 2 , then Hence, if .
4 !!
!
4
The Generalized Conjugate Residual Methods Algorithm can eliminate outlying eigenvalues by placing polynomial zeros directly on them.
Therefore the convergence of the Conjugate Gradient Methods is related to: (1) number
of distict eigenvalues. (2) Ratio between the max and the min eigenvalues.
28
Chapter 3
Solving Systems of Non-Linear
Equations
3.1 Introduction
Previous chapters have outlined methods for solving equations derived from circuits containing devices that are modeled using linear models. In practical circuits, not all elements
are linear in nature. For example, the current through a diode is modeled using the following
equation:
< 4 (3.1)
When writing the BCE, KVL, and KCL equations for Sparse Tableau Analysis (STA)
or Modified Nodal Analysis (MNA), non-linear elements such as diodes result in a system
non-linear equations.
The solution of these equations allows us to calculate the operating point of a circuit (see
Figure 5.1). It also allows us to perform the first step in SPICE: DC analysis of a circuit.
where is a
Generically, the equation that needs to be solved is given by: voltage and is a current. This can be expressed as which is equivalent
4
to solving 2 . Generally, it is difficult to find an analytical solution to 2 so
iterative methods are employed.
This chapter focuses on finding a solution to non-linear equations in the form of 2 using iterative methods. The focus is on one type of method known as the NewtonRaphson (NR) Method. Newton-Raphson methods have been used in almost all simulators
available to day. It should be noted that when using NR methods, a correct solution is not
guaranteed and neither is convergence.
29
operating
point
Figure 3.1: Graphic representation of finding operating point of circuit
3.2 Iterative Methods
Iterative methods for solving equations of the form 82 where may either be a scalar
and generate a sequence of values , , , ,
or vector start from an initial value
,
which hopefully converge to the exact solution "! .
The iterations are generated according to an iteration function where the current iteration is a function of previous iterations: $#
(3.2)
Two important questions to ask when using iterative methods to solve a system of equations are:
1. Does the method converge to the correct solution?
2. If the method converges, What is the convergence rate?
Methods that are appropriate for practical use should converge to the correct solution
and should converge quickly.
30
Figure 3.2: Graphic representation of Newton-Raphson Method
3.3 Newton-Raphson (NR) Method
Intuitively, the NR method linearizes the system of equations using the Taylor Series expansion. In the formulation, high order terms are neglected.
!
!
4
!
high order terms
(3.3)
Using Equation 3.3, the iteration function is then defined as follows by writing the
truncated Taylor Series about :
Since we want 4
(3.4)
32 , the above equation simplifies to:
4 (3.5)
In the case where is a scalar, Equation 3.5 can be interpreted as finding the tangent
. is the solution obtained
line at some and using this to obtain the next solution
by extending the tangent line until it crosses the x-axis as shown in Figure 5.2. Iterations
continue until the function converges to the exact solution ! .
Note that at each step the function and its derivative are evaluated at the previous time
step.
The algorithm that implements Newton’s method is simple. Given an initial guess at a
solution for , say , one iterates over Equation 3.5 until convergence.
31
3.3.1 Convergence
Convergence analysis needs to be carried out to evaluate conditions under which NR will
is said to converge with order if there
converge to the the correct solution. An iteration
exists a vector norm such that for eack ]
:
4
4
!
!
(3.6)
That is, the error in the next step of the iteration is smaller than the error from the
previous step.
The Taylor Series and the Mean Value Theorem are used to evaluate the convergence
properties for the NR method by expanding ! about as shown in Equation 3.7. Using
this equation and the definition of the NR iteration, Equation 3.8 is derived which shows
that convergence for NR is quadratic.
2
!
!
4
!
4
!
!
9
!
!
4
4
!
!
for some
!
(3.7)
(3.8)
Quadratic convergence is only possible given an initial guess sufficiently close to the
exact solution, ! . Unfortunately, there is no way to ensure that the initial guess is sufficiently close to the exact solution. In addition, the following two conditions stated in the
Local Convergence Theorem must be satisfied:
The first derivative of The second derivative of at ! is bounded away from zero.
is bounded.
9
These two conditions imply that is bounded as well.
4 the convergence is quadratic and for
For example, for the function is linear because the first derivative in this case is not bounded away from zero
(i.e. 32 ). To see the difference it is recommended that the reader work through the
4 and derivation of the convergence for both .
for a particular function, there is a need to evaluate both
To evaluate
convergence
4
and to ensure that they are both less than or equal to some given
threshold. This avoids false convergence.
For example, if the delta between iterations is
small,
it is still possible that the may be far away from zero. Conversely,
4
maybe close to zero but
may be large and moving away from
the exact solution.
32
Other non-convergence problems are experienced when the initial guess is not close
enough to the exact solution causing solutions to the oscillate around local minima. Functions that asymptotically approach infinity are also problematic in that solutions tend to
slowly converge to infinity.
3.3.2 Generalization of NR to Multidimensional Case
The generalization of Newton’s method to the multidimensional case is straightforward.
The iteration has the form:
4
#
(3.9)
# . is the Jacobian
In Equation 3.9 is a vector and # is defined as # #
matrix of partial derivatives of # with respect to each scalar in .
The problem we are trying to solve is to find ! such that # ! 82 . The implementation of the iteration defined by Equation 3.9 is difficult since the inverse of the Jacobian is
not sparse. So instead of computing the inverse, at each iteration a solution to the following
system of linear equations is obtained:
4
4
#
(3.10)
Two conditions are required for convergence. These are analogous to the scalar case.
1. The
inverse of the Jacobian is bounded; that is the Jacobian is far from singular.
2. The
function
4
is
smooth:
4 Lipshitz
continuous.
That
is:
As in the scalar case the initial guess must be close enough to the exact solution to
ensure convergence.
3.3.3 Application of NR to Circuits
When applying NR to circuit equations, it should be observed that the coefficients of KCL,
KVL, and BCE of all the linear elements remain unchanged with respect to iteration .
Non-linear elements are represented by a linearized version of their branch constitutive
equations at iteration . The linearized version of a nonlinear element is known as its
companion network. An example is shown in Figure 5.4.
The system of equations can be interpreted as the STA of a linear circuit (companion
network) whose elements are specified by linearized BCEs.
33
2
1
Figure 3.3: Circuit Example
! " NON−LINEAR
LINEAR
Figure 3.4: Companion network for diode
The general approach to applying NR to circuit equations is to first construct the STA
form of the circuit equations, linearize it and then solve each linear system at each iteration
using either STA or MNA.
The coefficients obtained for the companion networks for non-linear elements are func
tions of the iteration count .
3.3.4 Example 1
In this section an example is presented to clarify concepts presented earlier. Figure 5.3
shows the circuit under analysis.
Since there are 5 branches and 2 nodes, there will be 12 equations and 12 unknowns if
we employ STA. In the example, writing the KCL equations for both nodes gives us:
#
#
32
4 4 3
2
The BCE for the first diode shown in Figure 5.3 is given by:
#$#
:
4 32
! 4
%
(3.11)
The Newton iteration is given by Equation 3.10 so the entry for the derivative of #&# with
respect to is given by:
:
4
%
34
(3.12)
, row 9 is given by:
: 4 : : 4 Therefore at iteration
%
4
%
%
(3.13)
This is the BCE for the companion network for the first diode in the example as shown
in Figure 5.4. The coefficients
are given by:
and :
4 %
: : 4 %
%
Note that the model for the companion network is the same at each iteration however
the coefficients for the elements are functions of iteration . The same result could have
been obtained by linearizing the diode current equation using the Taylor Series expansion
around iteration .
3.3.5 Example 2
The NR method can be used generally to develop companion networks (and the corresponding MNA templates) for non-linear elements. In this example, we develop the companion
network for a Level 1 MOS using NR.
The MOS Level 1 drain current expression in triode is given by:
9 <4
.4
< < <
(3.14)
By NR, we can express an update function for the current:
<
< < < ! <
4
<
< < ! < 4
<
We recognize the partial derivative terms as the MOS transconductance
conductance < at iteration . The above equation is equivalent to:
4
4
<
(3.15)
and output
< 4 < 4 < <
(3.16)
Therefore, the drain current at iteration can be represented by a (linear) voltage
<
< <
controlled current source, a resistor, and a current source, the values of which are a function
of k (the previous iteration).
The modified nodal analysis stamp for this linearized element is then:
4
<
<
4
<
< 4
< 4 4 < 4 < 4 < <
< < < <
35
3.4 Modified Newton Methods
Several types of functions can lead to convergence problems with the Newton Raphson
method if the initial guess is insufficiently close. The presence of local minima (singular
Jacobian) can trap the algorithm far from the true solution. Similarly, functions with small
derivatives can lead to large iteration steps that overshoot the solution and prevent convergence. When the evaluated function is rapidly increasing, even a small overshoot can result
in overflow.
Here we focus on improving the multi-dimensional iterative Newton method to aid in
convergence. Two classes of modified Newton algorithms are discussed – limiting schemes
and continuation schemes.
3.4.1 Limiting Schemes and the Damped Newton Method
As discussed before, with the multi-dimensional NR method we seek a solution to the following equation at each iteration:
4
#
(3.17)
To avoid large overshoots, the step size is limited:
(3.18)
Limiting schemes can be either direction corrupting or non-corrupting, based on how
the original step vector is altered. Limiting schemes are heuristic, and offer no guarantee of
global convergence. A corrupting scheme is given below:
if !
!
(3.19)
(3.20)
otherwise. Notice that, for example, if only the component of the step vector exceeds the
limit , only the component of the step vector is changed – the direction of the limited
or
step is altered (hence the designation “corrupting”). A non-corrupting scheme is given by:
where
is given by
!!
!!
(3.21)
(3.22)
Here the step vector is scaled if its norm exceeds the limit , preserving the step direction.
36
The damped Newton method relies on a non-corrupting limiting scheme where the vector scaling is re-evaluated at each iteration in order to minimize the target function. The
limiting scheme is given by:
(3.23)
is picked so as to minimize # where
6 # .
Recall that for the general NR method, local convergence is guaranteed provided that
the Jacobian is Lipschitz continuous and its inverse is bounded. For the damped New 2 such that
ton method, these same conditions imply that there exists a set of
. Every step reduces # ,
! !# ! !
! !# ! !
! ! # ! ! with
implying global convergence.
A singular Jacobian can still disrupt the damped Newton method, however. In fact, if
a local minimum lies within the range of a step, the limiting scheme actually pushes the
iterate directly to that minimum, where the Jacobian is singular. This is a problem.
3.4.2 Continuation Schemes
We have seen that the success of the Newton method often relies on the value of the initial
guess. Therefore, it would be useful to find a way to approach the problem such that the
initial guess is an informed one. Suppose that the solution of a system 2 is relatively straightforward, and we seek the solution of a more complex system # 32 . The
continuation scheme involves
defining
a
function
# such that is an independent
parameter which maps # from to # and from to . In the language of homotopy
theory, this represents the mapping of a continuous function from one topological space to
another.
For example, suppose we choose # as
#
#
4 (3.24)
As we progress from , we trace a series of solutions which take us
2 to from the solution of the 2 system to the more difficult # 2 . Provided is sufficiently smooth (hard to ensure), we will approach # from very close to the final
system solution.
A step in represents movement from a system with a known solution to an altered
system. Based on how well the new system converges, the step size can be altered dynamically. In a general algorithm for implementing the Newton method with continuation,
an initial step a is chosen. If, after applying Newton’s method to the new system, the
method converges, then the step size can be increased (say, to 7a ) to speed up the algorithm. Conversely, if the method does not converge, this indicates that the system was
37
altered too drastically, and a smaller step size is used ( a ) to advance from the last solution.
An example of an application for continuation schemes would be to solve a system
of circuit equations including independent sources, by first looking at the system with the
sources removed (source stepping). The source vector would be a simple linear function of
, , where 2 . As advances, Newton’s method yields a solution to the
system, which serves as an initial guess for the next iteration. Once reaches , the final
solution of the system is reached. Because the continuation parameter appears only in the
independent sources, source stepping does not alter the Jacobian.
Another approach would be to devise a continuation scheme such that the Jacobian is
altered from the identity matrix to that of the system of interest. This could be implemented
as:
# # 4 (3.25)
This scheme has the easily identifiable initial solution 2 2 , and the initial Jacobian is
definitely non-singular.
Finally, the continuation scheme can be improved somewhat by refining the initial guess
for the next step (update improvement). With a familiar trick, the current state of the system
can be expressed in terms of the last state by:
#
Pa #
\a #
a
(3.26)
is the solution of the system at ).
4 \a #
The first term of the right-hand side is zero (
We would like to determine the best choice for the initial guess for the next state based on
a which makes the left-hand side zero as
the current state; therefore, we solve for well. This implies:
or
#
ea 4
#
4
a 4
#
#
a a (3.27)
(3.28)
Using the continuation scheme mentioned above,
#
then
#
#
#
which is easily computed. Since the Jacobian at step
a better initial guess for the next step.
38
4 4
(3.29)
(3.30)
is also known, we can then compute
Chapter 4
Solving Ordinary Differential
Equations
To analyze the transient response of dynamic circuits, which contain capacitance and/or
conductance, the corresponding branch constitutive equation include ordinary differential
equations. For example, if one analyze the signal transmission in interconnection metal lines
between two logic gates, the wire itself is a resistor and there also exists capacitance between
the signal wire and ground plane. In order to numerically simulate signal transmission of
this wire, it is a common procedure to divide a whole wire into serially connected resistors
and capacitors, which have the same resistance and capacitance, respectively.
To further exploit this signal transmission problem, let’s analyze a simple x IC model
as shown in the figure below. The corresponding constitutive equations are,
and the conservation laws are,
32
4 3
2
therefore, the nodal equation system for this x signal transmission problem can be written
as,
:
2
For simplicity, let
2
: 4
4 4
2 , the equation system can be
39
rewritten as,
where
4 2
are the unknown node voltages,
2
4 Before we have in–depth analysis of this problem, some knowledge about eigenanalysis
of matrix is required. Let’s introduce a general ODE,
=1
2
=
It is known the NxN matrix 1 can be decomposed into the following three components,
1
..
.
..
.
..
.
..
.
..
.
2
2
2
..
2
..
.
2
.
2
..
.
..
.
..
.
..
.
..
.
..
.
where
and are corresponding eigenvalue and eigenvector of the matrix, respectively.
A new variable can be introduced as,
dubstititing it back to the original ODE, we get,
Multiply the above equation by
1
=1
=
2
, it gives,
2
..
2
2
2
2
.
2
It is noted that in this equation systems, all variables are decoupled and can be solved
very easily,
=
=
2
Therefore, there are four steps to solve a general ODE,
40
1. Determine
and
2. Compute
2
3. Compute
2
2
2
4. Compute
.
2
2
..
2
2
Accordingly, the eigenvalues and eigenvectors of matrix
sion problem can be expressed as,
1
4
4 2 2
2
4 1
4
in this x signal transmis-
where 4 2 and 4 are two eigenvalues of the nodal matrix.
If the initial values of the system are [1,0], which is an average of two corresponding
eigenvectors, it can be noted from the time response of this system that the system has
come together quickly because of the smaller
a two time scale response: first, and
eigenvalue, 4 ; second, once and meet together, they decay to zero very slowly.
However, the above method is still not of much practical use. It requires the one knows
the eigenvalues, but the computation of a eigenvalues from a general matrix is one of the
best-known stupidities of numerical analysis. Accordingly, practical numerical methods
need to be introduced to solve real circuits.
In general, the sparse tableau equations for dynamical circuits can be written compactly
as,
#
2
32
where is the vector of circuit variables.
For the sake of simplicity, we shall focus on the first order ODEs with initial values in
the form,
in an interval given the initial condition . Typically, there are no available analytical solutions for this initial value problem (IVP), numerical method, therefore, has to be
employed to solve it.
41
Although there is not necessarily a solution exists and is unique for a general ODE, It
turns out that under rather mild conditions on the continuity and differentiability of # , it
can be proven that there exists a unique solution. Before the finite difference method is
introduced, Let’s assume there is a unique solution for ODEs and for simplicity, only linear
case is discussed here,
81
Generally, there three steps in finite differential methods:
1. Discrete time, i.e., divide the period T into N small intervals
2. Represent exact solution
with approximate numerical solution
at time
3. Approximate the differential using the discrete numerical values around time
By employing different methods to approximate the differential, there are three commonly used finite differential methods:
Forward Euler Approximation (explicit method, simplest):
which is equal to,
and the error at time
1
-4
1
can be given as,
-4
1
Therefore, the algorithm for FE can be written as,
1
Backward Euler Approximation (implicit method, more expensive):
3
1
which is equal to,
and the error at time
1
4
can be given as,
Accordingly, the value at time
-4
1
need to be solved via Gaussian elimination method,
4 1 42
Trapezoidal Rule Approximation (implicit method, more accurate):
which equals,
and the error at time
1
1
1
-4
1
-4
can be given as,
-4
1
also need to be solved via Gaussian elimination method,
4 1 1 Accordingly, the value at time
From the numerical integration point of view, the difference among these methods can
be clearly shown in the schematic figure in lecture 10. Despite the differences among them,
.
they all belong to one–step method, i.e., is determined only by
More common numerical method for solving ODEs is called multi–step method. For
nonlinear differential equations,
Its –step multistep approach can be expressed as,
& &(' &
&'
& & & where & and & are coefficients of multistep methods.
It is easy to note that the corresponding coefficients for three one–step methods are:
FE:
BE:
4 3
2 4 32
and for Trapezoidal Rule:
4 It can also be noted from the multistep approach that:
1. If 2
, the multistep method is implicit
43
2. A –step multistep method uses
entials
3. A normalization is needed,
previous solution values and corresponding differ-
4. A –step multistep method has is the common approach
free coefficients
Let’s focus on convergence properties of multistep method now. A finite difference
method for solving initial value problems on 2 is said to be convergent if given any 1
and any initial condition, it satisfies,
4
* 2
_
2
_
A multi-step method for solving initial value problems on
convergent if given any 1 and any initial condition, it satisfies,
4
* 2 is said to be order
for all
less than a given . It can be verified that Forward–Euler and Backward–Euler
are order 1 convergent, whereas Trapezoidal Rule is order 2 convergent.
In numerical simulations, there exist two kinds of errors:
1. The Local Truncation Error (LTE) of an integration method at is the difference
and the exact value of the solution , assumbetween the computed value
ing no previous error has been made.
2. The Global Truncation Error (GTE) of an integration method at is the differ and the exact value of the solution ,
ence between the computed value
assuming that only the initial condition is known exactly.
In order to check the convergence of a multistep method, one needs to look at the maximum error over the whole time interval, i.e., GTE; it is not sufficient to look at LTE only. As
a matter of fact, As one takes smaller and smaller timesteps , one would like the solution
to approach exact solution better and better over the whole time interval, even though he
has to add up LTE from more timesteps. There are two criteria to check the convergence of
a method:
1. Local Condition(Consistency): One step errors are small, which is typically verified
by Tayer series .
2. Global Condition(Stability) : The single step errors do not grow too quickly. All
one-step methods are stable in term of stability.
44
A one–step method for solving initial value problems on an interval
consistent, if for any 1 and any initial condition,
4
2
_
2 is said to be
2
_
Now, let’s analyze the convergent property of forward Euler method. From its definition,
it can be expanded in t about zero, which yields,
where bouded by,
2
2
2 . The last term in the above equation is LTE, which is the ”one-step” error
where
2 8
6
GTE of forward method is defined as,
4
1 taking norms and using the bound on , it gives,
1 1 which is equal to,
To get a bound on the global error, the following lemma is helpful. !
Lemma: If ! ! !
82 " 2 , then ! ! ! ! .
To prove this lemma, first write as a power series and sum
! ! & ! ! !4 ! ! &(' !4 , therefore . Thus,
4 4 Note that
! !
4
! !
! !
Mapping the global error equation of Forward-Euler to the lemma,
1
45
it can be seen that terms, a bound of
_
1
, and
is found to be
Finally, noting that _
. Applying the lemma and cancelling
1
, the following formula can be derived.
6
1
Some observations about convergence analysis for Forward-Euler method are listed below.
1. Forwad-Euler is order 1 convergent.
2. The bound grows exponentially with time interval.
3. C is related to the solution second derivative.
4. The bounds grows exponentially fast with the norm of matrix A.
The convergence analysis of multistep methods is more complicate. First, let’s see some
definition and observations. Generally, a multistep equation can be written as
&(' &
& &'
&
& &
where ’s are the function values and ’s are the derivative values. It can be seen that
1. If 32 the multistep method is implicit.
2. A -step multistep method uses
previous ’s and ’s.
3. A normalization is needed,
4. A -step method has
is common.
free coefficients.
A typical question is ”How does one pick good coefficients so that the highest accuracy
can be acquired?”. To answer this question, let’s consider a simplified problem for analysis.
A scalar ODE (Odinary Differential Equation) is given as
2
(4.1)
Where, is a complex number. The reasons why such a simple test problem is choosen
are 1) nonlinear analysis has many unrevealing subtleties, 2) a vector ODE is equivalent to
a series of scalar ODE’s for multistep methods. The equivalence of a vector ODE to a series
of scalar ODE’s is shown below.
46
The multistep discritization of
can be written as
Let
31
& &(' &
&1
&'
& , the discritization of the equation becomes
&(' For a properly choosen
, such that
&'
&
1
&('
&'
& &
& &
& 1
is a diagonal matrix, the vector ODE becomes
&
..
.
& which is actually a series of discritized scalar ODE’s.
The multistep formula for the scalar ODE as shown in (4.1) is
& &(' The solution of (4.1) is
Therfore,
if
2
2
&
the solution is growing with time.
the solution is oscillating with time.
the solution is decaying with time.
2
3
&
&'
"
&
The multistep formula of (4.1) is
&
&' &
4
&
&('
The exact solution satisfies
&('
Where,
&
&
-4
&('
&
& 32 & is the local truncation error (LTE). The global error is defined as
4 47
(4.2)
(4.3)
Substracting equation (4.3) from equation (4.2) yields the difference equation for global
error
4 4 4 (4.4)
Obviously, small LTE is desired for high accuracy. Multistep methods are said to be of
order p, if they are designed to be exact for polynomial of order p. The higher the order p,
, then
the smaller the LTE. Suppose &
4
4
&
4
Thus, equation (4.4) becomes
&('
&
If
&'
4
&'
&('
&('
4 4
& &('
4 4
& &
&
4
4
4
then,
. if (4.5) holds for all 32 for a locally accurate Taylor series in t, the LTE is &
32
(4.5)
then for any smooth , which has
. Mathematically,
Example: The exactness constraints for 2-step methods (k=2). If k=2, (4.5) becomes
32
2
2
2
2
2 4 4 4 2 2 4 4 2 2 2 4 4 2 2 2 4 4 2 2
1. For Forward-Euler method, , 4 , 82 , 82 , , and 2 .
.
Thus it satisfies 82 and , but not . 2. For Backward-Euler method, , 4 , 2 , , 2 , and
.
8
2 . Thus it satisfies 3
2 and , but not . 3. For Trapzoidal Rule method, , 4 , 2 , 2 , 2 , and
8
2 . Thus it satisfies 3
2 , , and , but not . .
48
In real application, usually
methods (k=2) become
is taken as 1. Then the exactness constraints for 2-step
2
2
4
2
4 2
4
2
4
4 4
4
2
2
4
2
4
2
2
4
4
4
4
4 If a 2-step method satisfies all of the above constraints, it is the 2-step method with lowest
LTE. The solution of the above linear equation system is , 2,
4 ,
, , and
. These parameters define the 2-step method with
lowest LTE, and the LTE is . Now, consider about the explicit methods, for which
2 . There are only 4 parameters which can be chosen. To satisfy the constraints up to
, the parameters must be chosen as , ,
4 , 2 , , and
. This method is called best explicit (BESTE) method, and has much higher one-step
accuracy than Forward-Euler and Trapzoidal Rule methods (see slides 27 of lecture 11).
Unfortuantely, it was found that BESTE method has an increasing global error as timestep
decreasing (see slides 29 of lecture 11). This means that BESTE method fails to converge,
although its LTE is so small. To explain this, let us look at the difference equation for
multistep methods (4.4). Obviously, we want the global error
to be small as long as the
LTE is small. But, this is not guaranteed by (4.4).
An aside on solving difference equations: Consider a general k-th order difference equation
% with k initial conditions
The update form of this equation is
Thus,
4 &'
4
& & can be related to by
If the polynomial equation then
& & &'
49
2
has distinct roots ,
Where,
&
&(')
&
% are coefficients.
& and 8
2 , then
Example for derivation of : Suppose
&
& & Thus,
&'
Important observations:
1. if ! !
! &
for all , then !
&
! , where C does not depend on &
! _
2. if ! !"
for any , then there exists a bounded such that ! 3
&
for all , and if ! !
& ! !
3. if ! !
, is distinct, then ! !
!
Definition: A multistep method is stable if and only if
* * for any .
Theorem: A multistep method is stable if and only if the roots of 2 are either less than one in magnitude or equal to one and distinct.
Now, go back to the multistep method difference equation (4.4)
!4 4 If the roots of
&
&('
4 &
&
4 2
are either less than one in magnitude or equal to one in magnitude
but distinct,
then
from
the above aside on differnce equations, it can be derived that
. i.e. the
multistep method is stable.
_
2 , roots of
As
&('
move to match the roots of
&
&('
4 &
&
50
&
&
32 2
Usually,
is denoted as , and the polynomial equation can be rewritten as
&'
&
4 &
&
32 (4.6)
Definition: A method is stable if all the solutions of the associated difference equation
obtained from (4.6) setting 32 remain bounded if d_
.
Definition: The region of absolute stability of a method is the set of q such that all the
solutions of (4.6) remain bounded if d_
.
Thus, a method is stable if its region of absolute stability contains the origin ( ?2 ).
Based on the above definition and the observations of the aside on solving difference equations, it can be shown that the region of absolute stability of a method is the set of such
. Where is the
that all the roots of (4.6) are such that ! !
number of distinct roots, and that the roots of unit modulus are of multiplicity of 1.
Definition: A method is A-stable (absolutely stable) if the region of absolute stability contains the entire left hand plane (in the space).
A more formal approach for convergence analysis for multistep methods is
1. Each method is a associated with two polynomials
&'
&'
&
&
&
&
82
82
2. Stability requires that roots of
and :
is associated with function past values
is associated with derivative past values
must stay in ! !
3. Absolute stability requires that roots of (
2
when
on ! !
4 and be simple on ! !
) must stay in ! !
and be simple
Dahlquist’s first stability barrier: For a stable, explicit k-step multistep method, the maximum number of exactness constraints that can be satisfied is less than or equal to k (note
there are 2k coefficients). For implicit methods, the number of constraints that can be satisfied is either k+2 if k is even or k+1 if k is odd.
In summary, a multistep method converges if and only if this method satisfies both
consistency and stability conditions.
1. Consistency (local) condition: one step errors are small. To satisfy this condition, the
methods must satisfy the exactness constraints up to ( "32 ). Mathematically, it
can be written as
* 51
for
2. Stability (global) condition: one step errors grow slowly. To staisfy this condition,
the roots of
&'
4 &
&
&
32
must be inside the unit circle. Mathematically, it can be written as
* If both conditions are satisfied, then
* * Therefore, the method converges.
4.1 Lecture 12: Large Timestep Issues
4.1.1 Last Lecture Review
This is simply a review of what has happened in previous lectures. Discussed were:
Transient Analysis of Dynamic Circuits; these are typically circuits which include
capacitances or inductances. Introduce ODEs which require numerical analysis techniques.
Solutions of ODEs; This introduced the notion of Initial Value Problems (IVP) solutions such as Forward Euler (FE), Backward Euler (BE), and the Trapazodial Rule
(TR). These were single step methods, but we were also introduced to Multistep methods. These were then analyzed for properties such as convergence (consistency).
4.1.2 Outline
The outline for this lecture will be:
Convergence for Multistep methods. The concept of Stability will include the notion
of Region of Absolute Stability and Dahlquist’s Stability Barriers.
Stiff Stability introduces issues with larger timesteps. Examples will be given as well
as analysis of FE and BE. New concepts such as Gear’s method and variable step size
will also be introduced.
There will be more on Implicit Methods (those which have 0). Solution with
Newton Raphson will be covered.
Finally, there will be a discussion of applications of multistep methods to circuit
equations.
52
4.1.3 Multistep Methods-Common Algorithms
This is just another review of the three Single step methods we have covered and what they
look like in a multi-step context.
First recall that:
Multistep Equation =
& &(' &
&'
& & & This leads to derivations of the coefficients and discrete equations for each method.
Forward Euler (FE):
4 4 Backward Euler (BE):
4 3
2 4 32
Trapezoid (TR):
4 4 4.1.4 Multistep Methods-Convergence Analysis
When dealing with the multistep methods, there are two criteria for convergence.
Local Condition: One step errors are small. This is the consistency portion. This is
typically verified using taylor series.
Global Condition: The single step errors do not grow too quickly. This is the stability
portion.
All one step methods are stable in this sense.
53
4.1.5 Multistep Methods - Stability
This slide is prefaced upon an example where a ”best” 2-step explicit method failed to converge. This would have meant that by ”best” that the coefficients were chosen to minimize
the LTE (as shown in previous lectures).
Examine the Multistep Method Difference Equation:
+( 4 =e
t ) +( 4 t )
t )
( 4 Ec v(l t)-v c Global Error
The e (LTE) is small so why is the Global Error so large?
4.1.6 Aside on Solving Difference Equations
Again consider a general kth order difference equation:
% There are 3 important observations that can be made:
&
! ! where C does not depend on l.
If ! !
for all i, then ! !
&
If ! !"
for all i, then there exists a bounded ! ! such that ! !7_
.
&
! !
If ! !
for all i, and if ! !
is distinct then ! !
4.1.7 Multistep Methods - Stability
This again refers to the difference equation:
+( 4 ( !4 t ) +( 4 t )
t
if and only if
Definition: A
multistep
method is stable
*
*
for any
6 )
=e
Theorem: A multistep method is stable if and only if the roots of 32 are either less then one in magnitude or equal to one and distinct.
4.1.8 Multistep Methods - Stability
This refers to the Stability Theorem Proof
Given the multistep method difference equation:
+( 4
t ) +( 4 t )
( !4 &
If the roots of &(' &
32 are either:
t
less then one in magnitude
equal to one in magnitude but distinct
from the aside
on diffence equations:
Then
from which stability easily follows.
54
)
=e
4.1.9 Multistep Methods - Stability
In the notes, this is simple a diagram of the unit circle on the real and imaginary axis. The
roots of:
&('
&
&
32
are shown to fall within this circle. As
mial.
_
2
, roots move inward to match
polyno-
4.1.10 Multistep Methods - Stability
This is a more formal approach which presents some definitions:
& & 32 _ & ' & 4 & & 32 (1)
&(' & 4 A method is stable if all the solutions of the associated difference equation obtained
from (1) setting q = 0 remain bounded as l _
The region of absolute stability of a method is the set of q such that all the solutions
of (1) remain bounded if
_
Note that a method is stable if its region of absolute stability contains the origin (q =
0)
4.1.11 Multistep Methods - Stability
It can be shown that the region of absolute stability of a method is the set of q such that all
where k is the number of distinct
the roots of (1) are such that ! !
roots, and that the roots of the unit modulus are of multiplicity 1.
Also note that a region of absolute stability is A-Stable if the region of absolute stability
contains the entire left hand plane (ie reals 0).
This slide also contains a picture of the unit circle (for stability in terms of z) as well as
an example of A-Stability.
4.1.12 Multistep Methods - Stability
This slide introduces the fact that each polynomial for a multistep method can be thought
of as two polynomials associated with and .
: associated with function values
: associated with derivative values
Again stability requires that the roots of must stay in ! !
and be simple on ! !
Absolute stability: roots of ( 4 ) must stay in ! !
and be simple on ! ! =1 when
Re(q)¡0
55
4.1.13 Dahlquists Stability Barriers
This introduces two points related to stability
For a stable, explicit k-step multistep method, the maximum number of exactness
constraints that can be satisfied is less than or equal to k (note that there are 2k coefficients). For implicit methods, the number of constraints that can be satisfied is either
k+2 if k is even or k+1 if k is odd.
There are no A-Stable methods of convergence order greater than 2, and the TR
method is the most accurate.
This points out why TR is very popular and used by spice.
4.1.14 Multistep Methods - Convergence Analysis
Conditions for convergence - Consistency and Stability
1) Local Condition - One step errors are small (consistency). Exactness contstraints up
to must be " 0) for
c
* 2) Global Condition: One step errors grow slowly (stability). Roots of &('
must be inside the unit
circle.
c
* Convergence result:
6 * * &
&
32
4.1.15 FE region of Stability
This shows that for Forward Euler’s region of absolute stabilty. Naturally the unit circle is
displayed for the z domain and for the domain you will require that ! !
.
z = (1+ so the right edge middle of the circle is at = 0 and the other at 4.1.16 BE region of Stability
This shows the Backward Euler region of absolute stability.
z = (1+ This will result in the region being the entire space with the exception of a circle which
defines the roots where z = 1 above.
4.1.17 Summary
This is simply a summary of what has been covered up to this point:
56
Convergence for one-step methods. This was show with consistency and stability
examples for forward euler.
Convergence for multistep methods. This was show to require consistency (exactness
constraints). This involved the selection of coefficients. And also stability. This was
the region of absolute stability and Dahlquist’s stability barriers.
4.1.18 Stiff Problems - Large Timestep Issues
First an example is introduced.
See the slide for the graph and the equations. The key to notice is that this function is
rapidly decreasing from [0, 5x10 ]. If the interval of interest is [0,5] then you would require
a very small step size to approximate the change in this region. This small step size is only
needed for a very small portion but will end up requiring a lot of steps for the whole interval
[0,5] which is very unnecessary.
4.1.19 Stiff Problems - Large Timesetp Issues
So from the previous example it would be nice if you can take small steps in the beginning
for accuracy and then take large steps later when you have a more ”flat” function you are
trying to approximate.
The stiff problem is:
Natural Time Constants
Input Time Constants
Interval of Intereste
If these are widely separated then the problem is stiff.
4.1.20 Application Problem
This shows an example of a signal transmission line in an IC. It is a 2x2 example. This is
examined and from this you get eignevectors for the problem.
4.1.21 FE on two time-constant circuit
This shows a graph on which FE is accurate for small timesteps but goes unstable when the
timestep is enlarged.
57
4.1.22 BE on two time-constant circuit
With BE it is easy to use small time steps for the fast dynamics and then swith to large
timesteps for slow decay. Recall the region of absolute stability for BE was much large then
FE.
4.1.23 Multistep Methods
This slide outlines the BE, FE, and TR methods on the scalar ODE problem
A scalar ODE example: 2 FE:
" and the solution grows even when 2
If
c BE:
If ) the solution decays even if " 2
2 Trap Rule: )) c
4.1.24 Regions of Stability and Stiff Problems
The next two slides show (as mentioned) how the regions of absolute stabiltity for FE and
BE greatly affect your ability to vary the time step ( ). For FE if you have a
of .1 you
will fall in the region, however if you change to
or 1 you will fall outside of it.
This is not the case with BE. You can change
to either .1 or 1 and stay within the
region of stability.
4.1.25 Stiff Problems
It has been shown that the analysis of stiff problems requires variable step sizes. Not
all linear multistep methods can be efficiently used to integrate stiff problems.
To be able to choose
base only on accuracy considerations, the region of absolute
stability should allow a large
for large time constants without being constrainted
by the small time constants.
Clearly A-Stable methods satisfy this requirement
4.1.26 Backward Differentiation Methods - BDF
This is an introduction to Gear’s Method
Here is a formula which represents a BDF:
& &
&('
3
2
Note that Gear’s first order method is BE
58
It can be shown that Gear’s methods up to order 6 are stiffy stable and are well suited
for stiff ODEs. Gear’s methods of order higher than 6 are not stiffy stable.
Less stringent than A-Stable
4.1.27 Gear’s region of stability
The next two slides just show that as the order of Gear’s method changes, the region of
stability (outside the closed curve) gets smaller.
4.1.28 Variable Step Size
When the step size is changed during the integration, the coefficients of the method
need to be recomputed at each iteration.
Example: Here is what Gear’s Method of order 2 looks like
where 4 32
4.1.29 Variable Step Size
Now based on the previous example of what an order 2 Gear’s Method looks like if you
consider the order 2-polynomial:
x(t) = Now you can rewrite how previous values of t as such:
4
4
4
Now you can write expression which will determine new coefficients for a new desired
step size (for the order
2 polynomial):
) ) )) 4.1.30 More Observations
To minimize the computation time needed to integrate differential equations, the
must be choosen as large as possible provided that the desired accuracy is achieved.
Several approximations are available, SPICE uses divided differences
At a certain time point, different integration methods would allow different step size.
Advantageous to implement a strategy which allows a change of method as well as of
.
59
4.1.31 Summary on Stiff Stability
FE: Timestep is limited by stability and not by accuracy
BE: A-Stable, any timestep could be used
TR: Most accurate A-stable method
Gear: Stiffy stable method up to order 6
The analysis of stiff circuits requires the use of variable timestep.
4.1.32 Multistep Methods - More on Implicit Methods
This slide show Forward Euler is an explicit method while Backward Euler is an implicit
method. Forward Euler requires only function evaluations. However, Backward Euler creates a nonlinear equation at each step. Therefore you need a stepwise Nonlinear equation
every time 32
4.1.33 Multistep Methods - Solution with Newton
This slide shows how the multistep equation can be rewritten so that you can use Newton
&
iteration. This involves recognizing the Jacobian as well as F )
4.1.34 Application of linear multistep methods to circuit equations
We have a set of mixed differential algebraic equations of the form:
F( 32
Using Gear’s method equation: linear equations
with
x( ) as unknowns:
#
( 32
( we obtain a set of algebraic non-
4.1.35 Transcient Analysis Flow
Here is the flow for transcient analysis as covered in this class:
1 Predict the values of variables at 2 Replace C and L with resistive elements via integration methods
3 Replance nonlinear elements with G and indep sources via NR
4 Assemble linear circuit equations
5 Solve linear circuit equations
60
6 Did NR converge? No goto 1, yes goto 7
7 Test solution accuracy
8 Save solution if acceptable
9 Select new
and compute new integration formula coeff.
10 Done; No goto 1
4.1.36 Summary
The last slides give a summary. Essentially this is the same as before but mentions that we
did not cover Runge-Kutta and Predictor-Corrector Methods/
Also it hits on an Appendix to circuit simulation this covers preconditoners:
The convergence rate of iterative methods depends on spectral properties of the coefficient matrix. Hence one may attempt to transform the linear system into one that is
equivalent, but has more favorable spectral properties.
A preconditioner is a matrix that effects such a transformation:
Mx = b c
1
81 The choice of the preconditioner is largely application specific
61
Part II
Analog Simulation
62
Chapter 5
Mixed-signal Behavioral Modeling
with Verilog-AMS
5.1 Introduction
5.1.1 Top-down Design Methodology for Mixed-Signal Design
Traditional circuit designs are divided into two separate domains: digital and analog. However, in recent years, there was a continuous increasing demand for mixed-signal designs,
in which designers have to deal with both digital and analog behaviors within a single design. As shown in Figure 5.1, it is estimated that by year 2006, 66% of design starts will be
mixed-signal designs.
And as the sub-micron technology advances, we reach the capability of making one
million transistors on a single chip. Bigger circuit means much more complexity. Modern
technology advancement creates a huge productivity gap between the maximum possible
number of transistors per chip and number of transistors that can be designed by a designer
Source: Mentor Graphics
Digital
Mixed−Signal
22%
66%
78%
33%
Year 1999
Year 2006
Figure 5.1: The Trend of Design Starts
63
in a month.
So mixed-signal system designers need a design methodology and associated CAD
tools, which can help to manage the overwhelming complexity in terms of the number
of transistors and the cross domain design.
A top-down design methodology, theoretically, can proceed from architecture down to
transistor level, with a hierarchy in which each higher level of design abstraction could be
mapped onto the lower level with design constrains refinement. The success of this design
methodology depends on the correctness of constrain propagation at each intermediate level.
A typically top-down design flow may consist of behavior level, RTL level, gate level,
transistor level, and physical layout.
The key idea of the top-down design methodology is the hierarchical propagation of
constraints based on behavioral modeling and optimization. At each level of the design
hierarchy, performance constraints are mapped onto constraints on the parameters characterizing the blocks of the subsequent level of the hierarchy. At the highest level, we can use
behavioral simulation and optimization to evaluate different architectures. Once we choose
architecture, the process is repeated until the layout is generated or a module meeting the
constraints is found in the library.
5.1.2 Mixed-Signal Behavioral Modeling
The purpose of behavioral modeling is to represent circuit function with abstract mathematical models that are independent of circuit architecture or schematics. In top- down designs,
designers can verify the system design before spending time in detailed circuit implementation, which enables them to explore the system design space rapidly and widely.
Circuit simulation with SPICE or SPECTRE can provide accurate results but it takes a
very long computing time. Especially for complex mixed-signal designs, accurate circuit
simulation is timely unaffordable. And it is often the case that at some design point the
detailed circuit implementation of some sub-blocks are not available. And the function
and performance of the overall block need to be verified and estimated, respectively. The
need of efficient simulation and the capability of mixed-level simulation (MLS) both lead
to behavioral modeling and simulation.
Behavioral modeling and simulation allow for early detection of design faults, efficient
exploration of the design space, rapid design optimization and facilitate concurrent designs.
5.2 Behavioral Modeling with Verilog
5.2.1 Introduction
For digital circuits, behavioral modeling and simulation can be performed using hardware
description languages such as VHDL or Verilog. And the analog mixed-signal (AMS) ex-
64
tension is now available in these languages. Simulink is another popular tool that is used
in behavioral modeling and simulation. Verilog-AMS, compared to the other two counterparts, provides a way of efficient simulation and a smooth transition from behavior level to
circuit level.
Verilog-AMS is a superset of Verilog-HDL and Verilog-A, and incorporates the digital
and analog behavior in a unified environment. It supports the top-down design hierarchy
in the sense that it enables modeling from system level to circuit level and any mixed-level
simulation between. Verilog-AMS combines discrete-event driven simulation (digital) and
continuous-time simulation (analog), and is capable of automatic interface element insertion
so that digital and analog models can be directly interconnected even if their terminal/port
type do not match.
In the digital simulation domain, a discrete-event simulation kernel calculates the discrete signal behavior. These signals describe signal changes at discreet points in time. On
the other hand, the analog simulation is mapped onto a differential equation kernel, which
also calculates signals at discreet points in time as a solution to the differential equations.
The signals are assumed to change continuously between individual time points by interpolation.
The difference in simulation kernels is reflected in the semantics of Verilog-AMS. The
basic behavioral description unit in Verilo-AMS is module.
5.2.2 Digital Signal Domain
In the discrete-event (digital) module, assign statements, initial blocks, and always blocks
are used to describe the behavior of the module. initial blocks set the initial value of the
signals and run only once. always blocks run repeatedly forever and are used with @ statements to defines the signal changes respect to the discrete events. Registers (reg) are internally defined as one-bit variables that hold values between processes.
A positive-edge triggered D flip-flop shown in Figure 5.2 is given below as an example.
Its behavior could be described as follow. Initially Q is set to Qinit. At each positive edge
of Clock, if Reset is not high then set Q to Data. At each positive edge of Reset, set Q low.
Always set Qb to be the inverse of Q.
module DFF1 (Q,Qb,Data,Clock,Reset);
output Q,Qb; input Data,Clock,Reset;
//Signals assume single-bit digital default
parameter Qinit = 0;
reg Q;
//A ’reg’ is a 1-bit variable
initial Q=Qinit;
//Initial section evaluates just once
always @(posedge(Clock))
//Always is a loop that runs constantly: Wait for positive clock edge.
65
D Q
clock
Qb
Reset
Figure 5.2: D flip-flop
if(!Reset) Q=Data;
//If the Reset signal isn’t high, Set register Q to equal the Data input.
always @(posedge(Reset)) Q=1’b0;
//Wait for positive edge of Reset, then Set Q to equal (1-bit binary) zero.
assign Qb = ˜Q;
//Define Qb output to be logical inversion of the Q output.
//Qb updates whenever Q changes.
endmodule
5.2.3 Analog Signal Domain
Additional constructs are used to model analog circuits in Veroilog. A new type of process
(analog module) is introduced, which is continuously applied (like empassign statement in
digital modules). And a new type of interconnection point (node) is added, which allows
multiple floating contributors. Its value varies continuously versus time and is resolved by
simultaneous application of Kirchhoffs laws. In order to support MEMS applications, the
signals could be either electrical or non-electrical (e.g. temperature, velocity, etc.).
Analog Models
Basically there are two types of models used in analog circuits and systems: signal-flow
models and conservative models. Both models are compatible in Verilog-A and can be
freely interconnected.
Signal-flow models relate potentials (voltages) only and are useful for top-level models
in a top-down design. In signal-flow models, there is a notion of input (excitation) and
output (response). Extensions of signal-flow models support both potentials and flows, and
signals could be floating or differential.
Conservative models relates potentials and flows (currents) and are used to model device
and loading at interfaces. In conservative models, there is no notion of input (excitation)
and output (response). The signal values are related in purely mathematical expressions.
66
Sx
Sy
Si
Conservative Model
So
Signal−Flow Model
Figure 5.3: Analog Models
Nodes, Braches, Ports and Modules
In analog modules, nodes are defined as points where branches interconnect and branches
are paths between nodes. Branch equations relate flow and potential of each branch and can
be given by user in the form of behavioral models. User connects branches to nodes using
his structural models. Simulator finds potentials and flows that satisfy branch equations and
Kirchhoffs laws.
Verilog allows you to represent the hierarchy of a design. The Verilog structures which
build the hiererchy are modules and ports. Module is a a collection of nodes and branches,
possibly in the form of submodules (children). Ports are interfaces that allows branches in
module to connect to nodes outside module. A port can be thought of as a wire that goes
from the outside of the module instance (the parent) to the inside of the module instance
(the child). These connections can be either input, output, or bi-directional.
Submodule
Port
Branch
Module
Node
Figure 5.4: Structure Hierarchy
Disciplines
disciplines are used to declares the types of nodes, ports, and branches. It composes one
or more nature definitions into the definition of an analog signal. nature defines a set of
attributes associated with a signal type. Required attributes include absolute tolerance, units
and access function. User- and simulator-defined attributes are optional.
nature Voltage
abstol = 1u;
67
units = V;
access = V;
endnature
discipline electrical
potential Voltage;
flow Current;
enddiscipline
discipline electrical
potential Voltage;
flow Current;
enddiscipline
Predefined conservative disciplines in Verilog-AMS are electrical, magnetic thermal, kinetic(position, velocity), and rotational (phase, velocity). In analog blocks, signals (potentials or flows) are associated with nodes or branches. Their values are determined by analog
kernel, and are correct upon entry to analog block. It means that it accounts for all contributors from all analog processes. The value is independent of order of access and it allows
implicit formulas. Values are affected by using contribution operators. All contributions to
the same signal at the same time point sum up. The conceptual simulation cycle could be
Step 1: Update the values of all input (probed) signals
Step 2: Evaluate all models at time t
Step 3: Resolve the values of all output (sourced) signals
- Sum up multiple contributions
Step 4: check if Kirchhoff’s laws satisfied, if not return to step 1.
Step 5: Advance to next time point
Signal on nodes, ports, branches are probed and set by using access functions defined in
the nature of the signal. Whether the access function probes or set the value of the signal
depends on the context where it is used. Access function used in expressions is to get
(probe) values. For example, x = V(a,b) means x gets voltage between a and b. Access
function used as target in contribution is to set (source) values. For example, V(a,b) + x
use means: set voltage between a and b to x.
The Contribution Operator ’ +’
The contribution operator accumulates potentials or flows to nodes, ports, and branches.
The order of contribution is not significant. And it supports implicit equations (e.g. x +
f(x))
68
module rlc (a, b);
electrical a, b;
parameter R = 1 exclude 0;
parameter C = 1;
parameter L = 1 exclude 0;
analog begin
I(a,b) <+ V(a,b) / R;
I(a,b) <+ C*ddt(V(a,b));
I(a,b) <+ idt(V(a,b) / L;
end
endmodule
a
C
L
R
b
Figure 5.5: Contribution Operator Used in a RLC Circuit
Indirect Assignment
Except for contribution operator, indirect assignment is also used to formulate implicit equations. In Verilog, we cannot both contribute and indirectly assign to a single branch
module ideal_opamp (out, pin, nin);
input pin, nin; output out;
voltage out, pin, nin;
analog
V(out): V(pin,nin) = = 0; //Find V(out) such that V(pin,nin)=0
endmodule
Braches
Modeled behavior is associated with branches, which are either defined explicitly or implicitly. Explicitly defined branches are referred to as named branches. Branch type is
determined by how branch values are accessed. If potential is set, it is a potential source
branch. If flow is set, it is a flow source branch. If neither is set, it is a probe branch. If
potential is probed, it is a potential probe branch. If flow is probed, it is a flow probe branch.
If both are set, it is a switch branch.
Potential source branch is a branch whose potential is assigned by contribution. It has
built-in potential source and built-in potential and flow probes. Flow source branch is a
branch whose flow is assigned by contribution. It has built-in flow source and has built-in
potential and flow probes. We cannot contribute both potential and flow to a single branch.
Probe branch is a branch to which no contribution is made. Potential probe branch is
a branch whose potential is used in an expression. It is a open circuit. Flow probe branch
69
is a branch whose flow is used in an expression. It is a short circuit. Named branches are
explicitly declared and are useful when defining distinct parallel potential branches.
Switch branch is a branch whose potential is occasionally, but not always, assigned by
contribution. It can switch between built-in potential and flow sources. And it has built-in
potential and flow probes.
Formulating Equations for Simulation
It is natural to use either Modified Nodal Analysis (MNA) or Sparse Table Analysis (STA).
In STA, potential and flow for each branch become unknowns. Each branch relation becomes an equation In MNA, unknowns include each node potential and the flow for each
potential branch, flow probe, and switch branch. Equations include the total flow contributions into each node, along with branch relations for each potential branch, flow probe, and
switch branch.
Mathematical Functions and Operators
Standard mathematical functions include Std math, logs, trig, hyperbolics. Random number
functions include uniform, Gaussian, exponential, Poisson, chi-squared, students-T, Erlang.
Analog operators include derivative, integral, absdelay. Analog filters include transition,
slew, Laplace, and z filter. User can define his own functions.
Cross event operator has syntax: cross( expr, direction, time-tolerance, expr-tolerance)
It generates event when expr crosses 0 in specified direction. Timepoint is placed just after
the crossing within tolerances. In order to know exact time of crossing, we need to use
last crossing().
module pfd_cp (out, ref, vco);
current out; voltage ref, vco;
output out; input ref, vco;
parameter Iout = 100u;
integer state;
analog begin
@(cross(V(ref)), +1)
if (state >-1) state = state - 1;
@(cross(V(vco)), +1)
if (state < 1) state = state + 1;
I(out) <+ transition(Iout*state);
end
endmodule
70
Looping and Conditional Statements
Verilog-A provides a complete set of loops and conditionals. if / else is for binary conditional. case is for n-ary conditional. repeat is to repeat a fixed number of times. while is to
repeat until false. for is to repeat until false with initialization and incrementing.
Analysis Function
It indicates whether analysis being run matches specified criteria. They use similar notations
to SPICE. ac is for .ac analysis. dc is for .op or .dc analysis. noise is for .noise analysis.
tran is for .tran analysis. ic is the initial-condition analysis that precedes a transient analysis.
static matches dc or ic.
Example: Capacitor with Initial Condition
module cap (a, b);
electrical a, b;
parameter real c=0, ic=0;
analog begin
if (analysis("ic"))
V(a,b) <+ ic;
else
I(a,b) <+ ddt(c*V(a,b));
end
endmodule
Small signal stimulus functions are used to model stimulus on small-signal analyses
(.ac, .noise). ac stim() is active only during AC analyses. white noise() describes noise constant with frequency, only active in noise analyses. flicker noise() describes noise inversely
proportional to frequency. noise table() determines noise amplitude versus frequency.
5.2.4 Mixed Signal Domain
There are two types of “domains” in Verilog-AMS. Discrete time is used to describe digital
circuits. Continuous time is used to describe analog circuit. It is used in partitioning analog
from digital and determines which solver (analog or digital) is used in solving its behavior.
The auto insertion of connect modules (IE’s) done directly in the simulator uses this partitioning. All analog and mixed signal modules require that ports and nodes associated with
behavioral code have a discipline declared for them.
In general, digital behavior is defined in the initial/always blocks and analog in the
analog block. All three types of blocks can appear in the same module. There can only be
71
one analog block but many digital blocks. Read operations of continuous-time and discretetime signals are allowed from either context. Continuous-time signals only written from
within analog context. Discrete-time signals only written outside of an analog context.
Exapmle-1: Analog Signal / Variable Appearing in Digital Expression
...
reg clock;
real r;
electrical x;
always @(posedge clock) begin
r = V(x);
end
Example-2: Digital Signal / Variable Appearing in Analog Expression
reg d;
electrical x;
analog begin
if (d === 0)
V(x) <+ 0.0;
else
V(x) <+ 3.0;
end
Example-3: Analog Event Appearing in Digital Event Control
electrical x;
reg d;
integer i;
always @(cross(V(x) - 4.5, 1))
i = d;
end
begin
Example-4: Digital Event Appearing in an Analog Event Control
real r;
reg d;
electrical x, y;
analog begin
@(posedge d)
r = V(x);
end
72
Mixed Signal blocks can be all behavioral, all structural, or mixed. They use full capability
of both Verilog-D and Verilog-A. They may have digital and analog sections interact by
sharing data and controlling each other’s events. It allows for event driven modeling of
analog blocks.
Model the D-to-A Interface
If a digital signal changes the analog behavior in any way, the use of the transition function
is strongly recommended. Rise/fall times will otherwise equal the current analog timestep
size. If the step causes a convergence problem, analog simulation may fail (since decreasing time step size won’tt help convergence). If using the digital value (not the event) in the
analog block, then every change of this value will cause an analog time step. When digital
signal changes, the analog sees the former state of the signal, then recomputes the point
with the new value of the signal. If digital signal goes to X or Z, analog simulation immediately fails. When using the analog value in the digital context, Reading of analog values
(accessing V(a) at p4 & p5) does not force analog time steps; the analog value is found by
interpolation (analog simulation always leads digital); the time point for the interpolation is
the current digital time tick converted to the analog real time.
5.3 Summary
Verilog-AMS provides the capability to model and simulate a complete mixed signal intergrated system using consistent Verilog syntax. It supports the top-down design methodology
in the sense that it enables modeling from system level to circuit level and any mixed-level
simulation between.
73
Bibliography
[1] Miller, I.; Cassagnes, T.: Verilog-A and Verilog-AMS provides a new dimension in
modeling and simulation Proceedings of the 2000 3rd IEEE International Caracas
Conference on Devices, Circuits and Systems pp. C49/1 -C49/6
[2] Frey, P.; O’Riordan, D.: Verilog-AMS: Mixed-signal simulation and cross domain
connect modules 2000 IEEE/ACM International Workshop on Behavioral Modeling
and Simulation pp. 103 -108
[3] Miller, I.; FitzPatrick, D.; Aisola, R.: Analog design with Verilog-A Verilog HDL
Conference, 1997.,IEEE International pp. 64 -68
[4] H. Chang, E. Charbon, U. Choudhury, A. Demir, E. Felt, E. Liu, E. Malavasi, A.
Sangiovanni-Vincentelli, and I. Vassiliou: A Top-Down Constraint-Driven Design
Methodology for Analog Integrated Circuit Norwell, MA: Kluwer Academic, 1997.
[5] Kundert, K.; Chang, H.; Jefferies, D.; Lamant, G.; Malavasi, E.; Sendig, F.: Design
of mixed-signal systems-on-a-chip IEEE Transactions on Computer-Aided Design of
Integrated Circuits and Systems Volume: 19 Issue: 12 , Dec. 2000 pp 1561 -1571
[6] Fernando De Bernardinis; Cong Liu Technical Report on Behavioral Modeling and
Simulation of RF LNA and Mixer Sept. 2001
[7] Ira Miller; Thierry Cassagnes: Verilog-AMS Eases Mixed Mode Signal Simulation
MSM 2000 Technical Program March 28,2000
[8] Verilog-AMS web home http://www.eda.org/verilog-ams/
[9] Verilog-AMS Language Reference Manual: Analog & Mixed-Signal Extensions to
Verilog HDL version 2.0. Open Verilog International http://www.ovi.org
74
Chapter 6
Steady State Analysis
6.1 Introduction
Steady State methods are powerful techniques for analyzing complicated, yet structured,
phenomena in circuits. They are essential for RF design and useful in many other contexts.
The numerical techniques discussed in this section include harmonic balance, finite differences, shooting methods, and Chebyshev interval methods, all which use iterative solution
methods and exploit problem structure to achieve efficient algorithms. In order to understand what Joel Phillips had talked about in class, we need at least to include the following
brief introdution about the background material.
When simulating analog circuits, the steady-state behavior of the circuit is of primary
interest. It is easier to characterize and verify certain aspects of system performance in
steady state. Examples of quantities that are best measured when a circuit is in steady state
include distortion, power, frequency, noise, and transfer characteristics such as the gain and
impedance. Therefore, any analog circuit simulator should be able to calculate the steadystate response of a circuit efficiently and accurately.
If an underlying circuit is aymptotically stable, then it is possible to use numerical analysis to find the steady-state solution by choosing arbitary initial conitions and integrating
the circuit equations until any transient decays. It is also possible to determine the steadystate solution of ordinary- and intergro-differential equations by other much faster methods.
In fact, these methods for finding steady-state solutions are more general in nature and useful in attacking many other circuit problems. They are classified as steady-state methods.
For example, because the periodic and quasi-periodic solutions represent steady states of
systems, the frequency domain and time domain methods for computing the periodic and
quasi-periodic solutions of systems are classified as steady state methods.
75
6.2 Steady State
Before introducing steady state methods, we should first clarify what the term steady state
means. In the most general case, the steady-state solution of a differential equation is the
one that is asymptotically approached as the effect of the initial condition dies out. It is
not necessary for the steady state to be unique, but for each steady-state solution there must
correspond a region of attraction, for a steady-state solution will eventually asymptotically
approach that value. For example, a circuit such as a flip-flop with more than one stable state
has distinct initial conditions which eventually result in different steady-state solutions. If
the differential equation is at a steady-state solution and is perturbed slightly and temporarily, it will return to the same solution. Such a solution is referred to as being asymptotically
stable. This definition excludes lossless linear integrators and LC oscillators.
There are several different kind of steady-state behavior that are of interest. The first
one is DC steady state. The DC solution is an equilibrium point of the circuit and is timeindependent. Recall what we have to do for DC analysis in a SPICE simulation. Asymptotically stable linear circuits driven by sinusoidal sources eventually exhibit sinusoidal steady
state solutions. Periodic Steady State (PSS) is the steady state response of a nonlinear circuit, which results either from self-oscillations or periodically varying input. If a nonlinear
circuit is driven with several periodic sources at unrelated frequencies, the steady state response is called quasiperiodic. A quasi-periodic steady state response consists of sinusoids
at the sum and difference frequencies of two or more fundamental frequencies and their
harmonics. There are also steady-state responses that do not fit well in any of the above
classification. These occur when either the input sources are not quasiperiodic or the circuit
is strange. Good examples of these are chaotic circuits and noise as the input when we are
doing noise analysis.
6.3 Steady State Methods
The goal of Joel’s lecture on steady state methods is to help us understand alternate ways of
analyzing differential equations. These methods are generally much faster than other methods in computation time and have many specific applications. Several different numerical
themes such as circuit theory, solutions of ordinary differential equations and differential
algebraic equations(ODEs/DAEs), Newton methods, iterative solvers and preconditioning
are tied to topics in the lecture.
Let’s first review our approach to solveing ODEs/DAEs. We start with an initial condition, on a discretized time axis, then pick up the a next time point and compute the solution
at that point using Newton or other methods. Then we repeat the operations until the end
of whole time interval. This approach suffers from several fundamental drawbacks. First,
transients may take a long time to decay and so require expensive calculations. Second, it
76
can be quite difficult to determine when the transient has died out. This kind of approach
is classified as transient analysis. You can see that the following questions are appropriate
to ask using these kinds of analyses: How does a circuit behaves when it is either driven
by a sinusoidal source, a step input or other unstructured inputs for only a short period of
time? It is even more likely to be applied to analyze other time-domain characteristics such
as delay, rise time, and overshoot calculations. However, it is quite difficult to answer a
different kind of question: How does a circuit behave when it is driven by some input for a
very long time? How much noise does the circuit introduced into a signal? Where does the
noise come from and where does it go? It is also not appropriate for many other frequencydomain questions we often meet in analog analysis regarding small signal stability, Bode
plots, and the positions of poles and zeros.
Historically, there are four general approaches to solve for the steady state response of
the circuits. The first is to accelerate the transient analysis by adapting it to the steadystate problems. It is not classified as one of the steady state methods we want to discuss
here. The other three are the shooting method, the finite-difference method, and harmonic
balance. And all of them were discussed in Joel’s lecture.
To make it clear why steady state methods are important to us, let’s first consider AC
analysis where we need to sweep the frequency of the sources. Transient methods clearly
do not have enough speed to do that. In another case, when we care about the distortion
caused by non-linear circuit elements, we cannot look for the results using transient analysis.
In addition, steady-sate methods provide us more insight into the small signal stability of
circuits. When analyzing the noise in RF circuits, we have no other choice, and steady state
methods are the only vehicles we can rely on.
AC analysis is one of the prototypical steady-state analyses. Again, the disadvantages of
computing AC results through transient method can be seen in the following. Let’s consider
the example: We apply a sinusoidal source at a single frequency to a linear circuit and sweep
the frequency of the source in order to get the Bode/Nyquist plot. We can do the following
in a transient approach: We apply the source at each frequency, then solve an initial value
problem using Trapazoidal, Euler, or some other methods. We will have to wait for a long
enough time for the steady state to be reached. Finally, we need a Fourier-transform to
convert the output to the frequency domain. From this we can see many problems. The
first one is the speed; we must wait many time steps in order to guarantee that the transients
died out and avoid corrupting the steady state. You can try this by applying a 1HZ sinusoid
to an RLC circuit. The second one is from the problem of considering the accuracy of
Fourier analysis. In general, we will interpolate the time domain steady-state results onto
uniform time points and apply an FFT. This is a widely used but very BAD way. Other
problems related to the method are: How to detect the ”onset” of steady-state? Polynomial
interpolation creates high noise floors in Fourier analysis. Truncation errors may corrupt
the spectrum. There are also the errors brought by the endpoints. One good example of
77
this was given in the lecture: Sample a sinusoid at 256 random points and interpolate onto
a uniform 1024-point grid. We cannot see the difference from the exact result in the time
domain plot. But after applying the FFT, we can see the interpolation method is very bad.
6.4 Steady State Analysis
Conventional steady-state analysis includes small signal analysis for linear and non-linear
circuits, stability and noise analysis. More generalized steady-state analysis mostly refers
to the ways of dealing with LARGE signal effects, i.e., NONLINEAR analysis. Examples
we often see are distortion analysis and frequency conversion. In the following sections, we
will go through the principles of those analyses in more detail.
6.4.1 AC Linear Analysis
&
1 % . In the frequency domain, for we have
Consider the linear problem: the Laplace transform sx(s) and the Fourier transform j x. Then we can easily get the
Fourier transform of the equation: j x( )=Ax( )+u. This is the fundamental equation for
AC analysis that works for all linear circuit elements.
6.4.2 AC Small Signal Analysis for Non-Linear Circuits
This analysis is separated in three steps:
Step 1: Find the DC operating point. Recall the ”.op” command in SPICE simulation.
This is evaluated by solving a set of equations given by Kirchhoff’s current/voltage
laws
(KVL and KCL) and the characteristics of circuit elements. We have ^c
2 32 . This DC point is important in circuit analysis, and the small signal property
of the circuit changes with the shift of the DC point.
Step 2: Linearize around the DC point. This is based on the assumption that all small signal
&
. We also assume that
inputs are perturbations around the DC point: the small signal response of circuits can be expressed as a Tailor expansion: .
2
4
! &
Step 3: Solve the AC analysis equation:
. Since we solve
%
equations in the frequency domain, the Fourier analysis of the above method is exact; we
get no truncation error, periodicity error and other errors.
6.4.3 Finding the Poles and Zeros of Linear Circuits
In linear circuit analysis, sometimes we want to know if the design is stable and what its
bandwidth is. We need to find out if all poles are located in the left half plane, the location
of the dominant pole, and the location of the second pole... In AC analysis, we have 78
.4 !
&
! & 2
, and the poles occur at the values of that satisfy the equation: .4
. The equation is often solved by a direct solver such as QZ or a Krylov
method.
6.4.4 Noise Analysis
Noise analysis is an important part of modern circuit design. As we know, a resistor can
produce thermal noise. Any lossy circuit element and unwanted random signal can also be
the noise source. The typical model for circuit noise is a stationary Gaussian process. We
can use the spectrum to characterize them: The spectra for white noise is a constant. Its
autocorrelation function, which is the inverse Fourier transform of power spectra, is a delta
function. To understand these properties, just consider the fact that the data value at any
time is only correlated with itself and has nothing to do with the circuit’s history. Because
of this, white noise should have unlimited bandwidth. Good examples of these are thermal
noise and shot noise. On the other hand, color noise will have limited bandwidth. Noise
propagating through the circuit becomes correlated in time because of the nonzero time
constant of circuit. In the frequency domain, the noise spectrum is shaped by the frequency
response of the circuit. Thus time correlation and frequency shaping describe the same
concept in time and frequency domain respectively. We can use frequency domain analysis
to find the noise power at the output of the circuit or the equivalent noise source at the input.
To do this, we need to compute the small signal tranfer function from each noise source to
the observation point(output). The number of noise sources is usually equal to the number
of the elements in the circuit. Calculating them one by one could be very troublesome. We
can apply adjoint analysis to improve the computation efficiency. What is adjoint analysis?
. Solving 1
is generally very
In standard AC analysis, we need to compute 1
is cheap based on the observation that 1
expensive, but computing 6 1 6 because it is a scalar. Suppose we have one or a few c’s but a lots of b’s, then we can solve
the much smaller matrix equation 1 6
. For all the sources compute . You can
see the difference: In the previous strategy, we computed the transfer function from each
source to all possible outputs, which is named forward analysis; But in adjoint analysis, we
compute the transfer function as the whole from all the sources to a given output. Since in
each circuit we only care about a few outputs, using adjoint analysis can improve the speed.
6.4.5 Distortion Analysis
In RF circuits, we have two kinds of distortion: The first one is harmonic distortion, which
is produced by the nonlinearity of the circuit to the sinusoidal signal at a single frequency.
The other one is called intermodulation distortion. The reason for intermodulation distortion is that additional intermodulation terms will be produced at the output because of the
nonlinearity of the circuit if the input signal has two or more frequencies. Please refer to
79
the class EE242. A major part of that class is to analyze the distortion in RF circuits.
6.4.6 Frequency Translation Analysis
Frequency translation analyses are performed on circuits such as mixers. A mixer is a
nonlinear circuit that is very commonly used in communications to translate a signal from
one frequency to another. A mixer has two inputs; one (RF) is the signal to be translated,
and the other (LO) is the translating frequency. The output predominantly contains two
signals, the RF translated to the sum and difference frequencies of the RF and LO signals.
Usually only the signal at one frequency is desired, and so the mixer is followed by a filter.
Mixers are very difficult to simulate for two reasons. First, the frequencies of the signals
present can be very widely separated. Second, the settling time of the output filter can be
much longer than the period of the lowest frequency present in the mixer. Generally, we
care about the conversion gain and noise of mixer circuits, which were not discussed in
detail in the lecture.
6.5 Periodic Steady State Analysis
Analyzing the harmonic distortion in circuits, computing a mixer conversion gain, and noise
analysis are actually small signal analyses superpositioned on a periodic steady state. All of
them fall into the class of periodic steady-state problems. The distortion problems caused by
intermodulation is are slightly different cases and may be named as quasi-periodic steadysate problems. The periodic steady-state computation can be carried out in the following
way: Apply a sinusoid or other periodic signal at the input, and then find the frequency
response of the circuit in either time domain or frequency domain. In the time domain,
we need to integrate the differential equations over one ”fundamental” time period. In the
frequency domain, we need to solve for the Fourier coefficients at the fundamental and at
harmonics. In both cases, we need to form ”steady state equations” first and then find a fast
and accurate way to solve them.
As we know, differential equations in general have infinite numbers of solutions and it
is necessary to place constraints on the solutions to allow one steady state solution. Based
on the kinds of contrains given in the problems, we have two cases:
1) Initial value problems (IVPs):
Initial value problems specify the value of x at t=0, and require f to be smooth enough to
assure that only one unique solution can be constructed over the entire time period: 2 .
2) Boundary-value problems (BVPs):
A problem of finding a solution to a system is a boundary-value problem if the solution is
required to satisfy subsidiary conditions at two or more distinct points in time. For example:
80
2
gives a periodic boundary-value problem. For the PBCs, if
the solution to the DAE is unique, the solution in one period determines the solution at
every time point. Both the shooting method and harmonic balance method take advantage
, the periodic
of this fact. Except for the extra equation in the above method, 2 boundary condition can also be enforced by the Fourier series over the time interval.
In the following section, we considered the algorithms for solving PSS problems in
more detail.
6.5.1 PSS Algorithm #1: Harmonic Balance
Harmonic balance has been around for many decades. It was originally considered an
approximate technique for finding analytically the near-sinusoidal solution of differential
equations. Later it was reformulated into an accurate method for finding numerically the
solutions of a differential equation driven by sinusoids without having to approximate the
nonlinearities with polynomials. The key ideal of the method is as follows: We pick up the
for the differensteady-state solution
in the form of Fourier series: $ ' . We actually perform a Fourier transform
2 tial equation on the signal to convert from the time domain to its frequecy domain representation, with
and the Fourier transform of the functions f and u, we get equations of coeffi
cients of the steady-state response, which are algebraic functions of the coefficients of the
stimulus. The resulting equations are harmonic balance equations. The harmonic method
is exact for sinusoidal signal and differs from traditional transient analysis in several ways:
It allows us to compute the periodic solution directly and in certain circumstances has significant advantages in terms of accuracy and efficiency. If the steady-state solution consists
of just a few dominant sinusoids, then the harmonic balance method needs only a small
data set to represent the response accurately. The advantage becomes particularly important
when the response contains dominant sinusoids at widely separated frequencies. The major
difficulty of the harmonic balance approach is in determining the response of the non-linear
devices. They are generally evaluated in the time domain. Thus, the harmonic balance
method is usually considered a mix-domain method. Its name was given to it because it was
viewed a method for balancing the currents between linear and non-linear subcircuits. Evaluating the non-linear devices in the time domain is not a fundamental part of the algorithm.
It is the formulation of the circuit equations in the frequency domain that give harmonic
balance its essential characteristics. Thus, harmonic balance can be summarized as being
the method where KCL is formulated in the frequency domain.
Many numerical methods that we have learned are weighted residual methods. We
can also use similar methods to solve the harmonic balance equations. The main idea of
, we can pick basis functions , let
the method is: Assuming we want to solve # $ ') 32 .
, then select weighting function & and force & 4 # $ ') 81
&
& for the
For the harmonic balance method we have # . We use 4
true spectra method (Galerkin). Recalling that a Fourier transform relates the solution at different discrete time points to Fourier coefficients, we can also apply pseudo-spectra method
to force F(x) at evenly-spaced time points and the DFT can be quickly evaluated using the
FFT algorithm. Then the BVP is converted to solve
, where J is the Jacobian matrix
as shown in the lecture slides. After reformulation, they are can be solved using iterative
method such as GMRES, in which good preconditioners are required, although it may be
hard to construct them.
6.6 Periodic Steady State: Finite Differences
6.6.1 The Algorithm
We can begin by writing all of the discretized circuit equations for the entire period in the
form
4
(6.1)
for all variables and equations 2
, which yields n+1 equations. This system of
&
equations must be solved at each time point
, leading to a set of equations in
2 time taking the form
&
& 4
&
&
&
&
(6.2)
The periodic boundary constraint (PBC) forces the waveform to take the same value at
. This implies that
at some time and a time point
one period away, or
(6.3)
The PBC equation can be written in a similar form to the other equations:
4
(6.4)
Now the entire system of equations can be solved. With N circuit equations and M time
points, the system is very large, involving O(MN) unknowns.
Newton for PSS-FD
This initial value problem (IVP) can be solved using Newton iterations to linearize the
equations, and then an iterative matrix solver such as GMRES to determine the solution.
Newton iterations take
the form:
4
4
82
4
(6.5)
where n is the timestep and k is the iteration number.
At each time point n, this equation
becomes
4
(6.6)
4 3a
4 The G entries to the Jacobian structure take the form
of
(6.7)
The
Jacobian
matrix is very sparse, consisting of main diagonal entries
in the form of
, entries on the diagonal line below the main diagonal of 4
, and one element
4
equal to 4
in the upper right corner corresponding to the PBC equation. This extra off
diagonal element causes problems with iterative solvers, and in fact the problem would be
much simpler if it wasn’t for this one entry. The system is very large, so each iteration is
expensive and an algorithm must be found that terminates in a small number of iterations.
6.6.2 GMRES
The General Minimal Residual algorithm (GMRES) is an iterative matrix solver algorithm
similar to some of the algorithms discussed in previous lectures.
The convergence rate of GMRES depends on the eigenvalues of the input matrix. GMRES converges the fastest when the eigenvalues are close together, but in the case of the
basic PSS-FD algorithm the eigenvalues in fact form a unit circle as h _ 0. Therefore, the
GMRES algorithm converges very slowly with this matrix. This convergence problem can
be resolved by using a preconditioner that multiplies the matrix and moves the eigenvalues
to more favorable locations.
6.6.3 Preconditioning
A good choice for the preconditioner matrix P is the inverse of the lower triangular portion
. The entire matrix J=L+U is difficult to invert because
of the PSS-FD matrix, denoted
of the element in the upper right corner due to the PBC equation. However, the lower
triangular piece L is fairly easy to invert. In addition, the upper matrix U consists of only
one nonzero entry in the upper right corner, and the rest is filled with zeros. The original
problem of solving for x=[ ] in the equation Jx=b is converted into the problem
6
, where
of solving PJx=Pb, or
. The new J matrix is now
L and U take the forms:
2
.
2
2
..
2
2
83
2
2
2
..
2
.
4
2
2
(6.8)
Note that each matrix element is itself a matrix. It is important to exploit the structure
of the L and U matrices to achieve maximum efficiency of the algorithm. U is used for
product computation and L is used for solving the system. The problem of calculating
can be split into three steps: First, y=Ux is determined by multiplying
by
the only nonzero element in U resulting in y=[
0
0]6 . Next,
is calculated. Examining the structure of L, it can be seen that a linear time algorithm will
suffice to calculate z. The two diagonal matrix L represents the equations
4
4 c
Finally, x is added to z to get
(6.9)
.
Compressed FD System
If the first or last point of the time interval was known, the rest of the points could easily be
. It turns
obtained by the forward substitution Jx = b. We want to solve for
has the form
out that
2
2
..
2
4
4
.
..
Inverting this matrix, we have
2
2
..
.
2
2
.
..
.
2
2
(6.10)
2
..
.
2
..
.
(6.11)
Now all we need to do is to use GMRES on the much smaller matrix
. The
original system of N equations and M time steps has been reduced (compressed) from size
MN to only size N, and the resulting new eigenvalue pattern leads to faster convergence of
GMRES.
6.6.4 PSS-FD Algorithm Outline
The overall PSS-FD procedure is outlined below.
.
. form equations
. foreach Newton iteration
.
compute the residual r
84
.
.
.
.
.
.
compute the Jacobian matrix J
solve J using the above algorithm
precondition by L and use GMRES on compressed equation
update entire interval through forward substitution update solution waveform on interval
6.7 The Shooting Method
If the value of the solution is known at one timepoint, then the solution over the entire
period can be constructed by solving the initial value problem (IVP). The basic idea of the
Shooting method is to exploit this fact by first guessing x(0), then integrating the waveform
to get x(T), and then updating x(0) to correct the residual. This method iterates on x using
Newton’s method.
6.7.1 Transition Function
2 , where x(0) is the initial condition
The transition function is such that for a forward integration in time. In addition, since the waveform is periodic, 2 .
For example, consider a simple homogeneous linear ODE 1 , which has a solution
2
c
(6.12)
The Shooting method uses Newton iterations to converge on the residual,
2 $ 4
2 (6.13)
which can be written as a differential equation:
4
a
2
4
(6.14)
6.7.2 Sensitivity Analysis
The sensitivity matrix
is defined as
2 (6.15)
The following equations explain how the sensitivity matrix is computed.
4
85
c
(6.16)
2
4
2 4
2
2
This can be written into a recursive formula for
starting condition of
2
2
2
2
c
(6.17)
2
(6.18)
2
(6.19)
The GMRES algorithm needs a vector product , compute, for ,
by replacing M with k+1 and using the
2
for some . Starting with
(6.20)
and for each k solve the equation
4
2
(6.21)
These equations are similar to those derived for the PSS-FD method.
6.7.3 Shooting Algorithm Outline
The Shooting method algorithm is shown below. Notice the similarities and differences
between the Shooting and PSS-FD algorithms.
.
. form equations
. foreach Newton iteration
.
compute the residual r in the periodicity condition
.
integrate the IVP forward to find x(T)
.
solve the Shooting update equation
.
solve the sensitivity matrix equation using GMRES
.
update the first time point x(0)
.
6.8 Algorithm Comparison
The three main algorithms discussed so far (AC, PSS-HB, and PSS-FD) are similar in their
compuation and equation forms, but represent different concepts and require different solution algorithms.
86
6.8.1 AC vs. PSS
AC
PSS-HB
4 9
PSS-FD
#
4
..
..
9
4
..
4
.
..
4
(6.22)
#
.
% .
&
4 4
(6.23)
.
..
.
(6.24)
where in the PSS matrices G is
(6.25)
6.8.2 Shooting vs. PSS-FD
The Shooting method and the PSS-FD method solve the same linear system and use GMRES on matrices with the same convergence properties. Also, both methods lead to the
same equations for computing the sensitivity update.
However, the nonlinear updates and thus Newton iteration convergence properties of
the two methods are different. the Shooting method updates only the end point, while PSSFD updates the entire interval. Shooting satisfies the ODE at every iteration and tries to
converge on the boundary condition. Alternatively, PSS-FD satisfies the boundary condition
at every iteration and tries to converge on the ODE solution.
6.8.3 GMRES Convergence Analysis
The convergence properties of the GMRES algorithm as used in the solution of steady
state problems is worth mentioning. As stated earlier, the convergence rate of GMRES
is dependent on the eigenvalues of the matrix. In a linear model problem, the transition
function matrix is in the form of an exponential, and the eigenvalues of an exponential
matrix are the exponentials of eigenvalues. From the stability standpoint, circuits tend to
be designed to be stable, so the eigenvalues have negative real parts, and all the eigenvalues
87
of the transition function are inside the unit circle. This means that stiff modes go away
quickly.
Differential algebraic equations (DAE’s) typical of RLC circuits described by KCL lead
to matrices where only the dynamic energy storage devices, i.e. capacitors and inductors,
determine the AC components of the matrix. Though these matrices pose problems to transient analysis methods, they in fact are quite easy for PSS analysis.
6.9 Other Topics
6.9.1 Multi-Interval Chebyshev Discretization
The general idea is to partition the waveform into different regions in time (subintervals)
and use higher order iterative analysis methods where possible but lower order methods
where needed. Regions in which the waveform is flat, smooth, and sharply transitioning
can be simulated with different order algorithms or decomposed further into subintervals if
too complex. This method is more efficient than using a purely high-order method.
6.9.2 Small Signal Analysis
Devices that are linear in the input of interest but highly nonlinear in other inputs can often
be approximated with a near-linear simulation. If a small sinusoidal signal is superimposed
on top of a larger, slowly varying signal, small signal analysis can be performed on this
circuit. The circuit is first linearized around the time varying operating point, for instance
using a Taylor series expansion, and then the transfer functions are solved for.
6.9.3 Cyclostationary Noise
Cyclostationary noise is periodically modulated noise resulting when circuits have periodic
operating points. Its autocorrelation and variance are periodic in t and the noise is correlated
in frequency. Generalizations to multiple periodicities and non-periodic variations exist.
Cyclostationary noise is caused by noise sources such as periodic bias currents generating
shot noise and periodic variation in resistance of transistor channels generating thermal
noise. Noise can also originate from the signal path, for example during the modulation
of gain by nonlinear devices and a periodic operating point. Transistors can also control
cyclostationary noise by only allowing it to propagate when the switch is closed.
6.9.4 Other Analyses
Other simulation types include oscillator analyses, quasi-periodic analyses, and envelope
methods. Quasi-periodic analyses can be grouped into steady state for intermodulation
88
distortion, AC and XF for solving for transfer functions, and noise analysis for circuits such
as mixers with blockers.
Two-frequency analysis deals with the case where a signal is composed of two sinusoids
at different frequencies. The simulation time is a function of the ratio of the larger frequency
. These cases can be analyzed using either multito the smaller frequency, or #
#
frequency harmonic balance or MFT shooting methods (see next paragraph).
Mixed Frequency-Time (MFT) algorithms sample quasi-periodic waveforms at the clock
frequency, and only a few cycles and samples are needed. The waveform envelope is represented with a Fourier series and a time-domain method is used to resolve any fast behavior
in each cycle.
Envelope analyses track the non-periodic carrier envelope without tracking all carrier
cycles. These methods are used for RF analog transient analysis.
6.10 Acronyms
AC
BC
BVP
DAE
DC
FD
GMRES
HB
IVP
MFT
MIC
ODE
PSS
RF
XF
Alternating Current
Boundary Condition
Boundary Value Problem
Differential-Algebraic Equation
Direct Current
Finite Difference
General Minimal RESidual
Harmonic Balance
Initial Value Problem
Mixed Frequency-Time
Multi-Interval Chebyshev
Ordinary Differential Equation
Periodic Steady State
Radio Frequency
Transfer Function
6.11 References
Harmonic Balance
http://www.aplac.hut.fi/publications/ecctd-1991-3/main.html
http://www.tec.ufl.edu/ flooxs/cur/floods/floods html/node21.html
89
PSS-FD
http://jamaica.ee.pitt.edu/Archives/ProceedingArchives/Iccad/Iccad2000/
papers/2000/iccad00/pdffiles/06b 1.pdf
MIC
http://www.sigda.org/Archives/ProceedingArchives/Dac/Dac2002/papers/2002/
dac02/pdffiles/33 2.pdf
GMRES
http://www.numa.uni-linz.ac.at/Staff/haase/Lectures/parvor e/node78.html
http://epubs.siam.org/sam-bin/getfile/SIMAX/articles/27909.pdf
Additional Info
See also MATLAB function gmres().
90