Parameterized Model Order
Reduction via Quasi-Convex
Optimization
Kin Cheong Sou
with Luca Daniel and Alexandre Megretski
Systems on Chip or Package
RF Inductors
Interconnect & Substrate
MEM
resonators
Courtesy of Harris semiconductor
Analog RF
Mixed Signal
ADC
I
LNA
ADC
10/22/2010
LO
Digital
DSP
Q
2
From 3D Geometry to Circuit Model
Fig. thanks to Coventor
• Need accurate mathematical models of components
• Describe components using Maxwell equations, NavierStokes equations, etc
10/22/2010
3
From 3D Geometry to Circuit Model
 4u
 2u
 2u
EI 4  S 2  Felec   ( p  pa )dy   2
x
x
t
0
w
dH
dt
dE
H  
dt
  E  
Z(f)
dv
C (v )
 G (v)  Bv in
dt
  ((1  6 K )u 3 pp )  12 
 ( pu )
t
dH
dt
dE
H  
dt
  E  
Z(f)
Z(f)
Z(f)
• Model generated by available field solver
• Field solver models usually high order
10/22/2010
4
RF Inductor Model Reduction
• Spiral radio frequency (RF) inductor
• Impedance Z  f   R  f   j 2 f L  f 
• State space model has 1576 states
• Reduced model has 8 states
-7
x 10
4
1.5
x 10
3
1
2
L
R
2.5
0
1.5
1
x full 1576 states
- reduced 8 states
10/22/2010
x full 1576 states
- reduced 8 states
-0.5
-1
0.5
0
1
0.5
-1.5
1.1
1.2
1.3
1.4
f
1.5
1.6
1.7
1.8
1.9
2
10
x 10
1
1.1
1.2
1.3
1.4
f
1.5
1.6
1.7
1.8
1.9
2
10
x 10
5
RF Inductor Parameter Dependency
• Parameter dependent RF inductor
• Two design parameters:
- Wire width w
- Wire separation d
d
w
-8
x 10
12000
5
8000
6000
3
2
L
R
4
d = 1um
d = 3um
d = 5um
10000
1
d = 1um
d = 3um
d = 5um
0
-1
4000
-2
-3
2000
-4
0
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
10
10/22/2010
f
x 10
-5
0
0.2
0.4
0.6
0.8
f
1
1.2
1.4
1.6
1.8
2
10
x 10
6
Parameterized Reduced Modeling
• One reduced model with
 d , w
explicit dependency on
parameters
• Fast generation of
Parameterized
reduced model for all
reduced model
parameter values
d
w
Gr(d,w)
reduced
model
ADC
I
LNA
ADC
10/22/2010
LO
DSP
Q
7
Parameterized Reduced Model Example
• Parameter dependent complex system
s

G  s, d  
s
99
99
 0.5  d   s  2d 
 0.499  d   s  d 
• Parameterized reduced order model
s  2d
Gr  s, d  
 G ( s, d )
sd
• Coefficients depend explicitly on d
• Low order, inexpensive to simulate
10/22/2010
8
Continuous/Discrete-time Setups
Continuous-time
s
z 1
z 1
left-half plane & imaginary axis
G  s  & G  j
10/22/2010
Discrete-time
z
s
s
unit disk & unit circle
G  z  & G  e j 
9
Parameterized Model Reduction Methods
• Parameterized moment matching methods
- References:
• [Grimme et al. AML 99]
• [Daniel et al. TCAD 04]
• [Pileggi et al. ICCAD 05]
• [Bai et al. ICCAD 07]
- Reduced model order increases with number of parameters rapidly
- Require knowledge of state space model
• Rational transfer function fitting methods
- Does not require state space model
- Reduced model order does not increase with number of parameters
- More expensive than moment matching in general
10/22/2010
10
Moment Matching Method
Reduced model
Full model
G  z   C  zI  A B  D
1
Gr  z   CV  zI  UAV  UB  D
1
Projection with
UV = I
moments matched
10
with the moment matching properties
d
G  zk  
dz
( nk )
10
Gr  zk 
Magnitude (abs)
d
dz
( nk )
10
10
10
10
user specified
10
Bode Diagram
2
8th order full
4th order MM
0
-2
-4
-6
-8
-10
10
-1
10
0
10
1
10
2
10
3
10
4
10
5
Frequency (rad/sec)
10/22/2010
11
Rational Transfer Function Fitting
• Idea from system ID – reduced model matching I/O data
input
p  z
Gr  z  
q  z
p  z  ?
output
q  z  ?
• Data in time domain or frequency domain
• Data from state space model or experiment measurement
10/22/2010
12
Explanations in Two Steps
• Will present a rational transfer function fitting method
• First describe basic non-parameterized reduction
• Then extend basic method to parameterized setup
10/22/2010
13
Non-Parameterized Model
Order Reduction
Non-parameterized Problem Statement
• Given transfer function G(z)
• Find parameterized reduced model of order r
p  z  pr z r 
Gr  z  

q  z  qr z r 
 p0
 q0
dec. vars.
• Reduced model found as the solution
minimize
p ,q
subject to
p  z
G  z 
q  z
q  z  stable
H norm error

roots inside unit circle
• Can obtain state space realization from p(z) and q(z)
10/22/2010
15
Difficulty with H Norm Reduction
• Difficulty #1: stability constraint not convex if r >2
q1  z    z 
but
q2  z    z 

3 3
5
q1 q2
27
3
 z 
z
2 2
25

3 3
5
convex combo. of stable poly.
not necessarily stable
• Difficulty #2: abs. value on the “wrong” side
p  z
G  z 
q  z
10/22/2010

iff
branching
solutions
G  e j  q  e j   p  e j    q  e j 


16
Idea From Optimal Hankel Reduction
min G  Gr
Gr
anti-stable
relaxation

min G  Gr  F
Gr , F
s.t. Gr stable
s.t. Gr stable, F anti-stable
order Gr   r
order Gr   r
suboptimal
solution
10/22/2010
redefine
dec. var.
Solve AAK problem
efficiently (Glover)
min G  Q 
Obtain Gr ,Han s.t.
G  Gr ,Han



n
  G 
i  r 1
i
Q
s.t.
Q  H  ( r )
17
Anti-Stable Relaxation in Rational Fit
• Similar to Hankel reduction, add anti-stable term
minimize
p ,q
subject to
f  z 1 
p  z
G  z 

q  z  q  z 1 

added
DOF
flip poles
of q(z)
q  z  stable, deg  f   r
• In Hankel MR, entire anti-stable term is decision variable
• Here, only numerator f is decision variable
10/22/2010
18
Combine Stable and Anti-stable Terms
• Combine stable and anti-stable terms in reduced model
f  z 1 
p  z
b  z   jc  z 


1
q  z q  z 
a  z
• New decision variables are trigonometric polynomials
a  z   a0  a1  z  z 1  
 ar  z r  z  r 
b  z   b0  b1  z  z 1  
 br  z r  z  r 
c  z 
10/22/2010
1
j

c1  z  z 1  
 cr  z r  z  r 

a  e j   a0  a1 cos   
b  e j   b0  b1 cos   
c  e j   c1 sin   
19
Stability and Positivity
• Can show
a  e j   0, 
q  z  stable
• Overcome Difficulty #1,
trigonometric positivity
 convex constraint
e.g. 1  a1 cos    a2 cos  2   0
1.5
G  e j  a  e j   b  e j   jc  e j 
  a  e j 
1
0.5
a2
• Overcome Difficulty #2,
the trouble making abs.
value is gone!
0
a1
-0.5
  a  e j 
-1
10/22/2010
20
-1.5
-1.5
-1
-0.5
0
0.5
1
1.5
Quasi-Convex Relaxation
• Original optimal H norm model reduction problem
minimize
p ,q
subject to
p  z
G  z 
q  z

q  z  stable
• Quasi-convex relaxation
minimize
a ,b,c
subject to
10/22/2010
b  z   jc  z 
G  z 
a  z
a  z   0, for z  1

Quasi-convex
problem,
easy to solve
21
From Relaxation Back to H Reduction
• Obtain (a,b,c) by solving quasi-convex relaxation
• Spectral factorize a to obtain stable denominator q*
a  z   Kq*  z  q*  z 1 
for some K
• With q* found, search for numerator p* by solving
p  z
p  arg min G  z   *
q  z
p
*
10/22/2010
convex problem

22
Quality of Suboptimal Reduced Model
• Minimizing upper bound of Hankel norm error
b  z   jc  z 
G  z 
a  z

1
p  z f  z 
 G  z 

q  z  q  z 1 

p z
 G  z 
q  z
H
• H norm error upper bound
p*  z 
G  z  *
q  z
10/22/2010


b  z   jc  z 
  r  1   min G  z  
 a ,b,c
a  z





23
Back to Big Picture – Model Reduction
min G  Gr
Gr

discussed
b  jc
min G 
a ,b , c
a

s.t. a  z   0,  z  1
s.t. Gr stable
order Gr   r
How to
solve it?
discussed
discussed
suboptimal p(z), q(z)
10/22/2010
optimal a(z), b(z), c(z)
24
Quasi-Convex Optimization
Quasi-Convex Optimization
• Quasi-convex function is “almost convex”
J(x)
Function not necessarily convex
All sub-level sets
are convex sets
x
Local (also global) minima
Local (but not global) minima
• [Outer loop] Bisection search for objective value
• [Inner loop] Convex feasibility problem (e.g. LP, SDP)
• Convex problem algorithms: 1) interior-point method
2) cutting plane method
10/22/2010
26
Cutting Plane Method
optimal point
covering set
iterate 2
Oracle
iterate 1
removed
kept
kept
removed
• Optimization problem data described by oracle
• What is the oracle in our model reduction problem?
10/22/2010
27
Model Reduction Oracles
• Given candidate a(z), b(z), c(z), check two conditions
Oracle #1 (objective value):
b  z   jc  z 
G  z 
a  z

for any fixed 

 G  e j  a  e j   b  e j   c  e j    a  e j 

Discretize frequency  finite number of linear inequalities, “easy”
Oracle #2 (positive denominator):
a  e j   0, 
10/22/2010
Cannot discretize frequency!
28
Positivity Check
• Check only finite number of stationary points
6
r = 8 case
5
stationary points
4
 
a e
j
3
2
1
0
-1
-2
0
0.5
1
1.5

2
2.5
3
3.5
t
• Much harder to check in the parameterized case
10/22/2010
29
Back to Big Picture – Model Reduction
min G  Gr
Gr

discussed
b  jc
min G 
a ,b , c
a

s.t. a  z   0,  z  1
s.t. Gr stable
order Gr   r
Solved with
cutting plane
method
discussed
discussed
suboptimal p(z), q(z)
10/22/2010
optimal a(z), b(z), c(z)
30
Parameterized Model Order
Reduction
Problem Statement
• Given parameter dependent transfer function G(z,d)
• Find parameterized reduced model of order r
p  z, d  pr  d  z r 
Gr  z, d  

q  z, d  qr  d  z r 
 p0  d 
 q0  d 
• Reduced model found as the solution
minimize
max G  z, d   Gr  z, d 
subject to
q  z, d  stable for all d
p ,q
10/22/2010
d
design
parameter

32
Parameterized Reduced Model Example
• Parameter dependent complex system
z

G  z, d  
z
99
99
  z  2d 
 0.499  d   z  d 
 0.5  d
• Parameterized reduced order model
z  2d
Gr  z, d  
 G ( z, d )
zd
• Coefficients depend explicitly on d
• Low order, inexpensive to simulate
10/22/2010
33
Parameterized Decision Variables
• Decision variables = parameterized trig. poly.
a  z , d   a0  d   a1  d   z  z 1  
 ar  d   z r  z  r 
b  z, d   b0  d   b1  d   z  z 1  
 br  d   z r  z  r 
c  z, d  
1
j

c1  d   z  z 1  
 cr  d   z r  z  r 
• When evaluated on unit circle, i.e.
c  e j , d   2c1  d  sin   
ze
j
 2cr  d  sin  r 
a  e j , d   a0  d   2a1  d  cos   
10/22/2010

 2ar  d  cos  r 
34
Parameterized Quasi-Convex Relaxation
• Parameterized quasi-convex relaxation
minimize
a ,b,c
subject to
b  z , d   jc  z , d 
G  z, d  
a  z, d 

a  z, d   0, for z  1 and d
• Solution technique similar to non-parameterized case
• Some extension requires more care, e.g.
check a  z, d   0, for z  1 and d
Parameterized positivity check is hard!
10/22/2010
35
Parameterized Positivity Check
denominator
a  e j , d   a0  d   a1  d  cos    a2  d  cos  2  
…
a simple
parameter
dependency
ai  d   poly of d
d  d1  d 2 cos  
denominator = multivariate trigonometric polynomial
e.g.
cos  3  cos    5cos  2  cos  
• Positivity check of multivariable trig. poly. is hard
• Another variant is multivariable ordinary polynomial
10/22/2010
our focus
36
Positivity Check of
Multivariate Polynomials
Checking Polynomial Positivity –
Special Cases
• Univariate case simple, check the roots of derivative
4
3
2
3
x

2
x

x
40 ?
Is it true for all x,
• Multivariate quadratic form is easy but important
 x1 
2 x12  x2 2  3x32  6 x1 x2  4 x1 x3   x2 
x 
 3
polynomial nonnegative
10/22/2010
T
 2 3 2   x1 
 3 1 0  x   0 ?

 2 
 2 0 3   x3 
matrix positive semidefinite
38
Checking Polynomial Positivity –
General Case
• Positivity check of general multivariate polynomial is hard
Question: 2 x14  5 x24  x12 x22  2 x13 x2  0 ?
[from Parrilo & Lall]
• What if we still write out “quadratic form”?
 x 


4
4
2 2
3
2 x1  5 x2  x1 x2  2 x1 x2   x 
 x1 x2 


2
1
2
2
Monomials of
relevant degrees
10/22/2010
T
 q11
q
 12
 q13
q12
q22
q23
q13   x12 
 2 

q23  x2 

q33   x1 x2 
= Q (Gram matrix)
39
Checking Polynomial Positivity –
General Case
• To find Q, equate coefficients of all monomials
x13 x2 :
2  2q13
x x : 1  2q12  q33
x x : 0  2q23
2 2
1 2
3
1 2
x14 :
2  q11
4
2
5  q22
x :
Generally, linear constraints
on Q, i.e. L(Q) = 0
• Gram matrix Q is typically not unique. If we can find Q ≥ 0
 x 


4
4
2 2
3
2 x1  5 x2  x1 x2  2 x1 x2   x 
 x1 x2 


2
1
2
2
10/22/2010
T




Q
2

x
 1 
  x2   0
 2 
  x1 x2 
40
Semidefinite Program/LMI Optimization
• Standard form:
Read Boyd and Vandenberghe’s SIAM review
minimize
L0  Q 
subject to
L Q   0
Q  QT  0
Q
linear objective
linear constraints
pos. def. matrix variable
• Efficiently solvable in theory and practice
• Polynomial-time algorithm available
• Efficient free solvers: SeDuMi, SDPT3, etc.
• Lots of applications
• KYP lemma, Lyapunov function search, filter design,
circuit sizing, MAX-CUT, robust optimization …
10/22/2010
41
Positivity Check is Sufficient Only
Quadratic case
General case
2 x12  x2 2  3x32  6 x1 x2  4 x1 x3
 x1 
  x2 
x 
 3
T
 2 3 2   x1 
 3 1 0  x 

 2 
 2 0 3   x3 
spans R3
2 x14  5 x24  x12 x22  2 x13 x2
 x 


 x 
 x1 x2 


2
1
2
2
T




Q
2

x
 1 
  x2 
 2 
  x1 x2 
does not span R3
• Requiring Q ≥ 0 sufficient but not necessary!
x12 x24  x14 x22  1  3x12 x22
10/22/2010
Positive? Can you find Q ≥ 0?
42
Sum of Squares (SOS)
• Finding Q ≥ 0 equivalent to sum of squares decomposition
• In our example, we can find
 2 3 1 
 2  2 
0 0
1   
1   


Q   3 5 0  3 3  1 1

 2   
2   
 1 0 5
 1   1 
 3  3
T
T
2
2
1
1 2
2
2
2 x  5 x  x x  2 x x   2 x1  3x2  x1 x2    x2  3x1 x2 
2
2
4
1
4
2
2 2
1 2
3
1 2
sum of squares  positive semidefinite Q  nonnegativity
10/22/2010
43
Wrap Up
4 Turn RF Inductor PMOR
d
• 4 turn RF inductor with substrate
• Circle: training data
• Triangle: test data
w
x full model
- QCO PROM
5
4.5
15
3.5
10
Q
d ( m)
4
3
5
2.5
2
0
1.5
1
1
10/22/2010
1.5
2
2.5
3
3.5
W ( m)
4
4.5
5
0
1
2
3
4
5
f (Hz)
6
7
8
9
10
9
x 10
45
Summary (1)
• Motivation for model reduction in design automation
• PDE  high order ODE  reduced ODE
• Parameterized reduced modeling facilitates design
• Model reduction based on rational transfer function fitting
• H problem difficult, resort to anti-stable relaxation
• Relaxation easy to solve, closely related to H problem
• Quasi-convex optimization
• Efficient algorithms exist (e.g. cutting plane method)
• Cutting plane method in model reduction setting
10/22/2010
46
Summary (2)
• Parameterized model reduction
• Reduced rational transfer function, coefficients are
function of design parameters
• Easily extended from non-parameterized case, except
positivity check is difficult
• Positivity check of multivariate polynomials
• Univariate case easy, quadratic case easy
• General case requires semidefinite programs, only
sufficient
• Related to sum of squares optimization
10/22/2010
47
Some References (1)
• Parameterized reduced modeling
• Moment matching: Eric Grimme’s PhD thesis
• Parameterized moment matching:
L. Daniel, O. Siong, C. L., K. Lee, and J. White, “A multiparameter moment matching model reduction
approach for generating geometrically parameterized interconnect performance models,” IEEE Trans. on
Computer-Aided Design of Integrated Circuits and Systems, vol. 23, no. 5, pp. 678–693.
• Parameterized rational fitting:
Kin Cheong Sou; Megretski, A.; Daniel, L.; , "A Quasi-Convex Optimization Approach to Parameterized
Model Order Reduction," Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions
on , vol.27, no.3, pp.456-469, March 2008
• MIMO rational fitting/interpolation:
A. Sootla, G. Kotsalis, A. Rantzer, “Multivariable Optimization-Based Model Reduction”, IEEE
Transactions on Automatic Control, 54:10, pp. 2477-2480, October 2009
Lefteriu, S. and Antoulas, A. C. 2010. A new approach to modeling multiport systems from frequencydomain data. Trans. Comp.-Aided Des. Integ. Cir. Sys. 29, 1 (Jan. 2010), 14-27
10/22/2010
48
Some References (2)
• Convex/quasi-convex optimization
• Convex optimization:
S. Boyd and L. Vandenberghe, “Convex Optimization”, Cambridge University Press, 2004.
• Ellipsoid Cutting plane method:
Bland, Robert G., Goldfarb, Donald, Todd, Michael J. Feature Article--The Ellipsoid Method: A Survey
OPERATIONS RESEARCH 1981 29: 1039-1091
• Multivariate polynomials and sum of squares
• Ordinary polynomial case: Pablo Parrilo’s PhD thesis
• Trigonometric polynomial case:
B. Dumitrescu, “Positive Trigonometric Polynomials and Signal Processing Applications”, Springer,
2007
10/22/2010
49