Model Reduction of Inhomogeneous Initial Conditions
Caleb Magruder
Abstract— Our goal is to develop model reduction processes for linear dynamical systems with non-zero initial conditions. Standard model reduction schemes optimize system
input/output characteristics but frequently destroy initial condition state space information. Additionally, existing model
reduction schemes for initial conditions inappropriately assign
weight to initial condition information. We propose a projectionbased model reduction scheme that combines subspaces from
differing model reduction approaches and compare it to existing
schemes.
I. P ROBLEM S TATEMENT
Given a LTI system with inhomogeneous initial conditions:


 ẋ(t) = Ax(t) + bu(t)
G:
y(t) = cT x(t)


x(0) = x0 6= 0
We desire reduced order system Gr and initial condition
xr,0 that emulates the full order system for a variety of
inputs including the zero input, or homogeneous differential
equation.


 ẋr (t) = Ar xr (t) + br u(t)
Gr :
yr (t) = cTr xr (t)


xr (0) = xr,0 6= 0
Via the Laplace transformation we can write the transfer
functions, G(s) and Gr (s) where Y (s) = G(s)U (s) and
Y (s) = Gr (s)U (s) assuming that x(0) = xr (0) = 0. Then
the transfer function of the two systems are
G(s) = cT [sI − A]−1 b
Gr (s) =
cTr [sIr
− Ar ]
−1
br
The transfer function of a dynamical system represents the
steady-state frequency response of the system. Consequently,
the transfer function ignores transient information including
the initial conditions.
Most model reduction techniques seek to minimize the
error system evaluated on purely imaginary frequencies,
G(jω)−Gr (jω). These methods then ignore initial condition
information thus motivating alternative approaches to model
reduction.
Remark To simplify notation, we assume the system to be
Single-Input, Single-Output (SISO), meaning that b and c
are column vectors. All theorems and proofs below generalize to Multiple-Input, Multiple-Output (MIMO) systems.
C. Magruder is with the Department of Mathematics, Virginia Tech,
Blacksburg, VA, 24061-0123, USA calebm@vt.edu
II. P ROJECTION - BASED M ODEL R EDUCTION
Let T
V and W be r-dimensional subspaces of Rn such
that V W ⊥ = {0}. (No vector in V is orthogonal to any
vector in W except the trivial 0 vector.) Choose matrices
V, W ∈ Rn×r so that V = Ran(V) and W = Ran(W).
Then we know WT V to be nonsingular. Without loss of
generality we assume WT V = Ir .
We can approximate the state space x(t) ∈ Rn with an rdimensional state, xr (t) ∈ Rr such that x(t) ≈ Vxr (t).
If we set the error Vẋr (t) − AVxr (t) − bu(t) to be
perpendicular to the W space we’ve constructed a PetrovGalerkin approximation to the linear dynamical system. Then
WT (Vẋr (t) − AVxr (t) − bu(t)) = 0
yr (t) = cT Vẋr (t)
Petrov-Galerkin approximation is a common technique in
projection-based model reduction. Then our reduced model
state-space parameters become Ar = WT AV, br = WT b,
and cTr = cT V. Clearly our choice of V and W determine
the accuracy of our reduced order approximation.
There are three leading categories of projection based
model reduction: SVD-based, Interpolation-based and Proper
Orthogonal Decomposition. All three approaches preserve
different characteristics of original system with varying
success. Proposed in this paper is a technique to combine
methods to preserve both initial condition information as well
as input-output behavior.
A. Projection of Initial Conditions
Consider the Laplace transform of the reduced system.
Write X(s) = L{x(t)}. Then
sX(s) − x(0) = AX(s) + bu
Then X(s) ≈ VXr (s) satisfies
WT (VXr (s) − x(0) − AVXr (s) − bU (s)) = 0
So our reduced model initial conditions can be written
xr (0) = WT x(0).
III. M OTIVATION
Current model reduction methods seek to maintain inputoutput (I/O) behavior of the full and reduced systems.
However, often these methods destroy initial condition information.
To demonstrate this consider the projection matrix W.
Denote N (X) to be the null space of the matrix X. Then
N (W) is an (n − r)-dimensional subspace of Rn . Since
r << n then the dimension of the null spaces of the
projection matrices V and W are large. Consequently,
components of x(0) in N (W) are destroyed in the projection
xr (0) = WT x(0).
Consider Figure III for a zero-input simulation, also known
as the solution to the homogeneous differential equation, for
full and reduced order systems constructed with a interpolatory model reduction algorithm. x(0) was chosen to be
nearly orthogonal to W. Clearly transient initial condition
information is not preserved.
1500
1000
In general P 6= Q. To perform a balanced truncation then we first balance the system, then truncate less
significant states. Setting balanced truncation in a PetrovGalerkin framework, let P = UUT and Q = LLT . Let
UT L = ZSYT be the singular value decomposition with
S = diag(µ1 , . . . , µn ). Let Sr = diag(µ1 , . . . , µr ). Let
W = LYr Sr−1/2
V = UZr Sr−1/2
Using W and V as Petrov-Galerkin approximations we
can construct a balanced truncated system. Balanced truncation was developed to minimize the error system H∞ norm,
kG − Gr kH∞ where
kGkH∞ = max |G(jω)|
ω∈R
In fact, the error system H∞ error is bounded
500
0
kG − Gr kH∞ ≤ 2
p
eig(PQ) and ηi+1 ≤ ηi are the Hankel
where ηi =
singular values of the dynamical system.
−1000
Fig. 1.
ηi
i=r+1
−500
−1500
n
X
B. Interpolatory Reduction
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Zero-Input Responses for Full (Blue) and Reduced (Red)
IV. M ODEL R EDUCTION M ETHODS
Three predominant methods for model reduction are introduced here. Their descriptions are intended to serve as
brief introductions to the methodologies. Further reading is
encouraged as each method has an enormous amount of
supporting literature. See [1] for a survey of model reduction
approaches.
A. Balanced Truncation
The balanced truncation method is a generalization of
SVD-approximation, or optimal 2-norm matrix approximation applied to linear dynamical systems.
We introduce a concept called a balancing transformation
where the states are transformed and ordered in a such a
way that leading states are easy to observe and easy to
reach. We then later truncate states that are difficult to
observe and difficult to reach as they are less influential in
I/O behavior. A more formal definition follows.
Definition 4.1: Define the observability gramian, Q, and
reachability gramian, P, be solutions to the Lyapunov equations:
AP + PAT + bbT = 0
AT Q + QA + ccT = 0
Then we say a system is balanced if P = Q.
The transfer function G(s) is a complex-valued rational
function of degree n. We seek a rational interpolant of degree
r, Gr (s), such that Gr (s) interpolates G(s) at a set of
complex frequencies {σi } ⊂ C.
Interpolatory model reduction derives from Krylovsubspace projection frameworks. Given a set {σi } ⊂ C we
can construct Krylov-subspaces, V and W:
V = [(σ1 I − A)−1 b, . . . , (σr I − A)−1 b]
W = [(σ1 I − AT )−1 cT , . . . , (σr I − AT )−1 cT ]
We can then show that Gr (s) is an Hermite interpolant of
G(s) such that
G(σi ) = Gr (σi )
G′ (σi ) = G′r (σi )
See [4].
We are not given however a choice of interpolation points
a priori. Write the dynamical system norm, H2 ,
kGkH2 =
sZ
∞
|G(jω)|2 dω
−∞
Meier and Luenberger, [7] show necessary conditions
for H2 optimal rational function, that is Gr (s) is a local
minimizer of kG − Gr kH2 :
G(−λi ) = Gr (−λi )
G′ (−λi ) = G′r (−λi )
where λi = eig(Ar ). Then {−λi } are our interpolation
points are not known a priori. Therefore finding interpolation
points that meet the necessary conditions above is difficult.
Gugercin, Antoulas, Beattie suggest an iterated rational
krylov algorithm (IRKA) to determine this optimal interpolation points {σi } to meet necessary conditions above. See
[4].
C. Proper Orthogonal Decomposition Methods
Like balanced trunction, proper orthogonal decomposition
(POD) is also a generalization of SVD methods applied
to linear dynamical systems. In fact it can be shown that
balanced truncation is a special case of POD.
Given a ”favorite” input u(t), let x(t) denote the solution
to the differential equation ẋ(t) = Ax(t) + bu(t). Choose
a discrete time scale tk = k∆t. Construct a snapshot matrix
X = [x(t0 ), x(t1 ), . . . , x(tN )]. Let our projection matrix
V be the leading left-hand singular vectors of X. So if
X = UΣZ = [u1 , . . . , ur , ur+1 , . . . un ]ΣZ then V =
[u1 , . . . , ur ].
We construct the projection matrix W in a similar way
from the Hermitian adjoint of the transfer function. Let x̃
˙
denote the solution to the adjoint x̃(t)
= AT x̃(t) + cu(t).
Let W be the leading singular vectors of the adjoint snapshot
matrix X̃ = [x̃(t0 ), x̃(t1 ), . . . , x̃(tN )].
POD can be used to create reduced order models that
approximate full scale systems for very specific inputs.
Unfortunately POD doesn’t work well for inputs, u dissimilar
to the choice of the ”favorite” input. In practice POD is run
on a family of inputs and projection subspaces are combined
together.
V. I NJECTION S YSTEM
Y(s) = cT (sI − A)−1 bU (s) + cT (sI − A)−1 x0
U (s)
T
−1
Y(s) = c (sI − A) [b x0 ]
1
We can then represent the dynamical system with nonzero
initial conditions as a multiple-input single-output (MISO).
The second input is fed with an impulse response δ(t) setting
the states variables at t = 0 to x(0).
b
x(0)
(sI − A)−1
cT
In a paper written by Heinkenschloss, Reis and Antoulas,
they refer to the MISO injection system above as an extended
system. See [5]. Model reduction is run on the MISO transfer
function:
Ĥ(s) = cTr (sI − A)−1 [b
x(0)]
To our knowledge, extended model reduction methods are
the only methods existing in literature currently. We propose
an alternative later in this paper and use extended methods
as a basis for comparison.
A. Extended Balanced Reduction
The following theorem is taken directly from [5].
Theorem 6.1: Let V, W be projection matrices and η1 ≥
· · · ≥ ηr ≥ ηr+1 ≥ · · · ≥ ηn ≥ 0 be the Hankel
singular values, generated by applying balanced truncation
model reduction to the appended system, H(s) = cTr [sI −
A]−1 [b x(0)]. Let γ = ηr+1 + . . . + ηn . Moreover, let
Σ̂ = diag(η1 , . . . , ηr ) be the controllability Gramian of the
reduced system and let Q = LT L be a factorization of the
observability Gramian Q of the extended system. Then
ky − ŷkL2 (t0 ,∞) ≤ 2γkukL2(t0 ,∞) +
1/3
γ 2/3
3 · 2−1/3 kLAx(0)k2 + kΣ̂1/2 Ar xr (0)k2
Then our time-domain error is bounded but dependent on
kx(0)k relative to kbk.
B. Extended Interpolatory Reduction
In the frequency domain we show:
u
VI. MISO E XTENDED M ETHODS
y
δ
Clearly then our choice of projections V and W determine
the efficacy of our input-output (I/O) and initial condition
(IC) systems alike.
Denote the transfer functions of I/O and IC systems
as H = cT (sI − A)−1 b and H̃ = cT (sI − A)−1 x(0)
respectively. Our reduced order I/O and IC systems then will
be written Hr = cTr (sIr − Ar )−1 br and H̃r = cTr (sIr −
Ar )−1 xr (0).
From MIMO H2 optimal model reduction we know that
kĜ − Ĝr kH2 meets the necessary conditions for optimality
if
Ĝ(−λ̃k ) = Ĝr (−λ̃k )
′
Ĝ (−λ̃k )b̃k = Ĝ′r (−λ̃k )b̃k
Where λ̃k and b̃Tk are the poles and residues from the
modal expansion
P of the extended reduced system Ĝr such
that Ĝr (s) = k s−1λ̃ b̃Tk , [4].
k
Similar to extended balanced reduction, the algorithm will
be run on the MISO system, Ĝ(s) = cT [sI−A]−1 [b x(0)].
Note that the choice of optimal shifts is ultimately dependent
on the magnitude of the IC system relative to the I/O system,
or kx(0)k/kbk.
A shortcoming of both extended methods is that they try to
minimize output of the initial condition error system, G̃− G̃r
across all possible bounded energy inputs. This is wasteful
for the initial condition system as it is fed with a very specific
input, δ(t).
To address this problem we propose a scheme that creates
projection subspaces for I/O and IC system separately and
combines them afterwards. This allows us to deliberately
assign dimensions of each approximation and approximate
the IC system fed with the zero-input and the I/O for a broad
range of inputs.
VII. P ROPOSED M ETHOD : A S UBSPACE D IRECT S UM
A PPROACH
We propose an algorithm to combine Petrov-Galerkin projection subspaces via a direct sum and demonstrate that this
approach outperforms extended model reduction frameworks.
First note that
Ran([V1 , V2 ]) = Ran(V1 ) ⊕ Ran(V2 )
We will let V1 = Ran(V1 ) be a subspace that approximates I/O behavior well and V2 = Ran(V2 ) be a subspace
that approximates the zero-input initial condition solution
well. Combining both subspaces with a direct sum aggregates
the better attributes of each subspace.
Recall that POD approximates output for a specific input. We then choose u(t) = 0 to compute the zero-input
response, x(t) = eAt x(0), also known as the solution to the
homogeneous differential equation.
Let X = [x(t0 ), . . . , x(tN )] where x(t) = eAt x(0) and
˜ = eAT t c
of its adjoint, X̃ = [x̃(t0 ), . . . , x̃(tN )] where x(t)
be snapshot matrices of the IC system and its adjoint respectively. Then construct POD-based Petrov-Galerkin projection
matrices
VP OD = Ran(VP OD )
WP OD = Ran(WP OD )
where VP OD and WP OD are the leading left-hand singular
vectors of the snapshot matrices X and X̃ respectively.
The dimension of the POD subspaces can be determined
by the decay of the singular values, µi and µ̃i of the snapshot
matrices. This is a major advantage over the extended methods as we can choose the accuracy of the zero-input response
approximation independent of our I/O model reduction.
The reduced order model constructed from VP OD and
WP OD subspaces then approximate the full order system
for initial condition information well but approximate other
I/O behavior poorly. Hence we turn to model reduction
techniques intended to match I/O behavior across all inputs
such as balanced truncation or rational interpolation.
Choose a model reduction technique and construct PetrovGalerkin projection matrices:
VMR = Ran(VMR )
WMR = Ran(WMR )
Then the reduced order model constructed from VMR and
WMR minimizes I/O error but very well may destroy initial
condition information. See Section III.
Combining IC system behavior of our POD projection
subspaces with I/O behavior of the alternate subspaces, we
take the direct sum of both
V = VMR ⊕ VP OD
W = WMR ⊕ WP OD
To accomplish this set V = [VMR , VP OD ] and W =
[WMR , WP OD ]. Then V = Ran(V) and W = Ran(W).
Our reduced order model then takes on attributes of both
model reduction frameworks.
Algorithm 7.1: Model reduction with initial conditions via
direct sum methods:
1) Choose a model reduction technique for I/O error
minimization. Construct matrices VMR , WMR .
2) Construct snapshot matrices, X = [x(t0 ), . . . , x(tN )]
and X̃ = [x̃(t0 ), . . . , x̃(tN )], from the state space
solutions to the homogeneous differential equation and
˜ = eAT t c.
its adjoint, x(t) = eAt x(0) and x(t)
3) Write VP OD and WP OD to be the left-hand singular
vectors of snapshot matrices X and X̃ respectively.
4) Let V = [VMR , VP OD ] and W = [WMR , WP OD ]
Correct the projectors so that they
T meet PetrovGalerkin requirements, Ran(W) Ran(V)⊥ . (Let
W̃ = (WT V)−1 W and Ṽ = V. Then W̃Ṽ = Ir ).
5) Finally, let the reduced order state parameters be Ar =
W̃T AṼ, br = W̃T b, and cr = cṼ.
VIII. N UMERICAL R ESULTS
We compare 6 model reduction schemes:
1) IRKA: Iterated Rational Krylov w/o zero-input approximation
2) IRKA ⊕ POD: IRKA with POD-based zero-input
approximation
3) IRKA Extended: IRKA with the initial conditions
treated as a second input
4) BalTrunc: balanced truncation w/o zero-input approximation
5) BalTrunc ⊕ POD: balanced truncation with PODbased zero-input approximation
6) BalTrunc Extended: blanaced truncation with the initial
conditions treated as a second input
Additionally, we apply the following metrics to compare
each algorithm:
(I/O)
(ZI)
kH − Hr kH2
kHkH2
kyzi − yzi,r k∞
kyzi k∞
where yzi and yzi,r are the zero-input responses to the full
and reduced order systems respectively.
We compare model reduction schemes for two models: the
CD Player and the Heat model:
A. CD Player Model
This system describes the dynamics between the lens
actuator and the radial arm position of a portable CD player.
The model has 120 states with a single input and a single
output.
B. Heat Diffusion Model
This system is a described by the heat equation with one
heat source and one point of measurement. The model has
an order of 197 obtained by a spacial discretization.
For more information about these models see [3].
Zero Input Response
Bode Diagram
1500
Full
IRKA, r=6, w/o POD
IRKA, r=4, w/ POD, r=2
40
Full
IRKA, r=6, w/o POD
IRKA, r=4, w/ POD, r=2
20
1000
0
500
Magnitude (dB)
−20
−40
0
−60
−500
−80
−100
−1000
−120
−140
0
2
10
10
Frequency (rad/sec)
4
6
10
10
−1500
0
0.1
0.2
(a) Bode Plot
IRKA
I/O
0.0104
0.0074
0.0042
0.0038
ZI
1.0574
1.0961
1.0929
0.9757
IRKA ⊕
I/O
0.0230
0.0104
0.0074
0.0042
POD = 2
ZI
0.1138
0.1138
0.0361
0.0321
IRKA
I/O
0.2284
0.0891
0.0525
0.0027
ZI
0.9634
1.2336
1.1644
0.4092
0.5
Time, t
0.6
0.7
0.8
0.9
1
IRKA ⊕
I/O
0.3499
0.2250
0.0686
0.0194
POD = 1
ZI
0.2825
0.2982
0.0297
0.0455
CD Player Model Reduction Comparison
IRKA
I/O
0.0366
0.0225
0.0087
0.0061
Fig. 3.
r
Metric
2
3
4
5
0.4
(b) Zero-Input Response Simulation
Fig. 2.
r
Metric
6
8
10
12
0.3
Extend
ZI
0.1282
0.0541
0.0576
0.0617
BalTrunc
I/O
ZI
0.0104 1.0542
0.0074 1.1010
0.0042 1.1014
0.0039 0.9238
BalTrunc ⊕ POD = 2
I/O
ZI
0.0230
0.1142
0.0104
0.1161
0.0074
0.0355
0.0042
0.0328
BalTrunc
I/O
0.0464
0.0223
0.0222
0.0078
Extend
ZI
0.1332
0.0461
0.0460
0.0513
BalTrunc ⊕ POD = 1
I/O
ZI
0.3831
0.3446
0.2250
0.3666
0.0770
0.0838
0.0195
0.0781
BalTrunc
I/O
0.2285
0.0972
0.0527
0.0028
Extend
ZI
0.9674
1.1748
1.1657
0.3924
Heat Model Reduction Comparison
IRKA
I/O
0.2284
0.0891
0.0525
0.0031
Extend
ZI
0.9634
1.2336
1.1644
0.3763
BalTrunc
I/O
ZI
0.2285 0.9674
0.0970 1.1710
0.0527 1.1659
0.0027 0.4058
One can see from the numerical results that while the
extended methods perform comparably to the direct sum
methods for the CD Player example, the extend methods
severly underperform the proposed methods for the Heat
model. This is largely due to the relative magnitues of
the initial condition and input vectors for the heat model.
−1
in this case.)
( kx(0)k
kbk ≈ 10
The improvement of results of subspace direct sum methods over extended methods can be attributed to the following
result:
Remark Traditional model reduction techniques seek to
bound I/O error for all inputs
R ∞ over all possible bounded
energy inputs, L(R) = {u : −∞ |u(t)|2 dt < ∞}.
Extended model reduction techniques then are minimizing
the x(0) input to the injection system above for all possible
bounded energy inputs. This is an unnecessary expense in our
model reduction since we know δ(t) to be the only input for
the IC system.
Hence treating a dynamical system with nonzero initial
conditions as an injection system constrains model reduction
if the second input is allowed to be fed any possible bounded
energy function.
IX. C ONCLUSION
In conclusion x(0) should be included in the model
reduction scheme. Existing extended method approaches
introduced by [5] try to do this but may assign inappropriate
weight to IC and I/O systems. We resolve this by introducing an approach that approximates each system separately
allowing us assign precisely the attention to each system
necessary.
Numerical results confirm that subspace direct sum
schemes outperform extended methods in several respects.
First, direct sum methods are not sensitive to the magnitude
of initial conditions, kx(0)k. Extended methods can be
thought of as averaging the I/O system with the IC system
instead of addressing them separately in the direct sum
methods.
Second, the appropriate dimension of the reduced order
IC system can be computed a priori by measuring the decay
rate of the singular values of the snapshot matrices, X and
X̃. With extended methods the weight assigned to the IC
system is unclear.
Additionally, the direct sum framework combining POD
and Interpolation preserves interpolation in the combined
reduced model. That is, the Petrov-Galerkin projection subspaces, V = VIRKA ⊕ VP OD and W = WIRKA ⊕ WP OD
still result in a reduced model, Gr (s), that interpolates
G(s) at the interpolation points, {σi }, originally chosen to
construct VIRKA and WIRKA .
R EFERENCES
[1] A. C. Antoulas, Approximation of Large-Scale Dynamical Systems,
Philadelphia: Society for Industrial and Applied Mathematics, 2005.
[2] A. C. Antoulas, C. Beattie, and S. Gugercin, Interpolatory model
reduction for large-scale linear dynamical systems, Efficient Modeling
and Control of Large-Scale Systems, J. Mohammadpour and K.
Grigoriadis Eds., Springer-Verlag, ISBN 978-1-4419-5756-6, in-press,
2010.
[3] A.C. Antoulas, D.C. Sorensen, and S. Gugercin, A survey of model
reduction methods for large-scale systems, Structured Matrices in
Operator Theory, Numerical Analysis, Control, Signal and Image
Processing, Contemporary Mathematics, AMS publications, 280:193219, 2001.
[4] S. Gugercin, A. C. Antoulas, and C. Beattie, H2 model reduction for
large-scale linear dynamical systems, SIAM J. Matrix Anal. Appl.,
vol. 30, no. 2, pp. 609-638, June 2008.
[5] M. Heinkenschloss, T. Reis, A. C. Antoulas, Balanced truncation
model reduction for systems with inhomogeneous initial conditions,
Automatica, vol. 47, pp. 559-564, 2011.
[6] Holmes, P. and Lumley, J. L. and Berkooz, G., Turbulence, Coherent
Structures, Dynamical Systems and Symmetry, Cambridge: Cambridge
University Press, 1996.
[7] L. Meier and D.G. Luenberger, Approximation of Linear Constant
Systems, IEEE. Trans. Automat. Contr., Vol. 12, pp. 585-588, 1967.
[8] Sirovich, L., Turbulence and the Dynamics of Coherent Structures,
Quart. Applied Math., vol. 45, pp. 561-571, 1987.