18th European Symposium on Computer Aided Process Engineering – ESCAPE 18
Bertrand Braunschweig and Xavier Joulia (Editors)
© 2008 Elsevier B.V./Ltd. All rights reserved.
REDUCTION OF COMPUTATIONAL LOAD ASSOCIATED WITH THE
INTEGRATION OF DAE SYSTEMS
Stefan C. de Graafa, Pål Kittilsena, Heinz A. Preisigb
a
Cybernetica AS, Leirfossveien 27, Trondheim N-7038, Norway
Norwegain University of Science and Technology, Department of Chemical
Engineering, Trondheim N – 7491, Norway
b
Abstract
Dynamic models come often in the form of Differential Algebraic Equations (DAEs).
The integration of DAEs has a high computational load, which is one of the main
obstacles for the real-time implementation of advanced control in chemical processes.
This paper emphasizes that the magnitude of these loads depends mostly on the
eigenvalues and the amount of time necessary for the computation of Jacobians of state
equations AND algebraic equations. Furthermore, it presents an overview of methods to
adjust these Jacobians by altering state equations and algebraic equations, and methods
to accelerate Jacobian computation.
Keywords: computational load, model reduction, DAE system
1. Introduction
The performance of chemical processes can often be improved markedly by
implementing advanced control. Product yields, operation near process constraints and
start-up and shutdown times all benefit from applying model-based control.
Unfortunately, advanced control algorithms often require integration of Differential
Algebraic Equations (DAE). Integrating DAEs is a computationally relatively complex
and consequently relatively slow, which often becomes the main obstacle hindering the
implementation of such control strategies.
With this problem being so central, much, though scattered research about fundamental
causes and possible reduction of high computational loads has been done. Maas (1993),
Hahn, et al. (2003) and Rowley (2005) associate the high computational load with the
number and complexity of state equations. Hedengren and Edgar (2005) and Sun (2006)
looked at the number and complexity of algebraic equations. Nejad (2005) and Averick,
et al. (1994) focused on the computational load of Jacobian matrices.
This research suggests a number of methods resulting in computational procedures that
are more efficient than solving the original problem. Though, with the literature being
as scattered as it is, the practitioner lacks guidelines on what method to choose in a
particular situation. There exists no overview that provides a clear representation of
what the cause of the computational problem is and how it can be resolved. This paper
attempts to fill some of this gap.
2
S.C. de Graaf, et al.
2. DAE-systems and their integration
In this paper it is assumed that the following DAE-system causes high computational
loads:
x  f  x, z , u, d 
x  t0   x0
0  g  x, z , u, d 
(1)
(2)
where x  nx , z  nz , u  nu , d  nd . The vector x is the system state vector, z the
vector with algebraic variables, d the disturbance vector and u the control input vector.
The function vector f is a system state function vector and the vector g is a function
vector for computing the algebraic variables z.
Note that the DAE system has index 1. If the system is of a higher index then the system
should first be reduced to an index 1 representation, because most software for
integration of DAE systems are developed for index-1 systems (Asher and Petzold,
1998).
A DAE or stiff ODE system is often integrated by using the backward difference
formulae (BDF) methods, which is a well-known multi-step method. The general form
of this method is:
 f (tn ) 
 x (tn )  k
 x (tn i ) 
 z (t )    i  z (t )   h0  g (t ) 
 n  i 1  n i 
 n 
(3)
where and αi and β0 are constants, h is the step size in time and n denotes the mesh
number. It shows that the BDF methods use several past values of the variable x and its
rates of change f and g with respect to time t to move the integration forward from tn-1 to
tn. This is done in an iterative way until a convergence criterion is met (Asher and
Petzold, 1998).
At a fixed integration horizon, the computational load associated to the application of
BDF methods depends on the step size, the convergence criterion and the Jacobian that
is required for solving equation 3 by using a Newton iteration. Note that this Jacobian
contains partial derivatives of state equations and algebraic equations with respect to
states and algebraic variables. The computational load is inversely proportional to the
step size, whose product with the eigenvalues of the Jacobian has to be small enough to
meet the convergence criteria. The load increases mostly with the number of non-zero
elements in the Jacobians, because they have to be computed for every move.
As the eigenvalues and number of non-zero elements of the Jacobian depend on the
equations, model adjustments that result in a reduction of the difference between the
largest and the smallest eigenvalues may reduce the computational load. But most of all,
a reduction of the number and nonlinearity of equations may reduce the computational
load.
Reduction of computational load associated with integration of DAE systems
3
3. Methods that reduce computational loads
The eigenvalues and computation of non-zero elements of Jacobians can be modified on
two complementary levels: (i) state and algebraic equations of DAE systems; (ii)
implementation of DAE-systems in integration routines.
3.1. Modifying state and algebraic equations of DAE systems
As the state and algebraic equations affect the eigenvalues and the number of the nonzero elements in Jacobians, they should both be reduced. Most reduction algorithms for
state equations create new algebraic equations, so it is wise to reduce the number of
state equations before reducing the number of algebraic equations.
3.1.1. State equations
Many of the methods achieve a higher computational efficiency by reducing the state
space. This is usually done in two steps: First the original model is transformed into
another state space, which enables the second step, namely to split the model into three
parts, one for which one assumes event dynamics, one that shows dynamic behaviour
and a third, which can be assumed to be constant in the dynamic scale of interest for the
application.
The state is transformed linearly:
x  Tx  Tf T 1 x , u , z , d 
(4)
The consequent splitting into the three different dynamic regimes is based on eigenvalue
criteria of a matrix associated with the transformation matrix (for example Hahn, et al.
(2002)) with the indices 1, 2, 3 representing the fast, the dynamic and the constant part,
the split model reads:
 T f T 1 x, u , z , d  
0  1 

 x   T f T 1 x, u , z , d 


2
2
  

 x3  

0


(5)
Beside the assumptions made about the dynamics, the three groups of states also have
other properties, that are determined by the approach chosen to compute the
transformation matrix. Properties that stem from three published approaches are
summarized in Table 1.
This transformation and splitting of the state equations reduces the computational load
only because of the assumption that was made for the states of the third group. The
algebraic and state equations for the dynamics of the states that belong to the first and
second group are not likely to reduce the number of non-zero elements in the Jacobian.
In the latter case a reduction is only achieved if the event dynamic part has an explicit
solution allowing for the complete elimination of the fast state variables. Reducing the
computational load for the algebraic equations (2) and (5) is the topic of the next
section.
4
S.C. de Graaf, et al.
Table 1. Three approaches to transform and split state equations in sub-models, and the properties
of these sub-models
References
x1
x2
x3
Maas (1993), Bykov, et al. (2005),
Skodje and Davis (2001)
Fast changing
states
Moderately changing
states
Slowly changing
states
Löffler and Marquardt (1991)
Fast changing
states that are
controllable
Moderately changing
states that are
controllable
States that are
uncontrollable
Hahn, et al. (2002), Hahn and Edgar
(2003), Willcox and Peraire (2001)
Fast changing
states that are
controllable and
observable
Moderately changing
states that are
controllable and
observable
States that are
uncontrollable or
unobservable or
both
3.1.2. Algebraic equations
Reduction of the complexity and number of algebraic equations can be seen as the
identification of new, but simpler algebraic equations that approximate the original set
sufficiently well in the context of the application. Linear equations can be approximated
in different ways: For example by (multiple) linear regression in connection with
discriminant analysis, principal component analysis and sub-space methods. Nonlinear
relationships can for example be captured in an artificial neural network.
The approximation with linear algebraic equations is preferable over the use of
nonlinear equations. This is because those elements in the Jacobians that stem from the
linear algebraic equations are constant and thus there is no need to recalculate them
when Jacobians are updated, which reduces the computational load.
3.2. Adjusting the implementation of DAE-systems in integration routines
Choosing a particular implementation of DAE-systems in the integration routines may
also reduce the computational load. Three possibilities are discussed here: solving
algebraic equations externally, use of pre-computed look-up tables and interpolation as
an alternative to evaluate the respective model equations for each step, and providing an
analytical representation of the Jacobians to the integrator.
3.2.1. Solving algebraic equations externally
Solving algebraic equations in functions that are programmed in another language is
relevant when equation-based languages, e.g. gPROMS, are used. Equation-based
languages solve all state equations together with the implicit and explicit algebraic
equations equation-system, which may lead to high computational loads. The only
alternative to reduce the computational load is to solve the algebraic equations
externally using a highly optimized C++ or Fortran code. Algorithm-based languages,
such as MATLAB, make it easy to use alternative or separated code.
Reduction of computational load associated with integration of DAE systems
3.2.2. Use of look-up tables
Algebraic equations can be solved off-line for different values of the dependent
variables. These solutions can be stored in tables and retrieved and interpolated during
on-line applications. The computational load associated to the off-line calculations is
high, but the computational load of the on-line applications is usually significantly
lower.
3.2.3. Efficient computation and storage of Jacobians
Providing coded analytical Jacobians and efficient storage of Jacobians may lead to
lower computational loads than one observes with the approximation of Jacobians by
numerical differentiation. The latter has the advantage that only the function is needed
as a black box. However, the accuracy of these approximations may be hard to asses.
The computational load is approximatley n+1 times the computational costs associated
with the computation of the function value f(x), where n is the number of independent
variables.
Manual coding of Jacobians produces accurate and efficient code. However, it is often
time consuming and error prone, especially if the function is complex. These
disadvantages can be partially overcome by using symbolic manipulation packages such
as Maple and Mathematica. It should be mentioned though that although symbolic
differentiation is a powerful technique, it quickly runs into resource limitations when
applied to even moderately sized problems.
Automatic differentiation is a better approach for generating code for derivatives. This
technique relies on the fact that every function, no matter how complicated, is executed
on a computer as a (potentially very long) sequence of elementary operations such as
additions, multiplications, and elementary functions. By applying the chain rule to the
composition of those elementary operations, one can compute derivative information of
f exactly and in a completely mechanical fashion. This approach lead to exact
derivatives and the computational load of the so-called reversed mode is independent of
the number of independent variables (Al Seyab, 2006).
The load related to computing and storing Jacobians may also be reduced when
elements of the Jacobians are zero or when directional derivatives that are used to
compute the elements are equal to zero. In these cases only the non-zero elements and
non-zero directional derivatives need to be computed and stored, which reduces the load
of computing and storing Jacobians. This is called sparse and compressed structure
storage of Jacobians and its computational load is relatively small for large DAE
systems. Software is available that scans Jacobians for these storage possibilities (Forth
and Edvall, 2006).
4. Concluding remarks
This paper emphasizes that the magnitude of computational load for integration of
DAEs depends mostly on the eigenvalues and the amount of time necessary for the
computation of Jacobians of state equations AND algebraic equations. Furthermore, it
5
6
S.C. de Graaf, et al.
presents various methods to adjust these Jacobians by altering state equations and
algebraic equations, and methods to accelerate Jacobian computation. A combination of
these methods will lead to a dynamic model with a lower computational load.
References
R.K. Al Seyab, 2006, Nonlinear model predictive control uising automatic differentiation (thesis),
Cranfield University, United Kingdom
U.M. Ascher, L. Petzold, 1998, Computer Methods for Ordinary Differential Equations and
Differential-Algebraic Equations, Society for industrial and applied mathematics
B.M. Averick, J.J. More, C.H. Bishof, A. Carle and A. Griewank, 1994, Computing large sparse
Jacobian matrices using automatic differentiation. SIAM Journal on scientific computing 15, 2,
285-294
V. Bykov, I. Goldfarb, V. Gol’dshtein, 2005, Novel numerical decomposition approaches for
multiscale combustion and kinteic models, Journal of physiscs: Conference series 22, 1-29
S.A. Forth and M.M. Edvall, 2006, User guide for MAD-- a Matlab Automatic Differentiation
Toolbox, TOMLAB/MAD, Version 1.4, The Forward Mode,
http://tomopt.com/tomlab/download/manuals.php
J. Hahn and T.F. Edgar, 2002, An improved method of nonlinear model reduction using balancing
of emperical gramians. Computers and chemical engineering 26, 1379-1397
J. Hahn, T.F. Edgar and W Marquardt, 2003, Controllability and observability covaraince
matrices for the analysis and order reduction of stable nonlinear systems, Journal of process
control, 13, 115-127
J. D. Hedengren and T.F. Edgar, 2005, Order reduction of large scale DAE models, Computers
and chemical engineering 29, 2069-2077
U. Maas, 1993, Automatische reduktion van reaktionsmechnismen zur simulation reaktiver
strömungen, thesis, Institut für Technische Verbrennung der Universität Stuttgart
H.-P. Löffler, W. Marquardt, 1991, Order reduction of non-linear differential-alebraic process
models, Journal of process control, 1, 32-40
L.A.M. Nejad, 2005, A comparison of stiff ODE solvers for astrochemical kinetics problems,
Astrophysics and space science, 299, 1-29
C.W. Rowley, 2005, Model reduction for fluids, using balanced proper orthogonal decomposition,
International journal of bifurcation and chaos, 15(3), 997-1013
R.T. Skodje, M.J. Davis, 2001, Geometrical simplification of complex kinetic systems, Journal of
physical chemistry, 105, 10356-10365
C. Sun, 2006, Model reduction of systems exhibiting two-time scale behaviour or parametric
encertainty (thesis), Texas A&M University
K. Willcox, J. Peraire, 2001, Balanced model reduction via the proper orthogonal decomposition,
Proceedings of the 15th AIAA Computational fluid dynamics conference, Anaheim, CA