Introduction to Differential Algebraic Equations
Dr. Abebe Geletu
Ilmenau University of Technology
Department of Simulation and Optimal Processes (SOP)
Winter Semester 2011/12
Introduction to Differential Algebraic Equations
TU Ilmenau
4.1 Definition and Properties of DAEs
A system of equations that is of the form
F (t, x, ẋ) = 0
is called a differential algebraic equation (DAE) if the Jacobian
n
matrix ∂F
∂ ẋ is singular (non-invertible); where, for each t, x(t) ∈ R
and


F1 (t, x(t), ẋ(t))
 F2 (t, x(t), ẋ(t)) 


F (t, x(t), ẋ(t)) = 
.
..


.
Fn (t, x(t), ẋ(t))
Introduction to Differential Algebraic Equations
TU Ilmenau
4.1 Definition and Properties of DAEs ...
Example: The system
x1 − ẋ1 + 1 = 0
(1)
ẋ1 x2 + 2 = 0
(2)
is a DAE. To see this, determine the Jacobian ∂F
∂ ẋ of
x1 − ẋ1 + 1
F (t, x, ẋ) =
ẋ1 x2 + 2
ẋ1
with ẋ =
, so that
ẋ2
! ∂F1
∂F1
∂F
∂F
−1 0
∂
ẋ
∂
ẋ
1
2
= ∂F2 ∂F2 =
, ( see that, det
= 0).
x2 0
∂ ẋ
∂ ẋ
∂ ẋ1
∂ ẋ2
⇒ the Jacobian is a singular matrix irrespective of the values of x2 .
Observe that: in this example the derivative ẋ2 does not appear.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.1 Definition and Properties of DAEs ...
Solving for ẋ1 from the first equation x1 − ẋ1 + 1 = 0 we get
ẋ1 = x1 + 1. Replace this for ẋ1 in the second equation ẋ1 x2 + 2 = 0
to wire the DAE in equations (23) & (23) equivalently as:
ẋ1 = x1 + 1
(x1 + 1)x2 + 2 = 0
(3)
(4)
In this DAE:
• equation (3) is a differential equation; while
• equation (4) is an algebraic equation.
⇒ There are several engineering applications that have such model
equations.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.1 Definition and Properties of DAEs ...
Suppose F (t, x, ẋ) = A(t)ẋ + B(t)x + d(t).
Hence, for the system F (t, x, ẋ) = 0 the Jacobian will be
I If A(t) is a non-singular (an invertible) matrix, then
∂F
∂ ẋ
= A(t).
[A(t)]−1 (A(t)ẋ(t) + B(t)x(t) + d(t)) = [A(t)]−1 0
⇒ ẋ(t) + [A(t)]−1 B(t)x(t) + [A(t)]−1 d(t)) = 0
⇒ ẋ(t) = − [A(t)]−1 B(t)x(t) − [A(t)]−1 d(t)).
This is an ordinary differential equation.
Remark
In general, if the Jacobian matrix ∂F
∂ ẋ is non-singular (invertible),
then the system F (t, x, ẋ) = 0 can be transformed into an ordinary
differential equation (ODE) of the form ẋ = f (t, x). Some numerical
solution methods for ODE models have been already discussed.
I Therefore, the most interesting case is when
Introduction to Differential Algebraic Equations
TU Ilmenau
∂F
∂ ẋ
is singular.
4.2. Some DAE models from engineering
applications
• There are several engineering applications that lead DAE model
equations.
Examples: process engineering, mechanical engineering and
mechatronics ( multibody Systems eg. robot dynamics, car dynamics,
etc), electrical engineering (eg. electrical network systems, etc), water
distribution network systems, thermodynamic systems, etc.
Frequently, DAEs arise from practical applications as:
I differential equations describing the dynamics of the process, plus
I algebraic equations describing:
• laws of conservation of energy, mass, charge, current, etc.
• mass, molar, entropy balance equations, etc.
• desired constraints on the dynamics of the process.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.2. Some DAE models from engineering
applications ... CSTR
An isothermal CSTR
A B → C.
Model equation:
= Fa − F
Fa
ĊA =
(CA0 − CA ) − R1
V
Fa
ĊB = − CB + R1 − R2
V
Fa
ĊC = − CC + R2
V
CB
0 = CA −
Keq
0 = R2 − k2 CB
V̇
Introduction to Differential Algebraic Equations
TU Ilmenau
(5)
(6)
(7)
(8)
(9)
(10)
4.2. Some DAE models from practical applications
...a CSTR...
• Fa -feed flow rate of A
• CA0 -feed concentration of A
• R1 , R2 - rates of reactions
• F - product withdrawal rate
• CA , CB , CC - concentration of species A, B and C, resp., in the
mixture.
Definining
x = (V, CA , CB , CC )>
z = (R1 , R2 )>
the CSTR model equation can be written in the form
ẋ = f (x, z)
0 = g(x, z).
Introduction to Differential Algebraic Equations
TU Ilmenau
Some DAE models from practical applications...a
simple pendulum
Newton’s Law:
mẍ = − Fl x
mÿ = mg Fl y
Conservation of mechanical
energy: x2 + y 2 = l2
(DAE)
ẋ1 = x3
ẋ2 = x4
F
x1
ml
F
ẋ4 = g x2
l
0 = x2 + y 2 − l 2 .
ẋ3 = −
Introduction to Differential Algebraic Equations
TU Ilmenau
Some DAE models from practical applications...an
RLC circuit
Kirchhoff’s voltage and current laws
yield:
conservation of current:
iE = iR , iR = iC , iC = iL
conservation of energy:
VR + VL + VC + VE = 0
Ohm’s Laws:
C V̇C = iC , LV̇L = iL , VR = RiR
Introduction to Differential Algebraic Equations
TU Ilmenau
Some DAE models from practical applications...an
RLC circuit...
After replacing iR with iE and iC with iL we get a reduced
DAE:
1
iL
C
1
iL
V̇L =
L
0 = VR + RiE
V̇C
=
(11)
(12)
(13)
0 = VE + VR + VC + VL
(14)
0 = iL − iE
(15)
(16)
Define x(t) = (VC , VL , VR , iL , iE )
Introduction to Differential Algebraic Equations
TU Ilmenau
Some DAE models from practical applications...an
RLC circuit...
The RLC system can be written as:
1
0 0 0
C
0 1 0 0
L

ẋ = 
0 0 0 0
0 0 0 0
0 0 0 0

0 0 1 0

1 1 1 0
0 =
0 0 0 1

0
0

0
x
0
0

 
R
0


0 x + 1 VE
−1
0
(17)
(18)
which is of the form
Introduction to Differential Algebraic Equations
TU Ilmenau
ẋ = Ax
(19)
0 = Bx + Dz.
(20)
4.3. Classification of DAEs
• Frequently, DAEs posses mathematical structure that are specific to
a given application area.
• As a result we have non-linear DAEs, linear DAEs, etc.
• In fact, a knowledge on the mathematical structure of a DAE
facilitates the selection of model-specific algorithms and appropriate
software.
Nonlinear DAEs:
In the DAE F (t, x, ẋ) = 0 if the function F is nonlinear w.r.t. any one
of t, x or ẋ, then it is said to be a nonlinear DAE.
Linear DAEs: A DAE of the form
A(t)ẋ(t) + B(t)x(t) = c(t),
where A(t) and B(t) are n × n matrices, is linear. If A(t) ≡ A and
B(t) ≡ B, then we have time-invariant linear DAE.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.3. Classification of DAEs...
Semi-explicit DAEs:
A DAE given in the form
ẋ = f (t, x, z)
(21)
0 = g(t, x, z)
(22)
• Note that the derivative of the variable z doesn’t appear in the DAE.
• Such a variable z is called an algebraic variable; while x is called a
differential variable.
• The equation 0 = g(t, x, z) called algebraic equation or a
constraint.
Examples:
• The DAE model given for the RLC circuit, the CSTR and the simple
pendulum are all semi-explicit form.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.3. Classification of DAEs...
A fully-implicit DAEs:
The DAE
F (t, x, ẋ) = 0
is in fully-implicit form.
Examples:
(i) F (t, x, ẋ) = Aẋ + Bx + b(t) is a fully-implicit DAE.
(ii) The equation (see equations (23) & (23) )
x1 − ẋ1 + 1 = 0
ẋ1 x2 + 2 = 0
is a fully-implicit DAE.
I Any fully-implicit DAE can be always transformed into a
semi-explicit DAEs .
Introduction to Differential Algebraic Equations
TU Ilmenau
4.3. Classification of DAEs...
Example (transformation of a fully-implicit DAE into a semi-explicit
DAE):
• Consider the linear time-invariant DAE Aẋ + Bx + b(t) = 0 , where
λA + B is nonsingular, for some scalar λ. Then there are non-singular
n × n matrices G and H such that:
I
O
GAH = m
O N
J
O
and GBH =
O In−m
(23)
where Im the m × m identity matrix (here m ≤ n), N is an
(n − m) × (n − m) nilpotent matrix; i.e., there is a positive integer
p such that N p = 0, J ∈ Rm×m and In−m is the (n − m) × (n − m)
identity matrix.
I Now, we can write Aẋ + Bx + b(t) = 0 equivalently as
(GAH) (H −1 )ẋ + (GBH) (H −1 )x + Gb(t) = 0.
Introduction to Differential Algebraic Equations
TU Ilmenau
(24)
4.3. Classification of DAEs...
• Use the block decomposing given in equation (23) to write
Im O
J
O
−1
(H )ẋ +
(H −1 )x + Gb(t) = 0.
O N
O In−m
(25)
• Use the variable transformation w(t) = H −1 x(t) to write
Im O
J
O
ẇ +
w + Gb(t) = 0.
(26)
O N
O In−m
w1 (t)
• Decompose the vector w(t) as w(t) =
, with w1 (t) ∈ Rm
w2 (t)
b1 (t)
n−m
and w2 (t) ∈ R
and correspondingly the vector Gb(t) =
so
b2 (t)
that:
Introduction to Differential Algebraic Equations
TU Ilmenau
ẇ1 (t) + Jw1 (t) + b1 (t) = 0
(27)
N w1 (t) + w2 (t) + b2 (t) = 0.
(28)
4.3. Classification of DAEs...
• Use now the nilpotent property of the matrix N ; i.e., multiply the
second set of equation by N p−1 to get
ẇ1 (t) + Jw1 (t) + b1 (t) = 0
p
N w1 (t) + N
p−1
w2 (t) + N
(29)
p−1
b2 (t) = 0.
(30)
From this it follows that (since N p w1 (t) = 0)
ẇ1 (t) = −Jw1 (t) − b1 (t)
0 = −N
p−1
w2 (t) − N
p−1
(31)
b2 (t).
Therefore, we have transformed the fully-implicit DAE
Aẋ + Bx + b(t) = 0 into a semi-explicit form.
• Similarly, using mathematician manipulations, any nonlinear
fully-implicit DAE can be transformed into a semi-explicit DAE.
Introduction to Differential Algebraic Equations
TU Ilmenau
(32)
4.4. Index of a DAE
• DAEs are usually very complex and hard to be solved analytically.
⇒ DAEs are commonly solved by using numerical methods.
Question:
Is it possible to use numerical methods of ODEs for the solution of
DAEs?
Idea:
Attempt to transform the DAE into an ODE.
• This can be chieved through repeated derivations of the algebraic
equations g(t, x, z) = 0 with respect to time t.
Definition
The minimum number of differentiation steps required to transform a
DAE into an ODE is known as the (differential) index of the DAE.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.4. Index of a DAE...
Example:
(DAE)
ẋ1 = x1 + 1
(x1 + 1)x2 + 2 = 0.
Here x2 the algebraic variable (i.e. z = x2 ). Differentiate
g(x1 , x2 ) = 0 to find a description for the time-derivative ẋ2 of the
algebraic variable. So
d
d
d
[g(x1 , x2 )] =
[0] ⇒
[(x1 + 1)x2 + 2] = 0
dt
dt
dt
⇒ ẋ1 x2 + (x1 + 1)ẋ2 = 0.
(x1 + 1)x2
ẋ1 x2
⇒ ẋ2 = −
= −
= −x2
(x1 + 1)
(x1 + 1)
I Only one differentiation step is required to describe ẋ2 . So, the
DAE is of index 1.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.4. Index of a DAE...
• The CSTR model is of index 2.
• The DAE model for the simple penudulum is of index 3.
I DAEs with index greater than 1 are commonly known as higher
index DAEs.
I The higher the index, the more difficult will be the DAE to solve.
I Transformation of a higher index DAE into a lower index DAE (or
to an ODE) is commonly known as index reduction. In general, index
reduction for higher index DAE simplifies computational complexities.
Two serious issues to consider when solving DAEs
• The solutions of the lower index DAE may not be a solution of the
original DAE. This is known as a drift off effect.
• Finding initial conditions that satisfy both the differential and
algebraic parts of a DAE may not be trivial, known as consistency of
initial conditions.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.4. Index of a DAE...
Therefore, computational algorithms for a DAE should:
be cable of identifying consistent initial conditions to the DAE;
as well as,
provide automatic index reduction mechanisms to simplify the
DAE.
I Modern software like Sundials, Modelica, use strategies for
index-reduction coupled with methods of consistent initialization.
In the following we consider only index 1 semi-explicit DAEs:
(DAE)
ẋ = f (t, x, z)
(33)
0 = g(t, x, z).
(34)
∂g dx
+ ∂g dz = 0. Hence, a semi-explicit DAE is of
∂x
h dti−1 ∂z dt
index 1 iff ∂g
exists. That is, one differentiation step yields the
∂z
h i−1 h i
h i−1
∂g
∂g
∂g
∂g
differential equation: dz
=
−
(f
(t,
x,
z))
−
dt
∂z
∂x
∂z
∂t .
Since
∂g
∂t
+
Introduction to Differential Algebraic Equations
TU Ilmenau
4.5. An overview of numerical Methods for DAEs
• Using mathematical principles and transformation of variables,
fully-implicit DAEs can be transformed to semi-explicit DAEs.
• Note that, in the semi-explicit DAE
ẋ = f (t, x, z)
(35)
0 = g(t, x, z).
(36)
h i
if Jacobian matrix ∂g
∂z singular (non-invertible), then the DAE is of
higher index.
I Since many applications have model equations as semi-explicit
DAEs, the discussion next is restricted to this form.
• In the DAE above, if both f and g do not explicitly depend on time
t; i.e. f (t, x, z) = f (x, z) and g(t, x, z) = g(x, z), then the model is
an autonomous DAE.
Introduction to Differential Algebraic Equations
TU Ilmenau
4.5. An overview of numerical Methods for DAEs...
I BDF and collocation methods are two most widely used methods
for numerical solution of DAEs.
(I) BDF for DAEs:
Consider the initial value DAE
ẋ = f (t, x, z), x(t0 ) = x0
(37)
0 = g(t, x, z).
(38)
Ideas of BDF:
I Select a time step h so that ti+1 = ti + h, i = 0, 1, 2, . . .
I Given xi = x(ti ) and zi = z(ti ), determine the value
xi+1 = x(ti+1 ) by using (extrapolating) values xi , xi−1 , . . . , xi−m+1
of the current and earlier time instants of x(t).
I Simultaneously compute zi+1 = z(ti+1 ).
Introduction to Differential Algebraic Equations
TU Ilmenau
4.5. An overview of numerical Methods for
DAEs...BDF
• There is a unique m-th degree polynomial P that interpolates the
m + 1 points
(ti+1 , xi+1 ), (ti , xi ), (ti−1 , xi−1 ), . . . , (ti+1−m , xi+1−m ).
• This interpolating polynomials P can be written as
P (t) =
m
X
xi+1−j Lj (t)
j=0
with the Lagrange polynomial
m Y
t − ti+1−l
Lj (t) =
, j = 0, 1, . . . , m.
ti+1−j − ti+1−l
l=0
l 6= j
Introduction to Differential Algebraic Equations
TU Ilmenau
4.5. ....BDF for DAEs...
• Observe that
P (ti+1−j ) = xi+1−j , j = 0, 1, . . . , m.
In particular P (ti+1 ) = xi+1 .
• Thus, replace ẋi+1 by Ṗ (ti+1 ) to obtain
Ṗ (ti+1 ) = f (ti+1 , xi+1 ).
(∗)
But
Ṗ (ti+1 ) =
m
X
xi+1−j L̇j (ti+1 ) = xi+1 L̇0 (ti+1 ) +
j=0
m
X
xi+1−j L̇j (t)
j=1
Putting this into (*) we get:
xi+1 = −
m
X
xi+1−j
j=1
Introduction to Differential Algebraic Equations
TU Ilmenau
L̇j (ti+1 )
1
+
f (ti+1 , xi+1 )
L̇0 (ti+1 ) L̇0 (ti+1 )
4.5. ....BDF for DAEs...
Define
aj =
L̇j (ti+1 )
1
, bm =
.
L̇0 (ti+1 )
hL̇0 (ti+1 )
• The values aj , j = 1, . . . , m and bm can be read from lookup tables.
An m-step BDF Algorithm (BDF m) for a DAE:
Given a1 , . . . , am ,bm
xi+1 = −
m
X
aj xi+1−j + bm hf (ti+1 , xi+1 , zi+1 )
(39)
j=1
0 = g(ti+1 , xi+1 , zi+1 ).
• Each iteration of the BDF requires a Newton algorithm for the
∂g
solution a system of nonlinear equations. Hence, the Jacobian ∂w
needs to be well-conditioned, where w = (t, x, z).
Introduction to Differential Algebraic Equations
TU Ilmenau
(40)
4.5. ....BDF for DAEs...
• Given x0 = x(t0 ), BDF requires consistent initial conditions to be
obtained by solving
g(t0 , x0 , z0 ) = 0.
to determine z0 = z(t0 ).
The m-step BDF algorithm converges if m ≤ 6; i.e.
xi − x(ti ) ≤ O(hm ),
zi − z(ti ) ≤ O(hm )
for consistent initial conditions.
• BDF is commonly used to solve DAEs of index 1 or 2
Software based on BDF for DAEs:
Matlab: ode15i
Open source: DASSL ; CVODE, CVODES, and IDA (under
Sundials); ODEPACK solvers, etc.
Introduction to Differential Algebraic Equations
TU Ilmenau
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal Collocation
• To collocate a function x(t) through another function p(t) = to
captur the properties of x(t) by using p(t).
• In general, we use a simpler function p(t) to collocate x(t); eg., p(t)
can be a polynomial, a trigonometric function, etc.
⇒ polynomial and trigonometric functions are usually simpler to work
with. (The discussion here is restricted to polynomilas)
Weirstraß’ Theorem
If x(t) is a continuous function on [a, b], then for any given ε > 0,
there is a polynomial p(t) such that
max |x(t) − p(t)| < ε
a≤t≤b
• Hence, we use the polynomial p(t) instead of x(t).
• However, Weirstraß’ theorem doese not specifye how to construct
the approximating polynomial p(t) .
Introduction to Differential Algebraic Equations
TU Ilmenau
• Suppose p(t) = a0 + a1 t + . . . + am tm is the approximating
polynomial to x(t) on [a, b].
Note: If the coefficients a0 , a1 , . . . , am are given, then the
approximating polynomial is exactly known.
Question: How to determine a0 , a1 , . . . , am ?
Question: How to construct the apprximating polynomial?
• There are several ways to construct p(t) to approximates x(t)
according to Weirstraß’ theorem.
Here we require p(t) to satisfy the following property:
•
p(ti ) = x(ti ) =: xi , i = 1, . . . , N.
for some selected time instants t1 , t2 , . . . , tN from the interval [a, b].
• This property relates p(t) and x(t) and is known as interpolatory
property.
Uniqueness of an interpolating polynamial
There is a unique interpolatory polynomial p(t) for the N data points
(t1 , x1 ),(t2 , x2 ), . . .,(tN , xN ) with degree deg(p) = m = N − 1.
Introduction to Differential Algebraic Equations
TU Ilmenau
• In the following we use N = m + 1.
Figure
Introduction to Differential Algebraic Equations
TU Ilmenau
According to the interpolatory property, we have
x1 = p(t1 ) = a0 + a1 t1 + . . . + am tm
1
(41)
am tm
2
(42)
x2 = p(t2 ) = a0 + a1 t2 + . . . +
..
.
(43)
am tm
m+1 .
xm+1 = p(tm+1 ) = a0 + a1 tm+1 + . . . +
(44)
 
 

a0
1
t1
t21
. . . tm
x1
1
2
m a 
 x2  1
t
t
.
.
.
t
2
2
2  1
 

⇒  .  = .
..
..
..   .. 
.
.
 .  .
.
.
...
.  . 
am
1 tm+1 t2m+1 . . . tm
xm+1
m+1
Thus, if we known x1 , x2 , . . . , xm+1 then we can compute
a0 , a1 , . . . , am and vice-versa.
• But, since x(t) is not yet known, x1 , x2 , . . . , xm+1 are unknown.
Introduction to Differential Algebraic Equations
TU Ilmenau


a0


 a1 


 
Hence, both 
 and  ..  are unkowns.


 . 
xm+1
am
• To avoid working with two unknown vectors, we can define the
polynamial p(t) in a better way as:
x1
x2
..
.


p(t) =
m+1
X
xi Li (t)
(45)
i=1
where Li (t) are Lagrange polynomials given by
Li (t) =
m+1
Y
j=1
j 6= i
=
t − tj
ti − tj
(t − t1 ) (t − t2 ) . . . (t − ti−1 ) (t − ti+1 ) . . . (t − tm+1 )
.
(ti − t1 ) (ti − t2 ) . . . (ti − ti−1 ) (ti − ti+1 ) . . . (ti − tm+1 )
Introduction to Differential Algebraic Equations
TU Ilmenau
Properties:
Li (tj ) =



1, if i = j
0, if i 6= j
? (satisfaction of the interpolatory property)
p(ti ) = xi , i = 1, 2, . . . , m + 1.
? (approximation of x(t) by p(t))
The polynomial p(t) in equation (45) can be made to satisfy the
Weirstraß’ theorem by taking sufficiently large number of time
instants t1 , t2 , . . . , tm+1 from [a, b].
Question: What is the best choice for t1 , t2 , . . . , tm+1 ?
Introduction to Differential Algebraic Equations
TU Ilmenau
Determination of collocation Points
An idea to determine collocation points:
Select values τ1 , τ2 , . . . , τN from the interval [0, 1] and define the ti ’s
as follows:
ti = a + τi (b − a), i = 1, . . . , N.
• This is a good idea, since the same values τ1 , τ2 , . . . , τN from [0, 1]
can be used to generate collocation points on various intervals [a, b].
Example:
(i) If [a, b] = [2, 10], then ti = 2 + τi (10 − 2) , i = 1, . . . , N ;
(ii) If [a, b] = [0, 50], then ti = 0 + τi (50 − 0) , i = 1, . . . , N ; etc.
Question: What is the best way to select the values
τ1 , τ2 , . . . , τN from [0, 1]?
Introduction to Differential Algebraic Equations
TU Ilmenau
Determination of collocation Points...
I Recall that: solution of a differential equation (or DAEs) is an
integration process.
⇒ Numerical methods for one-dimensional integrals can give us
information on how to select τ1 , τ2 , . . . , τN .
⇒ Gauss quadrature rules are one of the best numerical integration
methods.
To approximate the integral
Z 1
I[f ] =
f (τ )dτ,
0
by the (iterpolatory) quadrature rule QN [f ] =
• the integration nodes τ1 , τ2 , . . . , τN ∈ [0, 1]
• weights w1 , w2 , . . . , wN
are constructed based on the interval [0, 1] .
Introduction to Differential Algebraic Equations
TU Ilmenau
PN
k=1 wk f (τk ),
where:
Determination of collocation points...
Thus we would like to obtain the approximation
Z 1
N
X
I[f ] =
f (τ )dτ ≈ QN [f ] =
wi f (τi ).
0
i=1
• Once a quadrature rule QN [·] is constructed it can be used to
approximate integrals of various functions.
Question: How to determine the 2N unknowns τ1 , τ2 , . . . , τN ,
w1 , w2 , . . . , wN ?
We require the quadrature rule to satisfy the equality:
Z
b
W (τ )p(τ )dτ =
a
N
X
wi p(τi )
i=1
for all polynomials p with degree deg(p) ≤ 2N .
Introduction to Differential Algebraic Equations
TU Ilmenau
That is, QN [·] should integrate each of the polynomials
p(t) = 1, τ, τ 2 , . . . , τ 2N
exactly. This implies
Z
1
1dτ
=
0
τ dτ
=
X
=
X
0
(46)
wi τi
(47)
wi τi2
(48)
i=1
1
Z
wi
i=1
1
Z
X
τ 2 dτ
0
i=1
..
.
Z
0
Introduction to Differential Algebraic Equations
TU Ilmenau
1
τ 2N dτ
=
(49)
X
i=1
wi τi2N .
(50)
We need to solve the system of 2N nonlinear equations:
1 = w1 + w2 + . . . + wN
1
= w1 τ1 + w2 τ2 + . . . + wN τN
2
(51)
(52)
(53)
1
3
2
= w1 τ12 + w2 τ22 + . . . + wN τN
..
.
1
2N
2N
= w1 τ12N + w2 τ22N + . . . + wN τN
.
(54)
(55)
(56)
From the equation 1 = w1 + w2 + . . . + wN we can solve for w1 and
replace for it in the remaining equations.
⇒ It remains to determine the 2N − 1 unknowns τ2 , . . . , τN and
w1 , w2 , . . . , wN from the reduced set of 2N − 1 equations.
• Unfortunately, this system of equations is difficult to solve.
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal polynomials
•There is a simpler way if we use orthogonal polynomials.
I The concept of orthogonality requires a definition of scalar
product.
• The scalar product of functions h and g with respect on the
interval [0, 1] is
Z 1
hh, gi =
h(τ )g(τ )dτ.
0
• Two functions h and g are orthogonal on the interval [0, 1] if
Z
hh, gi =
1
h(τ )g(τ )dτ = 0.
0
• In this lecture, we are only interested in set of polynomials that
are orthogonal to each other.
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal polynomials...
• Orthogonal polynomials on [0, 1] (as defined above) are known as
shifted Legendre polynomials.
Theorem (Three-term recurrence relation)
Suppose {p0 , p1 , . . .} the set of shifted Legendre orthogonal
polynomials on [0, 1] with deg(pn ) = n and leading coefficient equal
to 1. The shifted Legendre polynomials are generated by the relation
pn+1 (τ ) = (τ − an )pn (τ ) − bn pn−1 (τ )
with p0 (τ ) = 1 and p−1 (τ ) = 0, where the recurrence coefficients are
given as
an = 12 , n = 0, 1, 2, . . .
bn =
n2
,n
4(4n2 −1)
Introduction to Differential Algebraic Equations
TU Ilmenau
= 0, 1, 2, . . .
Orthogonal polynomials...
The first four shifted Legendre orthogonal polynomials are
p0 (τ ) = 1, p1 (τ ) = 2τ − 1, p2 (τ ) = 6τ 2 − 6τ + 1, p3 (τ ) =
20τ 3 − 30τ 2 + 12τ − 1.
Further important properties of orthogonal polynomials:
Given any set of orthogonal polynomials {p0 , p1 , p2 , . . .} the following
hold true:
• Any finite set of orthogonal polynomials {p0 , p1 , . . . , pN −1 } is
linearly independent.
• The polynomial PN is orthogonal to each of p0 , p1 , . . . , pN −1 .
• Any non-zero polynomial q with degree deg(q) ≤ N − 1 can be
written as a linear combination:
q(τ ) = c0 p0 (τ ) + c1 p1 (τ ) + . . . , cN −1 pN −1 (τ )
where at least one of the scalars c0 , c1 , . . . , cN −1 is non-zero.
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal polynomials...
• We demand the quadrature rule is exact for polynomials up to
degree 2N − 1.
• Let P (τ ) be any polynomial of degree 2N − 1.
Hence,
Z 1
N
X
I[P ] = QN [P ] ⇒
P (τ )dτ =
wi P (τi ).
0
i=1
• Given the N -th degree orthogonal polynomial pN , the polynomial P
can be written as
P (τ ) = pN (τ )q(τ ) + r(τ )
where q(τ ) and r(τ ) are polynomials such that 0 < deg(r) ≤ N − 1
and
• deg(q) = N − 1, since
2N − 1 = deg(P ) = deg(pN ) + deg(q) = N + deg(q).
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal polynomials...
R
P
1
• Now, from the equation 0 P (τ )dτ = N
k=1 wk P (τk ) it follows that
Z b
N
X
(pN (τ )q(τ ) + r(τ )) dτ =
wi (pN (τi )q(τk ) + r(τi )) .
a
i=1
Z b
N
N
b
X
X
)dτ =
⇒
pN (τ )q(τ )dτ + r(τ
w
wi pN (τi )q(τi ) +
i r(τi ).
a
a
i=1
i=1
|
{z
}
Z
=0
Observe the following:
Rb
P
• Using polynomial exactness: a r(τ )dτ = N
i=1 wi r(τi ).
R• bOrthogonality implies < pN , pk >= 0, k = 1, . . . , N − 1. Thus,
a pN (τ )q(τ ) =
Z b
Z b
N
−1
N
−1
X
X
pN (τ )
ck pk (τ )dτ =
ck
pN (τ )pk (τ )dτ = 0.
a
a
i=1
k=1
{z
}
|
=0
Introduction to Differential Algebraic Equations
TU Ilmenau
OrthogonalPpolynomials...
It follows that
N
k=1 wk pN (τk )q(τk )
= 0.
(?)
Theorem
If the quadrature nodes τ1 , τ2 , . . . , τN are zeros of the N −th degree
shifted Legendre polynomial pN (τ ), then
• all the roots τ1 , τ2 , . . . , τN lie inside (0, 1);
• the quadrature weights are determined from
Z 1
wi =
Li (τ )dτ,
0
where Li (τ ), i = 1, . . . , N is the Lagrange function defined using
τ1 , τ2 , . . . , τN and wi > 0, i = 1, . . . , N .
• The quadrature rule QN [·] integrates polynomials degree up to
2N − 1 exactly.
Hence, equation (?) holds true if p1 (τ1 ) = pN (τ2 ) = . . . , p(τN ) = 0.
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal polynomials...
• Therefore, the quadrature nodes τ1 , τ2 , . . . , τN are chosen as the
zeros of the N -th degree shifted Legendre orthogonal polynomial.
Question:
Is there a simple way to compute the zeros τ1 , τ2 , . . . , τN of an
orthogonal polynomial pN and the quadrature weights
w1 , w2 , . . . , wN ?
The answer is give by a Theorem of Welsch & Glub (see next slide).
Introduction to Differential Algebraic Equations
TU Ilmenau
Orthogonal polynomials...
Theorem (Welsch & Glub 1969 )
The quadrature nodes τ1 , τ2 , . . . , τN and weights w1 , w2 , . . . , wN can
be computed from the spectral factorization of
JN = V > ΛV ; Λ = diag(λ1 , λ2 , . . . , λN ), V V > = IN ;
of the
symmetric
tri-diagonal Jacobi matrix
√


a
√0
 b1




JN = 




b1
a1
√
b2
√
.
.
b2
.
.
.
..
.
.
a
p N −2
bN −1
p
bN −1
aN −1










where
a0 , an , bn , n = 1, . . . , N − 1 are the known coefficients from the
recurrence relation. In particular ...
Introduction to Differential Algebraic Equations
TU Ilmenau
Theorem ...
τk = λk , k = 1, . . . , N ;
2
wk =
e> V ek , k = 1, . . . , N,
(57)
(58)
where e1 , ek are the 1st and the k−the unit vectors of length N .
A Matlab program:
n = 5; format short
beta = .5./sqrt(1-(2*(1:n-1)).^(-2)); % recurr. coeffs
J = diag(beta,1) + diag(beta,-1)
% Jacobi matrix
[V,D] = eig(J);
% Spectral decomp.
tau = diag(D); [tau,i] = sort(tau); % Quad. nodes
w = 2*V(1,i).^2;
% Quad. weights
Introduction to Differential Algebraic Equations
TU Ilmenau
• There are standardised lookup tables for the integration nodes
τ1 , τ2 , . . . , τN and corresponding weights w1 , w2 , . . . , wN .
The collocation points t1 , t2 , . . . , tN ∈ [a, b] can be determined by
using the quadrature points τ1 , τ2 , . . . , τN ∈ [0, 1] through the
relation:
ti = a + τi (b − a).
Example: Suppose we would like to collocate x(t) on [a, b] = [0, 10]
using a polynomial x̂(t) with deg(x̂) = m = 3.
• Required number of collocation points = N = m + 1 = 3 + 1 = 4.
⇒ First, determine the zeros of the 4-th degree shifted Legendre
polynomial p4 (τ ); i.e., find τ1 , τ2 , τ3 , τ4 .
• These value can be determined to be:
τ1 = 0.0694318442,
τ2 = 0.3300094783,
τ3 = 0.6699905218,
τ4 = 0.9305681558.
Introduction to Differential Algebraic Equations
TU Ilmenau
• Next, determine the collocation points :
t1 = 0 + (10 − 0) ∗ τ1 = 0.694318442,
3.300094783,
t3 = 0 + (10 − 0) ∗ τ3 = 6.699905218,
t4 = 0 + (10 − 0) ∗ τ4 = 9.305681558.
Introduction to Differential Algebraic Equations
TU Ilmenau
t2 = 0 + (10 − 0) ∗ τ2 =
The collocation polynomial is p(t) =
L1 (t) =
L2 (t) =
L3 (t) =
L4 (t) =
P4
i
xi Li (t). where
(t − t2 )(t − t3 )(t − t4 )
(t1 − t2 )(t1 − t3 )(t1 − t4 )
(t − t1 )(t − t3 )(t − t4 )
(t2 − t1 )(t2 − t3 )(t2 − t4 )
(t − t1 )(t − t2 )(t − t4 )
(t3 − t1 )(t3 − t2 )(t3 − t4 )
(t − t1 )(t − t2 )(t − t3 )
.
(t4 − t1 )(t4 − t2 )(t4 − t3 )
Hence, we approximate x(t) using the third degree polynomial
p(t) = x
b(t) = x1 L1 (t) + x2 L2 (t) + x3 L3 (t) + x4 L4 (t)
It remains to determine the coefficients x1 , x2 , x3 , x4 .
Introduction to Differential Algebraic Equations
TU Ilmenau
(59)
(60)
(61)
(62)
Discretization of DAEs using orthogonal collocation
Given the DAE:
ẋ = f (t, x, z), t0 ≤ t ≤ tf
(63)
0 = g(t, x, z)
(64)
where x(t)> = (x1 (t), x2 (t), . . . , xn (t)) and
z(t)> = (z1 (t), z2 (t), . . . , zm (t)).
• Determine the a set of collocation points t1 , t2 , . . ., tN .
• corresponding to each differential and algebraic variable define
collocation polynomials:
xk (t) −→ p(k) (t) =
zj (t) −→ p(j) (t) =
N
X
i=1
N
X
i=1
Introduction to Differential Algebraic Equations
TU Ilmenau
(k)
(65)
(j)
(66)
xi Li (t), k = 1, . . . , n;
zi Li (t), j = 1, . . . , m.
... orthogonal collocation of DAEs
• In the DAE, replace xk (t) and zj (t) by q (k) (t) and p(j) (t) so that
ṗ (t) = f1 (t, p (t), p (t), . . . , p (t) , q (1) (t), q (2) (t), . . . , q (m) (
ṗ(2) (t) = f2 (t, p(1) (t), p(2) (t), . . . , p(n) (t) , q (1) (t), q (2) (t), . . . , q (m) (
(1)
(1)
(2)
(n)
..
.
ṗ(n) (t) = f1 (t, p(1) (t), p(2) (t), . . . , p(n) (t) , q (1) (t), q (2) (t), . . . , q (m) (
0 = g1 (t, p(1) (t), p(2) (t), . . . , p(n) (t) , q (1) (t), q (2) (t), . . . , q (m) (
(1)
(2)
(n)
0 = g2 (t, p (t), p (t), . . . , p (t) , q (1) (t), q (2) (t), . . . , q (m) (
..
.
0 = gm (t, p(1) (t), p(2) (t), . . . , p(n) (t) , q (1) (t), q (2) (t), . . . , q (m)
Introduction to Differential Algebraic Equations
TU Ilmenau
... orthogonal collocation of DAEs
• Next discretize the system above using t1 , t2 , . . . , tN to obtain:
(1) (2)
(n)
(1) (2)
(m)
ṗ(1) (ti ) = f1 ti , xi , xi , . . . , xi
, zi , zi , . . . , zi
(1) (2)
(n)
(1) (2)
(m)
ṗ(2) (ti ) = f2 ti , xi , xi , . . . , xi
, zi , zi , . . . , zi
..
.
(1) (2)
(n)
(1) (2)
(m)
ṗ(n) (ti ) = fn ti , xi , xi , . . . , xi
, zi , zi , . . . , zi
(1) (2)
(n)
(1) (2)
(m)
, zi , zi , . . . , zi
0 = g1 ti , xi , xi , . . . , xi
(1) (2)
(n)
(1) (2)
(m)
0 = g2 ti , xi , xi , . . . , xi
, zi , zi , . . . , zi
..
.
(1) (2)
(n)
(1) (2)
(m)
0 = gm ti , xi , xi , . . . , xi
, zi , zi , . . . , zi
,
i = 1, 2, . . . , N.
Introduction to Differential Algebraic Equations
TU Ilmenau
... orthogonal collocation of DAEs
• Solve the system of (n + m) × N equations to determine the
(k)
(n + m) × N unknwons xi , k = 1, . . . , n, i = 1, . . . , N and
(j)
zi , j = 1, . . . , m, i = 1, . . . , N .
Example: Use a four point orthogonal collocation to solve the DAE
ẋ1 = x1 + 1, 0 ≤ t ≤ 1.
0 = (x1 + 1)x2 + 2.
Solution: we use the collocation polynomials
p(1) (t) =
4
X
(1)
xi (t)Li (t) and p(2) (t) =
i=1
to collocate x1 (t) and x2 (t), respectively.
Introduction to Differential Algebraic Equations
TU Ilmenau
4
X
i=1
(2)
xi (t)Li (t)
Advantages and disadvantages collocation methods
• can be used for higher index DAEs.
• efficient for both initial value as well as boundary values DAEs.
• more accurate
Disadvantages:
• Can be computationally expensive
• The approximating polynomials may display oscillatory properties,
impacting accuracy
• computational expenses become high if t1 , t2 , . . . , tN −1 are
considered variables.
Introduction to Differential Algebraic Equations
TU Ilmenau
Resources - Literature
References:
• U. M. Ascher, L. R. Petzold : Computer Methods for Ordinary
Differential Equations and Differential-Algebraic Equations. SIAM
1998.
• K. E. Brenan, S. L. Campbell, and L.R. Petzold: (1996). Numerical
Solution of Initial-Value Problems in Differential-Algebraic Equations.,
SIAM, 1996.
• Campbell, S. and Griepentrog, E., Solvability of General Differential
Algebraic Equations, SIAM J. Sci. Comput. 16(2) pp. 257-270, 1995.
• M. S. Erik and G. Sderlind: Index Reduction in
Differential-Algebraic Equations Using Dummy Derivatives. SIAM
Journal on Scientific Computing, vol. 14, pp. 677-692, 1993.
• C. W. Gear : Differential-algebraic index transformation. SIAM J.
Sci. Stat. Comp. Vol. 9, pp. 39 – 47, 1988.
• Walter Gautschi: Orthogonal polynomials (in Matlab). Journal of
Computational and Applied Mathematics 178 (2005) 215– 234
Introduction to Differential Algebraic Equations
• Lloyd N. Trefethen: Is Gauss Quadrature Better than
TU Ilmenau
Clenshaw.Curtis? SIAM REVIEW, Vol. 50, No. 1, pp. 67 – 87, 2008.
Resources - Literature .. Software
• Pantelides Costas. (1988) ”The consistent initialization of
differential algebraic systems.” SIAM Journal on Scientific and
Statistical Computing, vol. 9: 2, pp. 213–231, 1988.
• L. Petzold: Differential/Algebraic Equations Are Not ODEs, SIAM
J. Sci. Stat. Comput. 3(3) pp. 367-384, 1982.
Open source software:
• DASSL - - FORTRAN 77:
http://www.cs.ucsb.edu/ cse/software.html
• DASSL Matlab interface:
http://www.mathworks.com/matlabcentral/fileexchange/16917-dasslmex-file-compilation-to-matlab-5-3-and-6-5
• Sundials - https://computation.llnl.gov/casc/sundials/main.html
• Modelica - https://modelica.org/
• Odepack - http://www.cs.berkeley.edu/ kdatta/classes/lsode.html
• Scicos- http://www.scicos.org/
Introduction to Differential Algebraic Equations
TU Ilmenau