FROM
TO
SYSTEMS
C. G. Cassandras
Boston University
cgc@bu.edu,
Christos G. Cassandras
http://vita.bu.edu/cgc
CODES Lab. - Boston University
OUTLINE
¾ HYBRID SYSTEMS IN COMPLEX SYSTEM…
… DECOMPOSITION AND ABSTRACTION
¾ WHAT’S A HYBRID SYSTEM…
¾ DECOMPOSITION: HYBRID SYSTEM → DES
- SOLVING OPTIMAL CONTROL PROBLEMS
¾ ABSTRACTION:
DES → HYBRID SYSTEM
- ANALYSIS OF STOCHASTIC FLOW MODELS (SFM)
¾ THE FUTURE…
Christos G. Cassandras
CODES Lab. - Boston University
DISCRETE
CONTINUOUS
1980
TIME-DRIVEN
SYSTEMS
EVENT-DRIVEN
SYSTEMS
1990
HYBRID
SYSTEMS
Christos G. Cassandras
CODES Lab. - Boston University
2000
LESS COMPLEX
MORECOMPLEX
TIME-DRIVEN
SYSTEM
HYBRID
SYSTEM
EVENT-DRIVEN
SYSTEM
Christos G. Cassandras
CODES Lab. - Boston University
LESS COMPLEX
MORECOMPLEX
TIME-DRIVEN
SYSTEM
What exactly
does that mean?
HYBRID
SYSTEM
EVENT-DRIVEN
SYSTEM
LESS COMPLEX
Christos G. Cassandras
DECOMPOSITION
CODES Lab. - Boston University
MORECOMPLEX
LESS COMPLEX
TIME-DRIVEN
zz
SYSTEM
ZOO
M
HYBRID
SYSTEM
LESS COMPLEX
Christos G. Cassandras
OUT
EVENT-DRIVEN
SYSTEM
ABSTRACTION
(AGGREGATION)
CODES Lab. - Boston University
MORECOMPLEX
LESS COMPLEX
TIME-DRIVEN
SYSTEM
HYBRID
SYSTEM
ABSTRACTION
(AGGREGATION)
Christos G. Cassandras
HYBRID
SYSTEM
EVENT-DRIVEN
SYSTEM
DECOMPOSITION
CODES Lab. - Boston University
WHAT IS THE RIGHT ABSTRACTION LEVEL ?
TOO FAR…
model not
detailed enough
TOO CLOSE…
too much
undesirable
detail
JUST RIGHT…
good model
Christos G. Cassandras
CREDIT: W.B. Gong
CODES Lab. - Boston University
WHAT’S A
HYBRID SYSTEM ?
Christos G. Cassandras
CODES Lab. - Boston University
WHAT’S A HYBRID SYSTEM?
z&2 = g 2 ( z2 , u2 , t )
z&1 = g1 ( z1, u1, t )
TIME
x1
x0
x2
x1
x1 = f1 ( x0 , z1, u1, t )
x2 = f 2 ( x1, z2 , u2 , t )
NE
Christos G. Cassandras
W
DE
O
“M
”
CODES Lab. - Boston University
WHAT’S A HYBRID SYSTEM?
CONTINUED
TIME DRIVEN :
z&i = gi ( zi , ui , t )
TIME TIME TIMETIME
x1
x0
x1
x2
x3
TIME
x4
xi+1
x2
x3
xi
xi
Christos G. Cassandras
xi+1
EVENT - DRIVEN :
xi +1 = f i ( xi , zi , ui , t )
CODES Lab. - Boston University
WHAT’S A HYBRID SYSTEM?
CONTINUED
Physical State, z
z&i = gi ( zi , ui , t )
hybrid
…
x1
Switching Times
Christos G. Cassandras
x2
xi
xi+1 = fi(xi,ui,t)
Temporal State, x
SWITCHING
SWITCHINGTIMES
TIMES
HAVE
THEIR
HAVE THEIROWN
OWN
DYNAMICS!
DYNAMICS!
CODES Lab. - Boston University
HYBRID SYSTEM EXAMPLES
1. Autonomous Switching, e.g., Hysteresis
x<Δ
x > −Δ
x≥Δ
x& = x + u
x& = − x + u
x ≤ −Δ
Christos G. Cassandras
CODES Lab. - Boston University
HYBRID SYSTEM EXAMPLES
CONTINUED
2. External Switching, e.g., Zeno’s bouncing ball
x& = v x , v&x = 0
y& = v y , v& y = −mg
Switching
Events
Christos G. Cassandras
CODES Lab. - Boston University
HYBRID SYSTEM EXAMPLES
CONTINUED
3. Controlled Switching, e.g., Interconnected tanks
u1(t)
HIGH
LOW
u2(t)
Christos G. Cassandras
u3(t)
HIGH
HIGH
LOW
LOW
CODES Lab. - Boston University
HYBRID SYSTEM EXAMPLES - MANUFACTURING
Key questions facing manufacturing
system integrators:
• How to integrate ‘process control’ with ‘operations control’ ?
• How to improve product QUALITY within reasonable TIME ?
PROCESS CONTROL
• Physicists
• Material Scientists
• Chemical Engineers
• ...
Christos G. Cassandras
OPERATIONS CONTROL
• Industrial Engineers, OR
• Schedulers
• Inventory Control
• ...
CODES Lab. - Boston University
HYBRID SYSTEM EXAMPLES - MANUFACTURING
Throughout a manuf. process, each part is characterized by
• A PHYSICAL state (e.g., size, temperature, strain)
• A TEMPORAL state (e.g., total time in system, total time to due-date)
PHYSICAL
STATE
Time-driven
Dynamics
NEW
PHYSICAL
STATE
OPERATION
OPERATION
TEMPORAL
STATE
Christos G. Cassandras
Event-driven
Dynamics
NEW
TEMPORAL
STATE
CODES Lab. - Boston University
COOPERATIVE CONTROL
TARGET
EVENT: threat sensed
THREATS
TIME-DRIVEN
DYNAMICS
BASE
Christos G. Cassandras
EVENT: info. communicated by team member
CODES Lab. - Boston University
GENERAL MODELING FRAMEWORKS
• Hybrid Automata
Σq : z&q =γq(zq,uq,t)
Gq
Fq
[Branicky
[Branickyetetal.,
al.,1998]
1998]
Aq
AUTONOMOUS JUMP SET
System must jump with
Gq : Aq ×Vq →S
• Mixed Logical Dynamical Systems
CONTROLLED JUMP SET
System may jump with
Fq : Cq →2S
inputs
symbols δi
automaton
automaton/ /logic
logic
[Bemporad
[Bemporadand
andMorari,
Morari,1999]
1999]
[Proc.
[Proc.ofofIEEE
IEEESpecial
SpecialIssue,
Issue,2000]
2000]
Christos G. Cassandras
continuous
states
symbols δ
D/A
A/D
• etc.
Cp
Ap
Cq
continuous
continuous
dynamical
dynamicalsystem
system
inputs
CODES Lab. - Boston University
TIME-DRIVEN AND EVENT-DRIVEN DYNAMICS
Time-Driven Dynamics (STATE = z):
z&(t ) = g (z , u, t )
or:
zk +1 = g k (zk , uk )
Event-Driven Dynamics (STATE = Event Times xk,i):
xk +1,i = max{xk , j + a k , j u k , j }
j∈Γi
Event counter k = 1,2,...
Event index i∈E = {1,…,n}
Christos G. Cassandras
CODES Lab. - Boston University
DECOMPOSITION
Christos G. Cassandras
CODES Lab. - Boston University
HIEARARCHICAL DECOMPOSITION
???
PLANNING
DISCRETE-EVENT
PROCESSES
PHYSICAL
PROCESSES
Christos G. Cassandras
CODES Lab. - Boston University
HIEARARCHICAL DECOMPOSITION
CONTINUED
MODEL
MODEL
TIME
TIMESCALE
SCALE
???
Weeks - Months
Minutes - Weeks
msec - Hours
Christos G. Cassandras
Diff. Eq’s, Flows, LP
PLANNING
DISCRETE-EVENT
PROCESSES
PHYSICAL
PROCESSES
Automata, Petri nets,
Queueing, Simulation
Diff. Eq’s,
Detailed Simulation
CODES Lab. - Boston University
HIEARARCHICAL DECOMPOSITION
CONTINUED
???
???
PLANNING
PLANNING
DISCRETE-EVENT
PROCESSES
PHYSICAL
PHYSICAL
PROCESSES
PROCESSES
Christos G. Cassandras
CODES Lab. - Boston University
HYBRID CONTROL SYSTEM
What exactly
does that mean?
DISCRETE-EVENT
PROCESSES
CONTROL
CONTROL
PHYSICAL
PROCESSES
Christos G. Cassandras
CODES Lab. - Boston University
OPTIMAL CONTROL
Christos G. Cassandras
CODES Lab. - Boston University
WHAT’S A HYBRID SYSTEM?
Physical State, z
z&i = gi ( zi , ui , t )
hybrid
…
x1
Switching Times
Christos G. Cassandras
x2
xi
Temporal State, x
xi+1 = fi(xi,ui,t)
CODES Lab. - Boston University
OPTIMAL CONTROL PROBLEMS
• Get to a desired final physical state zN in minimum time xN,
subject to N-1 switching events
• Minimize deviations from N desired physical states: (zi - qi)2
and
deviations from target desired times:
(xi - τi)2
In general:
N
min ∑ ∫ Li ( zi (t ), ui (t ))dt
u
i =1
xi
xi −1
Temporal state
Christos G. Cassandras
z&i = gi ( zi , ui , t )
s.t.
xi+1 = fi(xi,ui,t)
Physical state
CODES Lab. - Boston University
OPTIMAL CONTROL PROBLEMS
Problems we consider:
N
min ∑ [φi ( xi , xi −1 ) + ψ i ( xi )]
u
i =1
Cost under ui(t) over [xi-1,xi]
where:
CONTINUED
Cost of switching time xi
xi
φi ( xi , xi −1 ) = ∫ Li ( zi (t ), ui (t ))dt
xi −1
Let:
si = xi - xi-1
Assuming stationarity:
Christos G. Cassandras
Time spent at ith operating
mode
φi ( xi , xi −1 ) = φi ( si )
CODES Lab. - Boston University
HIERARCHICAL DECOMPOSITION
z&i = gi ( zi , ui , t )
N
min ∑ [φi ( si ) + ψ i ( xi )]
u
xi+1 = fi(xi,ui,t)
i =1
HIGHER
LEVEL
PROBLEM:
LOWER
LEVEL
PROBLEMS:
s.t.
N
[
min ∑ φi ( si ) + ψ i ( xi )
s
i =1
*
si
]
min φi ( si ) = ∫ Li ( zi (t ), ui (t ))dt
0
ui
s.t.
xi+1 = fi(xi,si,t)
s.t.
z&i = gi ( zi , ui , t )
FIXED si
Christos G. Cassandras
CODES Lab. - Boston University
HIERARCHICAL DECOMPOSITION
zi0
CONTINUED
zif
zif+1
zi0+1
zi0+ 2
si
si+1
xi-1
xi+1
xi
min
φi ( si )
f
0
min
f
0
u +1i ( zi +1 , zi +1 , si +1 )
u i ( z i , z i , si )
u ∗i z 0i , z fi , s i 
 i z 0i , z fi , s i x ≡ min
u i  i z i , u i , s i 
x
1
THE
THEREALLY
REALLY
CHALLENGING
CHALLENGING
PROBLEM!
PROBLEM!
Christos G. Cassandras
φi +1 ( si +1 )
…
2
[
]
s.t.
min
θ i ( z , zi , si ) + ψ i ( xi )
∑
0 f
xi+1 = fi(xi,ui,t)
z ,z , s
i =1
N
0
i
f
CODES Lab. - Boston University
HYBRID CONTROLLER STRUCTURE
f, ψ
Hybrid controller steps:
Event timing
xi+1 = fi(xi,si,t),
i = 1,…,N
• System identification
s*
• Lower-level solution
Higher-level Controller
• Higher-level solution
• Lower-level solution
• Operation...
s*
φ*(s)
Lower-level Controller
u*(s*)
&z = g ( z, u, t ) g, φ
Physical processes
Christos G. Cassandras
CODES Lab. - Boston University
TWO TYPES OF PROBLEMS…
z&1 = g1 ( z1 , u1 , t )
z1f
z&2 = g 2 ( z 2 , u2 , t )
zdf
z0
x0
s1
x1
s2
x2
1. A single event process controls switching dynamics: xi+1 = xi + si(zi,ui)
z&1 = g1 ( z1 , u1 , t )
z1f
0
2
z0
z
x0
x1
s1
2. Multiple event processes control
switching dynamics:
Christos G. Cassandras
z&2 = g 2 ( z 2 , u2 , t )
zdf
EXTE
EVEN RNAL
T PR x2
O CE
si+1
SS
xi +1 = max( xi , ai +1 ) + si ( zi , ui )
CODES Lab. - Boston University
EXAMPLE
Physical State, z
led
p
u
co s…
e
D de
mo
z&i = gi ( zi , ui , t )
…
IDLING
a1
a2
x1
x2
a3
x3
xi
ai+1
Temporal State, x
x i = max {x i − 1 , a i } + s i ( z i , u i )
External event process: ith mode cannot start before ai+1
Christos G. Cassandras
CODES Lab. - Boston University
MANUFACTURING SYSTEMS
Throughout a manuf. process, each part is characterized by
• A PHYSICAL state (e.g., size, temperature, strain)
• A TEMPORAL state (e.g., total time in system, total time to due-date)
PHYSICAL
STATE
NEW
PHYSICAL
STATE
Time-driven
Dynamics
z&i = gi ( zi , ui , t )
OPERATION
OPERATION
TEMPORAL
STATE
Christos G. Cassandras
xk +1,i = max{xk , j + a k , j u k , j }
j∈Γi
Event-driven
Dynamics
NEW
TEMPORAL
STATE
CODES Lab. - Boston University
MANUFACTURING SYSTEMS
CONTINUED
x i = max {x i − 1 , a i }+ s i ( u i )
EVENT-DRIVEN
COMPONENT
Part
Arrivals
Part
Departures
ai, zi(ai)
TIME-DRIVEN
COMPONENT
Christos G. Cassandras
xi, zi(xi)
ui
z&i (t ) = g (zi , ui , t )
CODES Lab. - Boston University
EXAMPLE
…a
to nd m
t
u
(e.g his s st be
., d tate
pro
e si
ces
red
sed
tem
per
atu
re)
STATE
STATE
rts
a
t
s
ar t
p
y
r
Eve is state
at th
zi(ui)
si(ui)
Christos G. Cassandras
PROCESS
PROCESS TIME
TIME
CODES Lab. - Boston University
LOWER LEVEL PROBLEM
Parameterized by switching times
LQ PROBLEM:
min φi ( zi , ui , si ) = h ( z fi − zdi ) + ∫
1
2
ui
s.t. z&i = azi + bui , zi (0) = ς i
si
1
0 2
2
2
i
ru (t )dt
Penalize final state deviation
STANDARD LQ SOLUTION METHOD:
φi ( si ) = h( z − zdi ) + ∫
*
Christos G. Cassandras
1
2
*
fi
2
si
1
0 2
*
i
ru (t )dt
CODES Lab. - Boston University
HIGHER LEVEL PROBLEM
N
[
min ∑ φi ( si ) + ψ i ( xi )
s
i =1
*
]
s.t. x i = max {x i − 1 , a i }+ s i ( u i )
Given arrival
sequence (INPUT)
Cost of optimal
process control over
interval [0, si]
Processing time
(CONTOLLABLE)
Cost related to
event timing
EXAMPLE :
Christos G. Cassandras
ψ i ( xi ) = ( xi − τ i )
CODES Lab. - Boston University
2
HOW DO WE SOLVE THE HIGHER LEVEL PROBLEM?
N
[
min ∑ φi ( si ) + ψ i ( xi )
s
i =1
*
Even if these are convex,
problem is still NOT convex in s!
]
s.t. x i = max {x i − 1 , a i }+ s i ( u i )
Causes nondifferentiabilities!
Even though problem is
NONDIFFERENTIABLE and NONCONVEX,
optimal solution shown to be unique.
[Cassandras,
[Cassandras,Pepyne,
Pepyne,Wardi,
Wardi,IEEE
IEEETAC
TAC2001]
2001]
Christos G. Cassandras
CODES Lab. - Boston University
SOLVING THE HIGHER LEVEL PROBLEM
a1 a2 a3
x1 x2 x3
a4 a5 a6
x4 x5 x6
a7 a8
x7
a9
CONTINUED
IME
T
N
I
T
JUS
x8
Each “block” corresponds to a
Constrained Convex Optimization problem
⇒ search over 2N-1 possible Constrained Convex Optimization problems
BUT
algorithms that only need N Constrained Convex Optimization
problems have been developed ⇒ SCALEABILITY
[Cho,
[Cho,Cassandras,
Cassandras,Intl.
Intl.J.J.Rob.
Rob.and
andNonlin.Control,
Nonlin.Control,2001]
2001]
Christos G. Cassandras
CODES Lab. - Boston University
INTERACTIVE JAVA APPLETS
• http://vita.bu.edu/cgc/hybrid
- Single-stage model
- Backward-recursive TPBVP solver
with critical job identification
• http://vita.bu.edu/cgc/newhybrid
- 3-stage model
- Bezier approximation with
standard TPBVP solver
Christos G. Cassandras
CODES Lab. - Boston University
http://vita.bu.edu/cgc/hybrid
Christos G. Cassandras
CODES Lab. - Boston University
ABSTRACTION
Christos G. Cassandras
CODES Lab. - Boston University
COMPLEX DES ABSTRACTION
DISCRETE-EVENT
SYSTEM
TIME-DRIVEN
FLOW RATE
DYNAMICS
EVENTS
HYBRID
SYSTEM
Christos G. Cassandras
CODES Lab. - Boston University
STOCHASTIC FLOW MODELS (SFM)
θ
CONTROLLER
α(t)
α(t) – p(x)
β(t)
γ(t)
x(t)
(t),ββ(t):
(t):arbitrary
arbitrarystochastic
stochasticprocesses
processes
αα(t),
(piecewisecontinuously
continuouslydifferentiable)
differentiable)
(piecewise
0
x(t ) = 0, λ (t ) − p (0) ≤ 0
⎧
dx ⎪
=⎨
0
x(t ) = θ , λ (t ) − p (θ ) ≥ 0
dt ⎪
otherwise
⎩λ (t ) − p ( x(t ))
λλ(t)(t)==αα(t)(t)--ββ(t)(t)
Christos G. Cassandras
feedback
feedback
CODES Lab. - Boston University
WHY SFM?
¾ “Lower resolution” model of “real” system intended to capture
just enough info. on system dynamics
¾ Aggregates many events into simple continuous dynamics,
preserves only events that cause drastic change
⇒ computationally efficient
(e.g., orders of magnitude faster simulation)
¾ Intended for developing CONTROL schemes
rather than for PERFORMANCE ANALYSIS
Christos G. Cassandras
CODES Lab. - Boston University
MOTIVATING EXAMPLE: THRESHOLD-BASED BUFFER CONTROL
K
ARRIVAL
PROCESS
“REAL” SYSTEM
LOSS
x(t)
̄ T K  R  L̄ T K
J T K  Q
θ
β(t)
α(t)
RANDOM
PROCESS
γ(t)
x(t)
RANDOM
PROCESS
Stochastic
Flow
Model
(SFM)
̄ SFM
J SFM
  Q
  R  L̄ SFM

T
T
T
Christos G. Cassandras
CODES Lab. - Boston University
MOTIVATING EXAMPLE
CONTINUED
25
“Real” System
23
J(K)
21
SFM
19
Optim. Algorithm
using SFM-based
gradient estimates
17
DES
SFM
Opt. A lgo
15
0
5
10
15
20
25
30
35
40
45
K
Cassandrasetetal,
al,2002
2002
Cassandras
Christos G. Cassandras
CODES Lab. - Boston University
OPTIMIZATION OF SFMS
¾ Stochastic Optimal Control problems too hard!..
¾ Parametric optimization:
use gradient estimates with on-line opt. algorithms
SFM
n1  n −nHnn,n ,
n  0,1,…
¾ Need efficient ways to estimate performance sensitivities
Christos G. Cassandras
CODES Lab. - Boston University
INFINITESIMAL PERTURBATION ANALYSIS (IPA)
“Brute Force” Sensitivity Estimation:
θ
Jˆ (θ
DES
θ+Δθ
)
J$ (θ + Δ θ )
DES
]
J$ ( θ + Δ θ ) − J$ ( θ )
⎡ dJ ⎤
⇒⎢ ⎥ =
Δθ
⎣ d θ ⎦ est
Finite Perturbation Analysis (FPA):
θ
Δθ
J$ (θ )
DES
FPA
FPA
J$ (θ + Δ θ )
]
J$ ( θ + Δ θ ) − J$ ( θ )
⎡ dJ ⎤
⇒ ⎢
=
⎥
Δθ
⎣ d θ ⎦ est
Infinitesimal or Smoothed Perturbation Analysis (IPA, SPA):
θ
J$ (θ )
DES
IPA/SPA
IPA/SPA
Christos G. Cassandras
⎡ d J ⎤
⎢⎣ d θ ⎥⎦
est
CODES Lab. - Boston University
INFINITESIMAL PERTURBATION ANALYSIS (IPA)
OBJECTIVES:
• Obtain sample performance derivatives that depend ONLY
on observed sample path data:
Performance
′
L T 
≡
dL T
d
Parameter
• Prove unbiasedness:
dE  L T 
d
E
dL T
d
• Then, use gradient estimates to drive on-line opt. algorithms:
θ n +1 = θ n + η L (θ n ), n = 0,1,K
'
n T
Christos G. Cassandras
CODES Lab. - Boston University
THRESHOLD-BASED BUFFER CONTROL
θ
β(t)
α(t)
γ(t)
x(t)
J T (θ ) = QT (θ ) + RLT (θ )
WORK
T
Q T    0 x;tdt
Christos G. Cassandras
LOSS
T
L T    0 ; tdt
CODES Lab. - Boston University
THRESHOLD-BASED BUFFER CONTROL
CONTINUED
θ
ξ1
η1
ξ2
u1,2
v1,2 u2,2 v2,2
BUFFERING PERIOD: B k   k , k ,
η2
ξ3
u1,3 v1,3 η
3
k  1, …, K
OVERFLOW PERIOD: F i,k  u i,k , v i,k , i  1, …, M
SET OF BPs WITH AT LEAST ONE OVERFLOW:
 : k ∈ 1, …, K : xt  , t − t  0 for some t ∈  k , k 
B  ||
Christos G. Cassandras
CODES Lab. - Boston University
IPA – MAIN RESULTS
Cassandrasetetal,
al,2002
2002
Cassandras
THEOREM 1 (Loss IPA derivative):
′
L T   − B
• Simple count of Buffering Periods with at least one overflow
• Nonparametric (independent of θ and any model assumption)
THEOREM 2 (Work IPA derivative):
Q T   ∑ k∈ k  − u k,1 
′
• Simple timer for each Buffering Period with at least one overflow
• Nonparametric (independent of θ and any model assumption)
Christos G. Cassandras
CODES Lab. - Boston University
IPA – MAIN RESULTS
CONTINUED
Let N(T) be the number of events in [0, T].
THEOREM:
dEL T
1. If E[N(T)] < ∞, then L T  is an unbiased estimator of
d
′
dEQ T
Q

2. T
is an unbiased estimator of
d
′
Christos G. Cassandras
CODES Lab. - Boston University
THE FUTURE…
¾ COMPLEXITY
- GETTING AROUND IT THROUGH STRUCTURAL PROPERTIES
- FINDING THE “RIGHT” MODELING RESOLUTION
¾ COMPUTATIONAL CHALLENGES
- REAL-TIME IMPLEMENTATIONS, EMBEDDED SYSTEMS
- SOFTWARE TOOLS (for Verification, Optimization, etc.)
¾ APPLICATION DOMAINS
COOPERATIVE CONTROL, AUTOMOTIVE,
COMPUTER NETWORKS, LOGISTICS, BIOMEDICAL
Christos G. Cassandras
CODES Lab. - Boston University
ACKNOWLEDGEMENTS
Thanks to several co-workers whose contributions
are reflected in this talk…
… All 13 Former
+ Current PhD Students…
…Colleagues…
Y.C. (Larry) Ho
Wei-Bo Gong
Yorai Wardi
Young Cho
Liyi Dai
Xiren Cao
Ted Djaferis
Doug Looze
Felisa Vazquez-Abad
…and Sponsors…
NSF, AFOSR, AFRL, DARPA, NASA,
EPRI, ARO, UT, ALCOA, NOKIA, GE