DISCRETE EVENT SYSTEMS
MODELING AND
PERFORMANCE ANALYSIS
C. G. Cassandras
Division of Systems Engineering
and Dept. of Electrical and Computer Engineering
and Center for Information and Systems Engineering
Boston University
Christos G. Cassandras
CODES Lab. - Boston University
OUTLINE
 Why are DISCRETE EVENT SYSTEMS important?
 Models for Discrete Event Systems (DES) and Hybrid Systems (HS)
 DES Simulation
 Control and Optimization in DES: Resource Contention problems
 “Rapid Learning”:
Perturbation Analysis (PA) and Concurrent Estimation (CE)
 Applications
 Dealing with Complexity – Fundamental Complexity Limits
 Abstraction through Hybrid Systems:
Stochastic Flow Models (SFM), the IPA Calculus
Christos G. Cassandras
CODES Lab. - Boston University
WHY DISCRETE EVENT SYSTEMS ?
 Many systems are naturally Discrete Event Systems (DES)
(e.g., Internet)
→ all state transitions are event-driven
 Most of the rest are Hybrid Systems (HS)
→ some state transitions are event-driven
 Many systems are distributed
→ components interact asynchronously (through events)
 Time-driven sampling inherently inefficient (“open loop” sampling)
→ event-driven control
Christos G. Cassandras
CODES Lab. - Boston University
REASONS FOR EVENT-DRIVEN
MODELS, CONTROL, OPTIMIZATION
 Many systems are stochastic
→ actions needed in response to random events
 Event-driven methods provide significant advantages in
computation and estimation quality
 System performance is often more sensitive to event-driven
components than to time-driven components
 Many systems are wirelessly networked → energy constrained
→ time-driven communication consumes energy unnecessarily
→ use event-driven control
Christos G. Cassandras
CODES Lab. - Boston University
TIME-DRIVEN v EVENT-DRIVEN CONTROL
REFERENCE
+
ERROR
-
CONTROLLER
MEASURED
OUTPUT
INPUT
PLANT
OUTPUT
SENSOR
EVENT-DRIVEN CONTROL: Act only when needed (or on TIMEOUT) - not based on a clock
REFERENCE
+
ERROR
MEASURED
OUTPUT
CONTROLLER
EVENT:
g(STATE) ≤ 0
Christos G. Cassandras
INPUT
PLANT
OUTPUT
SENSOR
CODES Lab. - Boston University
SYNCHRONOUS v ASYNCHRONOUS BEHAVIOR
Indistinguishable events
INCREASING TIME GRANULARITY
Wasted clock ticks
More wasted clock ticks
…
Even more wasted clock ticks
Christos G. Cassandras
CODES Lab. - Boston University
SYNCHRONOUS v ASYNCHRONOUS COMPUTATION
x
y
x
y
TIME
us
o
n
ro
h
c
n
Asy events
t1
x+y
t2
TIME
Time-driven (synchronous) implementation:
- Sum repeatedly evaluated unnecessarily
- When evaluation is actually needed, it is done at the wrong times !
Christos G. Cassandras
CODES Lab. - Boston University
MODELING DES AND HS:
-Timed Automata
- Hybrid Automata
OTHER MODELS NOT COVERED HERE:
- Timed Petri Nets
- Max-Plus Algebra
- Temporal Logic
TIME-DRIVEN v EVENT-DRIVEN SYSTEMS
TIME-DRIVEN
SYSTEM
STATES
STATE SPACE:
x(t)
X 
DYNAMICS:
x  f  x , t 
t TIME
EVENT-DRIVEN
SYSTEM
STATES
STATE SPACE:
X   s1 , s2 , s3 , s4 
s4
s3
s2
x(t)
s1
Christos G. Cassandras
t1
t2
t3
t4
t5
e1
e2
e3
e4
e5
t TIME
DYNAMICS:
x '  f  x, e 
EVENTS
CODES Lab. - Boston University
AUTOMATON
AUTOMATON: (E, X, , f, x0)
: Event Set
X : State Space
(x) : Set of feasible or enabled events at state x
f : State Transition Function f : X  E  X
(undefined for events e    x  )
x0 : Initial State, x 0  X
e1 , e2 ,
Christos G. Cassandras
f  x , e   x'
x1 , x2 ,
CODES Lab. - Boston University
TIMED AUTOMATON
Add a Clock Structure V to the automaton: (E, X, , f, x0 , V)
where:
V  v i : i  E 
and vi is a Clock or Lifetime sequence: v i  v i 1 , v i 2 ,  
one for each event i
v1
f  x , e'   x'

vN
NEXT EVENT
Christos G. Cassandras
x1 , x2 ,
Need
Needan
aninternal
internalmechanism
mechanismtotodetermine
determine
NEXT
NEXTEVENT
EVENTe´e´and
andhence
hence
x'  f  x , e' 
NEXT
NEXTSTATE
STATE
CODES Lab. - Boston University
HOW THE TIMED AUTOMATON WORKS...
 CURRENT STATE
xXwith
withfeasible
feasibleevent
eventset
set(x)
(x)
xX
 CURRENT EVENT
thatcaused
causedtransition
transitioninto
intoxx
eethat

CURRENT EVENT TIME
associatedwith
withee
ttassociated
Associateaa
Associate
CLOCKVALUE/RESIDUAL
VALUE/RESIDUALLIFETIME
LIFETIME yyi
CLOCK
i
witheach
eachfeasible
feasibleevent
event i    x 
with
Christos G. Cassandras
CODES Lab. - Boston University
HOW THE TIMED AUTOMATON WORKS...

NEXT/TRIGGERING EVENT e' :
e '  arg min y i 
i  x 

NEXT EVENT TIME t' :
t'  t  y *
where: y *  min  yi 
i   x 

NEXT STATE x' :
x'  f  x , e' 
Christos G. Cassandras
CODES Lab. - Boston University
HOW THE TIMED AUTOMATON WORKS...
Determinenew
newCLOCK
CLOCKVALUES
VALUES yi
Determine
forevery
everyevent
event i    x 
for
 yi  y * i    x, i    x , i  e


yi  vij
i    x    x   e
0
otherwise

KS
C
O
LES
L
B
C
A
I
T
R
N
EVE TATE VA
S
ARE
v1

vN
Christos G. Cassandras
where : vij  new lifetime for event i
x '  f  x , e ' , e '  arg min { y i }
y '  g y , x, V 
i ( x )
x1 , x2 ,
CODES Lab. - Boston University
TIMED AUTOMATON - AN EXAMPLE
a
0
2
1
d
E  a , d 
X  0 ,1, 2 , 
d
d
a
a
a
a

n+1
n
d
d
  x   a , d , for all x  0
 0   a 
 x  1 e  a
f  x ,e  
 x  1 e  d , x  0
Given input : v a  v a 1 , v a 2 , , v d  v d 1 , v d 2 , 
Christos G. Cassandras
CODES Lab. - Boston University

TIMED AUTOMATON - A STATE TRAJECTORY
e1 = a
x0 = 0
e2 = a
x0 = 1
t0
t1
e3 = a
x0 = 2
e4 = d
x0 = 3
t2
t3
x0 = 2
t4
a
a
d
a
d
a
d
Christos G. Cassandras
CODES Lab. - Boston University
STOCHASTIC TIMED AUTOMATON
 Same idea with the Clock Structure consisting of Stochastic Processes
 Associate with each event i a Lifetime Distribution based on
which vi is generated
Generalized Semi-Markov Process
(GSMP)
In
Inaasimulator,
simulator,vvi iisisgenerated
generatedthrough
throughaa
pseudorandom
pseudorandomnumber
numbergenerator
generator
Christos G. Cassandras
CODES Lab. - Boston University
HYBRID
AUTOMATA
HYBRID
AUTOMATA
Gh  (Q, X , E , U , f ,  , Inv, guard ,  , q0 , x 0 )
Q:
set of discrete states (modes)
X:
set of continuous states (normally Rn)
E:
set of events
U:
f:
set of admissible controls
vector field, f : Q  X  U  X
:
discrete state transition function,  : Q  X  E  Q
Inv:
set defining an invariant condition (domain), Inv  Q  X
guard: set defining a guard condition, guard  Q  Q  X
:
reset function,  : Q  Q  X  E  X
q0:
initial discrete state
x0 :
initial continuous state
Christos G. Cassandras
CODES Lab. - Boston University
HYBRID
AUTOMATA
HYBRID
AUTOMATA
Key features:
Transition MAY occur
Guard condition:
Subset of X in which a transition from
q to q' is enabled, defined through 
Transition MUST occur
Invariant condition:
(domain)
Reset condition:
Christos G. Cassandras
Subset of X to which x must belong in order
to remain in q. If this condition no longer holds,
a transition to some q' must occur,
defined through 
New value x' at q' when transition occurs
from (x,q)
CODES Lab. - Boston University
HYBRID
AUTOMATA
HYBRID
AUTOMATA
Unreliable machine with timeouts
1
x  u
  1
[  T ]
0
x  0
  0
x'  0
 ' 0
Reset
 T
BUSY

IDLE
x'  0
 ' 0
DOWN
2
x  0
  0
, x  K
Invariant

Guard
x(t) : physical state of part in machine
(t): clock
 : START,  : STOP,  : REPAIR
Christos G. Cassandras
CODES Lab. - Boston University
HYBRID
AUTOMATA
HYBRID
AUTOMATA
BUSY

IDLE
1
x  u
  1
[  T ]
0
x  0
  0
x'  0
 ' 0
, x  K
Reset
e 
1
 (0; x, ; e)  
0 otherwise
e
0
 (2; x, ; e)  
2 otherwise
Christos G. Cassandras
 T
x'  0
 ' 0
DOWN
2
x  0
  0
Invariant

Guard
 T
2

 (1; x, ; e)  0 x  K , e  
1
otherwise

CODES Lab. - Boston University
DES
SIMULATION
DISCRETE EVENT SIMULATION
…is simply a computer-based implementation of the DES sample path
generation mechanism described so far
INITIALIZE
INITIALIZE
STATE
STATE
TIME
TIME
xx
SCHEDULED
EVENT LIST
x'
UPDATESTATE
STATE
UPDATE
f(x,e)1)
x'x'==f(x,e
1
e1
e2
...
x'
tt
t'
UPDATE
UPDATETIME
TIME
t't' ==t1t
t1
t2
1
t'
DELETE
DELETEINFEASIBLE
INFEASIBLE
(e(ek, ,tkt) )
k
RANDOM
RANDOMVARIATE
VARIATE
GENERATOR
GENERATOR
Christos G. Cassandras
ADD
ADDNEW
NEWFEASIBLE
FEASIBLE
(e(ek, ,t't'+v
+vk)k)
k
AND
ANDRE-ORDER
RE-ORDER
k
new event lifetimes vk
CODES Lab. - Boston University
DISCRETE EVENT SIMULATION - EXAMPLE
e1 = a
x0 = 0
e2 = a
x0 = 1
e3 = a
x0 = 2
x0 = 3
t0
t1
t2
t3
a t1
a t2
a t3
d t4
d t4
a t5
d t4
SCHEDULED
EVENT
e4 = d
x0 = 2
t4
SCHEDULED
EVENT LIST
(EVENT CALENDAR)
SCHEDULED
TIME
Christos G. Cassandras
CODES Lab. - Boston University
DISCRETE EVENT SIMULATION - EXAMPLE
QUEUE
SERVER
Arrival
Events
Departure
Events
Event-Based
Random Number
#n
t
IN
OUT
OUT
IN
FIFO Queue1
Server 1
Route with equal probability
to either queue
OUT
OUT1
IN1
OUT2
IN2
OUT
IN
Output Switch
Arrival Process1
Event-Based
Random Number1
#n
Entity Sink1
Path Combiner
t
IN
OUT
OUT
FIFO Queue2
IN
IN
IN
Server 2
w
OUT
IN
OUT
Start Timer1
Read Timer1
Christos G. Cassandras
CODES Lab. - Boston University
Average
Syatem Time
DISCRETE
EVENT SIMULATION - SimEvents
HYBRID AUTOMATA
www.mathworks.com/products/simevents/
[Cassandras, Clune, and Mosterman, 2006]
SOFTWARE IMPLEMENTATION ISSUES
• Object-oriented design (Libraries)
• Flexibility (user can easily define new objects)
• Hierarchical (macro) capability [e.g., a G/G/m/n block]
• Random Variate generation accuracy
• Execution speed
• User interface
• Animation (useful sometimes – distracting/slow other times)
• Integrating with operational control software
(e.g., scheduling, resource allocation, flow control)
• Web-based simulation: Google-like capability !?
Christos G. Cassandras
CODES Lab. - Boston University
SELECTED REFERENCES - MODELING
Timed Automata, Timed Petri Nets, Max-Plus Algebra
– Alur, R., and D.L. Dill, “A Theory of Timed Automata,” Theoretical Computer
Science, No. 126, pp. 183-235, 1994.
– Cassandras, C.G, and S. Lafortune, “Introduction to Discrete Event Systems,” Springer, 2008.
– Wang, J., “Timed Petri Nets - Theory and Application,” Kluwer Academic Publishers, Boston, 1998.
– Heidergott, B., G.J. Olsder, and J. van der Woude, “Max Plus at Work – Modeling and
Analysis of Synchronized Systems: A Course on Max-Plus Algebra and its Applications,”
Princeton University Press, 2006
Hybrid Systems
– Bemporad, A. and M. Morari, “Control of Systems Integrating Logic Dynamics
and Constraints,” Automatica, Vol. 35, No. 3, pp.407-427, 1999.
– Branicky, M.S., V.S. Borkar, and S.K. Mitter, “A Unified Framework for Hybrid Control: Model and Optimal Control Theory,”IEEE Trans. on Automatic
Control, Vol. 43, No. 1, pp. 31-45, 1998.
– Cassandras, C.G., and J. Lygeros, “Stochastic Hybrid Systems,” Taylor and Francis, 2007.
– Hristu-Varsakelis, D. and W.S. Levine, Handbook of Networked and Embedded
Control Systems, Birkhauser, Boston, 2005.
Christos G. Cassandras
CODES Lab. - Boston University
CONTROL AND
OPTIMIZATION
IN DES
DES CONTROL
ENABLE/DISABLE controllable events in order to achieve desired
LOGICAL BEHAVIOR
• Reach desirable states
• Avoid undesirable states
• Prevent system deadlock
SUPERVISORY
CONTROL
ENABLE/DISABLE controllable events in order to achieve desired
PERFORMANCE
• N events of type 1 within T time units
• nth type 1 event occurs before mth type 2 event
• No more than N type 1 events after T time units
Christos G. Cassandras
CODES Lab. - Boston University
RESOURCE CONTENTION
PROBLEMS
IN DES
RESOURCE CONTENTION
USERS competing for limited RESOURCES in an event-driven dynamic
environment
Examples:
• MESSAGES competing for SWITCHES in networks
• TASKS competing for PROCESSORS in computers
• PARTS competing for EQUIPMENT in manufacturing
TYPICAL GOALS:
• User requests satisfied on “best effort” basis
• User requests satisfied on “guaranteed performance” basis
• Users treated “fairly”
Christos G. Cassandras
CODES Lab. - Boston University
RESOURCE CONTENTION
1. ADMISSION CONTROL
Admit or Reject user requests
(e.g., to ensure good quality of service for admitted users)
Arrivals
ADMISSION
CONTROL
Departures
Admit
Reject
2. FLOW CONTROL
Control when to accept user requests
(e.g., to “smooth” bursty demand, prevent congestion)
Arrivals
Christos G. Cassandras
FLOW
CONTROL
Stop/Go
SYSTEM
CODES Lab. - Boston University
RESOURCE CONTENTION
3. ROUTING
User selects a resource
(e.g., route to shortest queue)

Arrivals
ROUTING

4. SCHEDULING
Resource selects user
(e.g., serve longest queue first)
Class 1
Arrivals
Class 2
Arrivals
Christos G. Cassandras
SCHEDULING
CODES Lab. - Boston University
RESOURCE ALLOCATION
Manufacturing system with N sequential operations:
(t)
…
…
…
…
THROUGHPUT
DELAY
THROUGHPUT
increases
(GOOD)
Christos G. Cassandras
average DELAY
increases
(BAD)
AV. DELAY
SYSTEM
CAPACITY
INCREASE (t)
THROUGHPUT
CODES Lab. - Boston University
RESOURCE ALLOCATION
KANBAN (OR BUFFER) ALLOCATION
(t)
…
…
…
K1
THROUGHPUT
…
J(K1,…,KN)
K2
KN
N
max J ( K1 ,..., K N ) s.t.  K i  C
K1 ,..., K N
Christos G. Cassandras
i 1
CODES Lab. - Boston University
RESOURCE ALLOCATION
xi(t)
…
…
…
…
…
STOP - GO CONTROLLERS
GO ifif xxi(t)
< KKi,i, STOP
STOP otherwise
otherwise
GO
i(t) <
CK
A
B
D
FEE
Christos G. Cassandras
No.of
ofKANBAN
KANBAN(tickets)
(tickets)
No.
allocatedto
tostage
stageii
allocated
CODES Lab. - Boston University
RESOURCE ALLOCATION
x1(t)
x2(t)
…
…
…
u1
xN(t)
u2
…
J(u1,…,uN)
uN
CONTROL PARAMETERS
D(u1,…,uN)
 D(u1 ,..., u N )  C
max J (u1 ,..., u N ) s.t. 
u1 ,...,u N
system dynamics
Christos G. Cassandras
CODES Lab. - Boston University
SOLUTION METHODOLOGIES
• Queueing models and analysis: Descriptive, not prescriptive
• Markov Decision Processes (MDPs):
- Decisions planned ahead
- Need accurate stochastic models
- Curse of dimensionality (Dynamic Programming)
• Approximate Dynamic Programming (ADP) techniques
• Data-driven techniques:
- Gradient Estimation
- Rapid Learning
Christos G. Cassandras
CODES Lab. - Boston University
STOCHASTIC CONTROL
AND OPTIMIZATION:
PERTURBATION ANALYSIS,
RAPID LEARNING
REAL-TIME STOCHASTIC CONTROL
AND OPTIMIZATION
GOAL:
max E[ L( )]
 
CONTROL/DECISION
(Parameterized by )
SYSTEM
PERFORMANCE
E[ L( )]
NOISE
 n 1   n   nL( n )
L( )
GRADIENT
ESTIMATOR
x(t)
L()
DIFFICULTIES: - E[L()] NOT available in closed form
-  L ( ) not easy to evaluate
-  L ( ) may not be a good estimate of  E [ L ( )]
Christos G. Cassandras
CISE - CODES Lab. - Boston University
THE CASE OF DES:
INFINITESIMAL PERTURBATION ANALYSIS (IPA)
Model
Sample path
x(t)
CONTROL/DECISION
(Parameterized by )
Discrete Event
System (DES)
PERFORMANCE
E[ L( )]
NOISE
 n 1   n   nL( n )
IPA
L( )
x(t)
L()
For many (but NOT all) DES:
- Unbiased estimators
- General distributions
- Simple on-line implementation
[Ho and Cao, 1991], [Glasserman, 1991], [Cassandras, 1993], [Cassandras and Lafortune, 2008]
Christos G. Cassandras
CISE - CODES Lab. - Boston University
RAPID LEARNING
Christos G. Cassandras
CODES Lab. - Boston University
LEARNING BY TRIAL-AND-ERROR
Design
Design
Parameters
Parameters
SYSTEM
Performance
Performance
Measures
Measures
OperatingPolicies,
Policies,
Operating
ControlParameters
Parameters
Control
CONVENTIONAL TRIAL-AND-ERROR ANALYSIS
(e.g., simulation)
• Repeatedly change parameters/operating policies
• Test different conditions
• Answer multiple WHAT IF questions
Christos G. Cassandras
CODES Lab. - Boston University
LEARNING THROUGH CONCURRENT ESTIMATION
Design
Design
Parameters
Parameters
SYSTEM
OperatingPolicies,
Policies,
Operating
ControlParameters
Parameters
Control
WHAT
WHATIF…
IF…
Performance
Performance
Measures
Measures
CONCURRENT
CONCURRENT
ESTIMATION
ESTIMATION
• •Parameter
Parameterpp11==aawere
werereplaced
replacedby
bypp11==bb
• •Parameter
Parameterpp22==ccwere
werereplaced
replacedby
bypp22==dd
••
••
Performance
PerformanceMeasures
Measures
under
underallallWHAT
WHATIFIFQuestions
Questions
ANSWERS TO MULTIPLE “WHAT IF” QUESTIONS
AUTOMATICALLY PROVIDED
Christos G. Cassandras
CODES Lab. - Boston University
LEARNING THROUGH CONCURRENT ESTIMATION
BUFFER
MACHINE
Part
Arrivals
Part
Departures
x(t)
2
Observed
sample path:
K=2
1
1
x(t)
2
3
x(t+) = 2
 x = -1
4
x(t+) = 0
 x = 0
2
Perturbed
sample path:
K=1
1
1
x = 0
Christos G. Cassandras
4
2
x = -1
x = 0
CODES Lab. - Boston University
PERTURBATION
ANALYSIS
Christos G. Cassandras
CODES Lab. - Boston University
IPA vs “BRUTE FORCE” DERIVATIVE ESTIMATION
“Brute Force” Derivative Estimation:


SYSTEM
SYSTEM
Jˆ 

J     
]
J (     )  J (  )
 dJ 
  

 d   est
DRAWBACKS: • Intrusive: actively introduces perturbation 
• Computationally costly: 2 observation processes
[ (N+1) for N-dim 
• Inherently inaccurate:  large  poor derivative approx.
 small  numerical instability
Infinitesimal Perturbation Analysis (IPA):

SYSTEM
IPA
IPA
Christos G. Cassandras
J  
 dJ 
 d  
est
CISE - CODES Lab. - Boston University
PERTURBATION GENERATION
 DES parameter  perturbed by 
 Suppose  is a parameter of some cdf F(x;)
 OBSERVED sample path: X()
 What is X() ?
dX ( )
 What is
?
d
EXAMPLE:
QUEUE
a
SERVER
d
- Mean Service Time: 
- Service Times: X1(), X2(), …
- What happens when  is
changed by  ?
Christos G. Cassandras
CODES Lab. - Boston University
PERTURBATION GENERATION
F(x;)
1
F(x;+)
U
X
X '  F 1[ F ( X ; );   )]
X  F 1 (U ; )
X  F 1[ F ( X ; );   )]  X
Christos G. Cassandras
U
F ( x; ) /  ( X , )
dX

F ( x; ) / x( X , )
d
CODES Lab. - Boston University
PERTURBATION PROPAGATION
QUEUE
A1, A2, …
SERVER
D1 , D 2 , …
Z1(), Z2(), …
Suppose  is perturbed by   Dk  Dk'  Dk
Lindley Equation for
any queueing system: Dk  max{ Ak , Dk 1}  Z k

Max-Plus Algebra
(One of many dioid algebras)
Cunningham-Greene, R.A.,
Minimax Algebra, 1979.
Dk  max{ Ak , Dk' 1}  max{ Ak , Dk 1}  Z k
 max{ Ak , Dk 1  Dk 1}  max{ Ak , Dk 1}  Z k
Iterative relationship that depends
ONLY on observed sample path data !
Christos G. Cassandras
CODES Lab. - Boston University
PERTURBATION PROPAGATION
Four cases to consider… After some algebra:
 Dk 1

0

Dk  Z k  
Ik

Dk 1  I k
where:
Ik
Zk
I k  Ak  Dk 1
if
if
if
if
Ik
Ik
Ik
Ik
 0 and Dk 1  I k
 0 and Dk 1  I k
 0 and Dk 1  I k
 0 and Dk 1  I k
(idle period length if > 0)
observed
calculated from Zk as described earlier
(PERTURBATION GENERATION)
Christos G. Cassandras
CODES Lab. - Boston University
PERTURBATION PROPAGATION
EXAMPLE: Perturbations in mean service time  Z1(), Z2(), …
2
I4
1
Z1
A1 A2
D1
A3 D2
D3 A4
A5
D4
D3-I4
2
1
1
D1 = Z1
1 2
4
D2
D4
1 2 3
D3 > I4
Christos G. Cassandras
CODES Lab. - Boston University
INFINITESIMAL PERTURBATION ANALYSIS
(IPA)
If  is so “small” as to ensure that Dk-1 ≤ Ik then
Dk 1 if I k  0
Dk  Z k  
otherwise
 0
Perturbation
Generation
Perturbation
Propagation
If derivatives are used, this can be rigorously shown:
dD
dDk dZ k  k 1

  d
d
d  0

Christos G. Cassandras
if I k  0
otherwise
CODES Lab. - Boston University
UNIVERSAL IPA ALGORITHM
1. INITIALIZATION: If  feasible at x0:
 :
dV ,1
d
Else for all other :  : 0
2. WHENEVER  IS OBSERVED (including  = ) :
If  activated with new lifetime V 
2.1. Compute dV/d
dV
2.2.  :   
d
IMPLEMENTATION: Simple, non-intrusive, overhead negligible
See
Christos G. Cassandras
http://people.bu.edu/cgc/IPA/
CODES Lab. - Boston University
INFINITESIMAL PERTURBATION ANALYSIS
(IPA)
QUESTION: Are IPA sensitivity estimators “good” ?
 Unbiasedness
 Consistency
ANSWER: Yes, for a large class of DES that includes
 G/G/1 queueing systems
 Jackson-like queueing networks
(e.g., no blocking allowed)
NOTE: IPA applies to REAL-VALUED parameters of some
event process distribution (e.g., mean interarrival and
service times)
Christos G. Cassandras
CODES Lab. - Boston University
IPA UNBIASEDNESS
Theorem. For a given sample function L(), suppose that
(a) the sample derivative dL()/d exists w.p. 1 for every   
where  is a closed bounded set, and
(b) L() is Lipschitz continuous w.p. 1 and the Lipschitz constant has
finite first moment.
Then,
d
 dL( ) 
E[ L( )]  E 

d
 d 
Christos G. Cassandras
[Rubinstein and Shapiro, 1993]
CODES Lab. - Boston University
COMMUTING CONDITION
States x, y, z1 and events , such that
p ( z1 ; x,  )  p ( y; z1 ,  )  0
Then, for some z2
p ( z 2 ; x,  )  p ( y; z1 ,  ) and p ( y; z 2 ,  )  p ( z1 ; x,  )
Moreover, for x, z1, z2 s.t. p(z1;x,) = p(z2;x,) > 0, z1 = z2

z1

y
x

Christos G. Cassandras
z2

[Glasserman, 1991]
CODES Lab. - Boston University
EXTENSIONS OF IPA
There are several generalizations, at the expense of more
simulation overhead (still, very minimal)
e.g.,
- routing probabilities
- systems with real-time constraints
- some scheduling policies
IMPLEMENTATION:
Still very straightforward and non-intrusive
Christos G. Cassandras
CODES Lab. - Boston University
CONCURRENT
ESTIMATION
Christos G. Cassandras
CODES Lab. - Boston University
THE CONSTRUCTABILITY PROBLEM
Consider a Discrete Event System (DES) operating
under parameter settings u1 , , u m 
INPUT:
 = all event lifetimes
u1 = a parameter setting
OUTPUT:
e

1
,
t
 observed sample path under u1
k
k
1
PROBLEM:
Construct sample paths under all u 2 , , u m  based only on
The input 

1 1
The observed sample path e k , t k

Observed state history
Christos G. Cassandras
CODES Lab. - Boston University
THE CONSTRUCTABILITY PROBLEM

u1
DES
DES





2 2
ek , t k
OBS
ERV
ED

u2

1 1
ek , t k
DES
DES
CONTINUED
um
Christos G. Cassandras
DES
DES
ekm , t km
TED
C
U
STR
N
O
C
CODES Lab. - Boston University
THE CONSTRUCTABILITY PROBLEM
CONTINUED
• OBSERVABILITY:

j
j
A sample path ek , t k
ek ,t k 
 is observable with respect to
 
if
 x k    x k  for all k = 0,1,...
j
j
INTERPRETATION: For every state xk
visited in the constructed sample path, all feasible
 
j
events i   x k are also feasible in the corresponding
observed state xk
Christos G. Cassandras
Ref:
Ref:Cassandras
Cassandrasand
andStrickland,
Strickland,1989
1989
CODES Lab. - Boston University
THE CONSTRUCTABILITY PROBLEM
CONTINUED
Define the conditional cdf of an event clock given the event age:
H t , zi   P  yi  t zi 
AGE
CLOCK
zi
yi
t
CONSTRUCTABILITY:

j
j
A sample path ek , t k
1.
 
 is constructable with respect to e ,t  if
k
 x k    x k  for all k = 0,1,...
k
j


 
j
j
j
2. H t , z i , k  H t , z i , k  for all i  Γ x k , k  0,1, 
Christos G. Cassandras
CODES Lab. - Boston University
CONSTRUCTABILITY: AN EXAMPLE
A simple design/optimization problem
a
ADMISSION
CONTROL
d
Reject if Queue Length > K
PROBLEM:
 How does the system behave under different
choices of K?
 What is the optimal K?
Christos G. Cassandras
CODES Lab. - Boston University
CONSTRUCTABILITY: AN EXAMPLE
a
ADMISSION
CONTROL
CONTINUED
d
Reject if Queue Length > K
CONCURRENT ESTIMATION APPROACH:
 Choose any K
 Simulate (or observe actual system) under K
 Apply Concurrent Simulation to LEARN effect of all
other feasible K
Christos G. Cassandras
CODES Lab. - Boston University
CONTINUED
CONSTRUCTABILITY: AN EXAMPLE
a
a


d
a


d
a
a

d
PERTURBED
PERTURBEDSYSTEM:
SYSTEM: KK==22
d


a
a
d
d


a
a
d
Christos G. Cassandras
a


d
d
NOMINAL SYSTEM: K = 3
AUGMENTED SYSTEM:
Check for
OBSERVABILITY
a

d
a
a

CODES Lab. - Boston University
CONSTRUCTABILITY: AN EXAMPLE
d

  0   a   1  a , d 

a
a
d
CONTINUED
d


a
a
d

d
a
a

OBSERVABILITY satisfied!
However, if roles of NOMINAL and PERTURBED are reversed,
then OBSERVABILITY is not satisfied...
Christos G. Cassandras
CODES Lab. - Boston University
SOLVING CONSTRUCTABILITY
 Constructability is not easily satisfied, in general
 However, the CONSTRUCTABILITY PROBLEM can be solved a
some cost
SOLUTION METHODOLOGIES:
 STANDARD CLOCK (SC) methodology for Markovian systems
Ref:
Ref:Vakili,
Vakili,1991
1991
 AUGMENTED SYSTEM ANALYSIS (ASA)
for systems with at most one non-Markovian event
Ref:
Ref:Cassandras
Cassandrasand
andStrickland,
Strickland,1989
1989
 TIME WARPING ALGORITHM (TWA) for arbitrary systems
Ref:
Ref:Cassandras
Cassandrasand
andPanayiotou,
Panayiotou,1996
1996
Christos G. Cassandras
CODES Lab. - Boston University
AUGMENTED SYSTEM ANALYSIS (ASA)
 Assume Markovian DES  only OBSERVABILITY required
 Observe sample path of 1
 As state sequence unfolds, check for OBSERVABILITY
for every constructed n = 2,…,m
 If OBSERVABILITY violated for some n,
suspend nth sample path construction
 Wait until a state of 1 is entered s.t. OBSERVABILITY
satisfied for suspended nth sample path and resume construction
Christos G. Cassandras
CODES Lab. - Boston University
EVENT MATCHING ALGORITHM
INITIALIZE: A = {2,…,m} = ACTIVE sample paths
1. Observe every event e in sample path of 1
2. Update state: x n 
f
x n
3. For each n = 2,…,m, check if
3.1. If 
 x 1  

,e

[is OBSERVABILITY satisfied?]
 x n  , add n to A (if nA, leave n in A)
3.2. Else, remove n from A and leave x n unchanged
3. Return to Step 1 for next event in sample path of 1
 Price to pay in ASA: long suspensions possible in Step 3.1
Christos G. Cassandras
CODES Lab. - Boston University
SOME
APPLICATIONS
BUFFER ALLOCATION IN A SWITCH
NETWORK SWITCH MEMORY ALLOCATION OVER OUTPUT PORTS
TOTAL
MEMORY B = b1 + b2 + b3
b1
b2
b3
Christos G. Cassandras
CODES Lab. - Boston University
BUFFER ALLOCATION IN A SWITCH
p1

p2
r1

1
r2

2
pN
rN
N
N
For N=6, K=24  Possible allocations =
Christos G. Cassandras
Allocate K buffer slots
(RESOURCES)
over N servers
(USERS)
so as to minimize
BLOCKING
PROBABILITY
N
subject to
 ri  K
i 1
118,755
CODES Lab. - Boston University
CONCURRENT ESTIMATION + OPTIMIZATION
1.4
Initial Allocation
1.3
[19,1,1,1,1,1]
1.2
[8,8,3,3,1,1]
Cost
1.1
Optimal
1
Convergence in ~ 20 iterations
vs.
118,755
0.9
0.8
20 iterations  630,000 events
0.7
0.6
0.5
1
11
21
31
41
51
61
71
81
91
101
111
121
131
141
151
Iteration


Christos G. Cassandras
 = 5.0,
i = 1.0, pi=1/6 for all i = 1,…,6
Optimal = [4,4,4,4,4,4]
CODES Lab. - Boston University
LOT SIZING IN MANUFACTURING
BATCHING QUEUE
Class 1
LOT QUEUE
SERVER
...
BATCHING QUEUE
...
LOT SIZE = r1
Class N
SETUP
PROCESS
...
...
LOT SIZE = rN
LOT SIZING: Determine Lot Sizes to minimize Average Lead Time
COMPLEXITY: For 3 product classes and
lot sizes in [1,100] :
Christos G. Cassandras
106
possible
allocations
CODES Lab. - Boston University
LOT SIZING IN MANUFACTURING
7
J
MEAN
SYSTEM
TIME
LOT
LOTSIZE
SIZErri itoo
toolarge
large

 All
Allother
otherclasses
classesdelayed
delayed
++class
classiiparts
partsdelayed
delayedwaiting
waiting
for
forlot
lotatatoutput
output
5
3
1
6
11
16
21
26
31
36
41
46
n SIZE
LOT
4.3
4
3.4
Christos G. Cassandras
n2
24
22
12
20
n1
18
15
16
14
3.1
12
LOT
LOTSIZE
SIZErri itoo
toosmall
small

 Too
Toomany
manysetups,
setups,
instability
instabilitypossible
possible
3.7
9
CODES Lab. - Boston University
4.00E+004.30E+00
3.70E+004.00E+00
3.40E+003.70E+00
3.10E+003.40E+00
CONCURRENT SIMULATION + OPTIMIZATION
convergence under different initial points
Mean System Time
7
6
Convergence in ~ 12
iterations
vs.
1,000,000
5
4
3
1
6
11
16
21
26
31
Iteration
(30.1,30.1,30.1)
Christos G. Cassandras
(30.1,20.1,10.1)
(11.1,10.1,30.1)
(30.1,15.1,15.1)
CODES Lab. - Boston University
ABSTRACTION
(AGGREGATION)
OF DES
Christos G. Cassandras
CODES Lab. - Boston University
THREE FUNDAMENTAL COMPLEXITY LIMITS
1/T1/2
LIMIT
NP-HARD
LIMIT
Tradeoff
between ACCURACY
one order increase
in estimation
INFO.
GENERALITY
and
EFFICIENCY
requires DECISION SPACE
STRATEGY
of an algorithm
two ordersSPACE
increase
in=LEARNING
SPACE EFFORT
and Macready,
IEEET)
TEC, 1997]
(e.g., [Wolpert
SIMULATION
LENGTH
NO-FREE-LUNCH
LIMIT
Christos G. Cassandras
CODES Lab. - Boston University
THREE FUNDAMENTAL COMPLEXITY LIMITS
1/T1/2
LIMIT
NP-HARD
LIMIT
Effect is
MULTIPLICATIVE!
NO-FREE-LUNCH
LIMIT
Christos G. Cassandras
CODES Lab. - Boston University
MORE COMPLEX
LESS COMPLEX
TIME-DRIVEN
SYSTEM
ZOOM OUT
HYBRID
SYSTEM
LESS COMPLEX
Christos G. Cassandras
EVENT-DRIVEN
SYSTEM
ABSTRACTION
(AGGREGATION)
CODES Lab. - Boston University
WHAT IS THE RIGHT ABSTRACTION LEVEL ?
TOO FAR…
model not
detailed enough
TOO CLOSE…
too much
undesirable
detail
JUST RIGHT…
good model
Christos G. Cassandras
CREDIT: W.B. Gong
CODES Lab. - Boston University
ABSTRACTION OF A DISCRETE-EVENT SYSTEM
DISCRETE-EVENT
SYSTEM
Christos G. Cassandras
CODES Lab. - Boston University
ABSTRACTION OF A DISCRETE-EVENT SYSTEM
DISCRETE-EVENT
SYSTEM
TIME-DRIVEN
FLOW RATE DYNAMICS
EVENTS
HYBRID
SYSTEM
Christos G. Cassandras
CODES Lab. - Boston University
THRESHOLD BASED BUFFER CONTROL
“REAL” SYSTEM
K
L(K): Loss Rate
ARRIVAL
PROCESS
LOSS
Q(K): Mean
Queue Lenth
x(t)
PROBLEM: Determine  to minimize [Q(K) + R·L(K)]
SFM

(t)
(t)
RANDOM
PROCESS
(t)
x(t)
RANDOM
PROCESS
SURROGATE PROBLEM: Determine  to minimize [QSFM() + R·LSFM()]
Christos G. Cassandras
CODES Lab. - Boston University
THRESHOLD BASED BUFFER CONTROL
CONTINUED
25
“Real” System
23
J(K)
21
SFM
19
Optim. Algorithm
using SFM-based
gradient estimates
17
DES
SFM
Opt. A lgo
15
0
5
10
15
20
25
30
35
40
45
K
Christos G. Cassandras
CODES Lab. - Boston University
REAL-TIME STOCHASTIC OPTIMIZATION
Model
Sample path
x(t)
CONTROL/DECISION
(Parameterized by )
 n 1   n   nL( n )
IPA
On-line optimization
L( )
DES
MODEL
L()
(ABSTRACTION)
- Unbiased estimators
- General distributions
- Simple on-line implementation
Christos G. Cassandras
PERFORMANCE
E[ L( )]
Not always true
in DES !
CISE - CODES Lab. - Boston University
REAL-TIME STOCHASTIC OPTIMIZATION:
HYBRID SYSTEMS
Sample path
CONTROL/DECISION
(Parameterized by )
HYBRID
SYSTEM
PERFORMANCE
E[ L( )]
NOISE
 n 1   n   nL( n )
L( )
IPA
x(t)
L()
A general framework for an IPA theory in Hybrid Systems?
Christos G. Cassandras
CISE - CODES Lab. - Boston University
PERFORMANCE OPTIMIZATION AND IPA
Performance metric (objective function):
J  ; x( ,0), T   E L ; x( ,0), T 
N  k 1
L   
k 0
 L ( x, , t )dt
k
k
IPA goal:
dJ  ; x( ,0), T 
dL 
- Obtain unbiased estimates of
, normally
d
d
dL( n )
- Then:  n 1   n   n
d
NOTATION:
Christos G. Cassandras
x t  
  
x , t 
,  k  k


CISE - CODES Lab. - Boston University
HYBRIDHYBRID
AUTOMATA
RECALL:
AUTOMATA
Gh  (Q, X , E , U , f ,  , Inv, guard ,  , q0 , x 0 )
Q:
set of discrete states (modes)
X:
set of continuous states (normally Rn)
E:
set of events
U:
f:
set of admissible controls
vector field, f : Q  X  U  X
:
discrete state transition function,  : Q  X  E  Q
Inv:
set defining an invariant condition (domain), Inv  Q  X
guard: set defining a guard condition, guard  Q  Q  X
:
reset function,  : Q  Q  X  E  X
q0:
initial discrete state
x0 :
initial continuous state
Christos G. Cassandras
CODES Lab. - Boston University
THE IPA CALCULUS
IPA: THREE FUNDAMENTAL EQUATIONS
System dynamics over (k(), k+1()]: x  f k ( x,  , t )
NOTATION:
x t  
  
x , t 
,  k  k


1. Continuity at events: x( k )  x( k )
Take d/d 
x' ( k )  x' ( k )  [ f k 1 ( k )  f k ( k )] 'k
d (q, q, x, ,  )
If no continuity, use reset condition  x' ( ) 
d

k
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA: THREE FUNDAMENTAL EQUATIONS
2. Take d/d of system dynamics x  f k ( x,  , t ) over (k(), k+1()]:
f (t )
dx' (t ) f k (t )

x' (t )  k
dt
x

Solve
f (t )
dx' (t ) f k (t )

x' (t )  k
over (k(), k+1()]:
dt
x

t
x(t )  e
 k
f k ( u )
du
x
 t f (v)  v f k (u ) du


k
x
k
e
dv  x( k )

 k 

initial condition from 1 above
NOTE: If there are no events (pure time-driven system),
IPA reduces to this equation
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA: THREE FUNDAMENTAL EQUATIONS
3. Get  k depending on the event type:
- Exogenous event: By definition,  k  0
- Endogenous event: occurs when g k ( x( , k ), )  0
1
g
 g

   g

 k    f k ( k )   x( k ) 
 x
   x

- Induced events:
1
 y ( ) 
 k    k k  yk ( k )
 t 
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA: THREE FUNDAMENTAL EQUATIONS
Ignoring resets and induced events:
Recall:
1. x' ( )  x' ( )  [ f k 1 ( )  f k ( )]  'k

k

k
2. x(t )  e

k
f k ( u )
du
k
x

t

k
x t  
 t f (v)  v f k (u ) du


k
x
k
e
dv  x' ( k )

 k 

 k 
x , t 

 k  

1
3.  k  0
x' ( )

k
or
1
3
Christos G. Cassandras
g
g


   g
 k    f k ( k )   x( k ) 
 x
   x

2
Cassandras
Cassandrasetetal,al,Europ.
Europ.J.J.Control,
Control,2010
2010
CISE - CODES Lab. - Boston University
IPA PROPERTIES
N  k 1
Back to performance metric:
L   
k 0
NOTATION: Lk x, , t  
 L ( x, , t )dt
k
k
Lk  x,  , t 

 k 1

dL  N 
Then:
   k 1  Lk ( k 1 )   k  Lk ( k )   Lk ( x, , t )dt 
d
k 0 

k

What happens
at event times
Christos G. Cassandras
What happens
between event times
CISE - CODES Lab. - Boston University
IPA PROPERTIES
THEOREM 1: If either 1,2 holds, then dL()/d depends only on
information available at event times k:
1. L(x,,t) is independent of t over [k(), k+1()] for all k
2. L(x,,t) is only a function of x and for all t over [k(), k+1()]:
d Lk d f k d f k


0
dt x dt x dt 
[Yao and Cassandras, 2010]
 k 1

dL  N 
   k 1  Lk ( k 1 )   k  Lk ( k )   Lk ( x,  , t )dt 
d
k 0 

k

IMPLICATION: - Performance sensitivities can be obtained from information
limited to event times, which is easily observed
- No need to track system in between events !
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA PROPERTIES
EXAMPLE WHERE THEOREM 1 APPLIES (simple tracking problem):
T

L
min E   [ x(t )  g ( )]dt 

1
 ,
x

0
f k
s.t. xk  ak xk (t )  uk ( k )  wk (t )
k  1, , N

xk
 ak ,
f k duk

 k d k
NOTE: THEOREM 1 provides sufficient conditions only.
IPA still depends on info. limited to event times if
xk  ak xk (t )  uk ( k , t )  wk (t )
k  1, , N
for “nice” functions uk(k,t), e.g., bkt
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA PROPERTIES
EVENTS
x  f ( x, u , w, t ; )
k
k+1
Evaluating x(t ; ) requires full knowledge of w and f values (obvious)
dx(t ; )
However,
may be independent of w and f values (NOT obvious)
d
It often depends only on: - event times k
- possibly f ( k1 )
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA PROPERTIES
In many cases:
- No need for a detailed model (captured by fk) to describe state behavior
in between events
- This explains why simple abstractions of a complex stochastic system
can be adequate to perform sensitivity analysis and optimization,
as long as event times are accurately observed and local system behavior
at these event times can also be measured.
- This is true in abstractions of DES as HS since:
Common performance metrics (e.g., workload) satisfy THEOREM 1
Christos G. Cassandras
CISE - CODES Lab. - Boston University
THRESHOLD-BASED ADMISSION CONTROL

(t)
(t)
(t)
0
x(t )  0,  (t )   (t )

dx(t ) 

0
x(t )   ,  (t )   (t )
dt
 (t )   (t )
otherwise

x(t)
J T ( )  QT ( )  RLT ( )
LOSS
WORKLOAD
 k 1
QT     x(t ,  )dt   
T
0
k
k
x(t , )dt
  k : xi (t )  0 for all t   k , k 1 
 k 1
LT     
k
k
 (t )   (t )dt
  k : xi (t )   i for all t   k , k 1 
Assume: {(t)}, {(t)} piecewise continuously differentiable, independent of 
Christos G. Cassandras
CISE - CODES Lab. - Boston University
IPA FOR LOSS WITH RESPECT TO 
 k 1
LT     
k
k
LT  
   ( k- 1 )   ( k- 1 )  k 1

k

   (
 (t )   (t )dt
k


k

)   ( k )  k
Need to obtain EVENT TIME derivatives
Apply:
1. x ' ( k )  x ' ( k )  [ f k 1 ( k )  f k ( k )]   ' k
t
2. x (t )  e
 k
f k ( u )
du
x

 t  f ( v )  v f k ( u ) du

k
x
k
e
dv  x ( k ) 




 k
1
3.  k  0
or
 g
  g g

 k   f k ( k )   x( k ) 
 x
   x

IPA WITH RESPECT TO 

 k 1
 k 1
k
Exogenous event:
Endogenous event:
 k 1  0
1
 g
  g g

 k    f k ( k )   x( k ) 
 x
   x

Here:
1  x( k )
g k  x( k )     k 
 ( k )   ( k )
LT  
    ( k )   ( k )  k

k


k

x ( k )  e  k 1
0 du
v
 k

  0 du
 k 1
dv  0   0
  0e
 k 1

LT  
 

Just count
Overflow
intervals
RESOURCE CONTENTION
GAMES
MULTIPLE USERS COMPETE FOR RESOURCE
PROBLEM: Determine (1,...,N) to optimize system performance
N
N user classes
Loss when class i
content > i
Capacity   i
i 1
FCFS
server
treats each class
differently
SYSTEM-CENTRIC OPTIMIZATION: vs USER-CENTRIC OPTIMIZATION:
Each user optimizes J i (1 ,, N )
, NN)
System optimizes JJ((11,,...,
by controlling only i
by controlling (1,...,N)
 Resource Contention Game
Christos G. Cassandras
CISE - CODES Lab. - Boston University
SYSTEM-CENTRIC v GAME SOLUTIONS
Generally, GAME solution is worse than SYSTEM-CENTRIC solution
→ “the price of anarchy’’
THE OPTIMAL LOT-SIZING PROBLEM
SYSTEM-CENTRIC OPTIMIZATION: Determine N lot size parameters
to minimize overall MEAN DELAY
USER-CENTRIC OPTIMIZATION: Determine ith lot size parameter
to minimize ith user MEAN DELAY
K. Baker, P.S. Dixon, M.J. Magazine, and E.A. Silver. Management Science, 1978.
P. Afentakis, B. Gavish, and U. Karmarkar. Management Science, 1984.
U.S. Karmarkar. Management Science, 1987.
J. Maes and L.V. Wassenhove. Journal of Operations Research, 1988.
G. Belvaux and L.A. Wolsey. Management Science, 2000.
N. Absi and S. Kedad-Sidhoum. Operations Research, 2007.
THE OPTIMAL LOT-SIZING PROBLEM
ri (t )  i (t ) a(t )  i and z (t )  sa (t )
dxi (t ) 

and xi (t )  y(t )  i (t )
dt
 r (t )
otherwise
i

y(t)
 a (t ) (t ) y(t )   a (t ) and z (t )  sa (t )
dy(t ) 

and xa (t ) (t )  y(t )   a (t )
dt
 r (t )
otherwise
 i
y(t )  0 if y(t  )   a (t  )
dz(t )
 1 if y(t )   a (t )
dt
z (t )  0 if y(t  )   a (t  )
[Yao and Cassandras, IEEE CDC, 2010;
IEEE TASE, 2012]
SYSTEM-CENTRIC v GAME SOLUTIONS
SYSTEM-CENTRIC solution…
…coincides with
GAME (USER-CENTRIC) solution!
→ ZERO “price of anarchy’’
Proof obtained for
DETERMINISTIC
version
CYBER-PHYSICAL SYSTEMS: THE NEXT FRONTIER
INTERNET
Data collection:
relatively easy…
CYBER
PHYSICAL
Control:
a challenge…
Christos G. Cassandras
CISE - CODES Lab. - Boston University