FROM TO SYSTEMS C. G. Cassandras Boston University cgc@bu.edu, Christos G. Cassandras http://vita.bu.edu/cgc CODES Lab. - Boston University OUTLINE ¾ HYBRID SYSTEMS IN COMPLEX SYSTEM… … DECOMPOSITION AND ABSTRACTION ¾ WHAT’S A HYBRID SYSTEM… ¾ DECOMPOSITION: HYBRID SYSTEM → DES - SOLVING OPTIMAL CONTROL PROBLEMS ¾ ABSTRACTION: DES → HYBRID SYSTEM - ANALYSIS OF STOCHASTIC FLOW MODELS (SFM) ¾ THE FUTURE… Christos G. Cassandras CODES Lab. - Boston University DISCRETE CONTINUOUS 1980 TIME-DRIVEN SYSTEMS EVENT-DRIVEN SYSTEMS 1990 HYBRID SYSTEMS Christos G. Cassandras CODES Lab. - Boston University 2000 LESS COMPLEX MORECOMPLEX TIME-DRIVEN SYSTEM HYBRID SYSTEM EVENT-DRIVEN SYSTEM Christos G. Cassandras CODES Lab. - Boston University LESS COMPLEX MORECOMPLEX TIME-DRIVEN SYSTEM What exactly does that mean? HYBRID SYSTEM EVENT-DRIVEN SYSTEM LESS COMPLEX Christos G. Cassandras DECOMPOSITION CODES Lab. - Boston University MORECOMPLEX LESS COMPLEX TIME-DRIVEN zz SYSTEM ZOO M HYBRID SYSTEM LESS COMPLEX Christos G. Cassandras OUT EVENT-DRIVEN SYSTEM ABSTRACTION (AGGREGATION) CODES Lab. - Boston University MORECOMPLEX LESS COMPLEX TIME-DRIVEN SYSTEM HYBRID SYSTEM ABSTRACTION (AGGREGATION) Christos G. Cassandras HYBRID SYSTEM EVENT-DRIVEN SYSTEM DECOMPOSITION CODES Lab. - Boston University WHAT IS THE RIGHT ABSTRACTION LEVEL ? TOO FAR… model not detailed enough TOO CLOSE… too much undesirable detail JUST RIGHT… good model Christos G. Cassandras CREDIT: W.B. Gong CODES Lab. - Boston University WHAT’S A HYBRID SYSTEM ? Christos G. Cassandras CODES Lab. - Boston University WHAT’S A HYBRID SYSTEM? z&2 = g 2 ( z2 , u2 , t ) z&1 = g1 ( z1, u1, t ) TIME x1 x0 x2 x1 x1 = f1 ( x0 , z1, u1, t ) x2 = f 2 ( x1, z2 , u2 , t ) NE Christos G. Cassandras W DE O “M ” CODES Lab. - Boston University WHAT’S A HYBRID SYSTEM? CONTINUED TIME DRIVEN : z&i = gi ( zi , ui , t ) TIME TIME TIMETIME x1 x0 x1 x2 x3 TIME x4 xi+1 x2 x3 xi xi Christos G. Cassandras xi+1 EVENT - DRIVEN : xi +1 = f i ( xi , zi , ui , t ) CODES Lab. - Boston University WHAT’S A HYBRID SYSTEM? CONTINUED Physical State, z z&i = gi ( zi , ui , t ) hybrid … x1 Switching Times Christos G. Cassandras x2 xi xi+1 = fi(xi,ui,t) Temporal State, x SWITCHING SWITCHINGTIMES TIMES HAVE THEIR HAVE THEIROWN OWN DYNAMICS! DYNAMICS! CODES Lab. - Boston University HYBRID SYSTEM EXAMPLES 1. Autonomous Switching, e.g., Hysteresis x<Δ x > −Δ x≥Δ x& = x + u x& = − x + u x ≤ −Δ Christos G. Cassandras CODES Lab. - Boston University HYBRID SYSTEM EXAMPLES CONTINUED 2. External Switching, e.g., Zeno’s bouncing ball x& = v x , v&x = 0 y& = v y , v& y = −mg Switching Events Christos G. Cassandras CODES Lab. - Boston University HYBRID SYSTEM EXAMPLES CONTINUED 3. Controlled Switching, e.g., Interconnected tanks u1(t) HIGH LOW u2(t) Christos G. Cassandras u3(t) HIGH HIGH LOW LOW CODES Lab. - Boston University HYBRID SYSTEM EXAMPLES - MANUFACTURING Key questions facing manufacturing system integrators: • How to integrate ‘process control’ with ‘operations control’ ? • How to improve product QUALITY within reasonable TIME ? PROCESS CONTROL • Physicists • Material Scientists • Chemical Engineers • ... Christos G. Cassandras OPERATIONS CONTROL • Industrial Engineers, OR • Schedulers • Inventory Control • ... CODES Lab. - Boston University HYBRID SYSTEM EXAMPLES - MANUFACTURING Throughout a manuf. process, each part is characterized by • A PHYSICAL state (e.g., size, temperature, strain) • A TEMPORAL state (e.g., total time in system, total time to due-date) PHYSICAL STATE Time-driven Dynamics NEW PHYSICAL STATE OPERATION OPERATION TEMPORAL STATE Christos G. Cassandras Event-driven Dynamics NEW TEMPORAL STATE CODES Lab. - Boston University COOPERATIVE CONTROL TARGET EVENT: threat sensed THREATS TIME-DRIVEN DYNAMICS BASE Christos G. Cassandras EVENT: info. communicated by team member CODES Lab. - Boston University GENERAL MODELING FRAMEWORKS • Hybrid Automata Σq : z&q =γq(zq,uq,t) Gq Fq [Branicky [Branickyetetal., al.,1998] 1998] Aq AUTONOMOUS JUMP SET System must jump with Gq : Aq ×Vq →S • Mixed Logical Dynamical Systems CONTROLLED JUMP SET System may jump with Fq : Cq →2S inputs symbols δi automaton automaton/ /logic logic [Bemporad [Bemporadand andMorari, Morari,1999] 1999] [Proc. [Proc.ofofIEEE IEEESpecial SpecialIssue, Issue,2000] 2000] Christos G. Cassandras continuous states symbols δ D/A A/D • etc. Cp Ap Cq continuous continuous dynamical dynamicalsystem system inputs CODES Lab. - Boston University TIME-DRIVEN AND EVENT-DRIVEN DYNAMICS Time-Driven Dynamics (STATE = z): z&(t ) = g (z , u, t ) or: zk +1 = g k (zk , uk ) Event-Driven Dynamics (STATE = Event Times xk,i): xk +1,i = max{xk , j + a k , j u k , j } j∈Γi Event counter k = 1,2,... Event index i∈E = {1,…,n} Christos G. Cassandras CODES Lab. - Boston University DECOMPOSITION Christos G. Cassandras CODES Lab. - Boston University HIEARARCHICAL DECOMPOSITION ??? PLANNING DISCRETE-EVENT PROCESSES PHYSICAL PROCESSES Christos G. Cassandras CODES Lab. - Boston University HIEARARCHICAL DECOMPOSITION CONTINUED MODEL MODEL TIME TIMESCALE SCALE ??? Weeks - Months Minutes - Weeks msec - Hours Christos G. Cassandras Diff. Eq’s, Flows, LP PLANNING DISCRETE-EVENT PROCESSES PHYSICAL PROCESSES Automata, Petri nets, Queueing, Simulation Diff. Eq’s, Detailed Simulation CODES Lab. - Boston University HIEARARCHICAL DECOMPOSITION CONTINUED ??? ??? PLANNING PLANNING DISCRETE-EVENT PROCESSES PHYSICAL PHYSICAL PROCESSES PROCESSES Christos G. Cassandras CODES Lab. - Boston University HYBRID CONTROL SYSTEM What exactly does that mean? DISCRETE-EVENT PROCESSES CONTROL CONTROL PHYSICAL PROCESSES Christos G. Cassandras CODES Lab. - Boston University OPTIMAL CONTROL Christos G. Cassandras CODES Lab. - Boston University WHAT’S A HYBRID SYSTEM? Physical State, z z&i = gi ( zi , ui , t ) hybrid … x1 Switching Times Christos G. Cassandras x2 xi Temporal State, x xi+1 = fi(xi,ui,t) CODES Lab. - Boston University OPTIMAL CONTROL PROBLEMS • Get to a desired final physical state zN in minimum time xN, subject to N-1 switching events • Minimize deviations from N desired physical states: (zi - qi)2 and deviations from target desired times: (xi - τi)2 In general: N min ∑ ∫ Li ( zi (t ), ui (t ))dt u i =1 xi xi −1 Temporal state Christos G. Cassandras z&i = gi ( zi , ui , t ) s.t. xi+1 = fi(xi,ui,t) Physical state CODES Lab. - Boston University OPTIMAL CONTROL PROBLEMS Problems we consider: N min ∑ [φi ( xi , xi −1 ) + ψ i ( xi )] u i =1 Cost under ui(t) over [xi-1,xi] where: CONTINUED Cost of switching time xi xi φi ( xi , xi −1 ) = ∫ Li ( zi (t ), ui (t ))dt xi −1 Let: si = xi - xi-1 Assuming stationarity: Christos G. Cassandras Time spent at ith operating mode φi ( xi , xi −1 ) = φi ( si ) CODES Lab. - Boston University HIERARCHICAL DECOMPOSITION z&i = gi ( zi , ui , t ) N min ∑ [φi ( si ) + ψ i ( xi )] u xi+1 = fi(xi,ui,t) i =1 HIGHER LEVEL PROBLEM: LOWER LEVEL PROBLEMS: s.t. N [ min ∑ φi ( si ) + ψ i ( xi ) s i =1 * si ] min φi ( si ) = ∫ Li ( zi (t ), ui (t ))dt 0 ui s.t. xi+1 = fi(xi,si,t) s.t. z&i = gi ( zi , ui , t ) FIXED si Christos G. Cassandras CODES Lab. - Boston University HIERARCHICAL DECOMPOSITION zi0 CONTINUED zif zif+1 zi0+1 zi0+ 2 si si+1 xi-1 xi+1 xi min φi ( si ) f 0 min f 0 u +1i ( zi +1 , zi +1 , si +1 ) u i ( z i , z i , si ) u ∗i z 0i , z fi , s i i z 0i , z fi , s i x ≡ min u i i z i , u i , s i x 1 THE THEREALLY REALLY CHALLENGING CHALLENGING PROBLEM! PROBLEM! Christos G. Cassandras φi +1 ( si +1 ) … 2 [ ] s.t. min θ i ( z , zi , si ) + ψ i ( xi ) ∑ 0 f xi+1 = fi(xi,ui,t) z ,z , s i =1 N 0 i f CODES Lab. - Boston University HYBRID CONTROLLER STRUCTURE f, ψ Hybrid controller steps: Event timing xi+1 = fi(xi,si,t), i = 1,…,N • System identification s* • Lower-level solution Higher-level Controller • Higher-level solution • Lower-level solution • Operation... s* φ*(s) Lower-level Controller u*(s*) &z = g ( z, u, t ) g, φ Physical processes Christos G. Cassandras CODES Lab. - Boston University TWO TYPES OF PROBLEMS… z&1 = g1 ( z1 , u1 , t ) z1f z&2 = g 2 ( z 2 , u2 , t ) zdf z0 x0 s1 x1 s2 x2 1. A single event process controls switching dynamics: xi+1 = xi + si(zi,ui) z&1 = g1 ( z1 , u1 , t ) z1f 0 2 z0 z x0 x1 s1 2. Multiple event processes control switching dynamics: Christos G. Cassandras z&2 = g 2 ( z 2 , u2 , t ) zdf EXTE EVEN RNAL T PR x2 O CE si+1 SS xi +1 = max( xi , ai +1 ) + si ( zi , ui ) CODES Lab. - Boston University EXAMPLE Physical State, z led p u co s… e D de mo z&i = gi ( zi , ui , t ) … IDLING a1 a2 x1 x2 a3 x3 xi ai+1 Temporal State, x x i = max {x i − 1 , a i } + s i ( z i , u i ) External event process: ith mode cannot start before ai+1 Christos G. Cassandras CODES Lab. - Boston University MANUFACTURING SYSTEMS Throughout a manuf. process, each part is characterized by • A PHYSICAL state (e.g., size, temperature, strain) • A TEMPORAL state (e.g., total time in system, total time to due-date) PHYSICAL STATE NEW PHYSICAL STATE Time-driven Dynamics z&i = gi ( zi , ui , t ) OPERATION OPERATION TEMPORAL STATE Christos G. Cassandras xk +1,i = max{xk , j + a k , j u k , j } j∈Γi Event-driven Dynamics NEW TEMPORAL STATE CODES Lab. - Boston University MANUFACTURING SYSTEMS CONTINUED x i = max {x i − 1 , a i }+ s i ( u i ) EVENT-DRIVEN COMPONENT Part Arrivals Part Departures ai, zi(ai) TIME-DRIVEN COMPONENT Christos G. Cassandras xi, zi(xi) ui z&i (t ) = g (zi , ui , t ) CODES Lab. - Boston University EXAMPLE …a to nd m t u (e.g his s st be ., d tate pro e si ces red sed tem per atu re) STATE STATE rts a t s ar t p y r Eve is state at th zi(ui) si(ui) Christos G. Cassandras PROCESS PROCESS TIME TIME CODES Lab. - Boston University LOWER LEVEL PROBLEM Parameterized by switching times LQ PROBLEM: min φi ( zi , ui , si ) = h ( z fi − zdi ) + ∫ 1 2 ui s.t. z&i = azi + bui , zi (0) = ς i si 1 0 2 2 2 i ru (t )dt Penalize final state deviation STANDARD LQ SOLUTION METHOD: φi ( si ) = h( z − zdi ) + ∫ * Christos G. Cassandras 1 2 * fi 2 si 1 0 2 * i ru (t )dt CODES Lab. - Boston University HIGHER LEVEL PROBLEM N [ min ∑ φi ( si ) + ψ i ( xi ) s i =1 * ] s.t. x i = max {x i − 1 , a i }+ s i ( u i ) Given arrival sequence (INPUT) Cost of optimal process control over interval [0, si] Processing time (CONTOLLABLE) Cost related to event timing EXAMPLE : Christos G. Cassandras ψ i ( xi ) = ( xi − τ i ) CODES Lab. - Boston University 2 HOW DO WE SOLVE THE HIGHER LEVEL PROBLEM? N [ min ∑ φi ( si ) + ψ i ( xi ) s i =1 * Even if these are convex, problem is still NOT convex in s! ] s.t. x i = max {x i − 1 , a i }+ s i ( u i ) Causes nondifferentiabilities! Even though problem is NONDIFFERENTIABLE and NONCONVEX, optimal solution shown to be unique. [Cassandras, [Cassandras,Pepyne, Pepyne,Wardi, Wardi,IEEE IEEETAC TAC2001] 2001] Christos G. Cassandras CODES Lab. - Boston University SOLVING THE HIGHER LEVEL PROBLEM a1 a2 a3 x1 x2 x3 a4 a5 a6 x4 x5 x6 a7 a8 x7 a9 CONTINUED IME T N I T JUS x8 Each “block” corresponds to a Constrained Convex Optimization problem ⇒ search over 2N-1 possible Constrained Convex Optimization problems BUT algorithms that only need N Constrained Convex Optimization problems have been developed ⇒ SCALEABILITY [Cho, [Cho,Cassandras, Cassandras,Intl. Intl.J.J.Rob. Rob.and andNonlin.Control, Nonlin.Control,2001] 2001] Christos G. Cassandras CODES Lab. - Boston University INTERACTIVE JAVA APPLETS • http://vita.bu.edu/cgc/hybrid - Single-stage model - Backward-recursive TPBVP solver with critical job identification • http://vita.bu.edu/cgc/newhybrid - 3-stage model - Bezier approximation with standard TPBVP solver Christos G. Cassandras CODES Lab. - Boston University http://vita.bu.edu/cgc/hybrid Christos G. Cassandras CODES Lab. - Boston University ABSTRACTION Christos G. Cassandras CODES Lab. - Boston University COMPLEX DES ABSTRACTION DISCRETE-EVENT SYSTEM TIME-DRIVEN FLOW RATE DYNAMICS EVENTS HYBRID SYSTEM Christos G. Cassandras CODES Lab. - Boston University STOCHASTIC FLOW MODELS (SFM) θ CONTROLLER α(t) α(t) – p(x) β(t) γ(t) x(t) (t),ββ(t): (t):arbitrary arbitrarystochastic stochasticprocesses processes αα(t), (piecewisecontinuously continuouslydifferentiable) differentiable) (piecewise 0 x(t ) = 0, λ (t ) − p (0) ≤ 0 ⎧ dx ⎪ =⎨ 0 x(t ) = θ , λ (t ) − p (θ ) ≥ 0 dt ⎪ otherwise ⎩λ (t ) − p ( x(t )) λλ(t)(t)==αα(t)(t)--ββ(t)(t) Christos G. Cassandras feedback feedback CODES Lab. - Boston University WHY SFM? ¾ “Lower resolution” model of “real” system intended to capture just enough info. on system dynamics ¾ Aggregates many events into simple continuous dynamics, preserves only events that cause drastic change ⇒ computationally efficient (e.g., orders of magnitude faster simulation) ¾ Intended for developing CONTROL schemes rather than for PERFORMANCE ANALYSIS Christos G. Cassandras CODES Lab. - Boston University MOTIVATING EXAMPLE: THRESHOLD-BASED BUFFER CONTROL K ARRIVAL PROCESS “REAL” SYSTEM LOSS x(t) ̄ T K R L̄ T K J T K Q θ β(t) α(t) RANDOM PROCESS γ(t) x(t) RANDOM PROCESS Stochastic Flow Model (SFM) ̄ SFM J SFM Q R L̄ SFM T T T Christos G. Cassandras CODES Lab. - Boston University MOTIVATING EXAMPLE CONTINUED 25 “Real” System 23 J(K) 21 SFM 19 Optim. Algorithm using SFM-based gradient estimates 17 DES SFM Opt. A lgo 15 0 5 10 15 20 25 30 35 40 45 K Cassandrasetetal, al,2002 2002 Cassandras Christos G. Cassandras CODES Lab. - Boston University OPTIMIZATION OF SFMS ¾ Stochastic Optimal Control problems too hard!.. ¾ Parametric optimization: use gradient estimates with on-line opt. algorithms SFM n1 n −nHnn,n , n 0,1,… ¾ Need efficient ways to estimate performance sensitivities Christos G. Cassandras CODES Lab. - Boston University INFINITESIMAL PERTURBATION ANALYSIS (IPA) “Brute Force” Sensitivity Estimation: θ Jˆ (θ DES θ+Δθ ) J$ (θ + Δ θ ) DES ] J$ ( θ + Δ θ ) − J$ ( θ ) ⎡ dJ ⎤ ⇒⎢ ⎥ = Δθ ⎣ d θ ⎦ est Finite Perturbation Analysis (FPA): θ Δθ J$ (θ ) DES FPA FPA J$ (θ + Δ θ ) ] J$ ( θ + Δ θ ) − J$ ( θ ) ⎡ dJ ⎤ ⇒ ⎢ = ⎥ Δθ ⎣ d θ ⎦ est Infinitesimal or Smoothed Perturbation Analysis (IPA, SPA): θ J$ (θ ) DES IPA/SPA IPA/SPA Christos G. Cassandras ⎡ d J ⎤ ⎢⎣ d θ ⎥⎦ est CODES Lab. - Boston University INFINITESIMAL PERTURBATION ANALYSIS (IPA) OBJECTIVES: • Obtain sample performance derivatives that depend ONLY on observed sample path data: Performance ′ L T ≡ dL T d Parameter • Prove unbiasedness: dE L T d E dL T d • Then, use gradient estimates to drive on-line opt. algorithms: θ n +1 = θ n + η L (θ n ), n = 0,1,K ' n T Christos G. Cassandras CODES Lab. - Boston University THRESHOLD-BASED BUFFER CONTROL θ β(t) α(t) γ(t) x(t) J T (θ ) = QT (θ ) + RLT (θ ) WORK T Q T 0 x;tdt Christos G. Cassandras LOSS T L T 0 ; tdt CODES Lab. - Boston University THRESHOLD-BASED BUFFER CONTROL CONTINUED θ ξ1 η1 ξ2 u1,2 v1,2 u2,2 v2,2 BUFFERING PERIOD: B k k , k , η2 ξ3 u1,3 v1,3 η 3 k 1, …, K OVERFLOW PERIOD: F i,k u i,k , v i,k , i 1, …, M SET OF BPs WITH AT LEAST ONE OVERFLOW: : k ∈ 1, …, K : xt , t − t 0 for some t ∈ k , k B || Christos G. Cassandras CODES Lab. - Boston University IPA – MAIN RESULTS Cassandrasetetal, al,2002 2002 Cassandras THEOREM 1 (Loss IPA derivative): ′ L T − B • Simple count of Buffering Periods with at least one overflow • Nonparametric (independent of θ and any model assumption) THEOREM 2 (Work IPA derivative): Q T ∑ k∈ k − u k,1 ′ • Simple timer for each Buffering Period with at least one overflow • Nonparametric (independent of θ and any model assumption) Christos G. Cassandras CODES Lab. - Boston University IPA – MAIN RESULTS CONTINUED Let N(T) be the number of events in [0, T]. THEOREM: dEL T 1. If E[N(T)] < ∞, then L T is an unbiased estimator of d ′ dEQ T Q 2. T is an unbiased estimator of d ′ Christos G. Cassandras CODES Lab. - Boston University THE FUTURE… ¾ COMPLEXITY - GETTING AROUND IT THROUGH STRUCTURAL PROPERTIES - FINDING THE “RIGHT” MODELING RESOLUTION ¾ COMPUTATIONAL CHALLENGES - REAL-TIME IMPLEMENTATIONS, EMBEDDED SYSTEMS - SOFTWARE TOOLS (for Verification, Optimization, etc.) ¾ APPLICATION DOMAINS COOPERATIVE CONTROL, AUTOMOTIVE, COMPUTER NETWORKS, LOGISTICS, BIOMEDICAL Christos G. Cassandras CODES Lab. - Boston University ACKNOWLEDGEMENTS Thanks to several co-workers whose contributions are reflected in this talk… … All 13 Former + Current PhD Students… …Colleagues… Y.C. (Larry) Ho Wei-Bo Gong Yorai Wardi Young Cho Liyi Dai Xiren Cao Ted Djaferis Doug Looze Felisa Vazquez-Abad …and Sponsors… NSF, AFOSR, AFRL, DARPA, NASA, EPRI, ARO, UT, ALCOA, NOKIA, GE