Distributed Control of
Multiagent Systems:
From Engineering to Economics
Prof. William Dunbar
Autonomous Systems Group
Computer Engineering
What are Systems? … ANYTHING
in Engineering, usually with
Dynamics.
Some familiar examples:
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
How do we describe (or predict) systems?
… with math!
(Images courtesy of http://www.cds.caltech.edu/~murray/cdspanel/ unless stated otherwise)
2
Math: Describing Diverse Engineering
Systems in a Common Way
Internet backbone
CA power grid
San Fran ATC
In these examples:
3
Control Systems are Hidden
Engineering Systems
“A Control System is
a device in which a
sensed quantity is
used to modify the
behavior of a
system through
computation and
actuation.”
4
Abstraction: Multiagent Systems
• The Internet
• Air traffic control
• Supply Chain
Control Problems with:
• Subsystem dynamics
• Shared resources
(constraints)
• Communications
topology
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
• Shared objectives
5
(SC image courtest of www.vipgroup.us)
Multiagent Systems: Distributed and
(presumed) Cooperative
Multiagent System:
• autonomous agents
• communication network
Agent
output
action
sensor
input
Environment
Distributed: local
decisions based on local
information.
Cooperative: agents agree
on roles & dynamically
coordinate.
6
A Relevant Decision Method: Model Predictive
Control (MPC)
MPC uses optimization to find feasible/optimal plans for near future.
objective
Minimize (distance to pump & fuel)
s.t. Car model (dynamics)
Without hitting wall (constraint)
To mitigate uncertainty, plan is revised after a short time.
actual
X
computed
7
Mathematics of MPC is Finite Horizon
Optimal Control
objective
Minimize (distance to
pump& fuel)
s.t. Car dynamics
Without hitting wall
(constraint)
8
Convergence of MPC Requires
Appropriate Planning Horizon
Theoretical conditions sufficient & in absence of explicit uncertainty.
*[Mayne et al., 2000]
9
MPC Compared to Other Techniques
• Gives planning &
feedback with builtin contingency
plans.
z(t0)
state
• Only technique that
handles state and
control constraints
explicitly.
• Tradeoff:
computationally
intensive.
t0
t0+d
z*(t;t0)
time
T
zk(t)
10
MPC Successful in Applications:
Process to Flight Control
Caltech flight control
experiment: Tracking
ramp input of 16 meters in
horizontal, step input of
1m in altitude.
MPC updates at 10 Hz,
trajectories generated by
NTG software package.
Movie
11
MPC Admits Cooperation
ok
follow
3
ok
2
follow
1
Get 1 to pump, 2 follow
1 & 3 follow 2.
Decoupled dynamics
Avoid collision
12
MPC of Multiagent Systems: What’s
Missing?
Enables autonomy of single
agent.
Amenable to cooperation for
multiple agents.
Missing?…Distributed Implementation*
Why not Centralized?…Local decision require Global information
Parallelization**?…If you can, but sometimes not applicable.
*[Krogh et al, 2000, 2001]
**[Bertsekas & Tsitsiklis, 1997]
13
My Contribution: A Distributed
Implementation of MPC
Distributed: local decisions based on local information.
Decoupled subsystem dynamics/constraints, Coupled cost L
Decomposition
14
Solution of Sub-problems requires
Assumed Plan for Neighbors
Agent 3 
z3(t0)
What 3 does
state
What 2 assumes
t0
t0+d
z3*(t;t0)
z3k(t)
15
time
Compatibility of Actual and Assumed
Plans via Constraint
Compatibility
constraint
Assumed plan
Bounds discrepancy
z3(tk)
state
tk
tk+d
time
16
Distributed Implementation Requires
Synchrony & Common Horizon T
17
Conditions for Theory are General
18
Convergence Conditions: Centralized
plus Bound on Update Period
*[Dunbar & Murray, Accepted to Automatica, June, 2004]
19
Venue: Multi-Vehicle Fingertip
Formation
4
2
1
d31
qref
3
20
Simulation Parameters
4
2
1
3
: Reference signal
: Actual COM of {1,2,3}
21
Centralized MPC:
Benchmark for Comparison
QuickTime™ and a
Microsoft Video 1 decompressor
are needed to see this picture.
22
Centralized MPC Simulation
23
Distributed MPC is Comparable to
Centralized MPC
QuickTime™ and a
Microsoft Video 1 decompressor
are needed to see this picture.
24
Distributed MPC Simulation
25
Naive Approach Produces Less
Desirable Performance
QuickTime™ and a
Microsoft Video 1 decompressor
are needed to see this picture.
26
Naïve Approach: Bad Overshoot
27
Summary of Contribution
Distributed implementation of MPC is provable
convergent, performs well, and is applicable to a
class of Multiagent Systems:
•
•
Distributed & cooperative structure:

Local decisions based on local information

Decomposition and incorporation of compatibility constraint

Coordination via sharing feasible plans
Applicable for:

Heterogeneous nonlinear dynamics

Generic objective function (need not be quadratic)

Coupling constraints and coupled dynamics
28
Supply Chain Management (SCM) is an
Attractive Venue for DMPC
•
Dynamics (Linear/Nonlinear) s.t. constraints and moving
set points.
•
Forecasts of measurable inputs often available, which MPC
can easily incorporate.
•
Dynamic time scales and inter-stage communication BW
are not limiting factors.
•
Active research area. Why? Companies don’t compete their supply chains do. Thus, SCM will make or break
companies. Examples: Dell, Walmart.
Challenge: distributed (asynchronous) coordination in the
presence of time delays.
29
Overview
1. Define three stage SCM problem from supply chain
literature
2. Distributed Problem ==> Distributed MPC Implementation
3. Nominal decentralized feedback policy from supply chain
literature
4. Numerical Experiments for Comparison
5. Conclusions and Extensions
30
SCM: Information Flows Upstream (orders) and
Material Flows Downstream (goods)
Three Stages: Supplier S, Manufacturer M, Retailer R
shipment rate
delay
acquisition rate
UP
stream
DOWN
stream
order rate
For each stage
Control: order rate
Measurable Exogenous Inputs:
demand rate
31
Bi-drectional Coupling in the Dynamics
For each stage
Dynamics:
Constraints:
Coupling: x depends on downstream order rate & upstream backlog
Objective:
Keep stock and unfulfilled order at desired levels
32
DMPC: Parallel Updates Assuming Remainder of
Previous Response for Neighbors
Q-cost with move suppression
DMPC Controller for each stage
1.
A.
B.
C.
2.
Move suppression ~ DMPC theory bounds
33
Experiments Show Comparable Performance
with Nominal Policy: Single Stage Case
Nominal:
Devised to
match observed
responses
Response to
initial stock
offset
Standard MPC
(not DMPC)
34
Step in Demand Rate: Comparable
Performance + Advantage of Anticipation
Add
anticipation
35
Three Stage with Pulse in Customer Demand:
Comparable then Better with Anticipation
Nominal
DMPC
36
Conclusions and Extensions
•
•
•
Realistic SCM problem (classic MIT “Beer Game”)
DMPC comparable to validated nominal feedback policy.
Clear advantage when customer demand can be reliably
forecasted (anticipation).
•
A detailed relative degree, controllability and stabilizability
analysis to come. Unfulfilled order in stages M and R exhibited
nonzero steady-state error.
Next leap: multi-echelon chains - at least two (and possibly
many) players operate within each stage, e.g., the S stage in
Dell's ``build-to-order" supply chain management strategy
might contain several chip suppliers such as Samsung, Intel and
Micron.
Extend theory asynchronous time conditions.
•
•
37