Robust Distributed Task Allocation for
Autonomous Multi-Agent Teams
Ph.D. Candidate:
Sameera Ponda
Thesis Committee: Prof. Jonathan P. How,
Prof. Mary L. Cummings,
Prof. Devavrat Shah
May 31, 2016
Motivation
 Modern missions involve networked heterogeneous multi-agent teams
cooperating to perform tasks
 Unmanned aerial vehicles (UAVs) – target tracking, surveillance
 Human operators – classify targets, monitor status
 Ground vehicles – rescue operations
 Key Research Questions:
 How to coordinate team behavior to improve mission performance?
 How to hedge against uncertainty in dynamic environments?
 How to handle varying communication constraints?
2016/5/31
2
Problem Statement
 Objective: Automate task allocation to improve mission performance
 Spatial and temporal coordination of team
 Computational efficiency for real-time implementation
25
40
10
Problem Statement:
 Maximize mission score
 Satisfy constraints
25
25
 Decision variables:
30
 Team assignments, Service times
 Key Technical Challenges:





Combinatorial decision problem (NP-hard) – computationally intractable
Complex agent modeling (stochastic, nonlinear, time-varying)
Constraints due to limited resources (fuel, payload, bandwidth, etc)
Dynamic networks and communication requirements
Robustness to uncertain and dynamic environments
2016/5/31
3
Planning Approaches
 Optimal solution methods are computationally intractable for large problems
 Typically use efficient approximation methods [Bertsimas ’05]
25
40
10
 Most involve centralized planning [Bertsimas ’05]
 Base station plans & distributes tasks to all agents
 Requires full situational awareness
 High bandwidth, slow reaction to local changes
25
25
 Motivates distributed planning [Sariel ‘05, Lemaire ‘04]
 Agents make plans individually & coordinate
with each other through consensus algorithms [Olfati-Saber ‘07]
 Faster reaction to local information
 Increased agent autonomy
30
 Key questions for distributed planning:
 What quantities should the agents agree upon?
 Information / tasks & plans / objectives / constraints
 How to ensure that planning is robust to inaccurate information and models?
2016/5/31
4
Distributed Planning
Centralized Problem:
 Maximize mission score
 Satisfy constraints
 Decision variables:
 Team assignments, Service times
Distributed Problem:
 Maximize mission score individually
 Satisfy constraints
 Decision variables:
 Agent assignments, Service times
 Main issues: Coupling & Communication
 Agent score functions depend on other agents’ decisions
 Joint constraints between multiple agents
 Agent optimization is based on local information
 Key challenge: How to design appropriate consensus protocols? [Johnson ‘10]




Specify what information to communicate
Create rules to process received information and modify plans
Performance guarantees – is distributed problem good representation of centralized?
Convergence guarantees – will algorithm converge to a feasible assignment?
2016/5/31
5
Distributed Planning – CBBA
 Consensus-Based Bundle Algorithm (CBBA)
[Choi, Brunet, How ‘09]
 Iterations between 2 phases: Bidding & Consensus
1
Phase 1:
Build Bundle &
Bid on Tasks
(individual
agents)
2
3
Phase 2:
Consensus
(all agents)
All agents
consistent?
Yes
N
No
 Core features of CBBA:
 Sequential greedy task selection – Polynomial-time, provably good approximate solutions
 Guaranteed real-time convergence even with inconsistent environment knowledge
Key Contributions – extensions to CBBA framework:
1) Time-varying score functions (e.g. time-windows of validity for tasks)
2) Guaranteeing connectivity in limited communication environments
3) Robust planning for uncertain environments
2016/5/31
6
CBBA with Time-Windows
e.g. monitor status,
security shifts
Arrival Time
Time-critical
e.g. rescue ops,
target tracking
Arrival Time
Score
Time-window
Score
Score
 In realistic continuous-time missions, have time-varying task scores
Peak-time
e.g. rendezvous,
special ops
Arrival Time
 Extended CBBA to continuous-time domains
[ACC 2010]
 Task optimization involves decisions on task
assignments and task service times
 Preserves convergence properties
 Embedded the algorithm into dynamic
planning architecture
 Real-time simulation framework for dynamic missions
 Experimental flight tests for UAV/UGV teams
 Demonstrates real-time feasibility
2016/5/31
7
Cooperative Distributed Planning
 Often have fleet-wide hard constraints on assignments
 Agent assignments coupled through joint team constraints
 Example: Maintaining network connectivity in dynamic environments
 Often have limited communication radius, line-of-sight requirements
 As agents move around environment – dynamic networks, potential disconnects
25
10
40
25
25
 Several issues:
Disconnected
Network
30
 Some tasks rely on continuous connectivity (e.g. streaming live video)
 Cannot perform consensus, cannot deconflict plans
 How to include network connectivity constraints into distributed planner?
2016/5/31
8
Example: Baseline Scenario
 Motivating example – Surveillance Mission around base station
 UAVs travel to tasks and stream live video back to base station
 Successful task execution relies on continuous connectivity
 Limited comm radius (RCOMM)
30
0
10
10
No connectivity!
15
0
No connectivity!
2016/5/31
9
Example: Network Prediction
 Conservative solution – predict network connectivity violations
 Drop tasks if disconnects will occur
 Only execute tasks in local vicinity – conservative
10
10
30
15
2016/5/31
10
Example: Planning with Relays
 Can use some agents as communication relays!
 Coordinated team behavior leads to higher mission performance
 Goal: Develop cooperative planning algorithms to coordinate team
30
30
10
10
Relay
Relay
15
2016/5/31
11
CBBA with Relays
 CBBA with Relays
[JSAC 2012, Globecom 2011, Infotech 2011, Globecom 2010]
 Generate CBBA assignments
 Predict network over mission duration
 Repair connectivity by creating relay tasks
 Key features:
 Explicit consideration of dependency constraints
 Predict network topology only at select missioncritical times – avoids discretizing time
 Leverages information available in CBBA
consensus phase
 Preserves polynomial-time and convergence
guarantees
 CBBA with Relays improves performance
 Agents accomplish higher value tasks
 Guaranteed network connectivity
 Demonstrated real-time applicability
2016/5/31
Real-time experiment
Field experiment
iRobot Create
Pelican quad
Distributed Planning Under Uncertainty
 Uncertainty in planning process
 Inaccurate models (simplified dynamics, parameter errors)
 Fundamentally non-deterministic processes (e.g. sensor
readings, stochastic dynamics)
 Dynamic local information changes
 Can hedge against uncertainty to improve planning
Agent Schedule
Target Identification
Mission
Late! involves several challenges
 Robust planning
Tasks
 Optimal solutions computationally intractable –
increased dimensionality of planning problem
 Non-trivial coupling of distributions – analytically
intractable
Time
 Current approaches involve many limiting assumptions
Distribution for Operator Target Identification
Figure from [D. Southern, Masters Thesis, 2010]
 Key questions:
 How to propagate uncertainty through planner to generate agent assignments?
 How to distribute planning given additional complexity due to uncertainty?
 How to ensure real-time performance and computational tractability?
2016/5/31
13
Distributed Planning Under Uncertainty
 Chance-Constrained CBBA – Extended CBBA to
incorporate risk into planning process [ACC 2012]
 Model coupling using numerical approx (sampling)
 Preserves polynomial-time
 Probabilistic performance guarantees for given risk
 Key features:
 Improved CBBA to handle non-submodular score
functions (e.g. stochastic scores) [CDC 2012]
 Approximate distributed agent risk given mission risk
using Central Limit Theorem assumption
 Improved performance under uncertainty
 Higher scores within allowable risk
 Distributed approximation on par with centralized
 Current work is exploring dynamic aspects
 Dynamic risk allocation
 Model learning using Nonparametric Bayesian techniques
[GNC 2012]
2016/5/31
14
Conclusion
 Distributed task allocation strategies for autonomous multi-agent teams
 Extended CBBA algorithm to include time-varying score functions
 Addressed cooperative planning in comm-limited environments using relay tasks
 Presented robust risk-aware distributed extensions to deterministic planning
 Acknowledgments:





Prof. Jonathan How for his invaluable advice and support
My committee members Prof. Cummings and Prof. Shah
My collaborators and colleagues at ACL, esp. Luke Johnson and Andrew Kopeikin
Aero/Astro faculty and staff
Graduate Aero/Astro friends!
2016/5/31
15