Decentralized Mission Planning for
Heterogeneous Human-Robot Teams
Sameera Ponda
Prof. Jonathan How
Department of Aeronautics and Astronautics
Massachusetts Institute of Technology
May 31, 2016
Motivation
• Modern day complex missions involve networked teams of heterogeneous
agents executing several tasks simultaneously:
– Unmanned aerial vehicles (UAVs) – target tracking, surveillance
– Human operators – classify targets, monitor status
– Ground convoys – rescue operations
• Key Research Questions:
– How can we coordinate team behavior to improve mission performance?
– How should planning strategies evolve as we acquire more information?
2
Problem Statement
• Goal: Automate task allocation to improve mission performance
– Spatial and temporal synchronization
– Reduce costs and improve efficiency
• Key Technical Challenges:
–
–
–
–
–
Combinatorial decision problem – computationally intractable (NP-hard)
Complex agent modeling & constraints (stochastic, non-linear, time-varying)
Limited resources (bandwidth, fuel, etc)
Task3
Task1
Dynamic networks and communication constraints
Task2
Unknown and dynamic environments
Task4
Agent5
Agent1
Agent4
Agent6
Agent2
Task6
Task9
Task5
Agent3
Task8
Task7
3
Planning Approaches
• Optimal solution methods are computationally intractable for large problems
– Typically use approximation methods
Agent2
Agent1
• Centralized Planning approach
– Mission Control Center (MCC) plans & distributes tasks for all agents
– High bandwidth, slow reaction, resource intensive
MCC
Agent3
Agent4
• Recent research in Decentralized Planning
– Individual agents make their own plans and coordinate with each other
– Faster reaction to local information changes
Agent1
– Trade-off between communication and computation
Agent2
• Key Questions:
– What quantities should the agents agree upon?
Agent3
• Information / tasks & plans / objectives / constraints
– How do we ensure that the planning is robust to inaccurate information and models?
Agent4
4
Consensus-Based Bundle Algorithm
• Decentralized task allocation approach called Consensus-Based Bundle
Algorithm (CBBA) [Choi, Brunet, How 2009]
– CBBA iterates between 2 phases: Bidding & Consensus
1
Phase 1:
Build Bundle &
Bid on Tasks
(individual
agents)
2
3
Phase 2:
Consensus
(all agents)
All agents
consistent?
Yes
N
No
• Core features of CBBA:
– Polynomial-time decentralized algorithm with provably good approximate solutions
– Consensus on task assignments, not information – guaranteed real-time convergence
even with inconsistent information
Key extensions to CBBA:
1) Temporal constraints – Time-windows of validity for tasks
2) Connectivity issues and constraints
3) Planning for teams with Humans-in-the-loop
5
CBBA with Time-Windows
e.g. monitor status,
security shifts
Arrival Time
Time-critical
e.g. rescue ops,
target tracking
Score
Flat
Score
Score
• In realistic missions, task scores often depend on arrival times and have
associated time-windows of validity:
Peak-time
Arrival Time
e.g. rendezvous,
special ops
Arrival Time
• Issue: Planning algorithms usually involve time discretization
– Extra planning dimension – computationally intractable!
• CBBA extended to include time-windows
– Solution does not discretize time!
– Preserves convergence properties
– Planner decides arrival times, producing
task schedules for agents
• Embedded CBBA with Time-Windows
into a real-time system architecture
6
CBBA with Time-Windows
• CBBA successfully used in real-time fight test environments
– Cooperative search, acquisition, and track (CSAT)
– Coordination of agents under dynamic network topologies
• Further information available online at: http://acl.mit.edu/projects/cbba.html
7
Connectivity: Network Challenges
• As agents move around in the environment, expect varying network topologies
– Limited communication radius between agents
– Potential broken comm links and/or disconnected networks
Task1
Task3
Task2
Task4
Agent1
Agent2
Agent5
Disconnected
Network
Agent4
Task9
Agent6
Agent3 Task
7
Task6
Task5
Task8
• Main issue: Planner cannot converge with a disconnected network, leading to
conflicting assignments
• Developed two solution approaches:
– CBBA with Relays – Creates relay tasks to ensure connectivity
– CBBA with Network Handling Protocols – Protocols to adjust task lists prior to planning
8
Connectivity: CBBA with Relays
• Extended CBBA to include relay tasks –
(Published in GlobeComm 2010)
– Employs underutilized agents as relays
– Key feature: Agents use bid info to predict
network structure at select times
– Guarantees connectivity
– Computationally efficient - converges in real-time
Relay Task
• CBBA with Relays improves team
performance and network connectivity
9
Connectivity: Network Protocols
• If preventing disconnects is too conservative: Network Handling Protocols to
adjust task lists for agents prior to planning – (Published in ACC 2010)
Baseline
Central
(no adjustment) Adjustment
Local
Adjustment
All tasks available
to all agents
MCC distributes
tasks to networks
at each replan
MCC distributes
new tasks to
closest agents
(once per task)
Low
Bandwidth
High
Bandwidth
Low
Bandwidth
Low
Computation
High
Computation
Low
Computation
• Local Adjustment improves
mission
performance
Conflicting
Guaranteed
Assignments
Deconfliction
––
lower
mission
missionscores
scores
and
and
with low bandwidth and higher
computation
requirements
wastedfuel
lower
fuel
consumption
10
Planning for Human-Robot Teams
• Most modern missions involve human-robot teams
– Human operators perform several tasks
(e.g. supervisory, target classification, monitoring)
– Need to coordinate robotic agents and operators
Predator UAV Operations – Associated Press
• Main Issue: Operator performance is stochastic
– Heterogeneous operator capabilities (“slow” vs. “fast”)
– Robustness to uncertainty in team performance
• Recent research has explored modeling operators
using probabilistic distributions – [Cummings et al ‘10]
Log-Normal Distribution for Operator
Target Identification
Figure from [D. Southern, Masters Thesis, 2010]
• Key Challenge: Incorporate uncertainty into planner to increase robustness
11
Planning for Human-Robot Teams
• Consider a time-critical mission with operators performing target classification
• As expected vs. actual service times differ, planner performance degrades
– Adding a margin of conservatism can mitigate this problem
– Tradeoff between late penalties and number of tasks assigned
Agent Schedule
Mission Performance
• Simulation Observations: Late!
– Performance is best when expected & actual
are close (ridge line)
– Steeper drop for overestimating (optimistic)
Optimistic
Conservative
vs. underestimating (conservative)
Planning
Planning
Time
– Conservative Planning performs better
than Optimistic Planning
Tasks
550
500
Mission Score
450
400
350
300
250
200
150
100
• Developing a Robust Planning Framework
– Explicitly embed PDFs of plan parameters
– Adapt as estimates improve
50
0
0
0.2
0.2
0.4
0.4
0.6
Expected Operator
Percentile
0.6
0.8
0.8
1
1
Actual Operator
Percentile
12
Planning for Human-Robot Teams
• Currently performing Human-in-the-loop
experiments at Cornell
– CBBA used to allocate targets to agents (MIT)
– Image processing and sensor fusion used to
update target PDFs (Cornell)
– Human-in-the-loop for target classification and
PDF updates through HRI (Cornell)
13
Conclusions
• Explored strategies to coordinate team behavior to improve mission performance
• Extended the Consensus-Based Bundle Algorithm (CBBA) to address the
demands of more realistic multi-agent mission planning
– Included task time-windows of validity
– Addressed connectivity issues and communication constraints
– Explored planning for heterogeneous human-robot teams
• Current research and expected thesis contributions:
1) Robust decentralized planning framework
• Embed distributions of parameters into planner
• Preserve computational tractability and scalability
(e.g. avoid discretization, explore efficient sampling techniques)
2) Flexible planner structure that adapts to dynamic uncertainty representations
• Modular uncertainty representations (Nonparametric Bayesian models, etc)
• Modify planning strategy without recomputing all scenarios
3) Efficient strategies for information consensus to improve planner performance
• Decide what information and when to share (e.g. hyperparameter consensus)
• Cooperative decentralized strategies to update global distributions
14