Scalable Intelligent
Control of Robot Teams
Tucker Balch
Ashley Stroupe (NASA/JPL)
Keith O’Hara
The BORG Lab
Thanks to: Daniel Walker, Matt Powers, and Frank Dellaert

Project 1 Grading Procedure
• Compile on CoC linux machine
• Test steps and path length against examples provided
• Test steps and path length against two additional maps
• Evaluate write up
• Next project: evaluate code readability
Tucker Balch
The BORG Lab

Project 1 Scoring
• Compiles:
• Solves some maps
• Solves all maps
+60
+10
+10
• Crashes sometimes
• Quality of write up
-10
+0 to +10
• Beats strawman on 1 public map +5
• Beats strawman on 2 public maps +5
• Beats strawman on 1 secret map +2.5
• Beats strawman on 2 secret maps +2.5
Tucker Balch
The BORG Lab

Test 1
• Thursday February 19
• In class, closed book
• Will cover Chapters 1, 2, 3, 4 maybe, 10
Tucker Balch
The BORG Lab

The BORG Lab
• Faculty
– Tucker Balch, Frank Dellaert, Sven Koenig
Thad Starner
• Staff
– Zia Khan (Biology/CS), Daniel Walker (EE)
• Resources
– 20 Robots, 3 ant colonies, 4 bee colonies, metal and wood working shop, electronics shop, workstations, number crunchers
• Research Thrusts
– Social insects
– Multi-robot systems
Tucker Balch
The BORG Lab

What is Special about Social Insects?
• Social insect colonies exhibit “super organism” capabilities beyond the ability of an individual agent
• How?
– Simple, local rules played out across thousands of interactions result in globally intelligent behavior (“emergent behavior”)
• Why should we care?
– Social insects provide inspiration for new, efficient, near optimal algorithms in many areas: network routing [Dorigo], robot navigation [Vaughn], job shop scheduling [Cicerello &
Smith]
Tucker Balch
The BORG Lab

Example: Optimal Path Finding by Ants
• Problem: find the shortest path to a food source
Tucker Balch
The BORG Lab

Solution: Pheromone Trails
Tucker Balch
The BORG Lab

Lessons From Insects
• Colony behavior is adaptive and intelligent but not usually optimal.
• No centralized control:
– Robust
– No single point of failure
– Scales very well w.r.t. number of animals.
• System behavior is mediated by
– local interactions between animals, or
– interaction with the environment: “stigmergy”
Tucker Balch
The BORG Lab

Rest of the Talk
• How can we borrow these ideas to control large teams of robots?
– Part 1: Synthetic Stigmergy
– Part 2: Move Value Estimation for Robot Teams
Tucker Balch
The BORG Lab

Part 1: Synthetic Stigmergy
• Idea: embed hundreds of simple computing and communication nodes in the environment
• These nodes can serve as physical repositories of navigation information
(e.g. for pheromones) with Keith O’Hara & Dan Walker
Tucker Balch
The BORG Lab

GNATs: A Physical Implementation
• Designed and built by Daniel Walker
• Features
– 4 independently powered IR emitters
– 4 IR receivers
– 4MHz PIC processor, 2K flash, 500 bytes RAM
– Very low power; up to 1 year on single battery
Tucker Balch
The BORG Lab

What can we do with GNATs?
• Assumptions:
– Nearby GNATs can communicate
– Obstacles to navigation block communication
– No sensors (other than IR) on the GNATs
• Embed information in the environment
• Build a network
• Distribute information
• Solve navigation problems
Tucker Balch
The BORG Lab

Solving Navigational Problems
• No localization required!
Tucker Balch
The BORG Lab

Solving Navigational Problems
Tucker Balch
The BORG Lab

Solving Navigational Problems
Tucker Balch
The BORG Lab

The Coverage Problem
Tucker Balch
The BORG Lab

A Complex Team Problem: Foraging
Tucker Balch
The BORG Lab

Research Problems
• How does network density impact performance?
• How does placement accuracy impact performance?
• What about dynamic environments?
• How do do this in a power efficient manner?
Tucker Balch
The BORG Lab

Network Density and Placement (ICRA 04?)
• Three experimental environments
• Wide range of densities
• 4 levels of randomness in placement
• Evaluate: time to complete task
Tucker Balch
The BORG Lab

Network Density and Placement: Results
Tucker Balch
The BORG Lab

What about dynamic environments? (AAMAS 04?)
Tucker Balch
The BORG Lab

The Impact of Replanning Rate
Tucker Balch
The BORG Lab

How to do it efficiently?
• Idea: “fast-path”
Tucker Balch
The BORG Lab

Fast-Path Animation (Gray == Fast-Path)
Tucker Balch
The BORG Lab

Fast-Path Results
Tucker Balch
The BORG Lab

Synthetic Stigmergy Summary
• Technique can be applied to many team navigation tasks
• Look, no hands! Localization is not required
• Computation is fast and distributed
• Reacts to dynamically changing environments
Tucker Balch
The BORG Lab

Next Steps
• Investigate application of GNATs to other team tasks
– Learning
– Multi-robot task allocation
• Implement on robots
Tucker Balch
The BORG Lab

Part 2: More Problems for Robot Teams
• How to move each robot to optimize team progress?
• Approach: Move Value
Estimation for Robot Teams
(MVERT)
A) B) with Ashley Stroupe (Journal of
Robotics and Autonomous
Systems)
Tucker Balch
The BORG Lab
C) D)

MVERT Algorithm for Each Robot
1.
Observe environment and localize
2.
Broadcast location and observations with uncertainties to teammates
3.
Receive information from teammates
4.
Update own world model based on teammates info
5.
Consider moves; for each candidate move
– Evaluate value of new location
6.
Choose highest value move; go to 1
A)
C)
B)
D)
Tucker Balch
The BORG Lab

MVERT Applications
•
•
•
•
Mapping: find minimum uncertainty location of landmarks in the environment
Multi-target tracking: observe multiple moving targets simultaneously
Sampling: visit targets and examine them (planetary exploration)
Communication: maintain line-of-sight communication networks (also with Matt Powers)
Tucker Balch
The BORG Lab

MVERT Results: Mapping
Tucker Balch
The BORG Lab

MVERT Results: Multi-Target Tracking
Tucker Balch
The BORG Lab

MVERT Results: Mapping and Sampling
Tucker Balch
The BORG Lab

MVERT Results: Robots
Tucker Balch
The BORG Lab

MVERT Features
• Close to optimal one-step team action selection
• Robust to agent failure
• Complex tasks can be composed by linear combination of value functions
• Computation for each robot scales linearly w.r.t. to number of robots, N
• Communicated information is limited, but number of messages scales as N 2
Tucker Balch
The BORG Lab

Closing
• Nature provides a library of effective robot team solutions
• Literal transfer from nature to robotics isn’t necessary
• Robust, effective team behaviors can be built using distributed computation and planning borg.cc.gatech.edu
Tucker Balch
The BORG Lab