Multi-Agent Exploration in
Unknown Environments
Changchang Wu
Nov 2, 2006
Outline
• Why multiple robots
• Design issues
• Basic approaches
– Distributed
– Centralized
– Market-based
Why Multiple Robots
• Some tasks require a robot team
• Have potential to finish tasks faster
• Increase robustness w/ redundancy
• Compensate sensor uncertainty by merging
overlapping information
• Multiple robots allow for more varied and
creative solutions
A Good Multi-Robot System Is:
• Robust: no single point of failure
• Optimized, even under dynamic conditions
• Quick to respond to changes
• Able to deal with imperfect communication
• Able to avoid robot interference
• Able to allocate limited resources
• Heterogeneous and able to make use of
different robot skills
Basic Approaches
• Distributed
– Every robot goes for itself
• Centralized
– Globally coordinate all robots
• Market-based
– Analogy To Real Economy
Distributed Methods
• Planning responsibility spread over team
• Each robot basically act independently
• Robots use locally observable information to
coordinate and make their plans
Example: Frontier-Based Exploration
Using Multiple Robots (Yamauchi 1998)
• A highly distributed approach
• Simple idea: To gain the most new information
about the world, move to the boundary between
open space and uncertainty territory
• Frontiers are the boundaries between open space
and unexplored space
Occupancy Grid
• World is represented as grid
• Each cell in the grid is assigned with a probability
of being already occupied/observed
• The initial probability is all set to .5
• Cell status can be Open (<0.5), Unknown (=0.5)
or Occupied (>0.5)
• Bayesian rule is used to update cells by merging
information from each sensor reading (sonar)
Frontier Detection
•
Frontier = Boundary between open and unexplored space.
•
Any open cell adjacent to unknown cell is frontier edge cell.
•
Frontier cells grouped into frontier regions based on adjacency.
•
Accessible frontier = Robot can pass through opening.
•
Inaccessible frontier = Robot cannot pass through opening.
Multi-Robot Navigation
• Simple algorithm: Each robot goes along the shortest
obstacle free path to a frontier region
• Robots share a common map: All information
obtained by any robot is available to all robots
• Robots are planning path independently
• Use reactive strategy to avoid collisions
• Robots may waste time for the same frontiers
An Exploration Sequence
Distributed Methods: Pros & Cons
• Pros
– Very robust. No single point failure
– Fast response to dynamic conditions
– Little or no communication is required
– Easy….Little computation required
• Cons
– Plans only based on local information
– Solutions are often sub-optimal
Centralized Methods
• Robot team treated as a single “system” with
many degrees of freedom
• A single robot or computer is the “leader”
• Leader plans optimal tasks for groups
• Group members send information to leader
and carry out actions
Example: Arena (Jia 2004)
• Robots share a common map and only
communicate with a leader
• Robots compete for resources by their
efficiency
• leader greedily assigns the most efficient
tasks
• Leader coordinate robots to handle
interference
Background
• World representation
– Occupancy grid
• Cost unit
– Moving forward one step = Turning 45 degrees
• Cost overflow
– Similar to minimum cost spanning tree
– Easy to compute the shortest path
– Easy to handle obstacle
Cost Overflow
3
4
5
12
1
2
4
5
2
3
3
5
6
7
8
3
4
4
4
6
7
4
5
5
5
5
5
6
6
6
Direction
priority
11
12
9
10
11
12
8
9
10
11
12
7
8
9
10
11
12
6
6
8
9
10
11
12
8
7
7
9
10
11
12
9
9
8
8
10
11
12
Cost of 45° turning ＝Cost of one cell’s step
Goal Candidates Detection
•
A goal point P should satisfy
i.
P is passable (Mark the cells in warning range or
obstacles/Wall/Unknown cells as impassable)
ii. Some unexplored cells lie in the circle with P as
the center and (R + K) as the radium, where R
is the warning radius and K is usually 1
Robot cell
paths cell
observation cell
candidate goal
Goal Resource
• Reserved goal candidates
– Robots obtained by competition
Goal candidates
• Recessive goal candidates
– The goal points in a given range to a
reserved goal point
– This distance can be adjusted
Recessive goals candidates
Path Resource
• Path resource is a time-space
term
• For a given time, the cells
close to any robot are marked
off for safety
• Looks just like a widened path
• Basically a reactive strategy
goal
path
resource
Revenue and Utility
• Revenue
– The expected gain of information that robots
observe at a goal point
• Utility used by many other approaches
– Utility = revenue – cost
• Utility in this paper
– Utility = Revenue / Cost
– Better connected to purpose of smallest cost
– No need to care about unit conversion
Greedy Goal Selection
• Try to maximize the global utility
• Coordination: robots obtain goal and path
resources exclusively
• Competition: repetitively select the pair of
free agent and goal with highest utility
• Sub-optimal
Simple Algorithm
•
Repeat until map is complete
– Repeat #free robots times
1. Cost computation (Also make sure no
interference with the busy robots)
2. Select the highest utility task (Compete)
3. Mark off the associated robot and goal points,
and nearby goal points
1st Competition:
1
2
3
1
1
2
3
2
2
2
3
4
5
5
3
3
3
3
6
4
4
4
4
4
3
3
5
5
4
3
Interval = 3 Competitor:
6
6
6
6
6
5
5
5
5
5
5
4
4
4
4
4
4
4
3
3
3
3
3
6
3
3
3
2
2
2
2
2
3
6
2
2
2
2
1
1
1
2
3
6
2
1
1
2
1
1
2
3
6
2
1
1st Competition Result:
6
4
4
4
4
4
4
3
3
3
2
2
2
2
2
2
1
Interval = 3 Competitor:
1
6
5
5
2nd Competition
1
2
3
1
1
2
3
2
2
2
3
4
4
4
3
3
3
3
4
4
4
4
4
4
4
5
5
5
5
Interval = 3
6
6
6
6
6
4
5
5
5
5
5
4
4
4
4
4
4
4
4
4
4
4
3
3
3
3
5
3
6
3
6
3
6
4
3
2
2
2
6
2
6
2
6
2
5
4
3
2
1
1
1
6
1
5
4
3
2
1
Competitor:
Satisfied:
2nd Competition Result
6
6
6
Competitor:
6
4
4
4
6
5
6
6
4
4
4
5
4
5
5
4
4
4
4
3
4
4
2
3
3
3
3
3
2
3
2
2
2
2
2
2
1
2
1
Interval = 3
6
1
Satisfied:
1
1
3rd Competition
1
2
3
11
6
11
6
11
6
12
1
1
2
3
10
6
10
6
11
6
12
13
2
2
2
3
4
4
7
4
8
4
9
6
9
10
5
6
11
6
12
13
3
3
3
3
4
4
7
4
8
4
8
5
9
10
4
5
11
5
12
13
4
4
4
4
4
4
7
4
7
4
8
4
9
10
3
4
11
4
12
2
13
5
5
5
5
5
3
6
3
7
3
8
3
9
10
3
11
2
3
12
2
13
6
2
6
2
7
2
8
9
10
2
11
2
12
1
2
13
7
1
7
7
1
8
9
10
11
12
1
13
Interval = 3
Competitor:
Satisfied:
3rd Competition Result
6
1
2
6
9
3
4
4
5
4
8
4
7
3
6
Competitor:
11
6
12
4
3
2
2
1
Interval = 3
10
5
6
1
Satisfied:
13
Planning Issues
• Do not transfer a reserved goal point to
another free agent (unless necessary).
Frequent change of tasks can cause
localization error.
• Quit an assigned task when the goal point is
unexpectedly observed by other robots
• Schedule at most one task for each agent
Possible Variations
• Still keep busy agents in competition. Remove
the goal resources they win from competition.
– This prevents those goal resources being
assigned to other agents
– It is too early to burden a new task on a robot
who has not achieved it current task
• No need to schedule them.
– New resources probably will be found when
they reach the goals
Handling Failure of Planning
• It may fail to plan safe paths
– When some robot get to a place where
• it is almost too close to other robot
• it has no good space to detour
– And it choose to just wait there for other robots to
move away, which is not known by other robots
• Avoidance of unexpected obstacle
– Robots have simple reactive mechanism
– Release resources and try to gain new task
Fail to plan safe paths
1
10
2
9
3
4
5
6
7
8
collision
16
15
14
13
12
11
10
4
3
2
1
Interval = 3
Competitor:
Satisfied:
Reactive Mechanism
Interval = 3
Competitor:
Satisfied:
Exchange Tasks
3
2
4
3
2
1
1
5
Interval = 3
Competitor:
Satisfied:
Some Statistics
Demo
Centralized Methods : Pros
• Leader can take all relevant information into
account for planning
• Optimal s islution possible!
• One can try different approximate solutions to
this problem
Centralized Methods: Cons
• Optimal solution is computationally hard
– Intractable for more than a few robots
• Makes unrealistic assumptions:
– All relevant info can be transmitted to leader
– This info doesn’t change during plan construction
• Vulnerable to malfunction of leader
• Heavy communication load for the leader
Market-Based Methods
• Based on market architecture
• Each robot seeks to maximize individual
“profit”
• Robots can negotiate and bid for tasks
• Individual profit helps the common good
• Decisions are made locally but effects
approach optimality
– Preserves advantages of distributed approach
Why Is This Good?
• Robust to changing conditions
– Not hierarchical
– If a robot breaks, tasks can be re-bid to others
• Distributed nature allows for quick response
• Only local communication necessary
• Efficient resource utilization and role adoption
• Advantages of distributed system with
optimality approaching centralized system
Architecture
• World is represented as a grid
– Squares are unknown (0), occupied (+), or empty (-)
• Goals are squares in the grid for a robot to explore
– Goal points to visit are the main commodity exchanged in
market
• For any goal square in the grid:
– Cost based on distance traveled to reach goal
– Revenue based on information gained by reaching goal
• R = (# of unknown cells near goal) x (weighting factor)
• Team profit = sum of individual profits
– When individual robots maximize profit, the whole team
gains
Example World
Goal Selection Strategies
• Possible strategies:
– Randomly select points, discard if already
visited
– Greedy exploration:
• Choose goal point in closest unexplored
region
– Space division by quadtree
Exploration Algorithm
Algorithm for each robot:
1. Generate goals (based on goal selection
strategy)
2. If OpExec (human operator) is reachable, check
with OpExec to make sure goals are new to
colony
3. Rank goals greedily based on expected profit
4. Try to auction off /bid goals to each reachable
robot
– If a bid is worth more than you would profit from
reaching the goal yourself (plus a markup), sell it
Exploration Algorithm
5. Once all auctions are closed, explore
highest-profit goal
6. Upon reaching goal, generate new goal
points
–
Maximum # of goal points is limited
7. Repeat this algorithm until map is complete
Bidding Example
• R1 auctions goal to
R2
Expected vs. Real
• Robots make decisions based on expected
profit
– Expected cost and revenue based on current map
• Actual profit may be different
– Unforeseen obstacles may increase cost
• Once real costs exceed expected costs by
some margin, abandon goal
– Don’t get stuck trying for unreachable goals
Information Sharing
• If an auctioneer tries to auction a goal point already
covered by a bidder:
– Bidder tells auctioneer to update map
– Removes goal point
• Robots can sell map information to each other
– Price negotiated based on information gained
– Reduces overlapping exploration
• When needed, OpExec sends a map request to all
reachable robots
– Robots respond by sending current maps
– OpExec combines the maps by adding up cell values
Advantages of Communication
• Low-bandwidth mechanisms for
communicating aggregate information
• Unlike other systems, map info doesn’t
need to be communicated repeatedly
for coordination
What Is a Robot Doing
• Goal generation and exploration
• Sharing Information with other robots
• Report information to OpExec at some
frequency
Experimental Setup
• 4 or 5 robots
– Equipped with fiber
optic gyroscopes
– 16 ultrasonic sensors
Experimental Setup
• Three test environments
– Large room cluttered with obstacles
– Outdoor patio, with open areas as well as walls and
tables
– Large conference room with tables and 100 people
wandering around
• Took between 5 and 10 minutes to map areas
Experimental Results
Experimental Results
Experimental Results
• Successfully mapped regions
• Performance metric (exploration efficiency):
– Area covered / distance traveled [m2 / m]
– Market architecture improved efficiency over no
communication by a factor of 3.4
Conclusion
• Market-based approach for multi-robot
coordination is promising
– Robustness and quickness of distributed system
– Approaches optimality of centralized system
– Low communication requirements
• Probably not perfect
– Cost heuristics can be inaccurate
– Much of this approach is still speculative
• Some pieces, such as leaders, may be too hard to do
In Sum
• Distributed vs. centralized mapping
• Distributed vs. centralized planning
• Revenue/Cost vs. Revenue – Cost
• Often sub-optimal solutions
• No common evaluation system for
comparisons
References
•
•
•
Yamauchi, B., "Frontier-Based Exploration Using Multiple Robots," In
Proc. of the Second International Conference on Autonomous Agents
(Agents98), Minneapolis, MN., 1998.
Menglei Jia , Guangming Zhou ,Zonghai Chen, "Arena—an Architecture
for Multi-Robot Exploration Combining Task Allocation and Path
Planning,“ 2004
Zlot, R., Stentz, A., Dias, M. B., and Thayer, S. “Multi-Robot Exploration
Controlled By A Market Economy.” Proceedings of the IEEE
International Conference on Robotics and Automation, 2002.
•
•
•
http://voronoi.sbp.ri.cmu.edu/presentations/motionplanning2001Fall/FrontierExp
loration.ppt
http://www.ai.mit.edu/courses/16.412J/lectures/advanced%20lecture_11.6.ppt
http://mail.ustc.edu.cn/~jml/jml.files/Arena.ppt