From: AAAI Technical Report WS-00-09. Compilation copyright © 2000, AAAI (www.aaai.org). All rights reserved.
Control of Multiple Small Surveillance Robots at AAAI2000
Paul E. Rybski,
Dean F. Hougen, Sascha A. Stoeter, Maria Gini and Nikolaos Papanikolopoulos
Center for Distributed Robotics, Departmentof ComputerScience and Engineering
University of Minnesota
{rybski,hougen,stoeter,gini, npapas}@cs,umn. edu
Abstract
Wedescribe Ourexperiencesdemonstratinga teamof miniature robots engagedin a surveillance missionat the AAAI
2000MobileRobotsExhibition.Wepresent the architecture
and the mainfeatures of our systemand discuss someof the
environmental
factors that affect its performance.
Introduction
Reconnaissanceand surveillance tasks can benefit from the
use of multiple small yet highly capable robots. Suchrobots
must be easily deployable and able to moveefficiently yet
traverse obstacles or uneventerrain. Theymust be able to
sense their environment,act on their sensing, and report their
findings. Theymust be able to be controlled in a coordinated
manner.
Wehave developed a set of extremely small robotic systems, called scouts (Hougenet al. 2000a), which were designed specifically to meet these requirements. The scout
robot, shownin Figure 1, is a cylindrical robot 11 cmin
length and 4 cmin diameter.
Scouts have two modesof locomotion designed to transport themover different kinds of terrain and obstacles. In the
first mode,the scouts use their wheelsto roll over smooth
surfaces (even upa 20 degree slope). Whenconfronted with
an obstacle taller that itself, the scout employsits second
locomotion mode, the jump. The spring-loaded tail of the
scout is compressedand released to propel the robot over
objects upwardsof 20 cm in height. Figure 2 showsa scout
jumpingover a barrier.
Scouts carry a small video cameraand video transmitter
whichthey use to capture information about their environment. They can also transmit and receive digital information over a separate RF communicationssystem which uses
a packet-based communicationsprotocol.
The small size of the scouts provides manyadvantages.
Theyare inexpensive and easily transportable, which makes
them ideal for use in large teams. This allows themto be
present throughouta wide area, forminga mobilesensor network.It also allows individual scouts to be expendablewithout jeopardizingan entire mission, scouts are well suited to
clandestine operations since they can be concealed easily.
Copyright(~) 2000,American
Associationfor Artificial Intelligence(www.aaai.org).
All rights reserved.
33
Figure 1: Twoscout robots (shown next to a quarter for
scale). Q2000by ACM,appeared in Rybski et al. (2000).
Their small size and cylindrical shape also allows themto
be deployed by launching or throwing by hand.
Related
Work
Automatic security and surveillance systems using cameras and other sensors are becoming more common.These
typically use sensors in fixed locations, either connected
ad hoc or, increasingly, through the shared communication lines of "intelligent buildings" (Porteous 1995). These
maybe portable to allow for rapid deployment(Pritchard et
al. 1998) but still require humanintervention to reposition
whennecessary. This shortcomingis exacerbatedin cases in
whichthe surveillance team does not have full control of the
area to be investigated, as happensin manylaw-enforcement
scenarios. Static sensors have another disadvantage. They
do not provide adaptability to changes in the environment
or in the task. In case of poor data quality, for instance, we
mightwant the agent to movecloser to its target in order to
senseit better.
Mobile robotics can overcomethese problems by giving
the sensor wheels and autonomy. This research has traditionally focused on single, large, independentrobots designed to replace a single humansecurity guard as he makes
his rounds (Kajiwara et al. 1985). Such systems are now
available commerciallyand are in place in, for example,factory, warehouse, and hospital settings (Kochan1997), and
Figure 2: A scout jumpingover a barrier (sequence starts from the upper left corner). ©2000by IEEE,appeared in Hougenet
aL (2000a).
research continues along these lines (Dillmannet al. 1995;
Osipov, Kemurdjian, & Safonov 1996). However, the single mobile agent is unable to be manyin places at once-one of the reasons whysecurity systems were initially developed. Further, large mobilerobots are unable to conceal
themselves, which they mayneed to do in hostile or covert
operations. Theymayalso be too large to exploretight areas.
Multiple robots often can do tasks that a single robot
would not be able to do, or can do them faster and more
reliably, as described in the extensive survey by Cao, Fukunaga, & Kahng(1997). Tasks traditionally accomplished
with multiple robots are foraging (Matari6 1997), formation
marching (Balch & Arkin 1998), map making (Burgard et
aL 2000), janitorial service (Parker 1996), security (Everett
& Gage1999), and exploration (Hougenet aL 2000b).
Traditionally, mobile robots have ranged in size from
roughly dog-sized to somewhatlarger than a human. For
small robots, Lego blocks and microprocessor boards, such
as the Handyboard(Martin 1998), are often used to quickly
prototype robots. Wehave designed multiple Lego-based
robots and used themfor a variety of navigation tasks (Rybski et al. 1998).
Recently a significant interest has arisen in designing
even smaller mobile robots for exploration and reconnaissance. These include the popular Kheperarobot (Mondada,
Franzi, & Ienne 1993) and other prototypes, such as ALICE(Caprari et al. 1998) and our ownscouts (Hougenet aL
2000a), developed by various research groups and not yet
commerciallyavailable.
The major challenge in designing miniature robots is in
fitting all the mechanicalparts and electronics into a limited
volume and in designing an adequate method of locomotion. Most miniature robots have wheels (Baumgartner et
al. 1998), but some can jump (Halme, Sch6nberg, & Wang
1996), roll (Chemel,Mutschler, &Schempf1999), fly (Wu,
Schultz, & Agah 1999), or swim (Fukuda, Kawamoto,
Shimojima1996). Our miniature robots have two rolling
wheelsbut they can also jumpto go over small obstacles, as
shownin Figure 2.
34
Dueto their small size, most miniature robots use proxy
processing, as in Inaba et al. (1996), and communicate
via
radio link with the unit wherethe computationis done. Limited communication bandwidth becomes a problem when
robots have to transmit large amountsof data, such as live
video, and whenmanyrobots need to share the bandwidth.
Power consumption is another major problem which limits
the mobility, ability to communicate,and long term survivability of miniaturerobots.
Scout Control Architecture
The scout’s small size and limited powersupply greatly restricts the kinds of on-boardprocessingthat can take place.
The only computations done on the scout’s two 8-bit microprocessors are what’s necessary for communicationsand
¯ actuation. All high-level decision makingand video image
processing must be relegated to an off-board workstation or
a humanteleoperator.
Autonomous
decision prgcess, such as planners, reactive
behaviors, or teleoperation controls, send low-level actuator
commandsto the scouts through an RF transceiver. Each
scout is assigned a unique network ID which allows packets to be routed to the correct robot. By interleaving the
transmission of packets destined for different robots, multiple scouts can be controlled simultaneously. The current
bottleneck in the systemis the numberof packets per second
that can be transmitted. The frequency at which commands
can be broadcast is currently between5 to 8 per second. The
limiting factor is the speedat whichthe scout can decodethe
messagesfrom the radio.
Scout video data is captured by a video receiver. This
can be either displayed on a monitoror fed into a digitizer.
Because the video is a continuous analog stream, only one
robot can broadcast at any instant. Signals broadcast simultaneously from multiple robots would interfere with each
other, makingboth signals useless. Scouts runningin parallel mustinterleave their video transmissions and limit their
transmissionsto short bursts.
Wehave developed a distributed, CORBA-based
(Group
The scout knowsthat it should stop whenits camerais
either pressed up against a dark object, or the scout is in
shadows.Scout motionin this behavior is continuousand
the scout does not check to see that it has movedby computing differences betweenframes. Other kinds of motion
makesuse of this check because the scout does not always
receive the commands
sent for it due to RFinterference.
Framedifferencing is not helpful in this behaviorbecause
the scout is unable to movevery quickly. The difference
between subsequent frames captured during forward motion is minimal,makingit very difficult for the robot to
detect forward motion.
1998) architecture to coordinate all of the individual hardware resources in our system. Access to robotic hardware
and other computationalresources is controlled through processes called RESOURCE CONTROLLERS(or RCS). Every
physical resource is given its ownRCprocess to manageit.
Behaviors and decision processes connect to these RCSto
send commandsand receive data from them. Behavior processes running on different computerscan access all of the
RCswithout even being aware that they maybe running on
different computers.
Scout Behaviors
The scouts are capable of operating autonomouslythrough
the use of simple behaviors. The only environmentalsensor
available to the scout is its video camera, the use of which
presents several problems. First, the scout’s proximity to
the floor severely restricts the area it can view at a time.
Secondly, since the video is broadcast over an RF link to
a larger robot or a workstation for processing, the quality
of the received video often degrades due to of range limitations, proximityof objects that interfere with transmission,
and poor orientation of the antennas. Figure 3 shows an
example image received from the scout’s camera. The RF
noise degradingthis imageincreases the difficulty of distinguishing the objects from the background.
Detect Motion:Detecting movingobjects is accomplished
using frame differencing. Oncethe scout has been placed
in a single location, it mustdeterminethe quality of the
video and set a threshold to filter out RFnoise. This is
accomplished by doing image differencing on a stream
of video and increasing the difference threshold until RF
noise is filtered out. Thescout then subtracts sequential
images in the video stream and determines whether the
scene changes at all (caused by movement
in the image).
Demonstrationsat the Exhibition
At the AAAI2000 Mobile Robot Exhibition, we demonstrated several aspects of our distributed scout control system on two networked laptop computers. The first laptop
ran Linux and handled all of the scout control programs.
The second ran Windowsand handled the video processing hardware and software. The scout’s command
radio was
connected to the Linux laptop while a video receiver was
connected to the Windowslaptop. In addition to the autonomousbehaviors defined above, a simple teleoperation
client was used which allowed a humanoperator to drive a
scout and operate its sensor payload and jumping mechanism. Figure 4 illustrates the implementationof the scout
control system.
Linux Laptop
Windows Laptop [
[Autonomous
ConsoleTele°perati°n
[Behavior
Ethemet
Figure 3: The worldfrom the scout’s point of view. Here the
scout is viewinga lab benchand twochairs at a range of 2m.
©2000by ACM,appeared in Rybski et al. (2000)
PCMCIA
Framegra~
A numberof different visually-based behaviors have been
implemented;see Rybskiet al. (2000) for the full list. The
behaviors demonstrated at the AAAI2000 Mobile Robot
Exhibition were:
SVideo
~
Drive TowardsGoal: Identifying a dark area to movetowardsis a simple matter of scanning across the image at
a fixed level on or about the level of the horizon and determining the horizontal position of the darkest area in
the image. The mean pixel values in a set of overlapping windowsin the image are determined. The scout
selects the darkest windowand drives in that direction.
~
~
"~
Signal
Transmitted
Commands
H
_
.
Scout
Robots
Figure 4: Diagramof the hardwareand software architecture
used to control the scout robots.
35
Several RCS were used by the behaviors and teleoperation consoles to control the scouts. The first, and arguably
the mostimportant, RCis in charge of controlling access to
the radio. The user interface consoles and behaviors have to
connect to the radio RC’s CORBA
interface to send packet
requests to it. The radio RCsupports two kinds of commandpacket requests. In the first request type, the decision
process must explicitly instruct the radio to send each command.If commands
are sent to the radio faster than they
can be transmitted, they are stored in a queueuntil the radio
can handle them. In the second request type, the decision
process gives the radio a single command
and instructs it to
repeat it as often as it can. This allows the systemto keep
inter-process communicationsto a minimum.If multiple decision processes set up repeated commands,the radio will
schedule transmission of each in a round-robin fashion. If
the queue of non-repeated commandshas pending requests,
the queuewill also be addedto the schedule.
The second RCis the framegrabberprocess, which is required for autonomousbehaviors. Images are captured by
the framegrabber on the Windowslaptop and are processed
based on the requests of the decision process. Several different image operators were implemented,including frame
differencing, connectedregion extraction and various statistical operators. To cut downon the amountof networkoverhead, all image processing was done by the framegrabberRC
and only the processed image information was passed back
to the decision process.
At the exhibition, the scout robots were demonstratedin
both teleoperated and autonomousmodes. In the teleoperation demo, we showed the rolling and jumping mobility
modesof the robot, as well as demonstratingits usefulness
as a remote camera. In the autonomousdemos, the scouts
served as motion detectors by transmitting video by doing
frame differencing operations and notifying a user’s console
whenevermotion was perceived in the image frame. Wealso
demonstratedthe scout’s ability to servo to a dark object.
For this behavior, imageswere analyzed to find the darkest
meanvalue in the robot’s visual field. The robot was commandedto drive itself towards the darkest objects and keep
themcenteredin its visual field.
Discussion & Lessons Learned
Because the scout robots are completely dependent on the
two RF communicationschannels for operation, any interference on those frequencies can seriously degrade performance. Because the commandRF channel is packet-based
(as opposedto streaming), it is somewhatresilient to local
interference. The video channel, on the other hand, transmits continuousanalog data whichis very sensitive to interference. Corruption in the video signal can makeit nearly
unusable by autonomouscontrol processes. The frequency
that we use for our video transmitters is the 900MHz
industrial/medical band and is, unfortunately, extremelypopular.
Several other groupsat the exhibition used this frequencyto
transmit video (most notably the teleoperated blimp used by
the BotBall competition). Wehad to work out a scheduling
arrangementwith the organizers of the exhibition to share
the communicationsband.
36
The small physical size and battery supply of our robots
severely limits the kind of RFtransceivers that we can use.
Weare also severely limited in the kinds of video capture
devices that wecan use. In our current design of the robots,
digital cameraswere not an option becauseof lack of available hardwaresmall enoughto fit into our robot as well as
lack of computationalpowerto process an image even if we
had such a camera. Moreattention will have to be paid to
this issue in future scout designs. Onesolution will be to
sacrifice robot size in order to include additional or more
powerful computational resources. Somesort of a spreadspectrum or frequency-hopping radio system would be extremely useful as well. Not only wouldthe higher frequencies involvedallow for faster data transmission(critical for
the transmission of video data), the ability to changefrequencies would allow more robots to operate in the same
area and wouldbe morerobust to interference.
Wehope to be able to address someof these issues in the
future as advances in technology make better and smaller
equipmentmoreavailable. Until this happens, we are focusing our efforts on algorithmsfor better video signal processing as well as decision processes which take the inherent
fragility of the RF communicationschannels into account
whencontrolling the robots.
Acknowledgements
Wewould like to acknowledge the University of Minnesota’s Doctoral Dissertation Fellowship programfor partial support of this research and the AmericanAssociation
for Artificial Intelligence for the AAAIRobot Competition and Exhibition Scholarship that allowedus to bring our
robots to AAAI2000 in Austin, TX.
This material is based upon work supported by the Defense AdvancedResearch Projects Agency, Microsystems
TechnologyOffice (Distributed Robotics), ARPA
Order No.
G155, Program Code No. 8H20, Issued by DARPA/CMD
under Contract #MDA972-98-C-0008.
References
Balch, T. R., and Arkin, R. C. 1998. Behavior-basedformation control for multiagent robot teams. IEEETransactions
on Robotics and Automation 14(6):926-939.
Baumgartner, E.; Wilcox, B.; Welch, R.; and Jones, R.
1998. Mobility performanceof a small-bodyrover. In Int’l
Symposiumon Robotics with Applications, WorldAutomation Congress.
Burgard, W.; Moors, M.; Fox, D.; Simmons, R.; and
Thrun., S. 2000. Collaborative multi-robot exploration.
In Proc. of the IEEElnt’l Conferenceon Robotics and Automation, 476-481.
Cao, Y.; Fukunaga, A.; and Kahng, A. 1997. Cooperative
mobile robotics: Antecedents and directions. Autonomous
Robots 4(1):7-27.
Caprari, G.; Balmer,E; Piguet, R.; and Siegwart, R. 1998.
The autonomousmicro robot ALICE:A platform for scientific and commercial applications. In Proc. of 1998
lnt’l Symposium on Micromechatronics and HumanScience (MHS’98).
Chemel, B.; Mutschler, E.; and Schempf, H. 1999. Cyclops: Miniature robotic reconnaissance system. In Proc.
of the IEEEInt’l Conferenceon Robotics and Automation,
2298-2303.
Dillmann, R.; Kaiser, M.; Wallner, E; and Weckesser, P.
1995. PRIAMOS:An advanced mobile system for service, inspection, and surveillance tasks. In Modellingand
Planning for Sensor BasedIntelligent RobotSystems, volume21 of Series in MachinePerception and Artificial Intelligence. Singapore:WorldScientific.
Everett, H. R., and Gage, D.W. 1999. From laboratory
to warehouse: Security robots meet the real world. Int’l
Journal of Robotics Research 18(7):760-768.
Fukuda, T.; Kawamoto,A.; and Shimojima, K. 1996. Acquisition of swimmingmotion by RBFfuzzy neuro with
unsupervised learning. In ALIFEV, 31-37.
Group, O.M. 1998. The CommonObject Request Broker: Architecture and Specification. Object Management
Group.
Halme, A.; Sch6nberg, T.; and Wang, Y. 1996. Motion
control of a spherical mobilerobot. In 4th lnt’l Workshop
on Advanced Motion Control.
Hougen, D. E; Benjaafar, S.; Bonney, J. C.; Budenske,
J. R.; Dvorak, M.; Gini, M.; Krantz, D. G.; Li, P. Y.;
Malver, E; Nelson, B.; Papanikolopoulos, N.; Rybski,
P. E.; Stoeter, S. A.; Voyles, R.; and Yesin, K. B. 2000a.
A miniature robotic systemfor reconnaissance and surveillance. In Proc. of the IEEElnt’l Conferenceon Robotics
and Automation, 501-507.
Hougen, D. E; Erickson, M. D.; Rybski, P. E.; Stoeter,
S. A.; Gini, M.; and Papanikolopoulos, N. 2000b. Autonomousmobile robots and distributed exploratory missions. In lnt’l Symposium on Distributed Autonomous
Robotic Systems.
Inaba, M.; Kagami, S.; Kanechiro, E; Takeda, K.; Tetsushi, O.; and Inoue, H. 1996. Vision-based adaptive
and interactive behaviors in mechanical animals using the
remote-brained approach. Robotics and AutonomousSystems 17:35-52.
Kajiwara, T.; Yamaguchi,J.; Kanayama,Y.; Yuta, S.; and
Iijima, J. 1985. Developmentof a mobile robot for security guard. In Proceedingsof the 15th Int’l Symposiumon
Industrial Robots, volume1, 271-278.
Kochan, A. 1997. HelpMateto ease hospital delivery and
collection.tasks, and assist with security. Industrial Robot
24(3):226--228.
Martin, E G. 1998. The HandyBoardTechnical Reference.
MITMedia Lab, Cambridge, MA.
Matari6, M. 1997. Behavior-based control: Examplesfrom
navigation, learning, and groupbehavior. Journal of Experimental andTheoretical Artificial Intelligence 9(2-3):323336.
Mondada,E; Franzi, E.; and Ienne, P. 1993. Mobilerobot
miniaturisation: A tool for investigation in control algorithms. In ExperimentalRoboticsI11, Proc. of the 3rd Int’l
37
Symposium on Experimental Robotics, 501-513. Kyoto,
Japan: Springer Verlag, London.
Osipov, A.; Kemurdjian, V.; and Safonov, B. 1996. Mobile robots for security. In Proceedingsof the 1996 2nd
Specialty Conference on Robotics for Challenging Environments, RCE-H,290-295.
Parker, L. E. 1996. Onthe design of behavior-basedmultirobot teams. Journal of AdvancedRobotics 10(6):547-578.
Porteous, J. 1995. Intelligent buildings and their effect
on the security industry. In Sanson, L. D., ed., IEEElnt’l
CarnahanConference on Security Technology, 186--188.
Pritchard, D.; White, R.; Adams,D.; Krause, E.; Fox, E.;
Ladd, M.; Heintzleman,R.; Sprauer, P.; and MacEachin,J.
1998. Test and evaluation of panoramicimaging security
sensor for force protection and facility security. In IEEE
lnt’l CarnahanConference on Security Technology, 190195. Larry D. Sanson, ed.
Rybski, P. E.; Larson, A.; Lindahl, M.; and Gini, M. 1998.
Performanceevaluation of multiple robots in a search and
retrieval task. In Workshopon Artificial Intelligence and
Manufacturing, 153-160.
Rybski, P. E.; Stoeter, S. A.; Erickson, M. D.; Gini, M.;
Hougen,D. E; and Papanikolopoulos, N. 2000. A team of
robotic agents for surveillance. In Proc. of the lnt’l Conf.
on AutonomousAgents, 9-16.
Wu, A. S.; Schultz, A. C.; and Agah, A. 1999. Evolving
control for distributed micro air vehicles. In Proc. IEEE
Int’l Symposiumon ComputationalIntelligence in Robotics
and Automation.