From: AAAI Technical Report FS-96-05. Compilation copyright © 1996, AAAI (www.aaai.org). All rights reserved.
TheTAO
Project: Intelligent wheelchairs
for the handicapped
Takashi
Gomi
Applied AI Systems, lnc. (Add)
340 A4arch Road, Suite 600
Kanata, Ontario, CANADAK2K 2E4
E-mail: gomi@dpplied-,4Lcom
HomePage: ht~o://fox.nstn.ca/-aai
Abstract
AnR&D
project to build a series of intelligent autonomouswheelchairs is discussed. A standardized autonomy
management
systemthat can be installed on commerciallyavailable well-engineeredpowerchairs has beendeveloped
and tested. Abehavior-basedapproachwasused to establish sufficient on-boardautonomy
at minimalcost and material
usage,whileachievinghighefficiency, maximum
safety, Iransparencyin appearance,and extendability.So far, the add-on
systemhas beeninstalled and tried on twopowerwheelchairmodels.Initial results are highly encouraging.
1. Introduction
In recent years, with the conceptof applyingrobots to service
tasks [Gomi,92] and with the accelerated rate of aging of the
populationbeing reported in manypost-industrial countries,
demandfor more robotic assistive systems for people with
physical ailments or loss of mental control is expected to
increase. This is a seeminglymajorapplication area of service
robots in the near future. For the past five years, we have
been developing a range of autonomousmobile robots and
their software using the behavior-based approach [Brooks,
86] [-Maes, 92]. In my experience the behavior-based
approach [Brooks86, 91a][Steels, 93] [Pfeifer and Scheier,
96] [Maes, 92] allows developers to generate robot motions
whichare more appropriate for use in assistive technology
than traditional Cartesian intelligent robotic approaches
[Gomi, 96a]. In Cartesian robotics, on which most
conventional intelligent robotics approaches are based,
planning for the generation of motion sequence and
calculation of kinematics and dynamics for each planned
motion occupy the center of both theoretical interest and
practice. By adopting a behavior-based approach, I felt,
wheelchairs which can operate daily in complexreal-world
environments with increased performance in efficiency,
safety, and flexibility, and greatly reduced computational
requirements can be built at less cost. In addition,
improvementsin the robustness and graceful degradation
characteristics wereexpected.
In the summerof 1995, an autonomymanagementsystem for
a commercially available Canadian-madepower wheelchair
was successfully designed and implemented. The system
looks after both longitudinal (forward and backward) and
angular (left and right) movements
of the chair as well
limited vocal interactions with the user. The results were
exhibited in August1995at the Intelligent WheelchairEvent
28
organized by David Miller at the International Joint
Conference on Artificial Intelligence (IJCAI’95) held
MontrealDespite a very short period (a little over a month),
the chair performedremarkablywell at the exhibition.
Encouragedbythe initial success, I implemented
a three year
plan to develop a fully autonomouspowerwheelchair for use
by people with various types and degrees of handicap. The
intelligent wheelchair project, now called TAOProject,
intends to establish a methodologyto design, implement,and
test an effective add-onautonomymanagement
systemfor use
in conjunction with most commoncommercially available
powerwheelchairs. In order to demonstratethe principle, the
project will build, during its life, an autonomymanagement
system for four well-established electric wheelchair models
currently available on the market throughout North America
and Japan.
In late 1995, a sister R&D
companywas established in Japan
exclusively for the development of intelligent robotic
technologiesfor the disabled. Withthe initiative of this new
R&Dgroup, the development of TAO-2 autonomous
wheelchairbegan in the spring of 1996.
Based on the experience gained in the past year, methods
used and some issues related to the application of the
behavior-basedapproach to realize an intelligent wheelchair
and possibly other assistive technologies are discussed.
2. Desirable characteristics
of robots for the
handicapped
2.1 Background
Since around 1992, AAIbegan a number of exchanges with
people with various handicaps and Individuals whoassist
them. This was proceeded by a few years of on-going
Interactions with the handicapped community through
marketing,installing, servicing, and training individuals on a
speech-to-text voice interface system for computers. This
device provedto be effective for people with several types of
handicap, particularly for individuals whohad lost arm/hand
usage. Since late 1995, voluntary work has been attempted
by membersof AAIat two institutions for the mobility
handicappedin Japan: a senior citizen’s hospice for severe
physical/mental problems,and an institution for people with
severe physical handicaps. A considerable amountof time
practising physical assistive workhas been carried out by
membersof the R&Dteam, including the designer involved
in the conceptual design of the robots, engineers and a
technician responsible for the construction of the robots, and
the project manager and administrators of the robotics
projects. In early 1995, an individual with a severe physical
disability (a quadriplegie) joined AAIas a regular data
entry/bookkeepingclerk and as a future tester of autonomous
wheelchairs.
Basedon these exposures, as well as earlier volunteer work
by myself and someof mycolleagues, a preferable approach
to robotics for service tasks [Gomi,96b]and a tentative list of
deskablecharacteristics for future robots built for the purpose
of interacting directly with severely handicappedor fully
disabled individuals has been compiled. Some of the
desirable characteristics are discussed below.
2.2 Softness and flexibility
Establishment of rapport between the handicapped person
and the earegiver is essential for a care to be successful. So
muchso, there will be a great deal of anxiety in the mindof
those treated by future robotized arms, support boards, and
wheels.Theneedfor set’messrealized at the physical interface
of the end effectors of such a robot and the humanbody
surface or limbs does not stop at simple paddingof otherwise
solid effector surfaces, or use of softer materials, or passiveor
active compliance of effectors. The softness must also be
architectural in that the entire physical supportstructure must
be able to alter, reconfigure, and evencompletelyrestructure
momentto momentreactions and responses to accommodate,
whenevernecessary, changesinnot only the physical but also
the perceivedpsychologicalsituation of the user.
Theflexibility of the systemas a whole,as well as that of the
end effectors, must essentially comefrom this "structural
softness". The flexibility must be foundedon the opennessof
the design of motionsthe systemcan generate so that it does
not rely on fixed modesof operation or rigid scenarios
defined a priori. Humans in general and in most
circumstances behave without a prepared set of motion
patterns, and since we are dealing with such an existence, a
man-madesystem itself must not act with a fixed set of
motionswhichare algorilhmieally describable. This places the
appropriateness of most existing systemcontrol methodsin
doubt as a tool to seriously deal with manytypes of physieaUy
handicapped people.
Learning has often been hailed as a schemewith which a
29
system can be mademore adaptable. Wewouldalso have to
question this relishable notion as a candidate that would
sufficiently increase adaptability of systems such as service
robots dealing directly with humans. Learning schemes,
particularly those so far studied to a great extent and depth in
symbolic AI community, have failed to make significant
contributions to robotic systems operating in highly dynamic
application areas. In general, learning research has focusedon
methodsto improvethe chosen performanceindex of systems
but variables involved in the schemeare most often not
groundedthrough sensors or actuators.
2.3 Fail safe and robustness
A robot arm holding a fragile humanbody cannot drop the
person whena bugis hit for the first time. Theconceptof fail
safe implies readiness of a systemagainst possible failure. In
traditional system engineering disciplines, such as Fault
Tolerant ComputerSystems(FTCS)research and practice, this
typically translates into the preparation of additional
capabilities in the form of a standbyin computerhardwareand
software. The concepts of hot-standby and cold-standby are
commonly
employedin system design. Since it is impossible
to prepare for every possa’ole failure, the provision of
readiness should exist, however, more in the form of
capabilities spread across the system in atomic form and
meshedfine grain with the competencestructure which also
functions in the normalexecutionof tasks. This is analogous
to the wayreadiness to failure is implementedin life forms
found in nature. If a small animal or an insect temporarily
loses the use of a limb, it tries to adjust to the situation by
immediatelyenlisting the use of other limbs or even other
portions of the body. Theadditional capability readied in this
formwouldbe quickly organized and mobilized the moment
a fault is detected.
2.4 Graceful degradation
A cousin to the conceptof fail safe, graceful degradationis
more important in systems that physically interface with
humans~hanin systemsthat deal with materials and artifacts.
A conlrol system designed as a monolith or componentswith
relatively larger granularity would have less chance of
realizing the concept fully. Whenone loses a limb, the
resulting transition is not smooth,causing great suffering to
the individual However,as we lose a large numberof brain
cells every day that we knowwon’t reproduce, we do not
deteriorate or lose capabilities as drastic as loosing a limb.
Systemscomposedof fmer grain active units seem to offer
moredesirable results.
2.5 Evolvability
Anotherreason for the failure of learning in symbolic AI
wouldbe the relatively short time the methodshavetypically
tried to achieve the"resulf’. In fact, weprobably do not know
what desirable results are as muchas we think we do. Both
shortcomings,this and the lack of grounding, are due mostly
to the very nature of being symbolicrather than pragmatic.
In evolution, changesoccur along a muchlonger time scale.
In situated andembodiedsystems, such as life formsin nature
and well-built autonomousrobots, a search through a very
high dimensional space of the real world for adaptation
demands"experiments" on a vast numberof combinations of
dimensional parameters, if such dimensionalization or
parameterization makesense at all. Evolutionary Robotics
(ER) is an emergingfield of science and technology[Harvey,
92], wherephysical or virtual robots’ autonomy
structures are
evolved to achieve collective trans-generational learning. ER
seems to be a schemethat could well be applied to robots
operating to tend and care for humansbecause of the open
nature of humanautonomyand ER’sbasic principle that can
provide long term learning. Here, the concept of learning
should probably be replaced by a more comprehensive
conceptof evolution, whichimplies perpetual adaptation of an
autonomoussystem to a constantly changing operational
environment rather than optimization of one or more
performanceindices of such a system.
2.6 The development plan
The development of autonomous wheelchairs at AAI is
carried out in the following four phases. Someof the phases
overlap in their execution.
(1) The basic safety phase,
(2) The mobility phase,
(3) The humaninterface phase, and
(4) The exploration phase.
Currently, we are in the secondphase of the project which
beganon April 1, 1996.Prior to the start of the project on July
20, 1995, a study was conducted to identify various
requirementsby potential users of the autonomouswheelchair
both in Canadaand Japan through interactions with people
with various types of handicap. Causes of the handicaps we
cameacross included gradual mobility loss by aging, recent
sudden loss of body control due to brain damage, and
prolonged motion limitations and bodily contortion due to
stroke suffered at ayoungage. Theproject continues to enjoy
cooperation from institutions for the handicapped and
individuals with disabilities. The TAO
project is scheduledto
end in the summer of 1998. For a description of the
developmentplan, please refer to [Gomiand Ide, 1996].
3. Implementationof the first prototype, TAO-1
A regular battery poweredwheelchair (a motorized chair)
produced and marketed in Canada (FORTRESS
Model 760V)
was used as the base of the first implementation of the
concept. A set of sensors, a computerized autonomy
managementunit, and necessary harnesses were built and
added to TAO-1(Figure 1) through the summerof 1995.
3O
3.1. Planned functions of the chair
The selection of functions to be implementedon TAO-1was
somewhat
influencedby the rules for exhibition set out for the
IJCAI’95robotics contest. However,the later demonstrations
of our prototype and observations madeat an institution for
the aged confirmedthe guideline was in fact appropriate. Of
the following functions which we nowfollow, only the fLrst
two were attempted at our IJCAI’95entry. However,all free
of themare currently pursued.
(a) Basic collision avoidance
This is achieved by behaviors which monitor and respond to
inputs from on-board ted camerasor those which respond to
active infrared (IR) sensors. Whenthe chair encounters
obstacle, it fkst reduces its speed, and then dependingon the
situation it faces, stops or turns awayfrom the obstacle to
avoid hitting it. Theobstacle can be inanimate(e.g., a column
in a halhvay, a light pole on the sidewalk, a desk, a standing
human)or animate (a passerby, a suddenlyopeneddoor in its
path, an approaching wheelchair). Encountering a moving
obstacle,the chair first hies to steer aroundit. If it cannot,it
stops and backs off if the speed of the advancingobstacle is
slow enough(e.g., 20 centimeters per second). Otherwise,
stays put until the obstacle passes away. Thus, if the chair
encountersanother wheelchair, both chairs can pass each other
smoothlyas long as there is enoughspace in the passage for
two chairs. A fast paced humanusually does not affect the
chair’s progress and at most causes the chair to temporarily
slow downor steer away.
(b) Passage through a narrowcorridor
Whensurroundedby walls on each side of the path, as in a
hallway, the chair travels autonomouslyfrom one end to the
other in parallel to the walls.
(c) Entry through a narrow doorway
The chair automatically reduces its speed and cautiously
passes through a narrow doorway which mayleave only a
fewcentimeters of space on each side of the chair. Sometypes
of ailment such as Parkinson’s disease or polio often deprive
a humanof the ability to adjust the joystick of a power
wheelchairthrough such a tight passage.
(d) Maneuverin a fight comer
Similarly, whenthe chair is surroundedby obstacles (e.g.,
walls, doors, humans), it is often dil~cult to handle the
situation manually. The autonomouschair should try to fmd
a break in the surroundings and escape the confinementby
itself unless instructed otherwiseby the user.
(e) Landmark-basednavigation
Twoccd color cameras on-board the chair are used for
functions explainedin (a), (b), and (c) above. Theyconstantly
detect the depth and size of free space aheadof the chair. The
cameras are also used to identify landmarks in the
environment so that the chair can travel from its present
location to a given destination by tracing them. An on-board
topological mapis used to describe the systemof landmarks.
3.2 Hardware Structure
As a standard powered wheelchair, model 760Vhas two
differentially driven wheels and two free front casters.
Although they are designed to rotate freely around their
vertical and horizontal axis, these casters typically give
fluctuations in delicate maneuvers due to mechanical
hysteresis that exists in thembecause of design constraints
(the rotating vertical shaft of the supportstructure of the caster
cannot be at the horizontal center of the caster). This
sometimescauses the chair to wiggle particularly whenits
orientation needsto be adjusted f’mely. Suchline adjustments
are necessary typically whena wheelchair tries to enter a
narrow opening such as a doorway.
Theentire mechanicaland electrical structure, the electronics,
and the control circuitry of the original powerwheelchair
were used without modification. The prototype autonomy
managementsystem still allows the chair to operate as a
standard manuallycontrolled electric wheelchair using the
joystick. The joystick can be used anytime to seamlessly
override the control whenever the user wishes even in
autonomy mode.
Physical additions to the chair were also kept to a minimum.
AI components added to the chair were made visually as
transparent as possible. Twoprocessor boxes, one for visionbased behavior generation and the other for non-vision
behaviorgeneration are tacked neatly underthe chair’s seat,
hiddencompletelyby the wheelchair’soriginal plastic cover.
Sensorsare hiddenunderthe footrests, inside the battery case,
and on other supporting structures. Onlytwoccd camerasare
a little morevisible: they are attached to the front end of two
armrests for a good line of sight. A small keypad and
miniaturetelevision set are installed temporarilyoverthe left
armrest to enter instructions and for monitoring.
The non-vision behavior generator is based on a Motorola
6833232-bit micro controller. A multi-tasking, real-time
operating systemwas developedand installed as the software
framewolk.This combinationgave the systemthe capability
to receive real-time signals from a large numberof sensors
and to send drive outputs to the two motorswhichgovernthe
wheels. The chair currently has several bumpsensors and 12
active infrared (IR) sensors whichdetect obstacles in close
vicinity (less than 1 meter) of the chair. Signals from the
camerasare processed by a vision-based behavior generation
unit based on a DSPboard developed by a group at MIT.
Vision processingis discussed in Section 5.5 below.
3.3 Software structure
Theover-all behaviorstructure of TAO-l is shownin Figure
3. Smaller behaviors are lumpedup to save space on the
diagram~Softwarefor the vision systemis also built according
to behavior-based principles. The major difference between
this and conventionalimageprocessingis that it consists of
behaviors, each of whichgenerates actualbehavior output to
the motors.It can presentlydetect depthand size of free space,
31
Figure 1 Autonomous
wheelchair TAO-1.Cover is removed
to showautonomyunit.
vanishing point, indoor landmarks, and simple motionsup to
10 meters ahead in its path. Indoor landmarksare a segment
of ordinary office scenery that naturally comesin view of the
cameras. Nospecial markings are placed in the environment
for navigation.
There are also a large numberof behaviors invoked by IRs
and bumperswhich collectively generate finer interactions
with the environmentVision-based and non-vision behaviors
jointly allow the chair to proceed cautiously but efficiently
through complexoffice spaces. Note that there is no main
programto coordinate behaviors.
~tly, the autonomy program occupies about 35 KBytes
for all ofthe vision related processingand 32 KBytesfor other
behavior generation and miscellaneous computation. Of the
35 KBytes for vision related processing, only about 10
KBytesare directly related to behaviorgeneration.Therest are
involved invarious forms of signal preprocessing: generation
ofdepthmap,calculation of the size of free space, estimation
of the vanishingpoint, and detection of specific obstaeles in
the immediatefront of the chair.
Of the remaining 25 KBytes, approximately 20 KBytes are
used in the neuralnetwolksystemfor detecting landmarksand
referencing atopological map. The current implementationof
the landmarksystem consumesonly 256 Bytes per landmark,
although this figure may change in the future as more
sophisticated landmarkdescription might becomenecessary.
The current systemhas space for up to 64 landmarksbut this
can also be adjusted in the future version.
Of the 32 KBytesof non-vision processing (i.e., processing
of inputs from IR’s, bumpsensors, voice I/o, etc.), again no
morethan several KBytesare spent for generating behaviors.
Altogether, there are some 150 behaviors in the current
version of TAO-1.A considerable amountof code has been
wlitten to deal with trivial periphery,suchas keypadinterface,
voice I/o, and LCDdisplay. The comparableinefficiency of
ceding is because these non-behavioral processing had to be
described in more conventional algorithms.
for a public demonstration in the town of Kosaka, Akita
Prefecture.
4. The second prototype,
TAO-2
Encouragedby the success of TAO-1,in late 1995 a sister
company of AAI (AAI Japan, Inc.) was established in
northern Japan. AAIJapan is dedicated to the developmentof
advanced intelligent robotics to aid people with various
handicaps. In May1996, AAIJapan purchased anew power
wheelchair (Suzuki MC-13P),which is a model widely used
in Japan. MC-13P
has a form of powersteering in which the
twofront casters alter their orientation in synchronywith the
drive wheelswhena turn is indicated by joystick. The servo
controller also halts the inside turn wheelof the two drive
wheels while the chair is makinga tight turn. This is a
significant departure from the way the FORTRESS
model
makesa turn. The latter simply turns the two differentially
driven mainwheelsin opposite directions, allowingthe chair
to turn in place. The intent of providing a powersteering
feature on the Suzukichair is obviously for ease of use, and
the user is freed from the wiggly caster problemdescn’bed
above. However,this prevented the chair from makingturns
in a tight turn circle. The feature wasfelt undesirablefor an
autonomouschair.
Immediatelyfollowing the purchase of the Suzukichair, the
development team began building an autonomymanagement
system for TAO-2;a new prototype autonomouschair based
on MC-13P.The over-all computer hardware and software
structures as well as sensors are almost identical to those for
TAO-1, except for a few changes listed below to
accommodate the above mentioned and other minor
differences in characteristics.
(1) The behaviors responsible for turning TAO-2needed
their parametersadjusted.
(2) The locations of touch sensors madeup of lhin piano
wires needed to be moved forward in order to
compensate
for a larger turn circle.
(3) The back bumperwas not activated since it was hardly
used. The difference in turning characteristics reduced
the chance of the Suzuki chair performing frequent
switch backs.
(4) Two prominent side bumpers were added to protect
bystanders when the chair makes a turn in their
directions. This wasnecessitated by the lack of structure
on which to mountsensors.
TAO-2is shownin Figure 2. It was fitted with the autonomy
management
system at AAIin Canadain the span of a week.
After twodays of testing, it wasshippedbackto Japanin time
32
5. Evaluation of the prototypes
5.1 Demonstrations
WhenTAO-1was demonstrated at IJCAI’95 in Montreal on
the 22rid of August, itwas the 33rd day of the developmentof
the first prototype. Everythingfrom the motherboard,vision
system, sensor arrangementsand their harnessing, operating
system (based on an earlier prototype), a large number
behaviors (some 60 by that time) were all developed and
Figure 2 TAO-2autonomouswheelchair
tested in that period. The chair could performfunctions (a)
and (b) in Section 3.1 well and functions (c) and
moderatelywell although they were not initially targeted.
Function (e) wasnot yet implemented.In all, it performed
well as other chairs at the exhibition mostof whichtook much
longer time to develop. All five functions are now
implemented on TAO-1and are undergoing continuous
improvement.
TAO-2was demonstrated on June 4, 1996 at a gymnasiumof
a local school in Kosaka. The chair ran smoothlythroughout
the 1 hour demonstrationpersistently avoiding by-bystanders,
other obstacles and the walls. Unsolicited, a severely
handicappedspectator whocould not even reach the joystick
volunteeredto test ride the chair. The chair performedto her
satisfaction and excitementas it went through the gymnasium
amonga large numberof spectators.
The success of the twoprototypes suggests that myintention
to build a standardized add-onautonomyunit is a valid one.
The concept has at least been provenin two powerwheelchair
types whichcomefrom drastdcaI~ different backgrounds.The
divergence in design philosophy and practical variances in
~] Speaker
Voice handler
/~ Joystick
r con~’o]ed)
vlslonsystem
Cameras
~a
Vanlshrng
po~t
I
Motors
I
@’°"’,
Depthmap
r~s- Frontovoid
L
LCD
j/
Cameral Keypad
~Itor
I
"- IR~
B4
,~~Rll
1
12
B3
B’left~
’~"
--IR 10
-IR 3
B2
B-right
IR 9
¢
B5
Camera2
Sensor arrangementfor TAO-1
Figure3 TAO-1
behavior
structure(Notall behaviors
arcshown)
implementation,some fairly significant, of a base power
wheelchaircan be absorbedby relatively minorhardwareand
software alterations madeon the standardizedadd-onunit.
TAO-2also showedthat the installation, testing, and
adjuslmentof a separatelybuilt autonomy
unit canbe madein
33
a very short period of time. In both TAO-1and TAO-2,no
cooperationfromthemanufactureswas sought. In eaoh ease,
charaotedsties of the joystick werestudied anda seamless
interface wasdesignedaroundit.
5.2 Development time
The extremelyshort development
time required for the initial
prototype for both TAO-1and TAO-2can largely be
attributed to the behavior-based approach. To achieve the
demonstratedlevel of mobility and flexibility wouldnormally
have required another several months to a few years in
conventional AI-based mobile robotics. In behavior-based
robotics, the operationalcharacteristics of the sensorsneednot
be as precisely uniformas in conventional mobilerobotics.
For example, emission strength and angular coverage of the
emitter, and the sensitivity and shapeof the reception coneof
the receptor of on-boardIR sensors need not be homogeneous
across all sensors, allowingthe use of inexpensivesensors and
simplertesting.
All sensors, including the ccd cameras,neednot be installed
at precise translational and angular coordinates. Theyalso do
not need calibration. They were placed on the chair in a
relatively ad hoc mannerat First, and continually moved
aroundfor better results as the development
wenton. In fact,
the camerasand someof the sensors are attached to the chair
by velcrodetachabletape, so that their location and orientation
can be adjusted easily. Suchloose treatmentof sensors is not
common
in conventional robotics where robot’s motions are
derivedafter high-precisionmeasuremeuts
of the relationships
betweenits extremities and environment.The large tolerance
for signal fluctuation is due also to flexibility of processing
and greater adaptability inherent in Subsumption
Architecture
[Brooks,86].
With the absence of "sensor fusion", sensor inputs are
directly linked to motor output only with a simple signal
transformation and amplification (e.g., from sensor output
voltage to motordrive current). The developer only needs to
adjust the appropriateness of the definition and performance
of the sensor-action pair or behavior in terms of its output
without a detailed and precise analysis of input signal
characteristics and elaborate planning and computation of
output signal generation. Readers not familiar with the
theoreticalbasis of behavior-basedAI are encouragedto read
[Brooks,91b]. Thesetheories are fuUyput into practice in our
development.
5.3 Software structure
During development,sensor-actuator pairs or behaviors are
simply "stacked up". Theyare addedto the system one by one
without muchconsideration for the design of the over-all
software structure. Our operating system provided an
adequate frameworkfor the incremental developmentprocess
allowing for shorter developmenttime.
Thus, software developmentwent totally incrementally side
by side with finer adjustment of the sensors. Only general
functions neededassigned to each sensor-actuator pair type
first. For example,depth map- motorpairs are excellent for
dealing with obstacles that suddenlyappear in the path of the
chair a few meters away. But the same sensor-actuator pair
type is not at all effective for the management
of the situation
in whichthe chair has actually madephysical contact with an
obstacle.
Sometimes,competition or contradiction occurs betweentwo
or morebehaviors. Suchcontradicting definitions of behavior
areinmost cases easily observable and corrected quickly. An
example of more complex contradiction occurs when two IR
collision-detection sensors placedon left and right front sides
of the chair detect an approaching doorway in quick
succession~ Since the doorwayis normally quite narrow, the
....
:
offnebul[d~
m
::~::::::
:~....................
~::
....~iiiiiiii~ii~
::iiiiiiii~iiiii~iiiiiif
~iiiiii:J
==============================
....
®
::::i::iii::iiiiiiiiiiiiiii::i::i~
"%~iiiiiiiiiiiiiiiiiiii
........
~::::::
cow .... ~iiiiii~i
......
~#i::i::i::i~::~::
o9
%iiiii
_
==============================================================
....~~
Figure 4 Theoffice space whichcontains the test loop
34
reflection of infrared signals
received by these sensors is
usually strong enoughto cause
the chair’s immediate evasive
action. As both sensors react
alternatingly, the chair can get
into an oscillatory motion,
commonly
known
as
"Braitenburg’soscillation" after
[Braitenburg, 84]. In this
specific
situation,
other
frontally-mounted IR sensors
take in "just go ahead" signals
that invoke behaviors which can
breakthe tie.
200
0
200
0
t~
a:
2O0
0
200
0
IR4
".51--~j1
~jAl~l’~’--°---~e
~----~j~l~
I R3
flv
_IR2
200
0
200
IR1
I
0
(11
(21
I
(3)
I
(4)
(5)
(6)
Figure5a Outputof active infrared OR)sensors
runningfast, medium,and slow speed, respectively. In another
5.4 Priority scheme
modeof obstacle detection using an IR, changesin value are
The priority arrangement is shownin top fight comerof
monitored for several sensor cycles. If the change is
Figure 4, whereseveral output lines to the motorsare joined
sufficiently large, detection of an obstacle is reported. TheIR
by O nodes or suppression nodes. Input from the left of the
sensors take invalues at 64Hzand several consecutive values
node is suppressed and replaced by one comingin vertically
are compared.Onceinvoked, a behavior corresponding to a
wheneverit is active. Inputs fromthe joystick take the highest
specific IR sensor generates a predeterminedreactive motion,
priority in deciding whichaction the chair should take. The
altering the speedand orientation of the chair.
electronically and mechanically seamless interface between
the joystick controller and the autonomymanagement
system
allows the chair to nmas a standard powerwheelchairsimply
5.5 Vision processing
by operatingthe joystick. Thesecondhighest priority is given
Inputs from 2 cc, d cameras are alternatively processed
to behaviorswhichtake in signals fromleft and fight bmnpers
through a single flame grabber into two primary vision planes
and somekey frontal IR sensors. Behaviors are bundled up
of 256 x 128 pixels each at about 8 frame sets per second.
in Figure 4 with implied logical relationships amonginput
Imagesin these primary vision buffers are averaged downto
lines to simplify the diagram. After several groups of
64 x 32 pixel secondary vision plane by combiningthe left
behaviorsthat mostlydependtheir invocation on signals from
and right vision inputs after dividing each primaryplane into
IR sensors, behaviorsinvokedby signals from the voice input
left, center, and fight. All vision processingdescribed below
systemare arrangednext, followedby vision-driven behaviors
occur using imagedata in this secondaryvisual plane.
as the lowest priority behavior groups. They are, in
Figure 5b plots three depth values (Y1, Yoand Yr) in terms
descendingorder of priority, depth map,vanishingpoint, and
of the number of pixels in the secondary visual plane
determinedaccording to HorswiU’shabitat constraint vision
free area.
Figure 5a showsIR signals from a test run in which TAO-1
processing [Horswill, 92]. In the absence of active bumper
wentaroundthe test loop in our office floor shownin Figure
and IRinvokedbehaviors, the parameterset directly dictates
3 (shaded area). Note that signals from only 6 of the 12
the orientation and speed of the wheels.
sensors are plotted here. The x axis in Figures 5a
# pixies
[3 o-5
[] 5-10
¯ 10-15 [] 15-20 [] 20-25 [3 25-30
through 5d shows the passage of time and its
length corresponds to the time required to
complete the loop from the workshopwhere the
chair began its run and back there counterclockwise. Note that checkpoints (1) through (6)
shown in Figure 3 are also marked on the
diagrams. Whenthere is no reflection from an
obstacle, output of an IR is kept at 255.
111 121
(31
(41
(51
(6)
Depending
on the strength of the reflected signal,
Figure 5b Depthmapparametersfrom the vision subprocess
a receptor mayreport lower values, 0 being the
lowest. Whenthe value becomes less than a
threshold, the sensor wouldhave "detected an obstacle." The
Outputfromthe vanishingpoint detector of the vision system
thresholdis set as a functionof speedof the chair, andin this
is shown in Figure 5c. The detector attempts to find a
specific test set at 210, 180, and 150, for whenthe chair is
vanishingpoint in the secondaryvisual plane and outputs its
IIIllllNliilli llt ll
IlJl
35
domestiotranaportation needs.
In April 1996, TAO-1was brought to a local shoppingmall in
Ottawa to freely roam around for an hour or so. TAO-1
skilfully skirted all internal structures of the mall such as
esoalators, flower pots, benches,signs, and showcases,as well
as allemoun shoppers. TAO-1
andits rider visited stores as if
lOO ~C~O
00~
~
O
O~
¢~>
he was window shopping or
X 9o
Bestfit lines sloped sameway just strolling the mall. Virtually
Invalid Vanishing
vanishing point set to appropriate
point- use Area to
8o
all fellow shoppers failed to
edge
turn
70
notice
that it was not driven
"6
Q.
manually.
This made me feel
6O
Mid
point
@
t:D
e°_
that with proper engineeringto
e5O
0
make the chair further
to
C
0
4O
0
t~
o ~>o
dependable, it can already
>
30
O
O
serve as an autonomouschair
O
o~o
t20
O
in limited application areas,
O
lO
O O
such as strolling or window
O
O
0
~ zt~D
0 ~>
IX.
shopping for the severely
(3}
(4}
(5)
t6~
(1) (2)
handioapped.
After
the
modest
Figure 5c Output of vanishing point detector
demonstration of TAO-2in
Japan, which was reported in
local television newsand several looal newspapers,we have
Figure 5d depicts output from the area deteetor. The number
receivedinquiries for the chair’s availability. Needlessto say,
of pixels representingfree space in the left, center and right
it wouldbe at least a few moreyears before even a modestly
visual fields are calculated by the detector, and steering and
autonomouschair can be released for use by the handicapped
speed of the chair are determinedby the size of available
at large and put into daily use only with affordable amountof
space as in depth mapprocessing. The behaviors associated
support. Maintenancewouldbe another issue if we proceed,
with area detection are invokedonly whenall other behaviors
not to mention various public liability
issues that,
are not invoked.
unfortunately but undoubtedly, foUow. I am not at all
As the project proceeds the vision systemwill be enhanced
optimlstio about the efforts required to establish an
to detect moreobjects and events such as outdoor landmarks,
infrastructure
for physical and moral support that
indoor landmarks that change in time, more complex and
encompasses
all
these
and other yet to be found issues.
dynamicobstaoles, and traffic signals in the path. In the fall
Nevertheless,
I
can
foresee
that we will be able to answer,in
of 1996, a popular Americanwheelchairwill be fitted with the
the near future, someof the sincere wishes that already come
autonomy managementsystem to become TAO-3.
from people whowouldbe most benefitted by the technology.
Gettinginto technioal issues, the list of things
yet to be done is still quite long. Landmark
# pixels
¯ 400-600
[] 0-200
[] 200-400
detection, for example, requires a lot more
left
work. Although
we have su~ed in
navigating the chair to go through a series of
landmarks arbitrarily chosen in the chair’s
present operational environment,this is still a
far cry frombeing able to state that it can run
freely in any environment traversable by a
141
151
11) 12~
131
wheelchair by detecting landmarks.
Apartfrom these and other shortcomings,I feel
Figure 5d Output of the area detector
the technologyas it is, is alreadyuseful in real
world applications by individuals with certain types of
6 Lessonslearned so far fromthe chair project
handicap. Persons with bodily contortions such as those who
Althoughthe experienceis stiU very limited, I can state that
suffered polio in earlier life, or individuals with involuntary
there is a strong expectation amongthe population for the
hand/annmovements
such as patients of Parkinson’s disease,
developmentof an autonomouswheelchair for assisting and
nowcould travelthrough confined and narrow spaces such as
eventually fully taking care of the handicapped person’s
corridors and doorwayswithout assistance. Other interface
x axis value whenit finds one. Thevalue 0 correspondsto the
left mostangle in the visual plane, and 63 to the right most.
Whenit fails to comeup with a vanishing point, value 99 is
output. Thecombinedhorizontal viewingangle of the left and
the right camerasis approximately100 degrees.
~:
~
36
mechanismssuch as neck control and voice recognizer would
also makethe inlroduction of the autonomouschair easier.
Less handicappedusers can use the chair as a manualpower
wheelchair whenever desired, while autonomymanagement
can assist in mundanesituations and emergencies.
Everybodywe have interfaced with so far, from a passer-by
at a shoppingcenter whereTAO-1was tested, fellow robotics
researchers, several handicappedpeople and caregivers who
heard about the project and cameto see and even volunteered
to try an early prototype,willing investors, to journalists all
gave us positive feedback. They agree in principle that
mobility should be provided as much as and as soon as
possible to those whootherwise are not capable of going to
places by themselves. Althoughthe developmentis still far
from complete, TAO-1and 2 have so far been covered by
several TV programs and a few dozen newspaper and
magazine articles in Europe, Japan, USA, and Canada,
indicating the keen level of interest the public has on this
subject.
I wishto report our next interim results at IJCAI’97(Kyoto,
Japan) and other occasions aroundthat time so that results
can be shared amongresearchers and the concept of the
autonomous wheelchair which is manually overridable
becomesfamiliar to the populationat large. Needsof the users
and capabilities that could be providedby the technologyare
matchedand better understood through frequent disclosures
through technical papers, presentations, and serious media
coverage.
7. Conclusions
Two prototype autonomous wheelchairs based on
commerciallyavailable motorizedwheelchairshave been built
using behavior-based AI. The initial prototyping went very
rapidly and the size of the software is significantly smaller
than control programsfor similar vehicles operating in real
world environment implementedusing conventional AI and
robotics methodologies.Oneof the chairs is nowcapable of
travelling to its indoor destinations using landmark-based
navigation~The performanceof the prototypes indicates there
is a cautiouspossibility todayto build a functionalintelligent
wheelchairthat is practical and helpful to peoplewith certain
types and degrees of handicap.
Acknowledgements
KoichiIde is responsible for the detailed design and most of
the implementation of both TAO-1and TAO-2. Reuben
Martin and Richard Edwardsassisted Ide in implementation
and testing of both chairs. AnnGfifl~th proofread earlier
manuscriptsof this paper and aided the author in editing.
References
[Braitenberg, 84] V. Braitenberg, "Vehicles: ExperimentsIn
Synthetic Psychology", MITPress, Cambridge,MA.1984.
37
[Brooks, 86] 1L A. Brooks, "A Robust Layered Control
System For a Mobile Robot", 1EEEdournalof Robotics and
Automation, RA-2(1986).
[Brooks, 91a] 1L A. Brooks, "NewApproachesto Robotics",
Science 253, 1991, pp 1227-1232.
[Brooks, 91b] 1LA.Brooks, "Intelligence WithoutReason".In
Proc. IJCAI-91, Sydney, Australia, August1991.
[Gomi, 96a] Takashi Gomi, "Non-Cartesian Robotics",
Robotics and AutonomousSystems 18, 1996, Elsevier, pp
169-184.
[Gomi, 96b] T. Gomi, "Aspects of Non-Cartesian Robotics"
in proc. Artificial Life and Robotics (AROB’96),Japan,
February 1996.
[Gomi, 92] T. Gomi, "Subsumption Robots and the
Application of Intelligent Robotsto the Service Industry",
AppliedAI Systems,Inc. Internal Report, 1992.
[Gomi and Ide, 96] T. Gomi, T., and K. Ide, "The
Developmentof an Intelligent Wheelchair". In proc. IVS’96,
September1996, Tokyo, Japan.
[Gomi et al, 94] T. Gomi, K. Ide, H. Matsuo, "The
Development of a Fully Autonomous Ground Vehicle
(FAGV), Intelligent Vehicles Symposium(1V’94), Pads,
France, October 1994.
[Harvey., 92] I. Harvey, "Evolutionary Robotics and SAGA:
the case for hill crawling and tournament selection", In
Artificial Life 3 Proceedings, C. Langton, editor, Santa Fe
Institute Studies in the Sciences of Complexity,Proc. Vol
XVI, pp 299-326.
[HorswiU,92] I. Horswill, "A simple, cheap, and robust visual
navigation system", Simulation of Adaptive Behavior
(SAB’92), Honolulu, Hawaii, 1992.
[Maes, 92] Pattie Maes, "Behavior-based Artificial
Intelligence", SecondInternational Conferenceon Simulation
of Adaptive Behavior (SAB’92), 1992.
[Pfeifer and Scheier, 96] 1L Pfeifer, C. Scheier, Introduction
to "NewArtificial Intelligence", Ins3itut fttr Informatikder
Universit/it ZtLrich,July 1996.
[Steels, 93] Luc Steels, "Building agents out of autonomous
behavior systems", The Biology and Technologyof Intelligent
Autonomous Agents, NATOAdvanced Study Institute,
Trento, 1993, Lecture Notes.