Search in the Context of Complex Robot-World Interaction
Sanjiv Singh
The Robotics Institute
Carnegie Mellon University
Pittsburgh, PA15213
ssingh@cmu.edu
From: AAAI Technical Report WS-97-10. Compilation copyright © 1997, AAAI (www.aaai.org). All rights reserved.
Abstract
In contrast to typical planningproblems,automatedearthmoving operationssuchas digginga trench or leveling a mound
of
soil posesomeadditionalchallenges.First, soil is diffuseand
thereforea uniquedescriptionof the worldrequiresa verylarge
numberof variables. Second,the interaction betweenthe robot
andthe worldis verycomplex
to the pointthat it is notpossible
to accuratelypredict the effect of a candidateaction, in this
paper, 1 pose planningfor earthmoving
operationsas a problem
of constraintoptimization.I discusswhyon-line searchis necessary and presentexperimentalresults froma testbed wherea
robotis taskedto diga trench.
Introduction
In our laboratory, we develop automation for machines that
operate in industries such as mining and construction. Three
characteristics distinguish such machines. They are capable,
e~cient and safe. Capability has to do with scale--forceful
interaction with the world requires machinesthat can generate
large forces. Efficiency boils downto performinga task as fast
as possible without compromisingoutput. Safety is a matter of
ensuring collision free operation and in some cases avoiding
tipover.
other state, through the use of a reasonably competentagent. To
be considered successful, the agent must independently cause
the transition for a Sufficiently large set of initial and goal
states. There are two maindifficulties with operating in this
particular domain. First, the dynamics of the interaction
between a robot earthmover and the world are very complex.
This meansthat it is difficult to produceaccurate forward models of actions that the robot might choose. Second,since soil
is diffuse, the numberof variables required to uniquely specify
the state of the world is extremely large. In other words, the
configurationspacethat the robot operates in is essentially infinite.
I have formulated the task of planning earthmovingoperations
as a problemof constraint optimization. The nature of the models in this domaindictates the use of search. Further, since the
large configuration space affects the choice of the control
parameters, there is no hope of doingthe search off-line.
Several researchers have adopted a similar view to planning for
diverse tasks such as robot juggling and robot pool playing
(Jordan 90, Moore 92, Schaai 94). Feiten and Bauer have
devised a planner for a mobile robot operating in a cluttered
environmentsthat plans in the space of actions rather than in
Used judiciously, modelscan greatly help in increasing effithe configuration space of the vehicle (Feiten 94). Kelly
ciency and safety. By "models" I mean physical, numerical
describes a methodologyfor navigation of a cross country
models of mechanismsand their interaction with the world.
robot based on like principles (Kelly 95). Myapproach is also
(Modelsof planning and sensing are also useful, but I do not
similar in motivationto workin classical optimal control (Kirk
address themhere.) That is, givefi a candidate input, these mod- 70) in that it seeks to determinethe control signals (plans) that
els predict the corresponding output of a (sub) system.
will both satisfy someconstraints as well as optimizesomepercourse we would like invertible models--models that compute formancecriterion.
the input necessary for a desired effect--but these are rarely
BelowI discuss five principles developed from myexperience
available. Good forward models are often the best we can
with a planner for earthmoving.
achieve. They provide the proaction that is necessary for saccessfui goal directed activity, at least in the environmentsin
¯ Develop a tractable representation. Since the configurawhich we must put our autonomousmachines. Wheninvertible
tion space for this domainis intractable, another frameworkis
necessary. Search is conductedin the space of control paramemodelsare not possible, search is necessary.
ters (action space) that abstracts the prototypical task that the
In this paper I will concentrate on one task---earthmoving-robot must perform. It is also necessary to develop the notion
that highlights the approach espoused above. I have a robot
manipulatorthat is equippedwith a scoop ("bucket") and a sen- of an atomic action and a forward modelthat predicts the effect
of an action on the world. Finally, we need an evaluation funcsor that allows it to measurethe shapeof the terrain aroundit.
I wouldlike the robot to dig a trench in a sandboxas per spec- tion to measuresthe utility of an action.
ification, or perhapslevel a moundof soil. Onthe surface, such ¯ Reduce the search space. On-line search dictates compacmessof the search space. In somecases an analysis of the
a problemis familiar to researchers in artificial intelligence.
The world is in someinitial state and must be modified to some mechanicswill often showthat not all the parametersare inde-
97
pendent.In other cases, if it is foundthat the utility function is
monotonicin relation to one of the parameters, then it needn’t
be searchedfor.
¯ Use force as well as geometry to constrain planning.
Geometryof the world and mechanismprovides useful constraints on the action space. However,force constraints that
arise from the mechanicsof robot-world interaction can effectively represent the boundson whatthe robot can actually do in
a world wherefriction and torques are significant. Noa priori
modelsmaybe available in which case it is necessary to learn
these models from experience.
¯ Use multi-resolution search and heuristics. Abstraction of
the search into levels of varying granularity makesthe search
more efficient. In somedomainsthe addition of commonsense
heuristics, can make the progression of the plans muchmore
natural.
¯ Take plans generated with a grain of salt. If meaningful
models of uncertainty can not be incorporated into the planning, the plans generated through the kind of process described
above should be thought of a means of being proactive. The
plans are best handedto lower levels controls that are tightly
coupled to the world with feedback.
Developing A Tractable Representation
parameters used to define an action that a robot is capable of
executing. Eachpoint in this space represents an atomicaction.
The space can be separated into two sets-- the set of all feasible actions and the set of actions knownto fail. Anaction might
fail becauseit is impossibleto achieve or it results in an undesirable effect¯ The task of an action space planner, then, is a
familiar problemin optimal control:
Maximize~Minimize
h(lk) subject to g(lli)
where hO is a utility function and gO are constraints that
delimit the set of feasible actions and lli spansthe set of actions
that a robot can execute.
Note that the constraints change wheneverthe state of the
world changes. Since we cannot predict the state of the world
in responseto robot actions and there is a large branchingfactor
(manyactions maybe chosenat any step), it is not practically
feasible to consider sequences of digging actions. Hence the
searching will involve identification of only the action that is
optimal over a single planning step.
An atomic action
Since an action space encodes actions, not mechanisms,our
task representation will not include any details about the configuration of the mechanism.Instead, we will encodethe trajectory followed by the excavator bucket (Fig. 2)
To develop a planner for a task such as digging a trench, we
need three things: a search space, a forward modeland an ~aluation function.
The Search Space
Instead of abstracting robot and world state, we could abstract
the task that the robot is to performsuch that atomicactions are
described by a compactset of parameters. The space spanned
by these parametersis called an action space. At every step, the
robot selects fromthe set of feasible actions, one that optimizes
somecost criterion. The chosen plan is guaranteed to satisfy
constraints (e.g. avoidcollisions) as well as optimizea cost criterion (e.g. minimizejoint torques). Fig. I provides a schematic
view of this optimization.
Fig. 2 Manipulatorarm equippedv,~ a bucket, creating a trench.
There are an infinite numberof digging trajectories that could
be used so it is necessaryto consider a smaller set--those that
are of a particular form. For example,we could base the notion
of an atomic action on the patterns used by humanoperators.
There are typically three phases to an operationm penetrate,
drag and curl. Trajectories of this formcan be parameterizedby
a smallerset (Fig. 3).
~ataP~neet
all constraints
andare"optimal"
Fig. l A ~hematlc
viewof constraintoptimization.I
More formally, an action space is spanned by the range of
98
This representation requires six variables and a functional to
represent uniquely: k is the distance from a fixed reference
frame to the point wherethe bucket enters in the soil, a is the
angle at whichthe bucket enters the soil. It travels along this
angle for a distance d!, and then follows a horizontal path for a
distance d2. Finally the bucket rotates throughthe soil along an
Using Force & Geometry to Constrain
,w
r
Planning
Recall that each point in an action space represents a unique
action. Wecan poseconstraintsin the regionsof the spacethat
correspondto plans that the plannershouldstay awayfrom.
/
,,=k
,
Geometric Constraints
....
-d; --
Fill. 3 A typical dig duringa trenchingoperation.A diggingtrajectory
of this formcanberepresented
by (k, oh1, d2, d3, r, PO).
Aplan maybe geometricallyinfeasible becauseit requires a
robot to exceedits rangeof motionsor becauseit violates some
geometriccriterion associatedwith successfulexecutionof the
task.
Theteachability constraint. Fig. 4 showsa graphical depiction of the set of diggingactions that are withinthe workspace
of the robotfor the veryfirst dig. Thesurfacethat is formedby
this three-dimensional
set representscritical pointsthat are on
The Forward Model
the edgeof being reachable. Since the actions are parameterOneinteresting aspect of this problemis that no computation- ized witha larger set of variablesthancanbe visualizedin three
ally tractable forwardmodelsare available. That is, given a dimensions,someof the critically reachableplans are in the
shapeof the terrain, anda candidateaction, there is nohopeof interiorof the set.
predictingthe resultant terrain. Finiteelementmodelsare availjoint limit
able to performsuch simulationsbut since a single run might
take tens of minutesto simulate,it is not possibleto use such
modelsfor the purposesof planning. Wemightsatisfy ourselves with something
less. For reasonableplans it is possible
to approximatethe resulting trajectories. Underthis assumption, it is possibleto calculatethe forcesexperienced
at the cutring edge (using a pseudo-static analysis) as well
approximatethe-amountsoil that might be swept into the
bucket.
arcof lengthd3 andradiusr. Wewill alsoneedto determine
the
valueof P0 (the pitch angle of the bucket,relative to a fixed
coordinateframe)throughoutthe dig.
The Evaluation Function
Typically,for earthmoving
operationsit is essential to optimize
the volumeof soil excavated.Sincethere is often morethan one
limit
actionthat will meetthis requirement,
it is possibleto addother
criteria such as minimizing
the time taken to completethe dig,
or minimizing
the torqueson the joints of robot. Notethat the
evaluationis goodfor only single actions and mustbe recomputedacross the rangeof the action spacewhenever
the state of Fig. 4 Twoviewsof set of feasible plans that meetthe teachability
the worldchanges.I assumethat the state of the world(topol- constraint.Thisset is specificto the joint limits of the robotusedandto
the terrain before the action Is commenced.
Thetmjectddesimplied by
ogyof the terrain) can be measuredusinga sensor.
threepoints are shownexplicitly. Theshadedregion showsthe amount
of soil sweptby eachtrajectory.
Reducing the Search Space
Takennaively, the search spaceas describedaboveis too large
to be searchedon-line. However,someanalysis of the space
can makethe search moretractable. For example,if weassume
that the amountof soil excavatedincreases monotonically
with
d2, its valuecanbe foundeasily if the othervariablesare instantiated. Valuesof P0, r, d3 can be foundfroma mechanical
analysis of contactbetweenthe tool andthe terrain (Singh95a).
Theshapingconstraint. TheshapingconsWaJnt
for trenching
keepsthe trajectory of the bucketfromgoingpast the shapeof
the desiredtrench(dashedline in Fig. 5). Thatis, all trajectories
that extrudepast these boundariesare excluded.
Thevolumeconstraint. Of the geometricallyfeasible actions,
somewill yield a partially filled bucket,somewill result in a
full bucket andyet others will sweepthroughthe terrain to
excavatemoresoil that can possiblybe held in the bucket. A
Theresult is that our action space can be spannedby a compact simple calculation of swept volumeis used to predict the
three-tuple
(k, ft., d!).
amountof soil excavatedby a diggingaction. All actions that
excavatea volumelarger than a preset percentageof the bucket
99
Force Constraints
If a robot earthmoveris infinitely strong, that is, it can muster
any torque required, then it is sufficient to consider only geometric constraints. Morerealistically, for robots with torque
limits, it is necessary to consider the required forces. If we
could estimate the forces required for candidate digs, we would
havea goodcriterion by whichto further restrict the set of digs
that are geometrically feasible. Unfortunately, the interaction
betweenan excavating tool and terrain is complexenoughthat
no simple physics-based models are available to predict the
resistive forces for arbitrary actions and terrain shapes. In earlier workI have showna methodthat learns to predict resistive
forces basedon observationof the resistive forces, tool trajecFill. S Twoviewsof set of feasible plansthat meetthe shaping tories and the shape of the terrain from previously conducted
constraint.Notethat all of thedigsherestaywithinthebounds
of the digging experiments (Singh 95b). Fig. 8 comparesthe geomettrench
that hasbeen
specified
(dashed
line)
ric and force constraints for the very first digging actionmfor
somevalues of the parameters, the robot is not limited geometcapacity, are excluded(Fig. 6).
rically but the corresponding actions can not be achieved
becausethe required forces are too great.
~
Co~lmlmJ
For~ Cmm~m
(a)
I
ml
Fig. 8 Comparison
of geometric
constraintsvs. forceconstraintsfor
onevalueof k.
Fill. 6 Twoviewsof set of feasibleplansthat meetthe volume
constraint.
All trajectoriessweep
onlysweep
upto asmuch
soil that the Todetermineif a candidate dig passes the force constraint, the
bucketcanhold
force prediction function is called. Thepredicted resistive force
and the robot’s trajectory are used to calculate the effective
The set of actions that meet all the geometric constraints is
joint torques required to overcomethe resistive forces. A canshownin Fig. 7.
didate dig is force-feasible if the joint torques due to resistive
forces do not exceedthe torque capability of the robot.
UsingMulti-resolutionSearchandHeuristics
So far I have examinedonly a planar problem, that of digging
a trench as wide as the excavator bucket. For more complex
problemssuch as the excavation of a foundation footing (Fig.
9), wecould abstract the problemsuch that at a coarser level the
ultimate goal is broken downinto a sequence of trenches. The
alternative is to makethe planner global by searching for additional variables that dictate base position. This has two problems. First, the search space becomesvery large and second, it
is difficult to developan efficient sequenceof actions.
~1~7 Twoviewsof set of feasibleplans that meetthe shaping
conslralnt.
Sometimes, there are commonsense procedures that can be
used at a coarser level of planning. For example,whendigging
into a cavity wemight alwaysstart near the top, before digging
further downinto the hole for the same reason as one starts
100
~Cla~
"g.~,~:~,
~ ea~ni
rangefinae~
~./bucket
~_
Fill. I0 Testbed
Fig. 9 Planviewof an excavatorbackhoe
digginga footing. 1"he
machine
startsat A anddigsonesideof thefootingto a desireddepth.
Second,the machine
movesto B fromwhicha secondside can be
excavated.
A third setup(C) is achievedby rotation of the base.
Excavation
of thecenteris achieved
by a fan-likepatternof digsfrom
(D).
sweepingstairs from the top and not the bottom. If relevant,
such procedures can be incorporated into the coarse planning
alleviating the need for search to generate such sequencing.
Taking Plans
Generated
With a Grain
of Salt
For real worldapplications, it is never possible to modelall the
effects of sensors and actuators. At best we might hope to capture first order or secondorder effects. It maybe possible to
fold uncertainty modelsinto the planning directly and limit the
feasible set by the marginof cumulative uncertainty. However,
this is not alwayspossible and after all the modeling,plans are
best thought of as a meansof proaction. Final execution of the
plan is best left to a lowerlevel control schemethat has a tight
coupling with the world. Control schemestypically are applicable in a narrowrange of operating conditions and in this light
planning can be thought of as a meansto specify actions that
fall within the operating conditions that where properties of
convergence hold.
Wewould like as muchfeedback as is possible--- state feedback before planning so as to plan properly, during execution
so that unmodeledeffects can be compensatedfor, and after
executionso as to set the stage for the next plan.
Implementation
duces an imageof the terrain such that the value of each pixei
in the imagecorrespondsto the distance fromthe scanner to the
world. This image is used to build a topographical mapof the
terrain in the sandbox.
The excavation cycle consists of three phases. First, a terrain
mapis producedfrom range images taken by the laser scanner.
Next, the planner chooses a digging action to execute that satisfies geometricand force constraints and optimizesutility criteria. Last, the action is executed by the robot and the cycle
repeats until the terrain is sufficiently close to the specified
goal.
Whenthe search space is extremely large and answers are
needed quickly, at the expense of optimality, a numerical
methodsuch as simulated annealing can be used. In fact I have
used an augmentedversion of simulated annealing to do the
optimization in the past (Singh 91). However,the stochastic
nature of the search means that the solutions found are not
repeatable and it is very hard to performcontrolled tests where
one of the constraints is modified. Further, exhaustive search
(test and generate) with intelligent ordering of constraints
works well enough for our purposes. While in the worst case
this meansthat each constraint has to be evaluated for each candidate point in the action space, in reality testing early for conditions such as reachability prunes the search space.=ffectively.
It is possibleto further select fromthis set basedon other criteria. such as time or joint torques required. For example,Fig. I l
showstwo different trajectories selected for the sameinitial
state of the terrain, given different evaluation functions. Note
that both plans sweepthe sameamountof soil.
Experimental
Wehave developed a testbed to conduct experiments in subsurface sensing and excavation. The testbed consists of a sandbox
(2.5m x 2.5m x lm), a hydraulic robot outfitted with an excavator bucket and force sensor, and a laser range finder. The
setup of our testbed is shownin Fig. 10. Weuse a large industrial manipulator with an end effector payload of approximately 125 Ibs. A small excavator bucket with a volume of
0.01m3 serves as an end effector. The laser range scanner pro-
Results
Wehave conducted over 2000 digging experiments with our
testbed. Fig. 12 showsa progression of one sequence in which
the goal is to producea trench. In this experiment’the digging
actions chosen sweepa volumeof soil very close to the bucket
capacity. However,the actual yield in the bucket is a function
of howcohesive the soil is--digging in loose, granular soil
results is someof the soil spilling out of the bucket. Experimen-
I01
Conclusions
116
.
1
1.6
: ......
Eanhmoving
is an exampleof an application where the interaction betweena robot and the world is significant. To perform
such a task efficiently, we will want to use every trick in the
book. This includes the use of good forward models, mechanical analysis and multi-resolution search.
..........
O.S
0.6
......
(a)
(b)
FIg. 11Twodifferent trajectoriesselectedfor the same
state of the
terrain (a) evaluationlunctlon minimizes
the largest torques(b)
evaluation
functionminimizes
thetotal joint torques.
1.5
I have shownhowthe task of digging a trench can be stated as
a problem of constraint optimization. Since the optimization
happens one action at a time (necessary due to the uncertain
forward models and large branching factor at every planning
step), the planner is greedy and thus requires a higher level
planner that ensures that proper subgoalsare specified.
.,
!
References
(Feiten 94)
Feiten, W. and Bauer, R., "Robust Obstacle Avoidance in Unknown& CrampedEnvironments;’ In
Proc. IEEE international. Conference on Robotics and
Automation, May1994. San Diego.
).s
°n’= I~gel
m,,~,
I
1A vo~um:O.0137
9
9A
q
1.6
).s..........i........:.-.-.:.~.-...... ~........
~a
I.S
oloDa
,,~, you,v,: o.m~
~
,.s 2 2.s
G
.........
:.......
".:.::.:.i:......i.......
3
1 I
.s
2 2.s
i
i........
,.s..........
::.-.:i.:
......i........
~;7 .~.
Dq;DSm volume: o.o130
(Moore 92)
Moore, A. and Atkeson, C., "An Investigation of Memory-baseFunction Approximatorsfor learning
Control;’ Technical Report, MITAI Lab, 1992.
~..o.m~
1 1.5
:)
2.S
......... i ......... ; ............................
i
O~,~8
mveptvt~u~,:0.012S
.8
1
1 .S
2
(Jordan 90)
Jordan, M. I, and Jacobs, R. A, "Learning
to Control an Unstable System with ForwardModeling;’ In
Advances in Neural Information Sciences 2, MorganKaufmann, 1990.
Kelly, A. J., "An Intelligent Predictive
(Kzily 95)
Control Approach to the High-Speed, Cross Country
AutonomousNavigation Problem", Ph.D. thesis, Robotics
Institute, CarnegieMellonUniversity, June 1995.
Kirk, D. E., Optimal Control Theory,
(Kirk70)
Prentice Hall, 1970.
2.S
Fill. 12Thefirst eightstepsin thecreationof a trench.Thedesired
trenchIs shown
by thedashed
line. Theshaded
regionis thevolume
of
soll Onm3) estimated
to beswept
by the excavator
buCkeL
tel results are encouraging. The planner examines approximately 2000plans in a few seconds and almost alwaysfind one
that fills the bucket. Theplanner becomesinefficient as the profile of the terrain approachesthe desired goal. This is because
the form of the prototypical plans does not produce efficient
trajectories. In this case, it is possibleto executeopenlooptrajectories that follow the outline to the desired trench to scoop
out the remainingsoil.
102
(Schnai 94)
Schaal, S., Atkeson C, "Robot Juggling:
Implementation of MemoryBased Learning;’ II~.~R Control Systems,Vol 14 (1), February,1994.
(Singh 91)
Singh, S. "An Operation Space Approach
to Robotic Excavation;’ In Proc. IEEESymposiumon Intelligent Control. Alexandria, August, 1991.
(Singh 95a)
Singh, S., "Synthesis of Tactical Plans for
Robotic Excavation," Ph.D Thesis, Robotics Institute, Carnegie Mellon Univ, January 1995.
(Singh 95b) Singh, S., "Learning to Predict Resistive
Forces During Robotic Excavation" In Proc. International
Conference on Robotics and Automation, Nagoya, May
1995.