From: AAAI-84 Proceedings. Copyright ©1984, AAAI (www.aaai.org). All rights reserved.
Task Frames
in Robot Manipulation
Dana H. Ballard
Department of Computer Science
University of Rochester
Rochester, NY 14627
disadvantage
Abstract
details
Most robotics computations
based
frame of reference;
refer to a single world-
however,
several
advantages
of this level is that important
are
suppressed.
For
depend
example,
CLEAR(x)
may
on
the
environment
and the manipulator.
geometric
the
predicate
geometry
of
the
Objects that might be
accrue with the introduction of a second frame, termed a
task frame. A task frame is a coordinate frame that can be
CLEAR with respect to a multiple degree-of-freedom
manipulator
might not be CLFAR to a loh-degree-of-
attached
freedom
to different
objects that are to be manipulated.
The task frame is related to the world-based coordinate
frame by a simple geometric transformation.
The virtues
Cartesian
Similar kinds of arguments can be made for the
lowest level of abstraction, the servomechanism itself. The
basic problem of the servomechanism
is to exert forces on
of such a frame are: (1) certain actions that are difficult to
specify in the world frame are easily expressed in the task
objects
frame: (2) the task-frame to task-uorld
transformation
provides a formalism for describing physical actions; and
in the world and transport
solved independently
from the symbolic plan using only
the inverse dynamics of the manipulator. At the symbolic
proprioception.
level, problems
*This research
was supported
Foundation
under
Grant
problems
levels of abstraction.
be analyzed
WCS-8203920.
can be solved
without
appealing
planning
it
symbolically
servomechanisms
is
are best considered
vvithin a given abstraction
helpful
without
involving
that
implement
independent
to
such
For
of
the
actions.
The
for
geometrical
actions
while
example,
property
in robot
consider
details
can
level
an
CLEAR(x)
action
CLEAR(x):
ON(x,z);
CLEAR(z):
be a necessary
symbolic
the necessary
level,
the
structure
that
Another element of
the
must
geometrical
level
with
that manipulators
On(x,y)
described
CLEAR(y).
can be created systematically.
may
the
the
is
that
it
servomechanism
be
able
level.
to
More
specifically, we argue that the geometrical level must
contain structure that functions as a command language
The principal advantage of such a system is that
symbolic plans involving several actions which achieve a
set of goal conditions
at
level contains
for the servomechanism
Postconditions:
of the context.
allows this property to be established.
communicate
standard symbolic description for the action move xfiom
y to z in a STRIPS-like expression [Fikes and Nilsson,
19711 is:
MOVW,
Y, d
Preconditions:
level problems
The geometric level provider an explicit representation of
space that includes geometrical and mechanical structure.
at different
to other levels. For example,
often
of the details;
at the servomechanism
The natural level of abstraction to introduce between
the symbolic level and servo level is a geometric level.
This is because many problems
effectively
can be solved independent
just as important,
in part by the National
1. Introduction
Robotics
them along desired
trajectories. The control problem of “given a trajectory,
find the actuator torques required to follou it,” can be
(3) the task frame can be related to the world frame by
Science
manipulator.
The
16
level. To first order,
we argue
need only these three levels, which are
in Table
1.
Table 1: Levels of Abstraction
(b) Trajectory
9
in Robot Manipulation
Level
Description
symbolic
STRIPS-like
description
of actions;
planning
done
by
chaining
appropriate actions to change current
state to goal state
geometric
representation
of space in which
actions
take
place:
command
language for servomechanism
level
servomechanism
detailed description
of manipulator:
inertial parameters,
friction models,
manipulator
internal geometry
as a transformation
locus.
A task frame is related to C surfaces but is different
in a crucial
way. Rather
than
thinking
about
the C-
surface in the world frame, the task frame is intrinsically
The central idea of this paper is that of a task fww.
A task frame is described
at the geometric
fixed to the object being manipulated
level and is a
geometric coordinate frame that is attached to the object
being manipulated. To understand the notation for a task
geometric
frame,
one
manipulation
the
computations
wor/Li frame.
compliance
constrained
must
appreciate
that
and control
strategies
to a single
An example
(for the duration
transformation.
That is, given any two of the
set (task frame, world frame, task-world
current
robot
tend to refer
reference frame, termed the
of such a strategy is that of
third
is easily
computed.
transformation),
Figure
lb
shows
19811, where the manipulator
is
to move along given geometric surfaces. Such
in the sense that either one could be transformed
into the
other. The important
the task
difference
is that uithin
frame formalism,
the problem of turning
surfaces have been termed C-surfaces [Mason, 19811. For
simply described
example,
meet
as “push in the el-direction
until
The expedient
of introducing
a crank.
the crank
handle
traverse a given path in the world frame, as shoun
Figure
la, which
is taken
the
characterization
of the task frame for the problem of
turning the crank. The two representations
are equivalent
[Paul,
when turning
of
the task) and is related to the world frame by the obvious
will
by
resistence.”
intrinsic
from [Brady et al.. 19821.
frame and separating
transformation
the crank can be
thz intrinsic
has the following
hou
an
frame from its
adv an tages:
1) many actions that are difficult to express in the
world
frame have very srmplr: expressions
respect to the task frame and tranbfortnation
fact, they can be described
respect
the
(in
with
to these two entities);
2) the frame-transformation
relate simply geomtitric
in
by invariants
with
world
descriptions;
with
decoupling
allous
and mechanical
corresponding
us to
changes
symbolic
and
betueen the task frame and
3) the transformation
world frame can be ccjmputed via both visic~n and
proprioception.
The crux of the rest of the paper is to describe these
advantages
in detail. Section 2 contains
from geometry and mechanics
subsequent
material. Section
between
the geometric
on the recognition
Figure 1: (a) Trajectory
-
-u-
\
I
.a
as a world-frame
locus.
involve
17
mechanics.
the basic notions
needed to understand the
3 describes the interface
and symbolic
levels. The focus is
and implementation
Section
4 describes
of actions
the
that
interface
between
the geometric
and servomechanism
focus is on the self-calibration
n). The external
necessary to relate the task
and world frames, and the automatic
the two levels introduced
the tip of the manipulator
levels. The
relations
joint
between
by the task frame formalism.
To
An orthogonal
vectors,
geometric
el,
coordinate
e2, e3, such
that
frame consists of
any two pair
dynamic
equations
drive
the arm,
(1)
S, f, n are assumed
known
and
the torques 7 can be related to (f, n) by a set of equations:
I. = f-l@,
f, n). Recently developed solution techniques
are
have
perpendicular
a right-handed
(ei * ej = 0, for i f j), they form
coordinate system (e3 = el x e2), and all
the vectors are unit vectors (ei . ei = 1). To denote
the world frame the subscript
w is used.
made
equations
it practical
in real time
to solve
[Luh
1980). The easier problem
the
the inverse
dynamics
et al., 1980: Hollerbach,
is readily also solved:
given 2, f, and n, determine
task frame the subscript t is used, i.e., elt, e2t, e3t, and to
denote
force and torque can be related to the
by general
Equation 1 is solved for the control torques 1. This way
of solving (1) is known as the inverse dynamics. That is,
A. Geometry
mutually
by a vector(f,
F (1, 8, f, n) = 0
2. Geometry and Mechanics
three
torques
can be described
that is,
8.
Each
frame has an origin x = (x, y, z), so that xt is the origin
of the task frame and xw is the origin of the world frame.
The world frame can be thought of as described in terms
of master coordinates.
In this case xw = 0, elw
= (1, 0,
0), e2w = (0, 1, 0), and e3w = (0, 0, 1). To denote
frame
a
(x, eL, e2, e3) we use E.
Given
the
task
transformation
frame
between
and
them
can
rotation
and a translation.
rotation
is done first since rotation
commute.
world
frame.
be specified
The understanding
The transformation
by a
is that the
and translation
is specified
the
do not
by an origin
‘T
change Ax = (AK, A}‘, AZ) and a rotation (n, 8). The later
notation
where
stands
n
is
for a rotation
expressed
transformation
in
8 about
world
can be computed
a unit vector n
coordinates.
directly
Figure 2: Two-link
The
as:
Besides the dynamics
Ax
=
xt
-
problem.
xw
Given
d@/dt, d2g/dt2,
and, assuming
a quaternion
[Per-tin and Webb,
n =
for rotations
manipulator
19831:
Normalize((elt
e = (-(n
representation
x q)(n
- eZ,))
independently
tip xp, dxp/dt,
d2xp/dt2.
l/2
In
of the
This is the easy
x find &-is harder
by designing
manipulators
but
with
dynamics
solutions.
the
manipulator,
The
expressed
links can be described by joint angles e (rotary joints)
which are controlled by applying torques 7. One such
is the two-link.
planar
manipulator
configuration
19831.
c’ery simple
analytical
is a series of links. Each link is
by its own servomotor.
in terms of @,
the motion
special geometries. One such geometry is a spherical wrist
[Feather-stone,
inverse
controlled
there is the kinematics
one must determine
can be solved analytically
B. Mechanics
A robot manipulator
problem
the state of the system
part. The reverse problem--given
- eLw) x (e2t
x qw>)
planar manipulator.
manipulator
and
inverse
For example,
joint
torques
geometries
kinematics
both
may
in the two-link
71 and
the
have
planar
72 may
be
as
71 = A + B + Cfx + Dfy + n
suggested
by [Horn,
19751. Figure
2 shows the
manipulator geometry. The force and torque applied at
?2 -- E + F + Gf,
18
+ Hfy + n
(2)
where
A, B, C, D, E, F, G, and
involving the manipulator
velocities and accelerations,
acceleration:
dependent
problem
the
letters
B and
on gravity.
These
equations
of
controlling
underdetermined.
(n,
However,
f)
This
this
if the problem
case
structure
has three
all
leads
is
naturally
level for describing
principal
invariants:
(2)
to
advantages:
the
a
actions.
(1) its
formalism
is
abstract that the same action can be used in a
and (3) its
variety of contexts;
interpreted by the servomechanism.
show that the
in
are
sufficiently
terms
frame
at the geometric
formalism
elements
on velocity
F denote
task
formalism
joint angles S, joint angular
mass, and inertial parameters.
The letters A and E denote terms dependent
and
The
H are expressions
structure
can
bc
is simplified
The format that we adopt for representing
slightly, e.g., the gripper is a finger such that n = 0, then
Each
has
a
set
of
a
control
preconditions
that must be true for the ,action to be
applicable, a set of while conditions
that must hold
Our example
assumes
such a gripper.
3. Geometry and Symbols:
Recognizing and Implementing
during the execution
Actions
This section shows how the geometrical
task frame and transform
conditions.
notions of a
can be of general
of actions; (2) the task frame and transformation
the task frame
allows checking
for collisions
and (3)
This
between
If an object
approaching
z-velocity
structure
development
geometric
of falling.
is falling
then
is
FALLING(obj)
In other
are used by the various conditions.
takes
which
constraints.
advantage
related
of
symbolic
Consider
the
previous
constraints
and
the example of closing a
symbolically
as:
DOORCLOSI%G(door)
that of the world frame origin such that the
is negative.
as:
then do ( whileconditions}
door. This can be expressed
the description
its origin
is described
(stoppingconditions)
where the parameters
objects.
To start with the first point. consider
of the action, and a set of stopping
an action
if (preconditions)
until
allows a
of tasks in terms of invariants;
Thus
ActionName(params)
use in a
symbolic planner. In particular: (1) the framework allows
the interrelation of symbolic and geometric descriptions
simple description
syntax.
action
the external forces at the tip can be directly related to the
torques.
STRIPS-like
actions has
if TOUCHING(door)
words,
until
< = > (vz < 0)
then do PUSH(door)
ARRESTED(door)
but also can be expressed
geometrically
as
DOORCLOSING(door)
where
the understanding
is that expressions involving
and changes in such are
positions and orientations
task-frame
world-frame
to
the
about
statements
transformation. Expressing the process of falling as a rate
of change
has the effect of making
static situation.
throughout
The logical expression
it comparable
if (E,O,O)then do (f,O,O) & .4LIGNED(Et,Ehandle)
until
This
illustrates
a number
of important
points
which we will nou elaborate. In the first place. note that
we have been able to decouple the force constraints from
to a
(vz < 0) must hold
the geometric constraints.
the falling process. The task frame orientation
may also play a role in the description.
syntax
(O,O,O)
C-surfaces
For example:
(A similar decoupling
is seen in
for the cases of pure force or pure position
control [Mason, 19811.) Thus rather than specify that force
RIGHT-SIDE-UP(obj)
< = > ALIGNED(e3,,
e3w)
control of the handle in the world frame [Mason, 19311
uhere it has a varying locus, it is specified in the task
In this as in the previous example, a first order logic
with expressions
consisting
of
is assumed
syntax
predicates denoted by upper case, terms denoted by
frame where it has a very simple structure. The notation
ft = (fl, f2, f3) stands for exertjbrce ft in the task frame
lower case. The point of these examples is that for
problems involving Newtonian mechanics, the task-frame
less than that required
to world-frame
systematically
large enough to start the object moving. The key virtue of
the task frame is that the force ft can be an invariant
geometrical
coordinate s?lstem. Ihe
transformation
provides the basis for
and
expressions
symbolic
relating
expressions.
during
19
the action.
quantity
E is a small contact force,
to move the object, v+hereas f is
action. Figure
The second part of the while condition
door expresses
and
some
the relationship
other
frame
between
expressed
for closing the
handled
the task frame
in
world
3 shows two different
by DOORCLOSING.
assumed to be perpendicular
frame
tasks which can be
In the
first, gravity
is
to the plane, i.e., the figure
shows a top view.
coordinates. For the earlier discussion one can appreciate
that
given
Et
and
predicate
Ehandle.
*e
ALIGNED(E,,Eh)
is easy to compute.
The
following
scenario
DOORCLOSING.
is imagined
for the action
Given that the handle
servomechanism
is grasped,
applies a force in the elt direction
task frame to move the door. The door moves
bumps
into the door
frame,
of the
until
torques.
satisfies
such
stopping
microdynamics
of
servomechanism,
condition.
the
Details
are
contact
left
to
the
of
The above strategy does not address the problem
of
details
until
we will defer
This
as the
the discussion
these
and
it
at which time the frame
exerts a force f to cancel the manipulator
the
the
the next section.
slamming the door. This can happen when the force is
too large. To deal with this example we will add one
condition
and change
while condition
notation
slightly.
First we add the
(11~11E [YO - AU, ~0 f
A”]). This states
that the speed of the task frame with respect to the world
frame is to be constrained in the interkal YO -t AU. As a
shorthand, we use capital letters to specify interFals, i.e.,
vo f AU = VO. The second change
relax the force specification
specification
The
we will make is to
in the task frame to just the
of the axis to be controlled,
while condition
in this case ‘1.
becomes:
Figure 3: Different tasks which can be handled by
{whilecondition)
=
FORCECO\TROL(etl)
DOORCLOSING.
and (11~11
f L’o)
and ALIGhED(E,,Eh)
The understanding
this
to
simplistic
is that the servomechanism
generate
the
possibility
appropriate
In these examples, recognizing
one of DOORCLOSING
can use
commands.
would
One
be difficult
plausibly,
is:
but
that the described
from the geometric
perhaps
the relevant geometric
which in this case are specified
if llvll > YO + AV
then
if llvll < ~0 + AU
then
fl : =
fl : =
known.
fl + A
fl - A
not
impossible.
action is
features
More
features of the problem.
by Et, may already
In any case, the geometric
be
level is the essential
starting point from which the relevant constraints
can be
synthesized.
Notice that although the while conditions have become
their essential
structure
has been
more complex,
maintained
in that they are invariants
We note that some details are being finessed
lebel of description.
with respect to the
what if the masses in
been established,
a wide variety of different
situations
by the same task frame description
cannot
achieve II\11 E Vg? This case of failure has to be resolved
at the planning level, and aside from characterizations
of
We now turn to the second advantage
of the
formalism, which is that, once appropriate bindings have
be described
For example,
these examples are such that the servomechanism
action.
at this
the failure mode, we are not addressing
can
problems
of the
20
in this paper.
these kinds of
Another
frames
problem
provide
problem.
is that of collision
a partial
mechanism
First instantiate
detection.
for handling
all the geometrical
initialized
Task
updated
this
objects
to check for solid material
occupying
One
2)
use g(t) to solve
3)
a’
4)
if la - all
=
tan-l((dyp/dt)/(dxp/dt));
Self-Calibration
for the task frame scheme
be some
between
way of computing
the task frame
the
and the world
to work there
< ~1, then CI := CX’:
else failure.
=
F =
into the real-time
condition.
If stopped,
B
0 in (2). This leads to:
transformation
frame.
1) compute
[r-r this
from
section we discuss ways of doing this and show how they
can be integrated
(3)
for x,(t);
Now we turn to the stopping
must
is:
space.
4. Geometry and Servomechanisms:
In order
way of updating
1) use I(t), f(t) to solve for e(t):
from two or more
the same physical
of the action and continuously
the action.
objects with
respect to the task frame, and then use the details of the
geometric
representation,
e.g.,
constructive
solid
geometry,
at the beginning
during
control
program
of
zs using B = F = 0 and f, measured
proprioception:
2) if ]x - zs] < ~2, then
stop.
the servomechanism.
One way of establishing
through
the desired transformation
visual input. This much-researched
problem
be done in constant time on a parallel machine
is
B-e can d.
can
if suitable
visual features can be identified [Ballard and Sabbah,
1983; Hrechanyk and Ballard, 19831. But from a robotics
context, a more interesting
dynamics and kinematics
To see how this might
method
is to use the inverse
start
SC0p
CO&O/
4
of the servomechanism
itself.
work, let us reconsider the
Siyngl
+
Sr’q nal
Low
c
problem of closing the door. From the inverse kinematics
of the manipulator
it is possible
to calculate
the end
effector velocity. In the normal case of door closing, once
the door moves, its velocity vector is available
frame coordinates.
In this case of compliant
1&
in worldmotion,
Toverse
the
door can only move in the direction oj’the el axis of the
task frame. Thus, el can be computed
as el =
Normalize(v),
where v can be measured
from the
kinematic
equations.
Since el is the crucial
axis in the
closing task, the other axes may not need to be updated
beyond
enforcing
the orthogonality
condition.
%f namics
I
T
I
;
I
If they
should be updated, an additional constraint is that, for
two different times tl and t2, then e3 can be computed as
e3 = Norrnalize(v(tl)
x
v(t2)), where the two velocities
must have different directions.
This is a natural constraint
in the door closing situation.
In pushing
velocity needs to wobble arbitrarily
establish
the second
Another
direction
way to calculate
c
the block the
Prop rio -
while being pushed to
Ception
vector.
the transform
is via force
9
proprioception. Assuming the velocity parallel to the door
surface is clamped at zero, the transformation
parameter,
which in 2D is a single angle (denoted
can be readily computed.
The transform
by, OLin Figure 3)
c
Figure 4 Details of servomechanism
is assumed to be
21
level.
planning,
These
kinds of computations
can be utilized
servo controller in the manner depicted
see how the controller works, consider
closing action. First, at the symbolic
description
into
constraints.
level
can
servomechanism
be
Second,
commands:
at
and
the
translated
the ifconditions
a control function,
showing
translated
actions
servo level, the controller
level. Third,
computes
the task frame. This command
a command
is translated
signal in
into world-
but assumes the torques
order to estimate
are checked
used
to
the world forces and velocities.
update
termination
where
and system state are known
the
condition
conditions
These
and also
The
to the symbolic
level
Horn, B.K.P., “Kinematics, statics, and dynamics of twod manipulators,”
MIT AI Lab, Working Paper 99,
June 1975.
issues that arise in robot
Hrechanyk,
L.M. and D.H. Ballard. “A connectionist
model for shape perception,”
Computer
Vision
Workshop, Ringe, NH, August 1982; also appeared as
“Viewframes:
A connectionist
model
of form
perception,”
DARPA
Image
Understanding
Workshop. Washington,
D.C., June 1983.
is propagated
is set to TRUE.
frames make many
planning and manipulation
simpler. The change from
earlier work has been inspired in part by recent work at
level which has allowed
the
the servomechanism
deh elopment
of
accurate
dynamically
plant
Luh, J.Y.S., M.W. Walker, and R.P.C. Paul, “On-line
computational scheme for mechanical manipulators,”
J. Dynamic Systems, Measurement, Control 102, 6976, 1980.
models
Mason, M.T., “Compliance
and force control
for
computer
controlled
manipulators,”
IEEE
Trans.
Systems, Man, and Cybernetics 1 I, 6, 418-432, 198 1.
[Mukerjee et al., 19841 and aforementioned
fast solutions
to the problems of inverse kinematics and dynamics. This
means that the main portion of manipulator control can
be carried out as an open loop rather than a closed loop.
Mukerjee, A., R.C. Benson, and D.H. Ballard, “Towards
self-calibration
in
robot
manipulator
sq stems:
Dynamics enhancement
through trajectory deviation
analysis,” Working
Paper, Depts. of Mechanical
Engineering and Computer
Science, U. Rochester,
March 1984.
Before these advances, manipulator control has had to be
segregated into a planning phase and an acting phase,
and
the
dynamics
introspected
plant
models
planning
linked.
of the acting
phase
could
not be
the planning
phase.
With
accurate
during
and
and
open
acting
It is important
loop
phases
easier.
problems
this
of the
Hollerbach, J.M., “A recursive formulation of Lagrangian
manipulator dynamics,” IEEE Trans. Systems, Man.
Cybernetics SMC- IO, 11, 730-736, 1980.
in
5. Summary
Task
other
Hopefully
will make the solution
transformation.
task-world
ARRESTED(door)
invariants.
strategy
Fikes, R.E. and X.J. Nilsson, “STRIPS: A new approach
to the application of theorem proving to problem
solving,” Art$cial Intelligence 12, 3/4, 184-209, 1971.
uses the inverse dynamics
against the termination
mechanical
as geometrical
Featherstone, R., “Position and velocity transformations
between robot end effector coordinates
and joint
angles,” Int. J. Robotics Research 2, 1983.
synthesized from the desired control signal. The actuator
torques have an effect on the plant which is monitored by
Proprioception
how they may be characterized
representational
The
Brady, M., J.M. Hollerbach, T.L. Johnson, T. LozanoPerez, and M.T. Mason. Robot Motion: Planning and
Cambridge,
MA: The MIT Press, 1982.
Control.
at the
frame coordinates by the task-world transformation.
The
inverse dynamics allows the actuator torques to be
proprioception.
information.
single actions and
Ballard, D.H. and D. Sabbah, “Viewer independent shape
IEEE Trans. on Pattern Analysis and
recognition,”
Machine Intelligence 5, 6, November
1983.
and the until conditions
at the servomechanism
spatial
6. References
are utilized
at the geometric level become set points for the
controller
of detailed
is limited to characterizing
into
are utilized in a termination monitor. In other words, the
invariants
from failures, and the representation
are utilized
to generate a start signal, the while conditions
to synthesize
exposition
level, the symbolic
automatically
recovering
of large amounts
in Figure 4. To
again the door
of the action can be automatically
task-frame
geometric
by a
control
can
to acknowledge
be
strategies,
more
Paul,
R.P.
Robot
Manipulators:
Mathematics.
Programming, and Control.
Cambridge, 41A: MIT
Press, 1981.
the
intimately
Pervin, E. and J.A. Webb, “Quaternions
in computer
vision and robotics,” Proc., IEEE Computer Vision
and Pattern Recap-nition Conf., 382-383, Washington,
that this paper does
not tackle many issues that must be solved to make robot
manipulation
practical. Some of these are: trajectory
DC, June
22
1983.