From: AAAI-82 Proceedings. Copyright ©1982, AAAI (www.aaai.org). All rights reserved.
REPRESENTING
SMOOTH
IMPLICATIONS
FOR
PLANE CURVES
FIGURE-GROUND
FOR RECOGNITION:
REVERSAL*
D. D. Hoffman & W. A. Richards
Artificial Intclligcncc
I .ahorntory
Mass&usctts
and Dcpartmcnt
Institute
Cambridge,
of Psychology
of’I’cchnology
Massnchusctts
02139
To be useful
A rcprcscntntion
of smooth
The basic rcprcscntation
“codons”,
IS a linked list of four primitive
which arc invariant
scaling.
Psychophysical
ground
reversals
pldnc curves for recognition
is proposed.
shapes,
under rot&ltions. translations
observations
are prcscntcd
regarding
to suggest
should
invariant
called
of figure-
the pcrccption
a description
scntation
could bc used by the human visual system.
I
shape without
constrain
bcnclit
based upon its curvature
and rotation
must compute
the set of posslblc
useful
for recognition.
predict
the contents
rccogniLcd.
matches
with memory
should
of a
indcpendencc
rules tllat provide
initi‘itc the recognition
process.
shape descriptions
where
translation
0 and translation
(u, v),
+ 21 ,
>
which can bc used to
However,
Such rules for smooth plant curves are
senting
because
curvature
is scale dependent.
K(S) in a scale invariant
orders
into memory.
ofn(a)
since the property
translations
maxiina,
for repre-
Kather,
In ad-
function
if
the representation
into units which can bc described
is to exploit
singular
points
miIlilni1 and Lcrocs of curvature
of being a singular
and uniform
a means
still must be sought.
not be a continuous
A r~ason~~l~lc approach
1 and 0, i.e.
II
manner
itself should
should provide aI1 articulation
qualit;lti\cly.
point iE in\,arinnt under
of
[3. 41,
rotations,
scaling.
MAXIM 2 ---L
MINIM/~ -ANI)
I’A’I‘
ION”
AL.
.-- CUKVI-‘
~_ --L “OI~II~N’
---1--.-__
Some shapes rccogniLable without benctit of context,
Which
points
orientation
quantity,
paper
I-aboratory
describes
research
and Psychology
Advanced
Rcscarch
dcr Oficc
of Naval Rcsc,uch
and AFOSR
done
at the Artificial
Department
contract
un-
and by NSF
D.D. Hoffman
was also sup-
doctoral
the cnrly descriptions
an
should dccouplc
rotation,
and allows their cffccts on the recognition
invariance
an oricntiltion.
of orientation
mints
of a shape
cur\cs
The oric‘ntation
cl~mgc
in
CilIlllOt
affect
‘I’hus knowing
vature. as
5
is usually
orientation
spcclficd
co~lvcx
~11~11 the
a sign to
of 3
for
in figure\
J~~IIC
curve the sign of
In J)articuJar maxima
adopted
hcrc is that
which side is figure dctcnnincs
choosing
an orientation
Minima
closed
for minima
curves,
in
the choice
dctcr-
in-c rhcn typically
of the figure, maxi1n.r with convcxitics
oricntrltion
by
in which the curl c is
‘1‘1~ con\ cntion
a curve or, convcrscly.
uith concavities
as in tlic case of
its
[l].
on the
to the right as the curve is travcrscd
figure 2). It is J,ossiblc howcvcr
for ex-
it is J>ossiblc to aG?n
cvc~~y~sl~rc a1011g the cur\c.
and vice-versa.
on
dcpcnds
cur\ aturc ic an unsigned
once one of the t\j’n possible orientations
which side is figure by con\cntion.
associated
process to bc discntanglcd.
for shape descriptions,
in no way implies that the rotation
intcrprctation
of plant
figure is to the left and ground
scaling and shape to bc made explicit scparatcly,
Thus the goal of rotational
Ijy a
chnngcs
minima
bcco~nc
the position,
of curvature
arrow on thc~ curve pointin, n in the direction
tbc curvature
and ovcrnll scaling of a shape from the shape itself. ‘l‘his allows
the position,
chosc11.
or minima
‘l’hough in gcncral
CuI’vC.
con+stcntly
to bc tra\,crscd.
fellowship.
arc maxilna
tllc
in the cast
the curve is
in part by the
of Dcfcnsc
of
the curvature
Institute
N00014-SO-C-0505
under grant 7923110-MCS.
**More prcciscly,
is prokidcd
Projects Agency of the Dcpnrtmcnt
ported by a Hughes Aircraft Company
Intelligence
of the Massachusetts
SuJ>port for this rcscarch
of ‘I’cchnology.
ample,
a rcpre-
Such
namely
a
ways,
+(s)
=Jws +0
of context-
it is to ‘serve as an index
rotation
conditions,
sin $(s)ds
dition the representation
“l‘his
K(S) is attractive.
[2]. For any rotation
Although
in many diffcrcnt
J
the subject here.
Figure 1.
which are
scaling and should
+u,
(Jcos~(s)ds
if they arc to be
implies the cxistcncc
the initial rules
given by
-i(s)=
highly
For cxaniplc. though one cannot rcasonnbly
of figure 1 prior to seeing it, the shapes arc readily
This simple demonstration
indcpcndcnt
its initial descriptions
Yet these descriptions
and uniform
satisfies two of the invariance
lN’I’ROI~UCTlON
of context.
rotations**
process
1 icld descriptions
= (z(s), y(s)) can bc specified
r(s) is uniquely
A vision system somctimcs
should
a first index into a table of shap’cs in memory.
plane curve r(s)
that a similar representation
the recognition
on images,
under translations,
provide
and unifonn
for triggering
be computable
(SW
to have positive curvature,
or maxima
to have ncgativc
of the convex closed curve is rcvcrsed.
cur-
If desired,
codons
of the maximum
31 2:
Figure 2.
malizcd
qualitatively
minima
and zeroes of curvature
arc all candidate
a curve into kinits in a manner
translations
and uniform
3-I) p;rrts arc joined
bc crcatcd
concavities
to the head,
imas
immcdiatcly
or 7crocs
segmentation
of curvature.
at minima
as is ncccssary.
in traversing
and the other minimum
of the maximum
For
a segment
label the “head”.
can bc given crudely as much closer
much closer to the tail, or approximately
in the middle.
This gives the “skew” of the curve. Zeroes can similarly bc described
closer to the maximum
point, closer to the head/tail,
in the middle bctwccn
the maximum
as
or approximately
and the head or tail.
points for
under
among
complex
them
rotations,
WC require
objects concakitics
Scgmcntation
cncodcs
of tl!c natural
PJO~CJ~Y
encountcrcd
nor-
be done
parts of shapes [5, 61. Fortunntcly,
to crcatc
in the silhoucttc.
thcrcforc
an important
invariant
To choose
scaling.
that the units should reflect natural
generally
This should
at first and then more quanGtativcly
the “tail” of the scgmcn!.
the positions
SEGMENTATION
partitioning
when
by noting
within the scgmcnt,
of the scgmcnt.
label the first minimum
Then the position
orientation.
Maxima,
described
of curvature
ZCJOCS
by lhc total arc length
example,
Minima of curvature (indicated by slashes). Arrows indicate curve
III
can bc further
and any
in a straightforward
manner
world that is not captured
This is our
will
of the image at
by max-
argument
ge1fe177Zpusiliu~~
Figure 5.
for
Cune
segments with identical smgularities but different shapes.
of curvature.*
As shown in figure 5. two segments
ima and zeroes, identical
appear
quite different.
bctwccn
invariant
can have identically
placed max-
at the maxima and minima,
The difference
is the behavior
and yet
of the curvature
points.
This behavior
manner
by the integral of curvature
the singular
propriately
curvatures
can be dcscribcd
in an ap-
between
each
of the singular points:
b
n(s)ds
=
B(b)
-
B(a),
a
I
Figure 3. Joming parts generally produces concavitlcs in the resulting silhouette.
where
1V ---A
Dt’SCl~IPTION
Minima
whcrcas
mcnt.
arc used
to break
There
four basic types of scgn~cnts.
arc
I’urthcrmorc,
only certain
CODONS
a curve
maxima and zcrocs arc used to describe
codons”.
arc
of curvature
OF PAl~l’S~ CONTOUR
into
vature
scgmcnts,
which WCcall “contour
There
connections)
First, all
with no
segments
curve
%CJOCS arc
type 2 codons.
cncountcrcd
contain
scgnlent
has
cxactty
either before (type l-)
thcrc
one
7~~0,
that when K(.s) =
or
0).
arc
when traversing
four
or
one
Scgmcnts
ZCJO, the zero
nfter (type l+)
the
curve
may
will give different
descriptions
for
on how codons
may be joined
at minima.
a, b E (0, l-,
a o b,
l+, 2)
to the tail of b. Note
connected
b o a and hence the codon sequence
is critical.
Not all c’onccivable codon joins are possible (figure 6).
The fact that not all conccivablc-codon
be
that the codon rcprcsentation
the maximum
in the chosen
points
that the head of a is smoothly
that in general a o b #
two points of zero cur-
0, K’(S) #
called type 0 codons, those with two zcrocs arc called
If a
point of the scgmcnt
tion. Thus
are restrictions
indicating
(This assumes
the singular
a codutl @in by the operation
Define
allowable.
vaturc.
between
curves A and B in figure 5.
the shape of each scg-
codon joins (pairwisc
B(s) is the angle of the tangent at r(s) given by B(s) =
which notes the integral of curA re p rcscntation
tan-‘(y’(s)/z’(s)).
niques.
orienta-
Consider,
for example,
all codon joins wcrc allowable
basic codon Lypcs (figure 4).
values
(0 l+
l-
2) rcgardlcss
the value of Cj would
1
however,
joins
may bc amenable
the codon string . . . cj-1
of its context.
the context
Figure 4. Contour codons (as dcfincd in section 4). Zcrocs of curvak~re are
indlcatcd by do& on the curves, minima ‘by slashes.
considered
6
here.
cI is actually uniquely
position
and dcscribc
tech-
cj c3+1 . . . . If
of the values of ~~-1 and cJ+l.
be indepcndcnt
one can show that in actuality
*When gcncral
parlition
suggests
then cj could take any one of the four
its range of possible values to two on average.
one third of the contexts
allowable
iiJC
to error correction
is violated
the resulting
special
Thus
Using figure 6,
of a codon restricts
One can also show that in
determined.
rules may bc nccdcd
image contour
to
[S].. This will not be
0
+
1
1-
The perceptual
2
significance
a simple modification
of minima
of Rubin’s
can also be demonstrated
ambiguous
the locations of minima are indicated
face-goblet
by occlusions,
then the perception
of the curve is biased toward one of its two orientations
0
face or goblet impression
becomes
by
illusion [9]. If
more apparent
and either the
(figure 8). However,
when this figure is viewed at a distance so the added lines are not visible,
then the classical
nor highlighting
1-
correspdnd
m
+
instability
returns.
the maxima
to natural
Neither
has comparable
highlighting
the zeroes
effect because they do not
points for segmentation.
+
+
1
2
Figure 8.
Fiigure 6.
Table of allowable codon joins Rows and columns are labclled by
odon type, with the intended codon join scqucncc at cxh table entry being (row
ype, column type). Legal joms are indicated by a +, others by a -.
VI
V
The rcprcscntation
RF,1 .A’TION TO PEKCI:PI’ION
dcpcnding
[7].
‘l’hc explanation
because its rcprcscntations
plctcly
difTcrcnt.
not invariant
under
parts by zcrocs of curvature
tour from its disposition
hcrc can
shape
description
scaling of the contour.
of a contour
of curvature
of traversal)
in its rcprescntation
contour
arc com-
shape description
arc
of a
The question
[8], or by minima
TRANSFORMS
to decouple
rotation,
the shape of a con-
translation
as dctnonstratcd
is not invariant
the
and uniform
in figure 7b, the shape
when the direction
It is not difficult
naturally
mirror rcvcrsal
of traversal
to convince
oncsclf
along the
also that the
under a mirror reversal of the contour.
arises, Arc thcrc simple rules that define how the
is transformed
or a reversal
when the contour
in direction
of traversal
undergoes
(change
a
in oricnta-
tion)i
nrzd maxima [3, 4, lo], the
parts would not differ under a change in orientation
under
llowcvcr,
codon string of a contour
can bc quite
(SW figure 7b). If one chnsc to define
is designed
is not invari,lnt
is rcvcrscd.
IUWERSAI,
in space and its overall size. Conscqucntly
is invariant
is that a curve looks diffcrcnt
Since the positions of the minima
a change in oricnt;ltion (direction
for the two orientations.
AN11 MIRROR
as figure and which as ground
under the two possible orientations
curve, the PJI ts of a curve as spccificd
different
proposed
that a curve can look very diffcrcnt
on which side is perccivcd
(SW figure 7a).
LOCK-KEY
‘l’hc codon rcprcscntation
of plane curves for recognition
explain the well known observation
Rubin’s face-goblet illusion segmented at minima to reduce the in-
stability.
of a curve.
In the case of a mirror
transform
of a codon
rcvcrsal
string
the rule is quite simple.
is obtained
by rcvcrsing
The mirror
the direction
in
which the string is read (right to left rather than left to right) and reversing the sign attached
of <
I+
2 l-
to each type 1 codon.
l+
0 l-
>
is <
l+
can also bc used to find symmctrics
cxamplc,
----------e--m
I
0
b
1’
1+
1’
symmetric
1+
within
1+ 2 l-
transform
>.
This rule
a single contour.
one half of a codon string is found to bc the mirror
of the rcmaindcr
m---e----
Thus the mirror
0 l-
of the string, a necessary
has been found.
long as zeroes of curvature
condition
If, for
transform
for the curve to be
Note that this applies to skew symmetry
arc not made to appear
or disappear
as
by the
skew.
When
string
but
codon shng :
1+21-1+01-
the
apparently
as a map
codon string :
01-1+022
The
codon
((00,
01+,
If}
sense
of traversal
transformation
rule,
not
from
simple.
pairs
which
l-l+)
and {(l+l-,
the lock-key
It is perhaps
of concatenated
doublets
l-0,
of a curve
called
1+2,
map
codons
to
t--+ O}, {(Ol-,
21-,
each
is reversed
the codon
transform,
is unique
most
easily
to codon
codon
02) H l-},
specified
singletons.
singleton
{(lt-0,
22) H 2}. This lock-key
are
20) i--r
mapping
can:
1
be used, for example,
Figure 7.
(a) Figure-ground reversal makes bounding curve appear different. (b)
Diffcrcnt codon string descriptions are assigned to a curve in its two orientations.
into the other.
7
to transform
each of the codon strings of figure 7b.
VII
An approach
to the representation
been sketched.
to break
quite
It is suggested
a curve into parts,
to describe
the parts.
different
the approach
Richards
SUMMARY
of plane curves for recognition
that minima
and that maxima
This approach
when
figure
has
can be used
and zeroes can be used
explains
and ground
to piecewise-smooth
[ll]. Extensions
of curvature
why a curve can appear
are reversed.
Extensions
curves are presented
in Hoffman
of
and
to surfaces are desirable.
ACKNOWLEDGEMENTS
We thank
J. Rubin,
Bobick for comments
J. Scheuhammcr,
on previous
S. True,
K. Stevens
and A.
drafts.
REFERENCES
[l] Rock, 1.: The perception
of disoriented
figures, Sci. nrjt., 230, 1, 7%
85 (1974).
[2] 110 Carmo,
M.: D&7hwtial geontelrqt of curves and surfices, Prcntice-
Hall, New Jersey (1976).
[3] Attneave.
Some
F.:
informational
aspects
of visual
pcrccption,
I’sychol. Rev., 61, 183-193 (1954).
[4] Duda,
R., Hart, P.: Patkrn chssijcarion
and scene analysis, Wiley,
New York, 337-341 (1973).
[5] Marr, D.: Analysis
of occluding
contour,
Proc. K. Sot. Land 13. 197,
441-475 (1977).
[6] Marr, D.: Vision: a compulational invesiigazion in/o lhe human reprosenrarion and processing of visual infkmation, Freeman,
San Francisco
(1982).
[7] Attncave,
F.: Multistability
in pcrccption,Sci.
&z., 225, 6, 63-71
(1971).
[8] Hollcrbach,
J.: Hierarchical shape descripiion of objects by selection
and modification of prototypes, Massachusetts
Institute
of Technology
AI-TR-346,47-51(1975).
[9] Rubin
E.: Figure
and ground,
In D.C. Bcardslcc
(Eds.), Readings in perccplion, D. Van Nostrand,
[lo] Brady
M, 1982 “Parts
Proceedings
description
of the Society
& M. Wcrtheimer
New York (1958).
and acquisition
of Photo-optical
using
vision”
and Inslrumentation
Engineers
[ll]
Hoffman
D, Richards
visual recognition”
W, 1982 “Representing
ANT AI Memo 630
plane
curves
for