NASSACHUISETTS INSTITUTE OF TECHNOLOGY
A.I. LABORATORY
June 1972
Artificial Intelligence
Meo No. 263
A HETERARCHICAL PROGRAM FOR RECOGNITION OF POLYHEDRA
Yoshiakl Shiraf
ABSTRACT
Recognition of polyhedra by a heterarchical program is presented. The
program is based on the strategy of recognizing objects step by step,
at each time making use of the previous results. At each stage, the
most obvious and simple assumption is made and the assumption is tested.
To find a line segment, a range of search is proposed. Once a line segment is found, more of the line Is determined by tracking along it. Whenever a new fact is found, the program tries to reinterpret the scene taking
the obtained fnformation into consideration. Results of the experiment
using an image dissector are satisfactory for scenes containing a few
blocks and wedges. Some limitations of the present program and proposals
for future developments are described.
Work reported herein was conducted at the Artificial Intelligence Laboratory, a Massachusetts Institute of Technology research program supported in part by the Advanced Research Projects Agency of the Department of Defense and monitored by the Office of Naval Research under
Contract Number N00014-70-A-0362-0003.
Reproduction of this document, in whole or in part, is permitted for
any purpose of the United States Government.
1. INTRCDUCTIION
We do not know how to
as
well as
a human
being.
One of
recognize objects
environment
may
in the
be recoqnized
of many computer
the shortcomings
context of
based on
his a
The
priori knowledge.
The
simple
that the
recognition proceeds to the
first and the
parts are recognized
A
environment.
the
programmed so
however, well
recognition rrocedure is,
cbvicus
visually
as , insky has pointed outt , their hierarchical structure.
rrowruams is,
hbuan may
make a program to recognize objects
more comrlicated details based on the previous results.
The work in this parer studies an example of a heterarchical program
to recognize
begin
polyhedra with
by trying
complete line
drawinm
to find
drawing.
an imace
dissector.
feature points
It
is
in a
very difficult
without knowledge about a scene.
errors, the
recognition by
complete
line drawing, such
mistakes.
Our work is
entire scene
to get
works
and make a
a complete
line
the line drawing has some
If
theory based
a
Most previous
on
of
the assumption
the
still more serious
as Guzaan'sk) might make
an attempt to recognize objects ster by ster, at
each time making use of the Irevious results.
We
assume in this
cbjects and
the
approximately.
jroderately
paper that the
background
At
complicated
is
present,
difference in brightnesc between
large enough
this
configurations
rrowram
of
to
detect
works
blocks
and
for
the
boundary
recognizing
wedges.
limitations and rroposals for future developrent are described later.
The
PAGE 2
2. GENEPAL STRATEGY
2.1 Priority Of Processing
For
convenience, we define the
fallirng into 3 classes.
end the cuter
lines AL,
edjes of the objects
A line formed at the boundary between the bodies
background is
a
_conto-ur line of
C
Cic,
M, 9,EF, Nt,ON, HI, IJ,
XY, YZ and ZV are contour lines.
of an object.
in a scene as
the bodies.
JK, XL, Li-.,
MN,
In
NO,
Fig. 1,
AO, V'i,UX,
A lboundary line is a line ca the bcrder
Contour liner are boundary lines.
In Fir. 1,
the bontwary
line~ are the contour lines and lines on the bourndary between twc bodies,
i.e. CP, Pf,
!Q, Qf, and P1,.
An internal line occurs at the intersection
of two planes of the sqme body.
Lines 3J,
1~, Q., F" , NT, AId,
A-,01,
M
and XV are internal linez.
The glotal strategy is shown in Fig.2.
are
extracted (b~cause
we assume a porori enough
objects and the Lackrround).
lig.1,
one contour for todies
the boundary lines and
contour.
At first, the contour lines
If more than
contrast between the
one contour is found, as
El, £2, 13 and
in
another for lody B4, then
internal lines are searched
one by one for eact
The g>lobal strategy in block 2 in Fig.2 is as follows.
A) Find boundary
lines before
finding internal
lines because boundary
lines often •ive good cues to Eues.: internal lines.
Kote that to find
boundary lines implies to find bodies.
I:) In searchinr
larter or
re•quirrd
for
smaller
lines, different situationr
areas.
In
our stratepgy,
to scarch lines, the hifher
require examination of
the
smaller
the
area
priority we 5ive to that search.
NOAE ?
In Fig.1, for instance,
line
IC,
it
iA
to determine the existence
enougb to search a
extension of the line.
all
To find
of a extensior:
small area whose
center is on the
line IQ, however, we should
possible directions of a line between IP and IJ.
of
consider
Thus the former
search has priority over the latter
The priority to extract
the most obvious
information first is
the
followinr order.
(
two boundary
1) If
lines make a
1i.'. 3),
try to find the extension of the!m.
found,
track alon:•
(b)
this line.
(such as
concave point
point B in
If only one extensio:n is
Most of such cases are like in F&i.S
W.ibere one body hides the other.
side
We can determine to which
of body this line belon r.
(2)
If no extensions of two concave lines are found,
line which starts from the concave point,
If only one line is
is not clear locally to
which body this line belons.
In Fig.e
line BD belongs to the upper body, but this is not always
(c),
is
That
the lines
found,
host of these cases are as in Fig.3 (c) where
track along this line.
it
try to find another
AB,
BC, BD
are
not sufficient
to
true.
decide the
relation.
( 3) If
hoth extensions cf two lines are found at a concave point, try to
find a t.ird one.
This
is the case as
boundary line.
If
only one line is :found,
shown in Fig.3 (d) where
track along this line.
the third line is
the
PAGE 4
.h.enever tracking terminates, an attempt
new line to the other lines
the
always made to
that were already found.
one line serment is found in (O), (2)
is
is
or (3),
connect
If more than
the tracking of those lines
put off hopefully to be clarified by the results of knowledge obtained
found at
two extensiors
Fig.4 illustrates
cases.
in simpler
concave
P. The interpretation of the two lines is put off to treat simpler
roint
cases first.
That is,
one wculd continue examining the contour and lines
found next;
AB and Cr miCht be
by a
then,
circular search at Toints B
(which is exrlained later), line BP would be found.
At this ste•e it is
easier to interpret lines AB and BP as boundary lines which separate
bodies.
Then
line
DP
might
be
found
similarly
and
two
interpreted
correctly.
( 4)
If an end of a boundary line is left unconnected as Ký in Fi,.5, try
end point (Q in this example)
to find the line startinr from the
circular
line is
search.
If multiple lines are
the boundary.
found,
by
try to decide which
If a boundary line is determined,
track alone
In Fig.5, the dotted lines are found by circular search and the
it.
arrows show the boundary lines to be tracked.
( 5) If no line is
(PQ
in this
found in the case (4)
example) by a
conected to other lins,
in (4).
is
This
certain leneth,
If not,
necessary
as stated above, extend the line
and test if
the line is
then apply circular search again as
because the
termination pcint
of
the
tracking is not always precise,
Note
ttat this
either t!:e line is
process can be
repeated until
successful (that is
connected to other lines or line se~ments are found by
FACE 5
circular search).
( 6) If the boundary
the
lines of a body are
boundary that might have
known, select the vertices
interral lines starting at them.
selection of vertices is based
find an
internal line
The
on heuristics such as seletin2
right vertex rather than lower right vertex.
which is
of
upper
At each vertex, try to
nearly parallel
to other
boundary
T
Fi 1.l
for
examrle, internal line JS is parallel to the boundary line KL or
IQ,
If
lines.
and (S ir
one line
is
found,
parallel to R1 or IJ.
XZ.
rarallel to
parallel
Thus it
track
along it.
In
Line FL is parallel to ED and XV is
is often
useful to
Note that search for
same body.
to boundary lines of the
lines
find internal
rarallels has small area.
( 7)
If
no line
is
found in
try
(6),
between adjacent boundary lines.
it.
to find one
search
by circular
When one line is found,
track alone
In fir,.6, circular search between BA and BC is necessary to find
the internal line BE.
( 8) If
two internal lines meet at a vertex, try to find another internal
line startint at the vertex.
is
where
no
line
internal
This process is
was found
in
used in two cases.
(7)
difference in brightress between adjacent faces.
that
because
One
of little
Suppose irt
Fig.1,
the internal line SJ was not found at vertex J, but that WI and
(S were found.
toward
found.
bodies.
J. If
Then
try tc
there is
The other case
find an
internal line
enocuh contrast
is where a body
near S,
starting at 0
a line
se.ment is
is partly hidden by
In Fir.6, the trianrular prism is
partly hidden.
other
After 'F
PAlF 6
and CE are found, EF is searched for.
of
direction
sometimes not.
If it is
sometimes predictable and
the line is
predictable, then try that direction.
the
In both cases, the
failed, then ayply
predicted direction
if
If it is unpredictable or
circular search between
If one line is found, track along it.
the two internal lines.
S9) If an end of an internal line is not connected to any line, try
find
lines starting from the end
to
If lines are
by circular searc~.
found, track along them one by one.
(10) If no line is found in (9),
extend the line by a certain length
in (5) and test if it is connected to other lines,
try circular search again as (9).
until successful.
line
M{'
if
not connected,
This process can also be repeated
Fig.7 illustrates
this process.
In Fir.7
is not connected to others at N', thus ster (9)
N' and fails.
aas
The line is extended to Ai
(a),
is tried at
and (0) is again
applied.
This process is repeated until the line is connected to line XL at N.
Fig.7
(b) shows that line H1 is extended by this process to P1 where
a new
line
is found
extended to }4.
by
circular search.
Similarly
line
CO
is
This process is useful so as to not miss a new body
sitting on ar obscure edge.
At
each
stage
when
an
above
step
is
finished,
information is interpreted as shown in Fig.2 (block 3).
tracking
along a line termirnates, a test
the
obtained
For instance,
if
is made whether the line in an
extension of other lines and/or the line is connecte6 to other lines at a
vertex.
If
a boundary line is
connected to another boundary line,
the
PAGE
lody havinc the lines is
split into two bodies and the propertiet of both
lines and vertices are stored in an appropriate structure.
exaamle, line N0'
is
is
umini
the
of these
obtained.
two lines.
Then,
boundary
12 and
two bodies
We
I3.
of B3 which hides a part of
it
is
Before the line was connectoe
CO, there were two bodies Bi and B2 as in Fig.8 (a).
tplit into
This line
I;i, and I'[i and N'P" are merged into one
equations
connected to CO and Fig.8 (b) is
to
In Fig.S, for
obtained by tracking starting at point 1J
interpreted as an extension of
straight line
7
line NO
can interpret
2,.
Now body j2 is
as
the
The other properties of lines
Fnd vertices are obtained similarly at this staMe.
2,2 Example
We illustrate the entire line-findin, procedure with the aid of the
examarle shown in Fig.9.
Ri,
At first, the contour lines AB, EC, CL, DE, ES,
;h, 1 1, 13, jX and RA are obtained as shown in Fig.9 (a).
described
in the previous section is tried
Step
for the concave points 0 and
J. In this example, the position of 0 is not precise enough to find
extension
of EG.
extension of
Cn
the line
the other
QJ.
hand, a line segrent
KJ is
(1)
extended by
trackin
the
is found as an
as
far as
L.
Locamise there is no ether point to which ster (1) is applicable, ster (2)
is
tried
for point
tracking teri:inates.
0. One line segment
Thus a line G':'
is found
and
extended till
is obtained as in FiQ-9 (h).
line is interpreted as an extension of FG and connected to JL.
This
Then the
rosition of I-oint F, 0, L are ad2usted to as shown in PFi.9 (C). Now two
bodies £1 and B2 are created by the boundary lines GL and JL. It is
important
to notice this, for it means that step (1) is again applicable
(to point L) at
this sta•e.
'1g1.? (d) (Note that line KN'
as
FL is extended as
has not yet been fcund).
an extension of FL but the end
point M is
LK is
new line LM.
f•re neither ster (1),
far as i in
interpreted
not connected to any ether
Thus vertices F, 0, L and end point M are adjusted
lines.
the
Thus line
considering
(2) nor (3) is arplicable, so
that (4) is now applied to M. Three lines are found by circular search as
Fic.9 (d).
trackingr.
UN' is
determined
as
a boundary
line
and
extended
by
Ghen it terminates, the line is connected to boundary line BC
at vertex H as in Fir~9 (e).
Body L1 sylits into body F1 Ard B1.
It is
known at thir staze that B1 is hidden by 83 and E2 is hidden by r2rtly E3
and yartly by Bi.
each
text, step (6) is applied to each body one by one
at
ti.•e selecting the easiest body for proposing the internal lines(in
this example, the order
E2).
is L3, El,
B2 because E3
hides BI which
hides
Internal lines CO and MO are found and connected at vertex 0, but
no line segment is found using step (6)and (7) applied to vertex E (this
stage is shown in Fig.9 (e)).
segment toward E is found.
in Pig.
Step (8) is applied to vertex 0 and a line
This is extended by trackinr
as far as E' as
(f).
Line OE' fails to be connected to any other lines which
activatet step (10).
After a few trials, OF' is extended to connect to
vertex E. Similarly, internal line AL is obtained for body El and line IF
is obtained for £2. When every ,tep has finished, three bodies are known
together wvth the relationships between them.
PAOG
9
3. ALGORITEMS
This
the algorithms that are used
section describes the details of
Some of them,
in findinc contours and in the steps stated in section 2.
such as tracking and circular search are used in more than one step.
algorithr used in more
than one ster may
be slightly different in
each
In the tracking algorithu,
but its essential rart is not changed.
step
An
used for
for examrle, some chanzes occur dependin- on whether tracking is
tount-try linrs or internel lines.
3.1 Contour Finding
.i.10 shows the
the procedure to
oLtline of
picture date obtained with an
The
find contour
lines.
image dissector usually consists of a
larce number of Toints (say about 100,000) each of which represents lirht
To speed up the proce.sing, one point for every 8 x 8
intensity level.
Foints
is smmpled.
This compressed
picture data consists
of 1/64 the
number of points in the original picture.
To find the contour, this data
found.
The judrement of contour point
is scanned till
a contour point is
is based upon the simple assumption that there is enough contrast between
the background and objects.
is
a
ncise point.
contour.
data
is
If
Thus a set of
It is then checked whether or not the point
it is
a
point, trace
real contour
contour points are found.
Then, the
is again scanned until a new contour point is found.
repeated for
all the
picture data.
When
all
alonp the
the
points hrve tnen found, each set is separately analysed.
pictutre
This process
sets of
contour
PAGE 1C
Suppose a certain set of contour points is to be analysed.
return
to
the
armroximately
high-resolution
original4
the position of the boundary
data which corresponds to the first
The
precise boundary point
picture.
aoint in
We
We
can
ruess
the oricinal picture
point found in the sampled
is searched for near
now
picture.
this point.
A set of
contour roints is obtained by tracinp from this point in the same way
in
the sample
picture.
A polygon is
formed after
we cormect contour
points one by one.
To classify the points of this 'curve"
the 'curvature'
the polyron
ct
::
ricture iK defined
the
is
used.
This curvature
re for convenience as shown in
figure represents a
contour point.
as
into reiments,
in a
Zig.11,
The curvature of
diy-it-al
Each cell in
a point P is
defined to be the difference in anmle between PR and PQ (a), where Q and
E. are a constant nunrber
If we plot the
of points away from P
curvature along the
(6 Toints in this case).
contour as shown
in FiC.12, we
ca,
tell what part is near a vertex and what part lies in a straight line.
that curvature is not very
Note
error.
result
sensitive to noise or diritization
If we intetrate the curvature
is nearly zero
in rpart of a straight line,
despite the effect
of noise.
If we
the
sum up the
curvature of consEecutive points whose absolute value is treater than some
threzhold, we can determine the existence of a vertex.
That is if
sum of the curvature of such points exceeds a certain threshold,
a vertex near those roints.
either
this
there is
Thus every contour point is classified to be
in the straifht part
of a line cr
near a vertex.
Usin. points
which belong to the .traight part of a line, the equation of the line
calculated.
Then each
vertex is
decided as
an intersection
is
of two
I ALF 11
ndjacent lncs.
3.2 Line setment Detoction
A
line Se~3~ent is detected given
This procedure is used
in most of the
cCnsiMts of two 1arts.
procedure
its direction and starting point.
steps stated in section
1.
The
to detect the possible feature
One is
The other is
roints which are to be regarded as elements of the line.
to
test whether or not obtained feature points cake a line segment.
In dotecting featurFe
ies,.
oints, we
an
edge into
proposed 3
detector
types of
is
(u,v),
boundary dctectors,
an ede-effect
U-V such
to be detected.
of
this paper,
In
U is
a roof
type-
edges can be detected by a
addition,
edge-effect types.
that
edge-effect, and
most roof
We
set up local
direction of
the
type
the
line
Let TIu,v) denote the light intensity at roint
and define the cortrast function Qi(v) at (u,v) as
Eg(v) = C
I2 1(1
+ iV
+ j)-I(u + i,v -
Suppose we have an intensity profile as
in
roof and
In
detector.
by step or
are accomnanied
Cartesian coordinaten
vegCment
namely step,
not considered because roof type
Ltep detector or
edges
3 typs,
various types
classified the light intensity profiles
:Herskovits and linford
across
should consider
Fig.13 (b) is the difference of
)
shown in Fir•13 (a),
Fk(v) at F
summed intensity between area Al and
4i(v) for a typical step tyle profile (Fit.13 (a)) is shoin in
Al.
The tyrical
'ip.13 (c) in which the edge is detected as the pepk.
-rofilas of YA(v) f'o- ott~r types are
shown in Fi7.14 where the edhe
is
PAGE 12
detected as the middle point between positive and negative peaks.
The
its position.
leak of QE(v) and
to detect the
basic procedure,therefore, is
The necessary properties for
(see
a peak are as follows
Fig. 15).
(a)
from v4 to vr
if WF(v) ranyes
, there must
exist the maximum of
FP(v) at vr other than v4 or vr.
where f" is threshold
(b) PF(v,) > fj
at v, between v4 and vq arl a
(c) There rust exist a minimum of Fg(v)
at v 1 tetween v,
miniaun of T(v)
where f1 is a threshold
F4 (vN) - F0(v2) > fA
If
is found,
such v1
and vy such that
the left of the peak (vq) and the right of the
teak (vt) are determined as the intersection of YA(v) with the line Fyv)
= ft as
Fig. 15.
shown in
The value
of ft
depends on
Fj(v,) and
is
represented as
ft
W cIF(Qv)
+ 0,
o, are constant and 0
where o, and
The
c<
c
< 1, C
>o 0
positiorn of the peak v, is obtained as the middle of v, and vy.
more than one peak
nearest
to the
is found between vq
middle of
similarly detected.
obtained
feature
as the middle ef
found, (although
positive
A
v( and
and vy , the
point vp
adopted.
A negative
vr is
point for
an
edge-effect or
the rositive and negative
the threshcld
fz is
or negative peak detection).
not
Thich
the same
If
is
peak is
roof
is
peaks, if both are
as in. the
simple
This method for the detection of
teaks and positions is nct aypreciably affected by noise.
PACE 13
The other part of the
linearity
line segment detection ie
at P0 arnd suppose the
lines Lo and L1 meet
tested as shown in Fig. 16.
concave boundary
Suppose
feature points.
the detected
for
line segment extending LO
is
is
Feature points are detected in a rectaruular
search area with given length and width whose direction is
of L,(=-),
co-
a test of the
equal to
that
at an appropriate place where the detection of feature points
not affected
detected alons'
sequentially,
corre.ponding to L1 .
by the edte
the
direction
v
at the
Feature points are
pcints
center
Py ,P. ,p..,P,
If Tositive peaks are found at INMc,...,M"
as shown in
the figure, the linearity of the points are tested as follows.
The nutxber of the feature yoints must exceed a threshold number n
(a)
(b) The
deviation
of the
dS
points in
line fitting
least
with the
square tethod should be less than a thresholdd4.
(c) let
L' denote
the
direction of
segment obtained. by line
line
fitting,
Iu'-
U1 < U,
where •l' - Uj denotes the diffrence in directions U'
Similar tests are made
If
for the different
types of feature
more than one type of line segment is found,
If an edge.-effect type is found,
(b) For the line segment with
then it
is
selected.
' and U', let the criterion
function C be
C = SJ
wu lU" - Uj
where w
points.
the selection depends on
r
criteria.
the followinr
(a)
and U
is a constant
PACP 14
The line segment selected is
the one wiht smaller C.
3.3 Circular Search
Circular search is used to search for lines starting
at a riven
The direction of the lines to be searched for is not known. The
point.
of directions in circular search depends upon the particular case.
range
Suppose two known lines L1 and L1 meet at P as in Fip.17 (a) and
we
and
L1
are slightly inside of
and Lj whose directions
between two lines L'
If lines
resrectively.
The search range ok is
between them.
wish to search for lines lyinh
at point
starting
suppose
P
L!
Le are
of line
searched for as in Fig.17 (b),
L' and LQ are similarly set inside of LO.
The
circular
center
F of
point
the
search is
always rrecisely
not
determined, especially when trackinr along a line has terminated at point
I as shown in Fig.17 (b).
Therefore
circular search should not be too
sensitive to the position of the center point.
It
malgt be natural to try to
section 3.2,
based
upon
line seements if there
arcs
is more than one
difficulty, a simple
method is
to
directions.
apply
This
segment detections
Oiroctior
along
around
the
The
center.
with this search is the classification of feetture points into
difficulty
this
F 1 (v)
detect feature points, as defined in
cthange
line
is
toward
algorithm is
segment
and search
area
deternined so that line segments of
used
detection
illustrated in
ul ,ut
as shown in Pi•. 18.
and u
To
in this paper.
successively
Fic.19,
are
(AI,AL and
where
applied.
A3
avoid
Its basic
in
various
successive line
The
in the
step
of
figure) are
any direction near the center
point
PACF 15
can
be found.
find lines
Fig.7 can
at roints
starting
the same as described
line setment detection is
respect to t1reshtolds
and search
directionr overlap
different
found in iifferent searches.
each other, tIhe
center
adjacent
The alporithm
in 3,2 except with
Because the
area.
search areas
for
sezment tay be
sane line
Fach time a line segment is
detection, a check should be made
~e.anent
between two
line L in Fig.2C starting between PI and Px).
roints (e.g.
fbr
Thus successive circular search along a line as shown in
found by
whether or not it
is
line
the same
e one obtained by the previouc detection.
ns the
If
centor roint
the
is
it
precisely,
are found (such as
has not
possible to find all
line Lz
the figure), a new
Li and L 3 in
line (L 0 ) and
been
determined
the lines startin, at
To avoid this inconvenience,
based on the known
calculated
circular search
Fig.21., for example,
search at Fo.
circular
ight
be missed
in
when line segments
center point FP
is
the obtained line sechents
Then circular search is arplied again at P1 .
and LI).
(LI
not always
In
the given point.
of
3.4 Tracking
TrackinE is used
until
it
when a line
terminvtes.
the line should not be
segment is given,
to track alongy
it
a tracking procedure are 1)
The requirements for
lost due to the effect
of other lines or
noise,
and 2) the procedure should terminate as precisely as possible at the end
of
the
line.
termination
These
requirements
are
contradictory
in
that
the
concition should be strict to satisfy the second requirement
which makes it
difficult to satisfy the first.
The following
al•oriths
PAGE 16
is a comiromise between these requirements.
basic procedure is
The
and to search for it
near
result of the search is
(a)
to predict the location
of a feature roint
the point using line segment
detEction.
The
classified into the followinr 4 cases.
there is no feeture point.
(b) a feature point is on the line.
(c) A feature point is not on the line.
(d) It is not clear whether or not a feature point is on the line.
In
case (a),
the
detection of a
serment dctxction except that the ty-pe
feature
The
decision between cases
(b),
distance d between the point and the line.
If d < d1
then case (b)
if
then case (c)
d > da
If dj< <d
d.d
similar to line
of edre is already known so
the thresholds stated in 3.2 can be.Edjusted
Ya(v).
•oint is
that
based on the averare peak of
(c) and (d)
is mn.de tsinc the
That is
then case (d)
The threshold d I chanzes derending en the state of trackin
.
The
state of tracking is represented by two intefers mi and r,. which are set
initially to 0.
(b),
The value of m and m are chanced for each case (a),
(c) and (d) as follows.
(a) a, = at + I
(b)
If
wm> *m,
Otherwise,
m, =
- 1 , (where m is a constant)
m -= 0, ml =0, and classify those feature points
into (b) which hpve teen clasified into case (d)
PA' ;
in the previous steps of tracking.
17
Adjust the
equation of the line with these points and the
•oint
present feature
(c)
If d
dz3 and m. > ma,
(where d is
=
- 1
a constant)
Otherwise no cYanre
m, = mX + I , and if
(d)
mi)
m,
= M
gr
Im -
The threshold d, is represented as
d| = d 0 + w, m
(where d o and w% are constants)
procedure is rereated and tracking rroceeds step by ster
Ihit
the line until the termination condition is satisfied.
cxtendinr
The termination condition of trackinr is either
am
> mI
The terminal point is
defined as the last point classified into case (b).
In
this aleorith works.
lig.22 illustrates hcw
lig. 2 2
of mn
these
however determired more
in the
The terminal point of
figure) which satisfies the termination condition.
trackinr is,
lines
two
Fig2•2 (a),
Irackinc might finish at some point beyond Pg (PI,
at P 0 .
cross
(where m~ and mt are cornstants)
+ a. > mt
or EI
precisely near PO(P i
cr
In
Pt).
(b), P1 ,• , 1 3 P Pare classified into case (d) increasing the value
points
which
Then the line is
into case (b).
which classifye P
are
now
classified
into
case
(b)
adjusted
with
and tracki4cn
rroceeds.
Fir.22 (c)
Trofile is
end (d)
illustrate that
even if
disturbed by roise or other lines,
a part of the
trackiner
intensity
does rot terminate
PACE 1Z
In Fig,.22 (d),
there.
however, if
the light intensity of the right side
of L* chanses across L l , the type of feature points might change across L,
.
eoints P3 ,P
Thus feature
rroceeds.
If
found,
If not found,
rosition of If
often
extension of
is found.
segment
,".
might not be obtained and tracking mipht
When tracking terminates,
terminate at P1.
applied at the
9
the line
the line segment detection is
to see
we adjust the
if
line
tracking finally terminates at point P1 and the
extends the line across other lines when it
vertical lines.
type of
line equation and tracking
is adjusted with the line equation.
a.t their crossingr
another
The above
rrocedure
terminates temporarily
as in Fig4. (b) where tracking along G'"M
crosses
many
PAGE 19
4. EXPFRIMFNTAL RQULT
To
test the
having relatively
The
proCram, experiments
muiform
A•&ND COMMENTS
with cubes
are made
white surfaces placed
on a black
imate dissector camera , used as an input device,
onto a 2C0
every
x
8 x
20000 (octal) grid.
L block
of Frid
In this
elements is
represented by 1C24 x 1024 grid
points.
and wedges
background.
dissects the scene
experiment, one pcint
sampled.
Thus, the
for
scene is
Objects occupy only a part
of
in the typical scene, the rectangular area which includes the
the scene,
cbectts cf interest mDay consist of about 400 x 400 points.
This area is
eivided into blocks each of which is made of 64 x 64 points and stored in
disk memory.
When a lieht intensity at some point is required, a
containira tGo :int and adjacent blocks are stored in core memory.
accessed for
core memory is
the iniut of
the light
.lock
The
intensity until a
joint outside of those blocks is refererenced.
Video input is at first converted into a 10 bit digital numLer which
is
an
inverse
converted
into
resolution is
linear measure
10
bit
logarithmic
the light
Some
measure.
by a little
It is again
intensity
level
In this experiment the
less than IC0
levels.
The
t edge in the dark tackground is blurred due
dKta for a clear trif.
to some limitations (mostly defocusing).
step
intensity.
lost in the logarithmic conversion.
light intensity is represented
input
of
function, there
is a
transient area
If
the intensity change is a
in the
input data
about 1I
Thus the resolution of the picture is regarded as 1C
roint.s wide,
ynints.
The arnmeteru In:d in line scfnment detection and trackinr are
PAGE 2C
Features of the picture involving resolution
based urcn this resolution.
of less thean 10 points are not usually found.
The difficulty or processing time
Some results are shown in Fig.23.
of the recognition depends not only
also
on the
SJ,
lines
of contour lines.
KS
for
as
the
and QS
less
reliable
in
easily
1.1 is applied.
after DY and rL
That is,
line
a
segment,
But tracking along VY
and
are
Circular
more
time
all the internal
in Fig, 2 3 (c )
Then step (10) stated ir
and F2 in Fiv.23 (c ) terminates in the middle.
section
it is not easy to fnd.
lines are determined,
lines are proposed in both cases.
proposed
at K and L respectively.
findinS
Once the boundary
consuming.
are
On the other hand,
circular search is necessary
search is
Fir.23 (c),
In
each stage.
in Fig.23 (c ).
boundary lines K0 or LN
found,
but
information extracted at
exn.qy:le, boundary
extension
on the complexity of the object
This is the most time consuming proce 1s (about
10 times more than the simple tracking process).
a hierarchical rrorgxam are shown
Some examples of the result of
lig.24.
and
Ieierarchical programs may look at the whole scene homoreniously
pick up feature points.
obtained in the
priori
in
the
Lines are
previous stage.
various thresholds
It
for
fitting and connection of lines.
In
found with those feature points
is very difficult
detection of
to determine a
feature
this beterarchica
points, line
progranm,
it
is
Tessible to adjust various thresholds with the context of the information
cbtained
previously.
case by case.
whether
Furthermore the algorithm
(For instance,
the line is a boundary
trackingr
itself can be modified
algorithm is
or internal.)
changed dependinr
on
The results of experiments
PAGE 21
with moderately complex scenes are mostly satisfactory.
many
Because of
the
checks for consistency of lines and vertices, the yrogran has small
false lines.
probability of findin;
However,
there are come limitations of this program at present.
of them is that lodies may be missed in some cases.
shown
in Fi-.25.
yroposed thovlh
Wbcause
The boundary
other contour
the
the resultin,
activate steo
necessary
A simple example is
EC in Fie.25
lines lB and
'neat'
that
(a) are not
lines are
lines and internal
recions are so
One
found,
no conceive vertices
in such a case when bodies are neatly stacked, it is
(1).
the -oundary line.
from some points on
boundary lines which start
to search for
To find a body
In !ig.25 (b) body E2 is not found.
that is included in a face of another body, it is necessary to search for
Though these two kind of search (search
line segrents inside the region.
adonc
the boundary line and
all the bodies in
search in the region)
are required to find
shown in Fig. 25,
they are still more
the scenes as
effective than the exhaustive search in the entire soene.
simpler
to interpret the scene
of the present
The other limitation
that
it
is
not always
Fig.26 (a) shows a simple oexnaple.
line
CBE
If all the bodies
boundary line as shown
bodies.
found by those searches.
is left to future work.
This procedure, towever,
introduction,
when a line is
Besides, it is
as
program is,
applicable
Line SD is
to
stated in
the
concave objects.
found as an extension
of
are convex, line ED is interpreted as the
in Fig.26 (b).
In this rrog•am, line ED is
This does
not hold for
concave
rewarded as a boundary line, and
then line DI can lbe found by circular search at L:. At this stare, however
PACE 22
LE -should be interpreted as sn internal line of the same body instea-d
the
boundary
line which
interpreted correctly,
line.
seperates
then line
the
BD can
body
into
two.
If
of
DE. is
be determined as an internal
This procedure should also be implemented in the present program'
PACE 23
CON CLUSIONf
A hetererchical program to
rrorran
be
detected
tackrround),
use of the previous results.
is
1) contour
lines
The order of the lines
Cboundary of bodies and
lines or among
of two faces
internal lines, the
of the sane
line, the range where
'most plausible lines' are
a line segment may exist
detected in a suitable way for
AmonC
body.
To find
rrorosed at each stale and an attemrt is made to find the line.
a
the
2) loundary lines which are the boundary between two bodies,
Y) internal lines (intersection
toundary
The
of recognizing objects step by step,
is based upon the strategy
at each time inakin
to
recognize polyhedra is presented.
is proposed and it is
the proposed range.
If a prorer
line
sepment is found, the end of the line is determined by tracking alone the
line.
then
scene taking
proposed
into consideration.
this line
Because
lines are
mostly
instead of found by exhaustive search in the scene, the program
is relatively
dissector
the line is determined, the program tries to understand the
effective.
Eesults
of
the experiment
using
an
image
are satisfactory for scenes including a few blocks and wedges.
Although the
present
program
has
limitations, some
overcome by developments proposed here for future work.
of
them
may
be
PAGE 24
RF~ERENCES
1) N. Minsky and S. Papert,
'Propioal
artificial intelligence at M.I.T.,
to ARPA for research on
1970-1971', MIT Project MAC
Artificial IntellIsence Memo No. 185, 1970.
2) A. Guzmman,
Computer Recornition Of Three-Dimensional Objects
In A Visual Scene',M•i
T MAO-TR-59 (Thesis),
1968.
3) A. Herskovits and T. finford, 'On Boundary Detection', MIT Project
MAC Artificial Intelligence Memo to. 183, 1970.
4) B. Horn,
'The Ima.e Dissector 'eye'
Intelligence Lemo No.
178, 1969.
', MIT Project MAC Artificial
Fig.
1.
fxample of Scene
1
2
Extract the most
Extract the most
obvious informa-
obvious informa-
tion based on
tion based on the
a priori
knaowledge ob-
knowledge
tained up to now
2
Interpret the
Information and
store in
an
appropriate form
Fig. 2.
Schematic diagram of the strategy
F--
C
A
-r
_rP,·
R
~·C
_Ccr
i`L
~
~
-~i
L
`t~51
(c)
Fig,
3.
Examples of concave boundary lines
~pb~
(d)
C~"I
Fig. 4.
llustrates two extension lines at concave point P.
d
*2
B
C
S'
Fig. 5.
Examples of line configuration found by circular search.
Fig. 6.
Example of circular search for internal lines.
(dotted lines are not yet found in this stage)
Fig7.7,
Examples of line verifying by circular search.
(a)
Fig. 8.
Illustrates the process after tracking
(b)
(e)
Fig. 9.
Illustrates the procedure to find lines,
Fig. 10.
Flow chart of contour finding.
Fig. 11.
Illustrates the definition of curvature
I"-----
··v
-
-
-r
,
L
*~y~y
__
-~-_^..~-~----
D
C
boundary points
Fig. 12.
Example of curvature.
.lI
E
-~- -
F
(a)
lighR
intensity level profile
U
Alb
oputeo
(b)
P
Areas to compute Fyat P
. . ..
_
.. ..
.
___
I
V
(e)
Fig. 13.
FP U (v) for (a)
Examnple of F u().
-'
L-~.~,_..,~._~_._~
V
-l
tdge-effect
Fig. 14.
v•e
I
Typical profile of F (v),
V
Vm V1
v --
Fig. 15.
Peak detection.
v'2
v
Po1.
6 P1 P etion
P Pa P
e
search area
Fig, 16.
Detection of feature points.
/I\
s
J' C
r
r
L
ri
L1
(a)
Fig, 17.
Range of circular search,
~rs
(b)
1L
~
K represents a feature point
Fig. 18.
Illustrates difficulty in classification
of feature points.
~1
Fig. 19.
S5uccessive line detection in circular search.
La
Fig. 20.
FI2
Exampie of successive circular search.
Po
LO
3
3I"
Figý 21.
Repeated circular search.
r,
ILL
ýP3
6P2~
LNb
P4
P2P
4--p
7
S,-
--
e3
Lo
L
I
(a)
Fig. 22.
(b)
Examples of tracking.,
(c)
(d)
0S
(a )
(ct)
C
(a2)
Objects,
Fig. 23.
(c2)
:
(c
):
Result
Examples of Experiment.
(a3)
(C)
(bg)
(4 )
(b00
(at):
Qci ) :
Objects,
(b i
:
Result of Hierarchical Prograa
Result of this p~roSgr
Fig. 24.
)
Examples of Comparison Betve~n Hierawrchical
and eteTrrcbhical Program.
(a)
Fig. 26.
(b)
ILtustrates difficulty for concave body..
(a)
Fig. 25.
I1luatrates lack of cu~es.