Vision Flash 4
December, 1970
Revised
February, 1971
The Object Partition Problem
Eugene C. Freuder
A.I.
Vision Group
Laboratory
M.I.T.
I.
We
a
provide
for
framework
theoretical
the
"object
partition problem".
begin by considering this
We
Consider the semantic domain of physical objects, and
context.
the
picture graphs.
syntactic domain of
yield partitions
operations that
interpreted In the semantic
simply
syntactic
which
picture graph,
We term
objects.
partitions"
problem
The
has
such
or
several
partitions, the one "best"
may seek all possible
We
specifying
realizable
partitions".
"realizable
The object partition
domaih correspond to possible
"physically
partitions
aspects.
of the
scene into physical
partitions of the
syntactic
of
terms
In
viewed
be
can
problem
when
problem In a rather abstract
partition, several plausible partitions, judge or rank
proposed
partitions, and so forth.
The most Important aspect of the problem, In sone sense, is
determining
the
"best"
or most
graph.
combined
choices on
These
to
choices
inform
the best
are
partition of
somewhat
global
Guzman
uses
physical world of objects to
understanding of the semantic
make local
realizable partition.
deal with this problem.
Guzman's SEE attempts to
his
likely
a given
Interrelated.
decisions
which
may
picture
They are
also
be
interrelated.
Guzman's
achievement.
SEE
is
a
However, as
remarkable
might be
heuristic
expected of
programming
tho
germinal
achievement in its
framework.
field, It lacks
There
motivations
Is
some
a satisfactory
difficulty
and implications
in
of Gukian's
theoretical
determining
heuristic decisions.
It is not always obvious just what semantic observations
the
the
selection of the syntactic partitioning choices.
inform
As it is
not clear which possible Interpretations are being discarded
each
stage
of
alternative
the
procedure,
partitions,
the
even
process
where
cannot
produce
several
interpretations are preseht in the physical scene.
at
plausible
To judge or
extend the woek, one must essbntlally rePeat Guzman's experience
with individual scenes.
The
program
does not
function as
heterarchical vision system.
which
to
relevant
launch
to the
flexibility to
dialogue
a
gobd element
for a
There Is no proper framework from
with
other
object partition
provide alternative
knowledge
problem.
analysis
structures
There Is
on the
not the
basis
of
higher level dissatisfaction.
It
may be argued that in order to establish a satisfactory
theoretical base
object
for a
partition, one must present
with all possible
graph,
It could
One criteria
physical scenes,
relevant property.
object partition
deal with
a given
as
would be
picture
This would require
to be "complete"
any physically
of completeness
such
a system capable of dealing
physical interpretations of
in terms of the
our theory of
that
property of
In the
sense
realizable partition.
that the
theory
could
all realizable partitions, though
produce
the system would not
necessarily be "generative" oriented.
A
partitions would
which
characterization
"complete"
framework
would
choices
these
In
among possible lbcal partitionings
Decisions
In determirlng the "best" possible global partition.
involving
object
realizable
an organized
hopefully provide
choices could be made
of
be
semantically
and
heterarchically Informed, and their Implications and motivations
would be clearly understood.
Alternative choices could be made,
in some plausible order.
Such a characterization would
by
eliminating rncontistent
cut down our decision
space
partioning choices.
or impossible
The nature and range of the remaining heuristic choices would be
clarified.
Recently Huffman has
analysis,
by
approached another
problem of
scene
the "configuration" problem, in somewhat this fashion
attempting
configurations.
to
identify
physically
unrealizable
The results of his venture recommend this type
of approach.
However, Huffman's success has also encouraged
that
his
theory
may
contain
characterization approach
The
"physically
realizable
realizable
partition"
Interdependent.
to
as
well
the "object
the
are
desired complete
partition"
configuration"
problems
speculation
and
problem.
"physically
Interrelated
However, they are not Identical, and it
and
would
be
a mistake
to base
an approach
to one
units that characterize the other,
problem
however;
has been begun
some
The treatment of the former
by Huffman.
relevant
upon the conceptual
Much remains
observations
will
to be done,
appear
in
future
papers.
The
"characterization" approach
to the
"object partition
problem" will be outlined below.
II.
We take for our
basic units of
analysis a line
predicate
and its negation.
The predicates are "belong to the same body"
and "do not belong
to the same body",
as applied to the
(two)
regions bordering a line.
This
or as
choice of our unit of
circular as
predicate
defined Is
The success of
that
may appear.
To
NOT dquivalent to
the link
the more fundamental
earlier.
that
It
analysis Is neither as trivial
begin with
a major
predicate we described
reason
was not used
in partition analysis
"link", while a useful concept,
basic
the predicbte "links".
predicate is probably
However, deep problems
the
reveal
is actually a handicap to
optimal thinking when used as the basic unit of analysis.
We note that "belong to the same body" does not even
that
imply
the line referenced corresponds to a physical edge of both
neighboring regions.
Consider line AB In the following figure!
by
begin
We
In
vertices,
terms
of
two
our
of
partlonings
realizable
obtained
"labellings"
or
interpretations
from all consistent
of
various
the
types
of
All physically
predicates.
given
a
realizable
physically
all
enumerating
picture
graph
then
are
combinations of local labellings.
(Using the criteria that an interpretation applies to an
entire
line, i.e. line segment, and thus a line cannot receive opposite
labels from its two vertices.)
We
will describe this process In some further detail.
we
demonstrate first how this approach circumscribes the realizable
interpretations, in achieving a solution to the "all
realizable
partitions" aspect of the object partition problem.
We then go
on to
indicate
decision
making
how this
approach
procedure which
partition and related
problems.
provides a
deals with
We will
basis
for
the
"best" realizable
gradually shift
our
focus from an abstract theory to a theoretical model embodied in
a
"partition system" embedded in a heterarchical vision system.
We will
find
it
easier
perhaps
to
talk
in
terms
of
labellings
and deal with picture graph elements so we Introduce
same body" and "M" for "do
the notation "I" for "belong to
belong
to
same
in
syntactic elements
interpreted
the
picture graph
If
are,
(These notations
body".
you like,
which
language
not
are
as the indicated physical relations.) We will refer
to "I" labellings as "ties" and "M" labellings as "breaks".
If
we simply consider any possible syntactic labelling of an n-line
vertex
we
have
obviously
However, we make
n
the
to
2
possible labellings.
"tie"
the restriction that
equivalence
is an
relation, using an observation of the Oropertles' df the "belong"
eliminates
an
For
predicate.
transitivity
vertex,
n-element
thus
all labellings with n-1 tie labellings and one break
labelling.
Consider theee
realizable
labellings
labellings
with
now
nd "ties",
breaks, and with three ties.
31/013!
The
line vertices.
+ 3!/112!
corresponds
All
number of
to
the
"breaks", with
physically
number
one
of
tie, two
I.e.
+ 31/3101
=1+3+1
= 5
A similar analysis can be carried out for n-line vertices.
In considering vertices of specific forms, however, we
find
that
different symmetries
further
reduce the
different labellings or group them into classes.
the
vertex
labellings
that
have
a
physically
may
number of
We will
call
realizable
interpretation simply "realizable
Examine
labellings".
the realizable labellings
for the different three
line vertex types, forks, arrows, and T's.
Forks
Arrows
T's
T
TTT
T
We can easily dispose of the two line vertex "L".
Consider also
the
of
Interpretable labellings
"K"
type
vertices.
X4 K
Yh
We
of different realizable labelllng
find that the number
classes even for the K is surprisingly manageable.
we
have placed no
restrictions on the
arbitrary planar polyhedra), or
picture
elements.
If we
physical domain (beyond
on possible Interpretations
wish to
do so
we may
number of realizable labellings 6ven further.
may
Impose
restrictions of
Recall that
physical
cut down the
For example,
objects to
of
we
degree three
polyhedra, and certain "general position" requirements.
(We might
totally
note
that this
analysis
does not
pertain
to
disconnected bodies, e.g. joining the two halves of the
partially hidden body in the following figure:
This Is a different class of problem.)
Applying
realizablb labellings
these
consistent
combinations
physically
realizable
problem.
partitibns'
that
We note
a
provides
to
finding consistent
involves checking foe "global trahsiltivity"..
be
asserted
transitivity
to
belong
of the
as
S.
We have
the
same
body
"tie" predicate while
"break" predicate asserts
body
to
that R
restricted
df
have to guard
aý
region
at the
our labellings
boundaries, as in the following figure:
'the pbatition
also
A region R cannot
does hot belong
against its occurence
"all
the
coribinations
inconsistency cannot occur around a single vertex;
still
to
solution
aspsct
in all
a scene
S
by
same time a
to the
so
same
that this
however,
we
across several line
III.
partition problem has
of the
complete characterization
A
been achieved.
dealing
laid for
also been
foundation has
Beyond this a
the "best possible realizable partion" and related aspects
with
The objective here is to produce the
of the partition problen.
partition,
plausible
most
with the
retrench and
to
ability
produce alternatives if required.
Any realizable partition of a picture graph may be produced
of realizable labellings depending on
a number of choices
have
choices, or rate them
on a plausibility scale.
These
in the sense that they affect our choices
factors may be global,
of arrow labellings, say, regardless of where the arrows
in
or
We consider factors Which may prohibit
the type of vertex.
dictate
we
At ahy vertex
by an approPrlatd labelling of..the vertites,
the picture graph, or
they may be local,
appear
In the sense that
they affect a decision at a particular vertex.
(Or their effect
may fall somewhere in between these extremes.)
Our first
context
the
general
polyhedra.
the
labellings
Cstat
Ist CsiI
be to
return to
the
semantic
in which our picture graphs are to be Interpreted.
most
as to
approach should
lM I
level
this
means
combinations
of
On
planar
By studying such combinations we can make judgments
relative
for the
C
ns
plausibility
of
different vertex
1A1
k
the
various
types.
#th
+t
realizabl]
Even in
A
-
i1
1
a gross
b
lI
i
g
e
e,
e
can
o
serve
a
er
may
observations
be
to
extended
certain
These
others.
or likely than
interpretations are more common
combinations
of
vertices.
this stage we may recognize that certain restricted
Beyond
physical contexts limit the range of Interpretable labellings or
otherwise
affect
the
Interpretations.
likelihood
of
certain
We observed above, for example, that limiting
the physical domain
effect.
relative
to degree three
polyhedra would have
this
Limitation to convex objects, or likelihood of concave
objects, would also affect labelling choices.
The fork possibility, for example, that contains two breaks
(
and one
With
tie,
Is highly unlikely
except in a concave
object.
the "K" possibilities clearly in hand we mar overcbme the
intimidation
of
possibilities
contexts.
K
analysis
and
qulckl
are highly unlikely or
obser d -that many
impossible in many common
In fact, as the unlikelihood of alternativd
choices
rises much faster for K's than for forks say, we may find K's at
times more helpful to our analysis than some "simpler" vertices.
And
we can afford to rate
certain choices as implausible since
that does not mean we have not dealt with them;
they are
still
available as alternatives In our complete analysis.
The
point to be
made here is not so much
that a limited
system could be designed that would function well in 6
specific
restricted
context.
decision
structure,
imbedded
Rather
in
a
complete
a heterarchical
partition
vision
system, could
employ Information from a
advise
context decision structure to
its labelling decisions or alter Its plausibility ratihgs.
an
benefit from
understanding
choices
labelling
were
partition
process
plausible
partition
analysis
being
implications
made.
In
patilcul ar,
difficulty
the
within
which
of
p roviding a
In
constraints
if the
the
of
contekt
provided It, the context analyser could be prompted to
In
a similar
fashion the
-partition
may complain that ah iýplausible choice Is.perhaps due
or It may
peAproceis r;
input .data from the
to improper
informed
of the
experienced
reconsider its findings.
analyser
in turn, could
context decision structure,
course the
Of
be
configuration' as a result
to expect certain unlikely
of missing lines, shadows, whatever.
Aside
labelling
from
what
we
choices may
right
call
be Informed
"contitt"' information,
an d knowledge
by knowledge
heterarchical
structules
of
structure.
Background informatloh, for example will
Many
differeht
determine
many labellings
There are
a variety
partitioning
decisions
(concave and
there are more
deal
a
which
are
relevant
convex fall
immediately
or sugg est others.
from the
predicates aside
global criteria
with these
within
and circurmscribe
of line
predicate
kinds
to
basic
partitioning
into this categ ory).
And
Future 0apers
will
Involved.
factors, their relation
to partition analysis
and to scene analysis in general.
Relevant
information
may
restrict
or
advise
our
partitioning
thoites
level,
Individual line
at
vertex
the
than
other
Decisions may be Indicated at the
level, of course.
labelling
dt
similarly
types
InVolving
higher levels
of
vertex codbinations, whatever.
beauty of
The
this Information and all
systematic,
complete
these decisions c6n
be done within a
the
All
framework.
that all
approach Is
our characterization
eealizable
ihterpretations are available for anblysis and comparison.
system
can make
decisions With
a clear
understanding of just
What the choices ari and what the Impi ca'tfohb
are.
The
d
thbe
ckoices
We know what Possibilities are Jiscbrded at egah
d6cision
point.
These featurbs of the charactetizat6on theory Also Indicate
why it is a great aid in organtzing, sttmuiating, and clarifying
our thinking, in determining precisely how the relevant
dutlined
above detertine the
factoes
ipecific decisioh Procedures used
by a partition system.
One of the difficulties in dealing with partitloh
is
that
the problem,
"potentially global".
certain
like
For
many scene
any
judgment
local configuration implying
ahalysiS
one
analysis
problems, is
makes
about a
a certain partidning, one
can usually find an exception by embedding the configuration
a
(
sufficiently complex environment.
theoretical approach is
that the
determination
in to
Is built
Another
basic aspect
the structure
In
advantage of our
of this
global
of the consistent
labelling approach, on a network level.
Local labelling decisions
global
relationship.
labelling
some (or
labelled by
vertex have been
the
Once
of
that
all) of
potentially
the lines
may
be
of a
deciiions,
neighbbring labelling
vertex
possibilities cut down.
16 a
affect others
determined,
or
the
The labelling 6f one or more tihes of a
vertex may direct our attention t6 the most plausible labelling,
for
the
vertex type,
which agrees
with the
alkbady labelled
lines.
The
guarantees
will
be
conflicts
interrelatlonship
that
built
potentially
donsidered.
When
and discarding
into
global
this
the
labelhing
system
determining relationships
n6cessitatei
decisions obr
reiotution
system will
know just
what decisionS were involved and be able to evaluat6 them.
appropriate alternatives will be available.
of
The
And our theoretical
base thsu.es that our options prOvide a comrplete set df possible
solutions.
Our system, in other words, has the "freedom to fail".
has long been an educational cliche that this is a
for accomplishment.
of principles, some bf
may fall at any given application.
heuristic
programming have
recover from or
this
capability
prereouisite
The very notion of "heuristic" prbgramming
implies, not algorithms, but sets
of
It
The most successful concepts
dealt
use these failures
which
with systems
in some
one is often forced
that could
fashion.
to tightly
Without
restlict the
.Ir
16
problem domain, or to deal with Ordbietf of hoýel6st complexity.
In
analysis
scene
particuldr,
Work".
the
the
global"
"p tentiallY
makes It difficUlt to
Our aroroadh to the
flexibility to #fil, with
~ake deo.lsloni
"blst pakti6t
in some
desired solution.
theoretical
in
that "haVe to
r.problett glVes
us
the added coh.idente that, sltce
we have characterized the set of realtlzabie
at least
problmi,
sensýe
be
artitions, we must,
able to
achieve
the