Abstraction of 2D Shapes in Terms of Parts
Xiaofeng Mi
Doug DeCarlo
Matthew Stone
Rutgers University
Department of Computer Science & Center for Cognitive Science
(a) Original
(b) Automatic abstraction
(c) Parts in (b)
(d) Douglas-Peucker
(e) Progressive Mesh
(f) Curvature flow
Figure 1: Abstractions and simplifications of a map of Australia
Abstract
Abstraction in imagery results from the strategic simplification and
elimination of detail to clarify the visual structure of the depicted
shape. It is a mainstay of artistic practice and an important ingredient of effective visual communication. We develop a computational method for the abstract depiction of 2D shapes. Our approach
works by organizing the shape into parts using a new synthesis of
holistic features of the part shape, local features of the shape boundary, and global aspects of shape organization. Our abstractions are
new shapes with fewer and clearer parts.
Keywords: non-photorealistic rendering, abstraction, symmetry
1
Introduction
Artistic imagery often exploits abstraction—the strategic simplification, exaggeration and elimination of detail to clarify the visual
structure of a depiction. Abstraction is both a distinctive visual style
[McCloud 1993] and a strategy for communicating information effectively [Robinson et al. 1995]. It allows artists to highlight specific visual information and thereby direct the viewer to important
aspects of the structure and organization of the scene.
In this paper, we present an approach to the abstraction of 2D shape.
Our approach creates a coherent new shape that specifically displays just those features of the original that matter. In Figure 1,
we illustrate the results of our approach and its contribution. Given
the map of Australia in Figure 1(a), and a threshold feature size to
be preserved, our method automatically produces the abstraction in
Figure 1(b). As you can see in the coastline at right, our method
idealizes prominent features while eliminating any trace of smaller
ones. It achieves this through an explicit representation of the significant parts of the shape, eight of which are shown in Figure 1(c).
We contrast abstraction with prior work in computer graphics on
simplifying imagery. The alternative renderings of Figure 1(d), (e)
and (f) show maps of Australia simplified by benchmark automatic
techniques. Two of these shapes are made by adding or removing
vertices based on a measure of error: Figure 1(d) uses the wellknown 2D simplification technique of Douglas and Peucker [1973];
Figure 1(e), progressive mesh techniques [Hoppe 1996; Garland
and Heckbert 1997]. As we see in the coast at right, these methods
cannot distinguish between changes that remove small features and
changes that slightly distort large features. Finally, Figure 1(f) is
simplified by curvature flow, a technique to locally soften sharp
detail [Desbrun et al. 1999]. Such methods allow residual effects of
small features to persist, even as simplifications gradually start to
distort the shape of larger features, as happens here. These methods
do not achieve the visual style we seek.
Our approach to abstracting shape directly aims to clarify shape for
the human visual system. We work with representations of shape
in terms of perceptual parts, inspired by findings from cognitive
science and computational vision. A part of a shape is a particular region with a single significant symmetry axis and (assuming
the shape is not a single part) a single attachment to the rest of the
shape. In the same way a finger attaches to a hand, this attachment
meets the part’s symmetry axis. A shape can thus be described in
terms of part attachments: fingers attach to hands, hands to arms,
and so on. To abstract a shape, we infer a representation in terms of
these parts, then create a new rendering that includes just the relevant parts from this representation. Although the idea is simple, the
execution involves sophisticated inference, since general examples
are far more complicated than the tree of limbs seen in articulated
figures. We can only approximately match the human visual system in its astonishing ability to recognize parts that integrate across
a shape, through a causal analysis of shape features and a holistic
understanding of the shape as a whole. Already, however, our realization of the approach yields lucid imagery that is qualitatively
different from existing alternatives.
Our work is the first attempt to abstract imagery that makes an explicit effort to preserve the form of the shape. In addition to this
demonstration, our work offers these specific contributions:
• The development of a new part-based representation of 2D
shape for computer graphics.
• The design of new techniques to infer such representations
from shape data and to render new shapes from them.
Our results thus illustrate the promise and the challenge of directly
designing imagery to fit human perceptual organization.
1.1
Abstraction
Our approach to shape abstraction is fundamentally different from
approaches to simplification based on local adaptation of detail. It
is better understood as continuing a line of research in nonphotorealism on clarifying the structure of images [DeCarlo and Santella
2002; Kolliopoulos et al. 2006; Orzan et al. 2007], video [Wang
et al. 2004; Winnemöller et al. 2006], and line drawings [Barla
et al. 2005; Jeong et al. 2005; Lee et al. 2007]. Broadly, these approaches aim to abstract the content of imagery. They reason about
what to show by using segmentation, clustering, and scale-space
processing to find regions or lines that can be omitted from imagery. They clarify depictions by replacing these omitted features
using smoothing and fitting. These techniques are not designed to
clarify a feature in imagery that they decide to show. This is the
problem of abstracting form. Without doing this, work often aims
for a sketchy style that conveys to the viewer that reconstructed
features are coarse or approximate. This loose visual style is often
effective—as in the smoothing result in Figure 1(f). Smoothing affords a wide range of elegant analyses of the presence of detail at
different scales [Mokhtarian and Mackworth 1992], but it seems
these analyses cannot be used to inform the way coarser shape
is rendered abstractly, without detail. In addition, smoothing can
be constrained and interleaved with sharpening so that pronounced
corners are not lost [Kang and Lee 2008]—enabling a wider range
of visual styles. Even so, when fine details disappear, they have
been averaged away into their local neighborhoods, not removed.
Different methods are required to achieve the clarity in form in (b).
Our contribution to abstraction is to model how each region is organized into perceptual parts. Our system tries to ensure that it selects
parts so the perceptual organization of a shape remains the same
even as smaller parts are omitted. Therefore we can now for the
first time clarify detail by abstracting regions we decide to preserve.
We abstract both form and content.
Researchers have also explored the control of abstraction by studying where visual information should be preserved. Importance can
be specified by a user [DeCarlo and Santella 2002; Durand et al.
2001; Hertzmann 2001; Orzan et al. 2007] or computed automatically from models of salience and distinctiveness [Lee et al. 2005;
Shilane and Funkhouser 2007]. This control can modulate abstractions of content and rendering style—and potentially shape also—
but does not in itself accomplish abstraction of form.
1.2
Part-based representations
We build on research in computer vision on part-based shape analysis [Kimia et al. 1995; Rom and Médioni 1993; Siddiqi and Kimia
1995]. This work infers a hierarchical representation that accounts
for the geometry of a shape as an assembly of simple parts described in terms of local symmetries. While similar in spirit to
traditional modeling in computer graphics, the goal in vision is to
reduce the problem of recognizing shapes to matching representations part by part. In this paper, we use these representations for a
different purpose: abstraction in computer graphics.
There is strong evidence for such part-based representations of
shape in the human visual system [Singh and Hoffman 2001]. Ultimately, we understand a part of a shape as a representation created
and used by the human visual system. To make effective abstractions of shapes, our definition of a part must align with the human
visual system as much as possible: it’s a perceptual model. We thus
design our system with reference to human and computer vision.
positive
curvature
maximum
transition
(a)
(b)
(c)
Figure 2: (a) Tracing a symmetry axis of part; (b) Partlike elements
and their SLS axes; (c) After removal of one part
The rest of this section summarizes the mathematical and algorithmic basis of the research we draw upon.
Computational approaches find parts through analyses of local symmetries. Axes of symmetry are defined in terms of bitangent circles, which touch the shape boundary in two locations, without
passing through the boundary of the shape. Local symmetry axes
are located by tracing out the centers of all such circles—this is
the medial axis transform (MAT) [Blum and Nagel 1978]—or by
tracing out the midpoints of chords that connect the two points of
tangency—this is the chordal axis transform (CAT) [Prasad 1997].
Given these axes, and the radii of the bitangent circles at each point
along them, the original shape can be reconstructed. These symmetries can link arbitrarily distant points into a common representation, because they put points on opposite sides of a symmetry axis
into correspondence. Thus, they naturally capture non-local shape
properties such as the thickness or length of the shape. A more general (and redundant) representation known as smoothed local symmetries (SLS) [Brady and Asada 1984; Giblin and Brassett 1985]
is like the CAT but permits the bitangent circles to pass through the
shape boundary. This representation additionally captures minor
axes of symmetry of the shape.
Parts can be defined using any symmetry representation. Some approaches detect part boundaries by locating cuts across symmetry
axes where the shape is at its thinnest (“necks”) or has coherent connections between negative curvature minima on the shape boundary (“limbs”) [Siddiqi and Kimia 1995]. Other approaches remove
parts one at a time until only a single part remains. In doing this,
the symmetric structure of larger parts is exposed as smaller parts
are taken away [Rom and Médioni 1993; Mi and DeCarlo 2007].
Parts have an identifiable connection to the rest of the shape, known
as the part transition. It is by locating this transition that the extent
of a part is detected, and the model of transitions introduced by Mi
and DeCarlo [2007] captures a number of cognitive theories of parts
within a single mathematical framework that uses SLS. The transition model not only locates the boundaries of partlike elements, but
also allows for gaps or overlaps between them—so that the specification of the part and the rest of the shape isn’t influenced by
the geometry of the transition between them. In decomposition, the
transition model provides a clean way of removing a part from a
shape, so that little or no trace of it is left behind.
Our approach builds most closely on algorithms and models in [Mi
and DeCarlo 2007]. In this work, the algorithm for part detection follows the SLS symmetry axes from where they start on the
shape boundary: at positive curvature extrema, as governed by the
symmetry–curvature duality [Leyton 1992]. As in Figure 2(a), bitangent circles start at the tip of an SLS axis and “slide” along the
boundary of the shape. The tracing continues along the SLS axis
until a part transition is detected (using local geometric properties).
This procedure is performed starting from all positive curvature extrema, thus detecting all potential parts of the foreground shape that
can be removed with one slice.
Given this set of potential parts, it removes the smallest by size,
as measured by bitangent circle radius at the base of the transition
[Siddiqi and Kimia 1995]. After committing to a part, it is removed
from the shape. This is not to create an abstract rendering but to
reveal the underlying structures and symmetries of the shape that
were obscured by the action of the part. The part transition model
dictates the exact locations on the shape where the part is “spliced
out”, and new geometry is inserted that smoothly fills the gap where
the part was. This process is demonstrated in Figure 2(b) and (c).
Notice how the single SLS axis of the long part in Figure 2(c) is
apparent only after the attached part was removed. Overlapping
parts, or parts that are directly across symmetry axis in the CAT that
are approximately the same size (by a factor of two) are removed as
a group. Following the removal, the part structure is recomputed.
This continues until a single primitive part remains.
It’s also possible to simplify shapes without computing parts at all,
but instead directly use the MAT or CAT. In this case, a simplified
shape is determined by trimming insignificant axes and then reconstructing the shape that corresponds to this pruned representation
[Gold and Thibault 2001; van der Poorten and Jones 2002]. Results resemble geometric smoothing, since this pruning leaves behind rounded “stumps” [Rom and Médioni 1993; van der Poorten
and Jones 2002]. However, there are also distracting differences
from the original shape since not all symmetry axes induce meaningful parts, and some symmetry axes contain several parts along
their length [Singh and Hoffman 2001].
2
Parts and Abstraction
A good analysis of a shape into parts offers an explanation that assembles the shape together by adding small numbers of parts in visually clear configurations. (See [Leyton 1992] for more on shape
analysis as explanation.) On our approach, an abstraction is just
a summary of this explanation. An abstraction is a new, simpler
shape which resembles the original in its essentials because it recapitulates the important steps in the process that gave rise to the
original. It therefore allows the viewer to see those steps more directly.
Our implementation consists of two modules: decomposition and
abstraction. The following summarizes our approach for decomposing the part structure of a shape.
Figure 3: Each row shows one step in the decomposition of the
shark. The shape at each stage of the algorithm is shown on the left,
with the next part to delete darkened. The set of partlike elements
computed geometrically at each step is shown on the right.
Part detection
1. Compute the CAT [Prasad 1997], which will be used in inferences involving neighboring parts.
2. Detect partlike elements of the foreground and background
based on the SLS [Mi and DeCarlo 2007], but also maintain
integrity of compound and bending parts. Many more elements are detected at this stage than will feature as parts in
the final organization of the shape.
3. Extrapolate the shape of each element so that they can be more
flexibly assembled during abstraction.
Part decomposition
4. Form a directed graph D where vertices correspond to partlike
elements and directed edges express dependencies between
neighboring elements that are derived using the heuristics in
Section 2.2. See Figure 13. These dependencies constrain
which elements are identified next as genuine perceptual parts
in the incremental analysis of the shape.
5. Find the strongly connected components of D. Our model
predicts that removing any of these components will preserve
the perceptual organization.
6. Identify the next component of D, which is the one whose
elements fit within the smallest bitangent circle radius, and
remove its elements from the graph D. Components often
contain a single part, but on occasion contain a set of dependent parts.
(a)
(b)
Figure 4: Abstraction of the shark: (a) selected parts; (b) result
7. We also allow a user to assist in disambiguating parts. If
the parts just found are consistent with user guidance strokes
(Section 2.3), then delete the parts and complete the gap with
a smooth curve—these parts become part of our representation. Otherwise, our part inference disagrees with the user’s
intuitions; continue at step 6 with the streamlined D.
8. If D is not empty, go to step 1.
The output of this process is a set of parts that collectively describe
the shape. The complexity of steps 1–2 is O(n log n) for a shape
consisting of n vertices. The potential parts are computed from
scratch after each part deletion, thus if there are p parts in the result,
then the complexity is O(p n log n).1
Figure 3 shows an example decomposition of a shark. The first two
part deletions remove small background parts, while the remaining
deletions remove fins (in the third and fourth step, pairs of fins that
are grouped together). The heuristics guide the process so that the
right part is deleted among the many that are shown on the right.
1 At the moment, we simply recompute the entire part structure in each
step; large gains in efficiency are possible by developing incremental algorithms for the SLS, such as those for adjusting the CAT [Shewchuk 2003].
The process of abstraction proceeds by working from this decomposition, but only includes some of the parts. Figure 4(a) shows the
set of parts used to produce the abstraction in (b).
detect compound parts as in Figure 6, where the attachment of a
large part to the rest of the shape makes one of its features look like
a short-cut.2
Abstraction
We also stop tracing the SLS axis when it intrudes too far towards
a major symmetry of the shape. This condition is needed to recognize the symmetry at the elbow of a bend as a feature of the shape
rather than a part. (Rom and Medioni [1993] also addressed this
case; they prioritize parts based on long translational symmetries.)
We implement the condition in terms of the major symmetries in
the CAT as in Figure 7. The boundary points a and b on either
side of the SLS axis lie distances da and db from their symmetric
counterparts in the CAT. Meanwhile, the completion curve for a cut
through a and b passes some minimum distance dmin to the shape
boundary. We measure the intrusion as 1 − dmin /min(da , db ), and
stop tracing the SLS if this measure exceeds a small value (0.05).
1. Select a subset of the parts from the decomposition using userspecified thresholds on part size and salience.
2. For each part in this subset, proceeding in the reverse order of
the decomposition (i.e. starting with the last part removed),
build up the resulting shape. At each step:
• If the part wasn’t attached originally (i.e. an island) then
simply add it in as a separate shape.
• Otherwise, attach the part to its appropriate shape by
finding intersection points or closest points using the
part attachment model described in Section 2.4.2.
2.1
Detection and Decomposition
negative
curvature minima
A
C
B
This section describes several adaptations to the approach of Mi
and DeCarlo [2007] to organize the shape into partlike elements.
We search the background for potential parts, adapt the search so
parts are more coherent, and additionally describe how each part
connects to the remainder of the shape.
2.1.1
Figure 6: Detection of compound part B by tracing further along
the SLS past the end of A. Tracing further from C along the SLS
exits the shape.
Background Parts
Partlike elements offer a range of organizations of the shape, not
one in particular. The possibility for ambiguity in part structure is
indicated by the top tail fin of a shark in Figure 5. The geometric
structure here licenses two explanations of the fin in terms of perceptual parts. The most natural explanation organizes the top fin
into a single coherent foreground piece. This piece has an indentation in it—part of the background. This indentation is removed in
the abstraction in the center. Geometrically, however, we can analyze the shark tail entirely in terms of foreground parts, as seen on
the right of Figure 5. The result is awkward, since the indentation
marks the transition between the tip of the fin (one foreground part)
and the base of the fin (a second foreground part). Yet this transition
is a natural part boundary: it is a neck [Siddiqi and Kimia 1995],
and is also characterized by the short-cut rule [Singh et al. 1999],
since it has a negative curvature minima on only one side.
Allowing for background parts is crucial for achieving an appropriate interpretation of a shape. We extend [Mi and DeCarlo 2007]
by tracing symmetry axes outward from negative curvature extrema
into the background until we reach a transition.
a
da
(a)
2.1.2
Detection
We also depart from Mi and DeCarlo in how far we search for transitions along an SLS axis. There are two differences. First, we
sometimes continue searching for additional transitions after the
first one we find. For example, in Figure 6, we find and report transitions both for the A region and for the B region along the same
SLS axis. We stop tracing the SLS axis when it passes through the
shape or when SLS has moved past a negative minimum of curvature on the shape boundary by a threshold distance (positive maxima for background parts). These additional transitions allow us to
b
db
(b)
Figure 7: (a) Various cuts along an SLS axis that intrude into a
bend; (b) Measuring the degree of intrusion of a particular cut.
part
part
transition
extended part
rest of the shape
rest of the shape
(a)
(b)
Figure 8: (a) Parts are extrapolated from the point where the transition starts, and their radius (the dotted line) is measured based
where they intersect the rest of the shape. (b) The radius of parts
connected with a neck transition is measured at the neck.
2.1.3
Figure 5: Removal of background and foreground parts
dmin
Extension
The transition allows us to describe each part separately from its
effects on the rest of the shape. Mi and DeCarlo [2007] reconstruct parts by completing their contour into a single closed curve,
as shown on the left in Figure 8(a). Instead, where possible, we reconstruct the part as implicitly continuing beyond where it attaches,
as shown on the right in Figure 8(a). This gives more flexibility in
reattaching the part and better estimates of size and salience.
Our extrapolated parts are constructed by applying a model of curve
completion to the main symmetry axis along which the part was
originally attached. In the transition region, starting from where the
2 More precisely, that in this situation, locations on the boundary are actually involved in two symmetries. The SLS axis traces out the symmetry of
one part of the shape (axis A), then backs up on one side ([Giblin and Brassett 1985] calls this an anti-symmetry—shown as a dotted line in Figure 6),
then proceeds forward again to trace out a different axis (axis B).
chord of the circle is most perpendicular to the axis, we extrapolate the symmetry axis with a logarithmic spiral [Singh and Fulvio
2005] so that its curvature fades to zero. The chord length is extrapolated linearly to determine part boundary curves. Should they
intersect, we abort the extension and connect the ends of the shape
with a Hermite curve as in Figure 8(b). Otherwise, the extrapolated
part ends as soon as the extrapolated axis and part boundaries pass
through the rest of the shape, or as soon as they extend outside the
convex hull of the transition region, whichever is closer.
2.2
Preserving Part Structure
Complex preferences are at work in leading our visual system to
prefer one organization of imagery over another [Feldman 2003].
In this section, we explain how our approach to decomposition attempts to abstract shapes robustly, using a set of heuristics that preserve symmetries and salient parts. These heuristics form a model
of perceptual organization of shape in terms of parts.
Without this model, we would have to identify parts by removing
partlike elements in size order. This approach has serious limitations. Consider the shark tail from Figure 5. We argued earlier that
to preserve the structure of the top of the tail, the small background
part must be removed first. In this case, this works: the indentation is the smallest part found. However, had the indentation been
larger, as in Figure 9, then the tip of the tail would have been the
smallest part, leading to a bad analysis. There are so many partlike
elements (see Figure 13 for an example) that no geometric measure that just describes a single element can successfully identify
correct parts. The analysis must consider relationships and symmetries across neighboring parts—both the shape of the indentation
and the tip of the fin. In what follows, we describe two heuristics
for such analysis. We call them heuristics since their use does not
guarantee that perceptual structure will be preserved. Not enough
is known about how people interpret shapes, or how approximate
symmetries can be determined when axes are not straight. This is
the best model of perceptual organization we could make without
introducing unwarranted complexity to our implementation.
The heuristics constrain the order of deletion of parts using a simple
graph algorithm described in Section 2.2.1. Before each deletion,
we summarize the evidence we have about the perceptual organization of parts using a directed graph D. Nodes in the graph are
possible parts. An edge in the graph from part A to part B suggests that the decomposition should remove B before A. Hence, B
is better seen as arising after A or in tandem with A.
Because of how we detect parts, it’s entirely possible that the entire structure of a part becomes visible only after removing other parts first—the shark tail is
an excellent example of this. Since we cannot rely on part size ordering, we must instead detect these situations. There are two cases,
shown in Figure 10(a)—there is either a part of the same type (foreground or background) across the shape, or there isn’t. We decide
which case applies by searching for a corresponding part across the
CAT axis that is approximately the same scale (the ratios of the areas, protrusions and radii are within a factor of two of each other).
Parts A and B on the shark tail, shown in Figure 10(b), illustrate
the first case. Figure 10(c) shows two pairs of curves that are meaH1. Don’t truncate axes prematurely.
1.
B
A
2.
(a)
A
(b)
(c)
Figure 10: (a) Two cases where H1 ensures the part on the left is not
removed; (b) An example where H1 prevents the premature deletion
of part B; (c) Curves to compare, before and after the hypothetical
deletion of A, to decide if A must be removed before B
sured before and after the hypothetical deletion of part A. We use
the curve matching energy from [Sebastian et al. 2003], which is
0 for a perfect match, and 1 for very different curves. We add an
edge from B to A if the curve matching energy decreases after the
deletion. For the second case, we have two parts that intrude on
B. We delete both, and add edges from B to both when B’s curve
matching energy decreases. This heuristic takes priority over the
next one (H2), so that if an edge is added from B to A here, we do
not consider adding an edge from A back to B due to H2.
r
B
B
d
A
(a)
(b)
Figure 11: (a) Measuring symmetry disruption of B due to A; (b)
the symmetry axis in the new version of B has moved significantly
A
B
Figure 12: Removing part A causes part B to disappear
Don’t distort parts. We also want to prevent situations
where the removal of A disrupts the structure of B. We measure
this in terms of the change of the symmetry axis of B when A is
removed. The situation is pictured in Figure 11(a). The symmetry
of B lines up a shared edge with A in with the opposite boundary of
the shape, which is unaffected by A. After deleting A, as pictured
in Figure 11(b), there is a new boundary which again lines up with
the opposite boundary of B. We compare corresponding points on
the old and new boundaries. We measure the relative change of the
d
axis at each point as d+r
, which ranges from 0 when A has no effect
(and is safe to delete) to 1 when A will completely disrupt the symmetry in B. We add an edge from A to B when the maximum of this
quantity goes above a “symmetry disruption” threshold of 51 . This
heuristic also applies in situations such as in Figure 6, preventing
parts such as A and C from disrupting the structure of B.
H2.
There is one exception when we do not add this edge. Sometimes
removing one part causes another part to disappear, as in the example in Figure 12, where the removal of A causes the background part
B to simply vanish. We don’t consider the structure of B to be disrupted by A in this case. This situation is detected by checking that
the curve segment containing B (in particular, around the curvature
extrema at its tip) is contained entirely in the boundary of A.
2.2.1
Figure 9: Deleting parts by size may lead to bad interpretations
B
B
Graph algorithm
We now have a graph D that formalizes our heuristic evidence for
global shape organization. We identify the parts in the shape representation by finding the strongly connected components [Cormen
et al. 2001] of this graph. Each component specifies a set of parts
that should be removed simultaneously. Of these components, some
have no outgoing edges. (At least one must exist—the graph that
is constructed by collapsing all vertices within a connected component into a single vertex is acyclic.) These are ordered by the
maximum radius of the parts that make them up. The parts from
the component with the least maximum radius are removed. (When
parts or transitions overlap, their boundaries can be easily removed
in one go.) An example constraint graph is shown in Figure 13.
to avoid excessive error in the reconstructed shape (Section 2.4.1).
The selected parts are then assembled back into an overall shape:
we view the shape as an aggregation of positive and negative parts,
connected together by transitions, and use a simple but flexible explicit model of transition geometry to reconstruct the shape under
a changing inventory of parts (Section 2.4.2). The result is a shape
that resembles the original in geometry up to the desired threshold,
and in addition respects its perceptual part structure.
2.4.1
Observe that our part representation does not necessarily order steps
in order of size or importance. Our technique often uncovers small
parts that make little difference to the overall shape (such as long,
shallow parts) underneath larger protruding parts. The small parts
are ultimately less important than the larger parts that were removed
before them. Thus, our abstraction can include parts from any step
of decomposition.
B
B
C
A D
F
C
F
D
A
E
E
G
H
G
H
Figure 13: Heuristics in use on the shark tail, with the corresponding constraint graph on the right. Partlike elements are shown as
dark regions. B waits for A and C to extend its symmetry; C and D
wait for A since they disrupt the symmetry of A. Similarly, D waits
for E, E and H wait for G and F waits for B. A doesn’t wait for
C or D since when A is removed, C and D disappear. Similarly, G
doesn’t wait for E or H. Thus, A and G are deletable. A has the
smallest radius, is deleted next, and is a part in the representation.
Simple orderings based on radius or area aren’t effective: part B has
the smallest radius, and C has the smallest area.
2.3
User guidance
The heuristics just described are just what they seem—the simplest rules we could design that improve the automatic extraction
as much as possible. Given appropriate data and models, we could
doubtless improve the process further by applying machine learning. Even so, any automated technique will sometimes fail and
necessitate user intervention. Examples of user intervention and its
results are given in Section 3.
To improve part decomposition, we enable the user to annotate the
shape with two types of strokes:
• Keep whole (blue). The decomposition process cannot cut
across these strokes when removing parts; any removed part
must contain the entire stroke.
• Not a part (red). Parts can only contain a small portion of
this stroke: we use an area-based threshold of 10%.
The intentions behind these strokes are enforced straightforwardly
during the decomposition process by disallowing particular operations, removing potential parts from consideration, or adding edges
to the graph. Users can easily place strokes to correct errors in decomposition (working off-line because the decomposition does not
proceed at interactive rates). However, it is possible for a set of
strokes to “deadlock” the process of decomposition. In this case we
show the impasse, and ask the user to remove and correct strokes.
2.4
Selecting Parts for Abstraction
Abstraction: Reconstruction from Parts
In making the selection of parts to include, we rely on models of
part salience. Hoffman and Singh [1997] argue that the protruding
parts are more visually distinctive than shallow parts. As a measure of this, they suggest protrusion: the ratio of the perimeter of a
part to its base. Their definition applies straightforwardly to parts
with neck transitions, as in Figure 8(b). Their protrusion is the ratio of the length of curve on the part-side of the neck to the width
of the neck. To compute protrusion for extrapolated parts, we delimit them by the extrapolated perimeter and the curve where the
base intersects the inferred boundary of the rest of the shape, as in
Figure 8(a).
To be included, a step of decomposition must meet two userspecified thresholds on the associated change in shape: there must
be a sufficiently large absolute area change (unsigned, so areas cannot cancel); and there must be a sufficiently salient part involved,
as measured by protrusion. We also allow the user to override these
thresholds by manually including a part by selecting it, so that small
but important parts are included in the abstraction.
2.4.2
Reconstruction
To reassemble the shape out of its parts requires adapting their transitions to one another. Our approach interpolates the shape contour
between the part and the rest of the shape in a way that is flexible
but is still determined by the geometry of the original attachment of
the part to the shape.
The left of Figure 14 shows the information we use for connecting
a part p (in white) to the whole shape w (in grey). Here, a and b are
the intersection points between the part p and the ongoing result
w, and signed distances dap and dbp measure the distance to the
transition boundary along the part to a p and b p , while daw and dbw
measure the analogous distance on w to aw and bw . If there is no
intersection between p and w, we instead determine the point c p
which is halfway the distance from a p to b p (on the original shape),
and cw which is the closest point on w to c—see Figure 14, right.
bw
bp
ap
dap
bp
dbw
We now have a representation that shows how the shape is built up
in successive steps that add parts and groups of parts to the shape.
Abstraction proceeds by identifying those steps that must be present
p
p
bp
a
b
w
aw
daw
ap
dbp c dap
p
aw
bw
dbw
cw
daw
w
Figure 14: Attaching a part p to the whole shape w.
Our goal is to adapt the geometry of the original shape in the transitions that connect aw with a p and bw with b p with a smooth curve—
these transitions are shown as thick lines in the figure. We start from
a description of the original transition using Hermite curves. The
Hermite curve endpoints are determined from the transition model
using distances measured when the part is detected; the endpoint
directions follow the tangents of the curve; and the magnitudes of
each of these tangents make least-squares fits to the original data.
Thus, each side of the transition has four parameters—two distances away from the intersection point along the curves to locate
the Hermite endpoints, and two tangent vector magnitudes.
Reconstruction proceeds in the reverse order of the decomposition
performed in Section 2.1. It starts by using the first part in its entirety. After this, any part that is to be included is intersected with
the relevant polygon in the ongoing result. This produces a and b,
or if no intersections are found, c p and cw . We traverse the boundary the specified distance to find the endpoints of the Hermite curve
(taking care to ensure they don’t cross over each other). The part
is inserted into the model by splicing out the portion from aw to bw
and inserting the Hermite curve at a, the portion from a p to b p , and
then the Hermite curve at b.
3
Results
We can now present results of our system, illustrating its visual effects and showing the contribution achieved by integrating positive
and negative parts, global shape organization, and user interaction.
To showcase the abstractions we achieve, we compare our results
to three simplification techniques: Douglas-Peucker (DP) [Douglas
and Peucker 1973], progressive meshes (PM) with quadric error
[Hoppe 1996; Garland and Heckbert 1997], and curvature flow (CF)
[Desbrun et al. 1999]. We create corresponding images by adjusting the parameters of each approach by hand so that set features at a
particular scale are visible. (Equalizing error metrics does not lead
to a good perceptual match—Hausdorff distance to the original is
typically smallest for DP closely followed by PM, while CF and
the abstractions are often 50-100% larger, depending on the target
size of detail.) Aside from these parameters (the size threshold for
the abstractions, the number of vertices for DP and PM, the amount
of smoothing for CF), all results are automatic unless clearly stated
otherwise. We leave the downsampling of decimation methods (PM
and DP), to portray the actual shapes these methods derive. The
difference in visual style is obvious, but it’s still possible to take
notice of how well the different approaches preserve the important
structures of the shapes.
In Figure 15, we present several automatic abstractions and simplifications. The shark in (a) has clearly delineated parts, and is
correctly decomposed without user interaction. The abstraction respects the part structure. By contrast, there are cases where the
simplifications do not. For example, both DP and PM hit the tips of
small fins, giving the impression of an extra bump on the underside
of the shark (DP) or exaggerating the back (PM). CF produces a
pleasant shape, but with several parts still visible as slight bumps.
The maple leaf, lighthouse, crab, and pine follow a similar pattern.
All of the methods produce reasonable results, at least some of the
time. In particular, the part structures of various results on the crab
are quite similar. However, DP, PM and CF do not produce the clean
results that our approach does. When producing the maple leaf and
crab, we adjusted our approach so it has a bias towards removing
foreground parts—the effective radii of the foreground parts (and
the symmetry disruption threshold) were boosted by a factor of 8.
While strokes would have been effective as well, this adaptation
is much easier and more appropriate for the widespread “spiky”
texture on the boundary of these two shapes, where the parts are
almost entirely foreground parts. Computing the decomposition of
these shapes here took a few seconds for the small models (like the
maple leaf), and close to a minute for larger models like the pine.
Figure 16 shows a range of approaches applied to a map of the UK
and Ireland, making results at a low and moderate level-of-detail.
We can see in each of the examples how our method does indeed
preserve and clarify the shape of significant features. Other results
are not unacceptable, but the imagery responds to a different stylistic objective. For example, careful inspection reveals that slight
undulations in the coastline in the CF result are in fact leftovers
of parts that are partially removed—something we saw earlier in
the comparisons in Figure 1. For large examples such as this, the
decomposition took about 10 minutes (2.66GHz single processor),
where each part detection step took 1-2 seconds. The reconstruction
is faster, taking 2-3 seconds for these examples.
We adapt this example with user strokes and manually selected
parts to match an existing map by an experienced cartographer, in
Figure 17. In making our result, 5 strokes were required (seen in the
insets). We also selected 14 parts (red dots) for inclusion that were
eliminated by the threshold. We feel our approach “captures basic
shapes” [Robinson et al. 1995] in a similar way as the cartographer
has done. However, our approach is silent about the many exaggerations that enhance and enlarge shapes well beyond their original
configuration—an exciting challenge for future work.
Figure 18 shows more examples of automatically produced abstractions. The wide variety of shapes and textures demonstrates
the generality of our approach. They also demonstrate limitations.
Sometimes our implemented model of part structure makes an incorrect analysis. Take for instance the part on the southwest corner
of Newfoundland in Figure 18(d). The abstract version on the right
eliminates the bay on the west coast in a way that appears to exaggerate the headland to its south. A better model of part structure
might reduce such errors. A different problem is seen on the northern coast of Iceland in Figure 18(b). The low detail map gives the
impression of a clear part, while the original gives the impression of
continuous texture. A similar but less drastic mistake can be seen
on the northeast coast of Baffin island in (e). Perceptual grouping [Feldman 2003] seems appropriate for detecting such regions
of shape texture. Finally, an explanation of a shape in terms of
parts will not be the most appropriate form of analysis, if the most
effective abstractions involve restoring material that has been cut
away, or repositioning movable articulations. Investigations here
seem like the next step in this line of research, both in computer
graphics and perceptual science.
3.1
Comparisons to degraded versions
Our technique involves three novel ingredients: (1) the heuristics
that preserve part structure, (2) admitting background parts for consideration during the decomposition, and (3) performing abstraction
by reconstructing a pruned representation of shape. With Figure 19,
we argue that these insights are crucial to our results.
In Figure 19, we disable these features selectively in abstracting
maps of Australia (the original in Figure 1(a)), processed at two
different resolutions. The six maps on the left filter parts and reconstruct the shape as in Section 2.4. They differ in how parts are
decomposed. The two maps on the left in (a) are our approach:
all features are enabled. In (b) we disable the heuristics from Section 2.2. Several parts, such as the features on the north coast, are
no longer good representations of the original. Finally, in (c), we
use only foreground parts. The results look unpleasantly distorted.
We quantify the differences between these images using a set of
strokes that “forces” all approaches to produce a map we felt was
free of mistakes. The method in (a) violated only 5 strokes (a total
of 9 times, since some strokes are violated more than once), out of a
(a)
(b)
(c)
(d)
(e)
Original
Abstracted
DP
PM
CF
Figure 15: Automatic abstractions and simplifications of various shapes: (a) shark (423 vertices, 12 parts); (b) maple leaf (576 vertices, 73
parts); (c) lighthouse (1898 vertices, 57 parts); (d) crab (1620 vertices, 288 parts); (e) pine (4298 vertices, 1035 parts)
Original
Abstraction
DP
PM
CF
Figure 16: Automatic abstractions and simplifications of the UK and Ireland (7600 vertices, decomposed into 950 parts; abstractions use 114
(top) and 31 parts (bottom)).
(a)
(a) Original with strokes (b) User guided
(c) Cartographer
Figure 17: User guided results from the strokes in (a), compared to
one produced by a cartographer (after [Robinson et al. 1995]).
(b)
total of 505 steps (i.e. about 2% of the steps are mistakes). That in
(b) violated 21 (35 times in total; about 7%). That in (c) violated 30
(50 times in total; about 10%). This is over five times the number
of mistakes as the full approach.
The six maps on the right show the contribution of flexible shape
reconstruction. These maps are produced by running the decomposition backwards (up to a step we felt included about the same
amount of detail as the left side). We keep the small parts we find
after removing larger ones. The clean style is gone.
4
(c)
Conclusion
In this paper, we have developed an approach to abstract depiction
of 2D shapes based on the analysis of shape into parts. Our computational techniques use local analysis of the boundary of the shape,
holistic exploration of the symmetries of shape regions, and global
inference to organize the shape into a sequence of additions of interrelated parts, both foreground and background, which explains how
a shape can be created. We can summarize, adapt and transform
this representation, automatically and interactively, and flexibly assemble the results into new shapes. These new shapes highlight the
potential for techniques in computer graphics to be more sensitive
to the meaningful organization of the things they depict, and complement existing techniques for the abstraction of the content and
style of imagery.
Our implementation involves a specific choice of representations
and computations. While we aim to accommodate insights from the
science of perception in the simplest models required, it’s inevitable
given the state of research that our choices are sometimes heuristic
and partial. As better models and algorithms become available, we
expect improvements. We are particularly optimistic that research
on approximate symmetry [Mitra et al. 2006] and computational
topology [Dey et al. 2008] can lead to better computational models
of relationships among parts, while approaches based on fitting implicit models [Shen et al. 2004] will lead to more flexible models for
part reassembly. Meanwhile, additional experience analyzing and
interacting with part structures will open new possibilities for empirical methods for part analysis, driven by Bayesian approaches to
shape inference and machine learning. The challenge of such inferences hints at the depth to which visual communication can exploit
our understanding of the world, and opens up new directions for
graphics research that aims to depict things more richly, deeply and
effectively to us.
Acknowledgments
This material is based upon work supported by the NSF under
grants SGER-0741801 and CCF-0541185. Thanks to Maneesh
Agrawala, Olga Karpenko, Manish Singh, Michael Leyton, Andy
Nealen, Adam Finkelstein and Jie Xu for helpful discussions.
(d)
(e)
(f)
Figure 18: More maps: originals (left), abstractions with moderate detail (center), abstractions with low detail (right). (a) eastern
China; (b) Iceland; (c) Luzon Island (Philippines); (d) Newfoundland; (e) Baffin Island (Canada); (f) the island of New Guinea.
References
BARLA , P., T HOLLOT, J., AND S ILLION , F. 2005. Geometric
clustering for line drawing simplification. In Proceedings of the
Eurographics Symposium on Rendering.
B LUM , H., AND NAGEL , R. 1978. Shape description using
weighted symmetric axis features. Pattern Recog. 10, 167–180.
B RADY, M., AND A SADA , H. 1984. Smoothed local symetries and
their implementation. Int. J. of Robotics Research 3, 3, 36–61.
C ORMEN , T., L EISERSON , C., R IVEST, R., AND S TEIN , C. 2001.
Introduction to Algorithms, 2nd edition. MIT Press.
D E C ARLO , D., AND S ANTELLA , A. 2002. Stylization and abstraction of photographs. In SIGGRAPH 2002, 769–776.
D ESBRUN , M., M EYER , M., S CHR ÖDER , P., AND BARR , A. H.
1999. Implicit fairing of irregular meshes using diffusion and
curvature flow. In SIGGRAPH 99, 317–324.
Selection and reconstruction
Reverse decomposition order
(a)
(b)
(c)
Low detail
High detail
Low detail
High detail
Figure 19: Disabling features: (a) with heuristics; (b) no heuristics;
(c) no heuristics, foreground only. Left column selects parts and
reconstructs. Right column uses reverse decomposition order.
D EY, T. K., L I , K., S UN , J., AND C OHEN -S TEINER , D. 2008.
Computing geometry-aware handle and tunnel loops in 3D models. ACM Transactions on Graphics 27, 3 (Aug.), 45:1–45:9.
D OUGLAS , D., AND P EUCKER , T. 1973. Algorithms for the reduction of the number of points required to represent a digitized
line or its caricature. The Canadian Cartographer 10, 2, 112–
122.
D URAND , F., O STROMOUKHOV, V., M ILLER , M., D URANLEAU ,
F., AND D ORSEY, J. 2001. Decoupling strokes and high-level
attributes for interactive traditional drawing. In Proceedings of
the 12th Eurographics Workshop on Rendering, 71–82.
F ELDMAN , J. 2003. Perceptual grouping by selection of a logically
minimal model. IJCV 55, 1, 5–25.
G ARLAND , M., AND H ECKBERT, P. S. 1997. Surface simplification using quadric error metrics. In SIGGRAPH 97, 209–216.
G IBLIN , P., AND B RASSETT, S. 1985. Local symmetry of plane
curves. The American Mathematical Monthly 92, 10, 689–707.
G OLD , C., AND T HIBAULT, D. 2001. Map generalization by skeleton retraction. In Proceedings of the International Cartographic
Association, 2072–2081.
H ERTZMANN , A. 2001. Paint by relaxation. In Computer Graphics
International, 47–54.
H OFFMAN , D. D., AND S INGH , M. 1997. Salience of visual parts.
Cognition 63, 29–78.
H OPPE , H. 1996. Progressive meshes. In Proceedings of SIGGRAPH 96, Computer Graphics Proceedings, 99–108.
J EONG , K., N I , A., L EE , S., AND M ARKOSIAN , L. 2005. Detail
control in line drawings of 3D meshes. The Visual Computer 21,
8-10 (September), 698–706. Pacific Graphics 2005.
K ANG , H., AND L EE , S. 2008. Shape-simplifying image abstraction. Computer Graphics Forum (Pacific Graphics 2008 Conference Proceedings) 27, 7, 1773–1780.
K IMIA , B., TANNENBAUM , A., AND Z UCKER , S.
1995.
Shapes, shocks, and deformations. I: The components of twodimensional shape and the reaction-diffusion space. IJCV, 15,
189–224.
KOLLIOPOULOS , A., WANG , J. M., AND H ERTZMANN , A. 2006.
Segmentation-based 3D artistic rendering. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 361–
370.
L EE , C. H., VARSHNEY, A., AND JACOBS , D. W. 2005. Mesh
saliency. ACM Transactions on Graphics 24, 3 (Aug.), 659–666.
L EE , Y., M ARKOSIAN , L., L EE , S., AND H UGHES , J. F. 2007.
Line drawings via abstracted shading. ACM Transactions on
Graphics 26, 3 (July), 18:1–18:5.
L EYTON , M. 1992. Symmetry, Causality, Mind. MIT Press, Cambridge, MA.
M C C LOUD , S. 1993. Understanding Comics: The Invisible Art.
Harper Perennial.
M I , X., AND D E C ARLO , D. 2007. Separating parts from 2D
shapes using relatability. In ICCV 2007.
M ITRA , N. J., G UIBAS , L. J., AND PAULY, M. 2006. Partial
and approximate symmetry detection for 3D geometry. ACM
Transactions on Graphics 25, 3 (July), 560–568.
M OKHTARIAN , F., AND M ACKWORTH , A. K. 1992. A theory
of multiscale, curvature-based shape representation for planar
curves. IEEE PAMI 14, 8, 789–805.
O RZAN , A., B OUSSEAU , A., BARLA , P., AND T HOLLOT, J. 2007.
Structure-preserving manipulation of photographs. In Int. Symp.
on Non-Photorealistic Animation and Rendering (NPAR).
P RASAD , L. 1997. Morphological analysis of shapes. Tech. Rep.
CNLS Newsletter, No. 139, LALP-97-010-139, Center for Nonlinear Studies, T-DOT, Theoretical Division, Los Alamos National Laboratory, July.
ROBINSON , A. H., M ORRISON , J. L., M UEHRCKE , P. C.,
K IMERLING , A. J., AND G UPTILL , S. C. 1995. Elements of
Cartography, 6th edition. Wiley.
ROM , H., AND M ÉDIONI , G. 1993. Hierarchical decomposition
and axial shape description. IEEE PAMI 15, 10, 973–981.
S EBASTIAN , T., K LEIN , P., AND K IMIA , B. 2003. On aligning
curves. IEEE PAMI 25, 1, 116–124.
S HEN , C., O’B RIEN , J. F., AND S HEWCHUK , J. R. 2004. Interpolating and approximating implicit surfaces from polygon soup.
ACM Transactions on Graphics 23, 3 (Aug.), 896–904.
S HEWCHUK , J. 2003. Updating and constructing constrained Delaunay and constrained regular triangulations by flips. In Proceedings of the Nineteenth Annual Symposium on Computational
Geometry (SoCG 2003), 181–190.
S HILANE , P., AND F UNKHOUSER , T. 2007. Distinctive regions
of 3D surfaces. ACM Transactions on Graphics 26, 2 (June),
7:1–7:15.
S IDDIQI , K., AND K IMIA , B. B. 1995. Parts of visual form:
computational aspects. IEEE Transactions on Pattern Analysis
and Machine Intelligence 17, 239–251.
S INGH , M., AND F ULVIO , J. M. 2005. Visual extrapolation of
contour geometry. Proceedings of the National Academy of Sciences, USA 102, 3, 939–944.
S INGH , M., AND H OFFMAN , D. D. 2001. Part-based representations of visual shape and implications for visual cognition. In
From Fragments to Objects: Grouping and Segmentation in Vision, T. F. Shipley and P. J. Kellman, Eds. Elsevier Science Ltd.,
401–459.
S INGH , M., S EYRANINAN , G., AND H OFFMAN , D. 1999. Parsing
silhouettes: The short-cut rule. Perception and Psychophysics,
61, 636–660.
VAN DER P OORTEN , P., AND J ONES , C. B. 2002. Characterisation and generalisation of cartographic lines using Delaunay triangulation. International Journal of Geographical Information
science, 8, 773–794.
WANG , J., X U , Y., S HUM , H.-Y., AND C OHEN , M. F. 2004.
Video tooning. ACM Transactions on Graphics 23, 3 (Aug.),
574–583.
W INNEM ÖLLER , H., O LSEN , S. C., AND G OOCH , B. 2006. Realtime video abstraction. ACM Transactions on Graphics 25, 3
(July), 1221–1226.