Models and Matching
Methods of modeling objects and their
environments;
Methods of matching models to sensed
data for recogniton
Stockman CSE/MSU Fall 2005
Some methods to study








Mesh models (surface)
Vertex-edge-face models (surface)
Functional forms: superquadrics (surface)
Generalized cylinders (volume)
Voxel sets and octrees (volume)
View class models (image-based)
Recognition by appearance (image-based)
Functional models and the Theory of
affordances (object-oriented)
Stockman CSE/MSU Fall 2005
Models are what models do
Stockman CSE/MSU Fall 2005
What do models do?
Stockman CSE/MSU Fall 2005
Vertex-edge-face models
Polyhedra and extensions
Stockman CSE/MSU Fall 2005
Vertex-Edge-Face model
Stockman CSE/MSU Fall 2005
Sample object
All surfaces are planar
or cylindrical
Stockman CSE/MSU Fall 2005
Matching methods





Hypothesize point correspondences
Filter on distances
Compute 3D alignment of model to data
Verify positions of other model points,
edges, or faces
LOTS of work in the literature on this!
Can work for many industrial objects
(and human faces perhaps!)
Stockman CSE/MSU Fall 2005
Triangular meshes
Stockman CSE/MSU Fall 2005
Texture-mapped mesh dog
Courtesy of Kari Puli
With each triangle is a mapping of its vertices
into pixels [r, c] of a color image. Thus any
point of any triangle can be assigned a color
[R, G, B]. There may be several images
available to create these mappings.
Stockman CSE/MSU Fall 2005
Meshes are very general
They are usually verbose and often are too detailed for many
operations, but are often used in CAD
Stockman CSE/MSU Fall 2005
Modeling the human body for
clothing industry and …
Multiple Structured light scanners used: could
this be a service industry such as Kinkos?
Actually cross sections
of a generalized
cylinder model.
Stockman CSE/MSU Fall 2005
Mesh characteristics
+ can be easy to generate from scanned data
Stockman CSE/MSU Fall 2005
Making mesh models
Stockman CSE/MSU Fall 2005
Physics-based models
Can be used to make meshes;
Meshes retain perfect topology;
Can span spots of bad or no data
Stockman CSE/MSU Fall 2005
Physics-based modeling
Stockman CSE/MSU Fall 2005
Forces move points on the
model; halt at scanned data
Stockman CSE/MSU Fall 2005
Fitting an active contour to
image data
Stockman CSE/MSU Fall 2005
Balloon model for closed
object surface
Courtesy of Chen and Medioni
Stockman CSE/MSU Fall 2005
Balloon evolution
• balloon stops at
data points
• mesh forces
constrain neighbors
• large triangles split
into 4 triangles
• resulting mesh has
correct topology
Stockman CSE/MSU Fall 2005
Physics-based models
Can also model dynamic behavior of solids (Finite Element Methods)
Stockman CSE/MSU Fall 2005
Algorithms from computer graphics
make mesh models from blobs



Marching squares applied to some
connected image region (blob)
Marching cubes applied to some
connected set of voxels (blob)
See a CG text for algorithms: see the
visualization toolkit for software
Stockman CSE/MSU Fall 2005
Volume model: voxels, octrees
Stockman CSE/MSU Fall 2005
Simple object and its octree
Stockman CSE/MSU Fall 2005
Generalized cylinders
Stockman CSE/MSU Fall 2005
Generalized cylinders
• component parts have axis
• cross section function describes variation along axis
• good for articulated objects, such as animals, tools
• can be extracted from intensity images with difficulty
Stockman CSE/MSU Fall 2005
Extracting a model from a
segmented image region
Courtesy of Chen and Medioni
Stockman CSE/MSU Fall 2005
Interpreting frames from video



Can we match a frame region to a
model?
What about a sequence of frames?
Can we determine what actions the
body is doing?
Stockman CSE/MSU Fall 2005
Generalized cylinders
Stockman CSE/MSU Fall 2005
View class models
Objects modeled by the distinct
views that they can produce
Stockman CSE/MSU Fall 2005
“aspect model” of a cube
Stockman CSE/MSU Fall 2005
Recognition using an aspect model
Stockman CSE/MSU Fall 2005
View class model of chair
2D Graph-matching (as in Ch 11) used to evaluate match.
Stockman CSE/MSU Fall 2005
Side view classes of Ford Taurus
(Chen and Stockman)
These were made in
the PRIP Lab from a
scale model.
Viewpoints in between
can be generated from
x and y curvature
stored on boundary.
Viewpoints matched to
real image boundaries
via optimization.
Stockman CSE/MSU Fall 2005
Matching image edges to model limbs
Could recognize car model at stoplight or
gate.
Stockman CSE/MSU Fall 2005
Appearance-based models
Using a basis of sub images;
Using PCA to compress bases;
Eigenfaces (see older .pdf slides 14C)
Stockman CSE/MSU Fall 2005
Function-based modeling
Object-oriented;
What parts does the object have;
What behaviors does it have;
What can be done with it?
(See plastic slides of Louise Starks’s work.)
Stockman CSE/MSU Fall 2005
Theory of affordances: J.J. Gibson




An object can be “sittable”: a large number of
chair types, a box of certain size, a trash can
turned over, …
An object can be “walkable”: the floor, ground,
thick ice, bridge, ...
An object can be a “container”: a cup, a hat, a
barrel, a box, …
An object can be “throwable”: a ball, a book, a
coin, an apple, a small chair, …
Stockman CSE/MSU Fall 2005