Matching and Recognition in 3D
Moving from 2D to 3D –
Some Things are Easier
• No occlusion (but sometimes missing data instead)
• Segmenting objects often simpler
Moving from 2D to 3D –
Many Things are Harder
• Rigid transform has 6 degrees of freedom vs. 3
– Brute-force algorithms much less practical
• Rotations do not commute
– Difficult to parameterize, search over
• No natural parameterization for surfaces in 3D
– Hard to do FFT, convolution, PCA
– Exception: range images
Matching / Recognition in 3D
• Project into 2D, do image matching
• Structural methods (i.e., part decomposition,
graph matching)
• Shape similarity methods
• Statistical methods
• Feature-based methods
3D Medial Axis and Shock Scaffolds
• Medial axis: locus of points equidistant from
2 surfaces
• Shock scaffolds [Leymarie & Kimia]: do
matching on “sheets” and “lines”
Shape Similarity
• Key difficulty – locating objects under any
rigid-body transformation
• Translation: relatively easy (match centroids)
• Rotation:
– Align objects to each other
– Align objects to canonical coordinate frame
– Rotation-invariant methods
Iterative Closest Points (ICP)
• Besl & McKay, 1992
• Start with rough guess for alignment
• Iteratively refine transform
ICP
• Assume closest points correspond to each other,
compute the best transform…
ICP
• … and iterate to find alignment
• Converges to some local minimum
• Correct if starting position “close enough“
Aligning Scans
• Start with manual initial alignment
[Pulli]
Aligning Scans
• Improve alignment using ICP algorithm
[Pulli]
Aligning Objects With Moments
• For each point on object, compute
( x  x )( x  x )T
• Canonical orientation based on eigenvectors
(ordered by eigenvalue)
Problem with PCA-Based Alignment
• If eigenvalues are close, axes unstable
Rotation-Invariant Descriptors
• Decompose model into spherical shells
• Decompose each shell into spherical harmonics
• Keep amplitude, throw away phase
3D Model
Rotation Independent
Components
Shape
Descriptor
Statistical Methods for Matching Shape
• EGI: extended Gaussian images
• For each direction, what fraction of normals
point in that direction
• Not rotation invariant, but tends to be peaky
Shape Distributions
• Osada, Funkhouser, Chazelle, and Dobkin
• Compact representation for entire 3D object
• Invariant under translation, rotation, scale
• Application: search engine for 3D shapes
Computing Shape Distributions
• Pick n random pairs of points on the object
• Compute histogram of distances
• Normalize for scale
Random
sampling
3D Model
Shape
Distribution
Comparing Shape Distributions
Similarity
Measure
3D
Model
Shape
Distribution
Shape Distributions for Simple Shapes
Robustness Results
7 Missiles
7 Mugs
Classification Results
Classification Results
Features on Surfaces
• Can construct edge and corner detectors
• Analogue of 1st derivative: surface normal
• Analogue of 2nd derivative: curvature
– Curvature at each point in each direction
– Minimum and maximum: “principal curvatures”
– Can threshold or do nonmaximum suppression
3D Identification Using Spin Images
• Spin images: Johnson and Hebert
• “Signature” that captures local shape
• More expressive than curvature
Computing Spin Images
• Start with a point on a 3D model
• Find (averaged) surface normal at that point
• Define coordinate system centered at this point,
oriented according to surface normal and two
(arbitrary) tangents
• Express other points (within some distance) in
terms of the new coordinates
Computing Spin Images
• Compute histogram of locations of other points,
in new coordinate system, ignoring rotation
around normal:
  p  n̂
  p  n̂
Computing Spin Images
Spin Image Parameters
• Size of neighborhood
– Determines whether local or global shape
is captured
– Big neighborhood: more discriminatory power
– Small neighborhood: resistance to clutter
• Size of bins in histogram:
– Big bins: less sensitive to noise
– Small bins: captures more detail, less storage
Spin Image Results
Range Image
Model in Database
Spin Image Results
Detected Models