Selecting Distinctive 3D Shape
Descriptors for Similarity Retrieval
Philip Shilane and Thomas Funkhouser
Large Databases of 3D Shapes
Molecular Biology
(Protein Databank)
Computer Graphics
Mechanical CAD
(Princeton Shape Benchmark)
(National Design Repository)
Shape Retrieval
3D
Model
Model
Database
Best
Matches
Local Matches for Retrieval
3D
Model
Model
Database
Best
Matches
Local Matches for Retrieval
3D
Model
Model
Database
Cost
Function
 C( X ,Y )
i
Best
Matches
Local Matches for Retrieval
3D
Model
Best
Using many local
Model
Matches
descriptors
is
slow.
Database
Cost
Function
 C( X ,Y )
i
Local Matches for Retrieval
3D
Model
Best
Using many local
Model
Matches
descriptors
is
slow.
Database
Many descriptors
Cost
Function
not represent
do
X ,Y )
parts.
 C (distinguishing
i
Local Matches for Retrieval
3D
Model
Best
Focusing on the
Model
Matches
distinctive
regions
Database
improves retrieval time
Cost
and accuracy.
Function
 C( X ,Y )
i
Related Work
Selecting Local Descriptors
• Random
Mori 2001
Frome 2004
Related Work
Selecting Local Descriptors
• Random
• Salient
Gal 2005
Lee 2005
Frintrop 2004
Related Work
Selecting Local Descriptors
• Random
• Salient
• Likelihood
Johnson 2000
Shan 2004
Distinction = Retrieval Performance
The distinction of each local descriptor is based on how well it
retrieves shapes of the correct class.
Query
Descriptors
Retrieval Results
Distinction = Retrieval Performance
The distinct descriptors that distinguish between classes are
classification dependent.
Query
Descriptors
Retrieval Results
Approach
We want a predicted distinction score for each
descriptor on the model.
Descriptors
Distinction
Approach
We map descriptors into a 1D space where we learn
distinction from a training set.
Descriptors
Distinction
Approach
Likelihood of shape descriptors is a 1D function that
groups descriptors with similar distinction.
Descriptors
Distinction
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
Likelihood of Descriptors
Multi-dimensional normal density [Johnson 2000]

density ( x ) 

1
2
2 
d
2
 1
 1   
exp   x     ( x   )

 2 


 t

x
d dimensiona l feature vector



mean feature vector
d x d covariance matrix
Likelihood of Descriptors
The likelihood function is proportional to the descriptor
density.


p( x )  ln( density ( x ))
 t
1
 1  
p( x )    x     ( x   )
2




x
d dimensiona l feature vector



mean feature vector
d x d covariance matrix
Map from Descriptors to Likelihood
Flat regions are the most common while wing tips and
the cockpit area are rarer.
Less Likely
More Likely
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
Measuring Distinction
Evaluate the retrieval performance of every query descriptor.
Query
Descriptors
Retrieval Results
Evaluation
Metric
0.33
Measuring Distinction
Some descriptors are better for retrieval than others.
Query
Descriptors
Retrieval Results
Evaluation
Metric
0.33
1.0
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
Build Distinction Function
Measure likelihood and retrieval performance of each descriptor.
Build Distinction Function
Measure likelihood and retrieval performance of each descriptor.
Build Distinction Function
Measure likelihood and retrieval performance of each descriptor.
Build Distinction Function
Retrieval performance is averaged within each
likelihood bin.
Descriptor Distinction
A likelihood mapping separates descriptors with
different retrieval performance.
Less Likely
More Likely
Descriptor Distinction
The most common features are the worst for retrieval.
Less Likely
More Likely
Predicting Distinction
The likelihood mapping predicts descriptor distinction.
Descriptors
Distinction
Distinction Function
System Overview
Training
Shape
DB
Local
Descriptors
Likelihood
Descriptor
DB
Distinction
Function
Classification
Retrieval
Evaluation
Query
Shape
Local
Descriptors
Likelihood
Evaluate
Distinction
Select
Descriptors
Match
Retrieval
List
Selecting Distinctive Descriptors
The k most distinctive descriptors with a minimum
distance constraint are selected.
Mesh
Descriptors
Distinction
Scores
3 Selected
Descriptors
Matching with Selected Descriptors
3D
Model
Best
Matches
Model
Database
k

k
i
X  Y   C( X ,Y )
k
i
Results
• Examples of Distinctive Descriptors
• Evaluation for Retrieval
Distinctive Descriptor Examples
Descriptors on the head and neck represent consistent
regions of the models.
Distinctive Descriptor Examples
Descriptors on the front of the jet are consistent as
opposed to on the wings.
Challenge
The wheels are consistent features for cars.
Shape Database
• 100 Models in 10 Classes from the
Princeton Shape Benchmark
• Models come from different branches
of the hierarchical classification
Shape Descriptors
• Mass per Shell Shape Histogram (SHELLS)
Ankerst 1999
• Spherical Harmonics of the Gaussian
Euclidean Distance Transform (SHD)
Kazhdan 2003
Radius of Descriptors Considered
0.25
0.5
1.0
2.0
Local vs. Global Descriptors
Using local descriptors improves retrieval relative to
global descriptors.
Global vs Local
1
Precision
0.8
0.6
0.4
Global
All Local
0.2
0
0
0.2
0.4
0.6
Recall
0.8
1
Focus on Distinctive Descriptors
Using a small number of distinct descriptors maintains
retrieval performance while improving retrieval time.
Global vs Local
1
Precision
0.8
0.6
Global
All Local
10 Distinct
3 Distinct
0.4
0.2
0
0
0.2
0.4
0.6
Recall
0.8
1
Alternative Selection Techniques
Selection Techniques
20%
% Improvement Precision
Johnson 2000 (DB)
15%
Random
10%
5%
0%
10%
30%
50%
-5%
Recall
70%
90%
Alternative Selection Techniques
Selection Techniques
% Improvement Precision
20%
Johnson 2000
(Model)
Johnson 2000 (DB)
15%
Random
10%
5%
0%
10%
30%
50%
-5%
Recall
70%
90%
Alternative Selection Techniques
Distinction improves retrieval more than other techniques.
Selection Techniques
20%
% Improvement Precision
Distinctive
15%
Johnson 2000
(Model)
Johnson 2000 (DB)
10%
Random
5%
0%
10%
30%
50%
-5%
Recall
70%
90%
Conclusion
• Method to select distinctive descriptors
• Distinctive descriptors can improve retrieval
• Mapping descriptors through likelihood and
learned retrieval performance to distinction is
better than other alternatives
• Distinction is independent of type of descriptor
Future Work
• Explore other definitions of likelihood
including mixture models
Future Work
• Explore other definitions of likelihood
including mixture models
• Consider non-likelihood parameterizations
Future Work
• Explore other definitions of likelihood
including mixture models
• Consider non-likelihood parameterizations
• Combine descriptors while accounting for
deformation [Funkhouser and Shilane, SGP]
Acknowledgements
Szymon Rusinkiewicz
Joshua Podolak
Princeton Graphics Group
Funding Sources:
National Science Foundation Grant CCR-0093343 and Grant
11S-0121446
Air Force Research Laboratory Grant FA8650-04-1-1718
The End