Query Interfaces for
a 3D Model Search Engine
Patrick Min
Tom Funkhouser, Misha Kazhdan, Joyce Chen,
Alex Halderman, David Dobkin, David Jacobs
DLI2 meeting, Edinburgh, 25 June 2002
Online Data

On-line multimedia data is changing the way
we get and use information
Registered Saddlebred out of
Famous Sultan Supreme line. 100%
sound. 16 year old, flashy, chestnut
w/white, loving, high energy horse,
needs experienced rider. Was shown
professionally in early years as
gaited saddlebred. Most recently
shown and always placed in training
and first level dressage
shows.Currently used as
dressage/pleasure horse, jumps,
loves trailriding.
Text
www.dreamhorse.com
2D Image
What about 3D Models?
Audio
3D Models
are becoming easier to create, use and distribute
3D Models
are becoming easier to create, use and distribute
• Laser range scanners: supply
Cyberware Scanner
3D Models
are becoming easier to create, use and distribute
• Laser range scanners: supply
• Fast graphics cards: demand
• World Wide Web: distribution
Georgia Tech
Cyberware Scanner
Protein Data Bank
Large Model Archive
3D Models
are becoming easier to create, use and distribute
• Laser range scanners: supply
• Fast graphics cards: demand
• World Wide Web: distribution
One day 3D models will be as ubiquitous
as images are now
Georgia Tech
Cyberware Scanner
Protein Data Bank
Large Model Archive
Research Challenges
Current research:
• How do we construct 3D models ?
Digital Michelangelo Project
Stanford Graphics Laboratory
Research Challenges
Current research:
• How do we construct 3D models ?
Digital Michelangelo Project
Stanford Graphics Laboratory
Future research:
• How do we find 3D models ?
• How do we understand them ?
Finding 3D Models
car
Project Goal
A Search Engine for 3D Models on the Web
car
3D Model Search Engine
World
Wide
Web
Crawler
Crawler
Crawler
Crawler
Repository of
3D Models
Off-line
On-line
Indexer
Text
Index
2D
Index
3D
Index
Text
Matcher
2D
Matcher
3D
Matcher
Query
User
Query
Interface
Matches
3D Model Search Engine
Acquisition
World
Wide
Web
Crawler
Crawler
Crawler
Crawler
Repository of
3D Models
Off-line
On-line
Indexer
Text
Index
2D
Index
3D
Index
Text
Matcher
2D
Matcher
3D
Matcher
Query
User
Query
Interface
Matches
3D Model Search Engine
Analysis
World
Wide
Web
Crawler
Crawler
Crawler
Crawler
Repository of
3D Models
Off-line
On-line
Indexer
Text
Index
2D
Index
3D
Index
Text
Matcher
2D
Matcher
3D
Matcher
Query
User
Query
Interface
Matches
3D Model Search Engine
World
Wide
Web
Crawler
Crawler
Crawler
Crawler
Repository of
3D Models
Off-line
On-line
Indexer
Text
Index
2D
Index
3D
Index
Text
Matcher
2D
Matcher
3D
Matcher
Query
User
Query
Interface
Matches
Matching
3D Model Search Engine
World
Wide
Web
Crawler
Crawler
Crawler
Crawler
Repository of
3D Models
Off-line
On-line
Indexer
Text
Index
2D
Index
3D
Index
Text
Matcher
2D
Matcher
3D
Matcher
Query
User
Query
Interface
Querying
Matches
3D Model Search Engine
Analysis
World
Wide
Web
Crawler
Crawler
Crawler
Crawler
Repository of
3D Models
Off-line
On-line
Indexer
Text
Index
2D
Index
3D
Index
Text
Matcher
2D
Matcher
3D
Matcher
Query
User
Query
Interface
Querying
Matches
Matching
Main Challenge & today's topic
Which Query Interfaces are Useful
for a 3D Model Search Engine?
Query
User
Query
Interface
Matches
Desired Properties of Query
Interfaces




easy to use
• also for novice users
possible to quickly specify query
efficient matching method
effective
• high number of relevant results
Available Data for Queries
Example of a 3D Model available on the Web:
http://www.3dspline.com
Available Data for Queries
Text
Appearance
Shape
Available Data for Queries
Text
Appearance
Shape
Text Queries
Text Queries
+ easy to use
+ possible to quickly specify query
+ efficient matching method
+/- effective
Effective Text Queries
747
caffeine
dog
dna
fork
Text Query Problems

objects are badly annotated:
• ferrar03.3ds, ferrair, rahmen, …

keywords are too common
• face, plane
Text Query Problems

objects are badly annotated:
• ferrar03.3ds, ferrair, rahmen, …

keywords are too common
• face, plane


keywords are unknown to user, or annotator
not possible to anticipate every relevant text query
• elf, fairy-tale creature, puppet, person, ...
Text Query Problems

objects are badly annotated:
• ferrar03.3ds, ferrair, rahmen, …

keywords are too common
• face, plane


keywords are unknown to user, or annotator
not possible to anticipate every relevant text query
• elf, fairy-tale creature, puppet, person, ...
We hypothesize shape can help
Outline







Introduction
Text Queries
3D Shape Queries
2D Shape Queries
Results
Conclusions
Future Work
3D Shape Queries
use 3D shape to find 3D shape
3D Model Queries

(create &) upload 3D model

select 3D model on results page
("shape iteration")
3D Matching Method
3D Matching Method
Shape Iteration User Study
Shape Iteration User Study
How does shape combine with text in interactive searches?

Test Design:
• 18 students in an introductory CS class
• input: on website, initial text keyword search,
then “find similar object”
• where similar means similar text or similar
shape (unknown to user)
Shape Iteration Results
Similarity


shape iteration finds target object more often
in fewer iterations and in less time
3D Model Query Problems


upload 3D model:
• such a model may not be available
• creating one with standard tools is hard
shape iteration:
• have to get such a result first
3D Sketch Queries

Sketch 3D model using simple modeling program
• e.g. Teddy, Sketch
Teddy: A Sketching Interface for 3D Freeform Design
Igarashi et al., SIGGRAPH 1999
3D Sketch Queries
Teddy: A Sketching Interface for 3D Freeform Design
Igarashi et al., SIGGRAPH 1999
Teddy Sample Results
3D Sketch Query Problems
Teddy:
 supports only coarse, blobby shapes of genus 0
 still hard to use for novice users
3D Sketch Query Properties
- easy to use
- possible to quickly specify query
+ efficient matching method
+/- effective
So we need a simpler query interface
2D Sketch Queries
2D Sketch Queries

“paintbrush” style interface:
• simple outline sketches
• easy to use for most users
Multiple 2D Sketches
2D Sketch Matching Method
5
4
6
3
7
1
Query sketch
2
7 projections per database model
2D Sketch Matching Method


compute 2D shape descriptor in similar fashion
intersect with circles instead of spheres
2D Sketch User Study
2D Sketch Interface User Study
Does shape add to text?

Test design:
• 43 students in introductory CS class
• goal: retrieve target objects
• input: text keywords and sketches on paper
Example Input
2D Sketch User Study Results

Distinguishing keywords:
• text works well
• even better with sketch
2D Sketch User Study Results

Class is too large:
• text is too inspecific
• sketch works best
2D Sketch User Study Results


text and shape can complement each other
sketch allows refinement within a class
2D Sketch Query Problems

The drawing is free-form:
• hard to draw accurately
• hard to get proportions right
Dog
DC10
2D Sketch Query Problems

The drawing is free-form:
• hard to draw accurately
• hard to get proportions right
Dog
DC10

Image matching method:
• user and database outline have to look similar
Search Engine Usage Results
Search Engine Results



online since early November 2001
20,707 models indexed, 2,873 commercial
one typical week of usage:
• 4,522 queries
• 1,346 unique hosts from 55 different countries
• 1,029 models downloaded
Search Engine Results
Text vs. Shape
text
shape
empty

about 70 % text, 30 % shape-based
Search Engine Results
Shape Query Type
2D
text & 2D
3D
text & 3D
similar shape
upload model

about 58 % shape iteration, 38 % 2D/text & 2D
Number of Hosts per Day
jan

feb
mar
apr
may
average about 175 hosts and 525 searches/day
% New Hosts per Day
jan

feb
mar
apr
about 20 % returning hosts per day
may
# Searches per User
# hosts
7000
6000
5000
4000
3000
2000
1000
1

3
5
7
9 11 13 15 17 19 21 23 25
# searches
about 40 % does 3 or more searches per day
Conclusions and Future Work
Conclusions


shape-based search can help
each interface has its strengths and weaknesses
• so useful to combine them
table
find similar shape
Future Work
2D Structural Interface
 primitive drawing user study
 evaluation of several interfaces
• oval + rectangle
• skeletal graphs
3D Structural Interface
Future Work II
3D Model Database


fully automated acquisition and model processing
support for more model formats
Acknowledgements
Funding
 NSF
(CCR-0093343, 11S-0121446, CCR-99-88173, DGE-9972930)
 Army Research Organization
(DAAD19-99-1-0205)
 Alfred P. Sloan Fellowship
(for Prof. Tom Funkhouser)
People
 Brian Kernighan and the students of CS109 and CS111
 Bernard Chazelle, Adam Finkelstein,
Szymon Rusinkiewicz, Emil Praun
 Princeton CS department technical staff
 Princeton Shape Analysis Group
3D Models
 Viewpoint Datalabs,
http://www.viewpoint.com
 De Espona Infografica,
http://www.deespona.com
http://shape.cs.princeton.edu
3D Model Search Engine
Test Database

50 models, 10 categories of 5 models each