How Do Humans Sketch Objects?
SIGGRAPH 2012
Mathias Eitz @ Technische Universität Berlin (柏林工业)
James Hays @ Brown University
Marc Alexa @ Technische Universität Berlin
Speaker: Xiaohua Xie
Mathias Eitz
• M.Eng. degree from Shanghai Jiao Tong University
• PhD student at Technische Universität Berlin
• Research on sketch:
-
How Do Humans Sketch Objects? SIGGRAPH 2012
Sketch-based Shape Retrieval. SIGGRAPH 2012
Learning to classify human object sketches. ACM SIG. Talks
Photosketcher: interactive sketch-based image synthesis. IEEE
CGA
- Sketch-based image retrieval: benchmark and bag-of-features
descriptors. IEEE TVCG
- An evaluation of descriptors for large-scale image retrieval
from sketched feature lines. CG
……..
#1:A large dataset of human object sketches
• 20,000 sketches of 250 object categories
• Using Amazon Mechanical Turk (AMT)
-paid “Human Intelligence Tasks (HIT)” to work
#2:Human Recognition
•
•
•
•
•
•
AMT again
Each HIT identifies 4 sketches.
Each worker takes no more than 100 HITs.
Categories are organized in 3-level hierarchy.
Average accuracy: 73.1%.
Usually confusions between semantically similar categories
(e.g. ‘panda’ and ‘bear’)
Easy tasks
Difficult tasks
#3:Machine Recognition
• “Bag-of-word” + ”SIFT” + “soft SVM” works best!
• Average accuracy: 56%.
• Extension to semantic sketch-based retrieval and the
recognition of artistic sketches.
Semantic search
Recognizing artistic sketch
Some middle conclusions
• Humans tend to follow a coarse-to-fine drawing strategy:
-first outlining with longer strokes and then adding detail.
• People seem to have more than one iconic (representation) for
each category of objects.
Conclusions
• Pay for collecting a sketch dataset.
• Pay for a study for human recognition on sketches.
• Study the computer recognition for sketches.
• Title question: How do humans sketch objects?
• How? How? How?.....
Open questions
• How to exploit the temporal order of strokes.
• How could the computer generate distinctive
sketches that are immediately recognizable by
humans?
• How universal sketching and sketch recognition is
among humans. (different cultures, ages, genders,
artistic expertise, etc.)
Thank you!!