How Do Humans Sketch Objects? SIGGRAPH 2012 Mathias Eitz @ Technische Universität Berlin (柏林工业) James Hays @ Brown University Marc Alexa @ Technische Universität Berlin Speaker: Xiaohua Xie Mathias Eitz • M.Eng. degree from Shanghai Jiao Tong University • PhD student at Technische Universität Berlin • Research on sketch: - How Do Humans Sketch Objects? SIGGRAPH 2012 Sketch-based Shape Retrieval. SIGGRAPH 2012 Learning to classify human object sketches. ACM SIG. Talks Photosketcher: interactive sketch-based image synthesis. IEEE CGA - Sketch-based image retrieval: benchmark and bag-of-features descriptors. IEEE TVCG - An evaluation of descriptors for large-scale image retrieval from sketched feature lines. CG …….. #1:A large dataset of human object sketches • 20,000 sketches of 250 object categories • Using Amazon Mechanical Turk (AMT) -paid “Human Intelligence Tasks (HIT)” to work #2:Human Recognition • • • • • • AMT again Each HIT identifies 4 sketches. Each worker takes no more than 100 HITs. Categories are organized in 3-level hierarchy. Average accuracy: 73.1%. Usually confusions between semantically similar categories (e.g. ‘panda’ and ‘bear’) Easy tasks Difficult tasks #3:Machine Recognition • “Bag-of-word” + ”SIFT” + “soft SVM” works best! • Average accuracy: 56%. • Extension to semantic sketch-based retrieval and the recognition of artistic sketches. Semantic search Recognizing artistic sketch Some middle conclusions • Humans tend to follow a coarse-to-fine drawing strategy: -first outlining with longer strokes and then adding detail. • People seem to have more than one iconic (representation) for each category of objects. Conclusions • Pay for collecting a sketch dataset. • Pay for a study for human recognition on sketches. • Study the computer recognition for sketches. • Title question: How do humans sketch objects? • How? How? How?..... Open questions • How to exploit the temporal order of strokes. • How could the computer generate distinctive sketches that are immediately recognizable by humans? • How universal sketching and sketch recognition is among humans. (different cultures, ages, genders, artistic expertise, etc.) Thank you!!