Automated Caricature of
Robot Expressions in
Socially Assistive
Human-Robot Interaction
Ross Mead and Maja J Matarić
Presented by David Feil-Seifer
Interaction Lab
University of Southern California
Outline
Motivation and Background
Approach and Methods
Robot Platform
Experimental Design
Q&A
2
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Motivation and Background
Children with autism spectrum disorder (ASD) suffer from a deficit that prevents them from
observing, interpreting, and learning social cues.
Clinical studies in social skills training have proposed methods, such as exaggeration, to
enhance autism intervention strategies (Rao et al., 2008).
Socially assistive robotics has potential to improve social activity (Feil-Seifer & Matarić, 2005).
The Transporters is an animated series that has shown an improvement in face-based emotion
recognition in children with ASD (Golan et al., 2009)…
•
However, expressive behaviors of the body still remain to be addressed.
3
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Approach and Methods
Take inspiration from several principles of animation (Thomas & Johnston, 1981)…
•
•
•
•
Staging
Exaggeration
Anticipation
Secondary Action
Goal: Automated caricature of social interaction behaviors of robots.
4
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Approach and Methods — Staging
Aims to provide some grounding for robot gestures in subsequent operations.
Process of presenting a communicative act in as clear a way as possible…
•
•
Minimize or eliminate conflicting signals.
Isolate features that uniquely identify the content of the expression.
Caricaturing highlights such features, providing building blocks for expressions.
Hypothesis: Well-staged expressions provide more clarity to a child with ASD than
poorly-staged expressions.
Source: http://familypants.files.wordpress.com/2009/02/preston11.jpg
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
5
13
Approach and Methods — Exaggeration
Amplify distinct features that identify the expression to make content of the behavior
more convincing and/or explicit.
Use feature parameterizations isolated during the staging process to produce
exaggerated expressions.
Hypothesis: A child with autism will be more capable of interpreting an exaggerated
expressive behavior than a non-exaggerated expressive behavior…
•
Supported by the peak shift principle (Ramachandran & Hirstein, 1999).
Source: http://www.animationbrain.com/2D/38principle.JPG
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
6
13
Approach and Methods — Anticipation
Anticipatory action often indicates or emphasizes the intent of the character.
Staging and exaggeration provide insights pertaining to the dynamics of a
communicative act…
•
We aim to utilize this to automatically generate motion paths for both micro and
macro anticipatory expressions that precede it.
Hypothesis: Anticipatory actions will provide a child with ASD a better understanding
of the intent of his or her social partner than non-anticipatory action.
Source: http://www.evl.uic.edu/ralph/508S99/gif/batter.gif
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
7
13
Approach and Methods — Secondary Action
The use of redundant signals in an expression to better communicate an idea.
Redundancy plays a key role in social interaction (Birdwhistell, 1970).
Signals must be isolated into distinct parts for proper staging, exaggeration, and
anticipation to occur.
We suspect that secondary action has potential with high-functioning children with
autism (particularly, those who have participated in the Transporters studies)…
•
Might be overwhelming for children that are far in the spectrum.
Source: http://www.awn.com/tooninstitute/lessonplan/images/walk17.jpg
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
8
13
Robot Platform
Currently being implemented on the Sparky
Minimatronic™ robot figure…
• Courtesy of Walt Disney Imagineering
Research & Development.
Uses two servo controllers and 18 servos…
• Lightweight and highly dexterous.
• Movements are fluid and natural.
• Articulated spine allows us to manipulate
posture-based expressions.
Will utilize an off-board sensor network that
includes color cameras, lasers, Nintendo
Wiimotes™, and desktop computer interface.
Being implemented as a tabletop agent,
interacting verbally and nonverbally with user.
9
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Experimental Design
Preliminary work discusses gesture repetition/persistence (Mead & Matarić, 2009).
Test techniques and hypotheses within the context of ASD social skills intervention.
Must validate with typically developed children…
•
Consider expressive behaviors that utilize the isolated features determined
during the staging process.
•
Consider their exaggerated, anticipatory, and secondary counterparts.
Conduct a study with children with ASD to determine the impact of each technique.
10
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Acknowledgments
This work is supported in part by the National Science Foundation under:
•
•
Graduate Research Fellowship Program (GRFP)
Grant CNS-0709296 (“CRI: IAD - Computing Research Infrastructure for
Human-Robot Interaction and Socially Assistive Robotics”)
•
Grant IIS-0803565 (“Personalized Socially-Assistive Human-Robot Interaction:
Applications to Autism Spectrum Disorder”)
•
Grant IIS-0713697 (“HRI :Personalized Assistive Human-Robot Interaction:
Validation in Socially Assistive Robotics for Post-Stroke Rehabilitation”)
We would like to thank Akhil Madhani and Walt Disney Imagineering Research &
Development for the use of the Sparky Minimatronic™ robot figure.
11
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Selected References
1.
S. Baron-Cohen, O. Golan, E. Chapman and Y. Granader,
6.
R. Mead and M. J. Matarić, “The Power of Suggestion:
Teaching Sequences Through Assistive Robot Motions,” in
the Proceedings of The 4th ACM/IEEE International
Conference on Human-Robot Interaction (HRI-09), pp. 317318, 2009.
7.
Z. Mo, J. P. Lewis, and U. Neumann, “Improved Automatic
Caricature by Feature Normalization and Exaggeration,”
SIGGRAPH 2004 Sketches and Applications, 2004.
8.
V. S. Ramachandran and W. Hirstein, “The Science of Art: A
Neurological Theory of Aesthetic Experience,” in the Journal
of Consciousness Studies: Special Feature on Art and the
Brain, vol. 6, no. 6-7, pp.15-51, 1999.
9.
P. A. Rao, D. C. Beidel, and M. J. Murray, “Social Skills
Interventions for Children with Asperger’s Syndrome or HighFunctioning Autism: A Review and Recommendations,” in the
Journal of Autism and Developmental Disorders, vol. 38, no.
2, pp. 353-361, 2008.
10.
A. Tapus, M. J. Matarić, and B. Scassellati, “The Grand
Challenges in Socially Assistive Robotics,” in IEEE Robotics
and Automation Magazine, vol. 14, no. 1, pp. 35-42, 2007.
11.
F. Thomas and O. Johnston, The Illusion of Life: Disney
Animation. Hyperion, 1981.
“Transported into a World of Emotion,” in The Psychologist,
vol. 20, no. 2, pp. 76-77, 2007.
2.
R. L. Birdwhistell, Kinesics and Context: Essays on Body
Motion Communication. Philadelphia, PA: University of
Pennsylvania Press, 1970.
3.
S. E. Brennan, “Caricature Generator: The Dynamic
Exaggeration of Faces by Computer,” in Leonardo, vol. 18,
no. 3, pp. 170-178, 1985.
4.
D. J. Feil-Seifer and M. J. Matarić, “Defining Socially
Assistive
Robotics,”
in
International
Conference
on
Rehabilitation Robotics, pp. 465-468, 2005.
5.
O. Golan, E. Ashwin, Y. Granader, S. McClintock, K. Day, V.
Leggett,
and
S.
Baron-Cohen,
“Enhancing
Emotion
Recognition in Children with Autism Spectrum Conditions: An
Intervention Using Animated Vehicles with Real Emotional
Faces,” in the Journal of Autism and Developmental
Disorders, DOI: 10.1007/s10803-009-0862-9, in press.
12
HRI2010 Workshop: What Do Collaborations with the Arts Have to Say about HRI?
13
Q&A
Please address praise
and easy questions to
rossmead@usc.edu.
Please address
criticism and
difficult questions
to Dave… 