Expressive Emotional
ECA
✔
Catherine Pelachaud
✔
Christopher Peters
✔
Maurizio Mancini
Expressive Emotional
ECA
✔
✔
Emotion behavior modeling:
✔
Signals
✔
Temporal course of signals
✔
Expressive qualities
Often in ECA systems:
✔
only modeling of signals is considered
✔
6 ‘universal’ expressions
Large variety of facial
expressions (Kollias et al)
✔
Archetypal expressions modeling using FAPs
MPEG-4
✔
Estimation of the range of variations of FAPs in
archetypal expressions
✔
Intermediate expressions: rule-based
computational model based on
✔
Whissell’s representation:
✔
✔
Activation, evaluation,
Plutchik’s angular measure
Laban Movement Analysis
✔
✔
5 major components:
✔
Body: part of the body in used
✔
Space: description of the directions and paths of
motion
✔
Shape: changing forms of the body
✔
Effort: dynamics of the body moves
✔
Relationship: modes of interaction with oneself,
others and the environment (facing, contact…)
Textual and symbolic language of movement
description
EMOTE (Badler et al)
EMOTE model for Effort and Shape
✔ Based on Movement Observation
Science
✔ Laban Movement Analysis
✔ Computational model of Effort and
Shape components
✔
Effort
✔
Four motion factors: space, weight,
time, flow
Space
✔ Weight
✔ Time
✔ Flow
✔
✔
Each factor ranges from:
Indulging in the quality
✔ Fighting against the quality
✔
Effort
✔
Space: indirect  direct
✔
✔
Weight: light  strong
✔
✔
Feather movement  punching
Time: sustained  sudden
✔
✔
Waving away bugs  pointing to a particular
spot
Stretching  grabbing a falling object
Flow: free  bound
✔
Waving widely  moving in slow motion
Shape
✔
Three distinct qualities of change in
the form of movement:
Shape flow: mover’s attitude toward the
changing relationship among body parts
✔ Directional movement: mover’s intent to
bridge the action to a point in the
environment
✔ Shaping: mover’s carving or molding
attitude with the environment
✔
Shape
✔
Three dimensions:
✔
Horizontal: spreading  enclosing
✔
✔
Vertical: rising  sinking
✔
✔
Opening arms to embrace  clasping
someone in a hug
Reaching for something in a high shelf 
stamping the floor with indignation
Sagittal: advancing  retreating
✔
Reaching out to shake hand  Avoiding a
punch
Expressive
Communicative Gestures
(Hartmann et al)
✔
Expressivity parameters:
✔
✔
✔
✔
✔
✔
✔
Spatial: amplitude of movement
Temporal: duration of movement
Power: dynamic property of movement
Fluidity: smoothness and continuity of movement
Repetitiveness: tendency to rhythmic repeats
Overall Activation: quantity of movement across
modalities
Expressivity modeling: modal and intermodal
Emotional Modeling
• Carmen's Bright IDEAS
( Stacy Marsella, USC)
• Interactive drama
• Agent exhibits gestures
based on their emotional
states and personality
traits
FantasyA (Ana Paiva,
Inesc)
• Role-playing
game
• The emotion
detected from user
drives the
behavior of one of
the agents in the
video game
SenToy (Ana Paiva,
Inesc)
• SenToy is a tangible
interface device,
used to influence
emotional behaviour
in FantasyA
• SAFIRA: EU project
Behavior Modeling
✔
Facial expression:
✔
Use facial analysis techniques/motion capture
to look at:
✔
Value of facial animation parameters (muscular
contraction, FAP…)
✔
Pattern of movement for a given parameter
✔
Relation of facial animation parameters within a
same expression
✔
Coarticulation of movements between expressions
Mocap
Mocap
FAPs – MPEG-4
Example
Gesture
✔
Gesture specification often based on sign
language gesture specification:
✔
✔
✔
✔
✔
✔
Hand description
Palm orientation
Wrist position
Movement: wrist, elbow, finger…
Parameters characterizing quality of
movement: Laban, sub-set…?
Parameters act on overall animation (Emote) or
on gesture phases (preparation, stroke, …)?
Behavior
✔
Gaze:
✔
✔
Body:
✔
✔
Direction, gaze length, mutual gaze, ratio of gaze
avoidance/gaze look at
Body posture, overall movement quality
Voice:
✔
Paralinguistic parameters
✔
Text
Signals analysis/generation
✔
Use analysis method to:
✔
Get information on parameters
✔
Multimodal integration
✔
Signals synchrony across modalities
✔
Build computational model for generation
✔
Get the expressive quality of behavior
✔
Signals generation should not be done on one
frame only but on a temporal interval
Conclusion
✔
Emotion is shown through:
✔
Signals
✔
Their temporal course
✔
Which parameters to look at? Which are the
perceptually relevant ones?
✔
How to describe:
✔
The signals?
✔
Their evolution through time?
✔
Correlation between modalities?
Conclusion
✔
Relation between intensity of emotion and
behaviors?
✔
Selection of behaviors
✔
Quality of behaviors
✔
Duration of behaviors
✔
Behaviors across modalities