Telematica Institute, Enschede,
LIACS, Leiden University.

•
What is emotion?
• What theoretical angles can we take to study emotion?
• What is affective computing + examples ?
• Why is affective computing useful ?
15 min coffee break.
• Interactive Human-Robot Interaction
• Human emotional expressions as feedback to a learning robot

• Common emotions: fear, anger, happiness, sadness, surprise, disgust
•
Short episode of synchronized system activity triggered by event:
– subjective feelings (the emotion we normally refer to),
– tendency to do something (action preparation),
– facial expressions,
– evaluation of the situation (cognitive evaluation, thinking),
– physiological arousal (heartbeat, alertness).
•
Affect ( affectie )= related to emotion and mood:
– emotion : short term, high intensity, object directed,
– mood : unfocused, long term, low intensity,
– affect terms of,
: sometimes seen as abstraction of emotion/mood in
• positiveness/negativeness and activation/deactivation ( Russell ).

•
Situational evaluation and communication.
•
Heuristic relating events to personal goals , needs and beliefs:
– evaluates personal relevance of event ( Scherer ) and helps decisionmaking ( Damasio ),
– fast reactions and action preparation ( Frijda ),
– influence information processing and decision making ( Damasio ):
• e.g, adaptation, attention, search.
• Communication medium:
– communicate internal state (show feelings),
– alert others (direct attention),
– show empathy (show understanding of situation).
– Give feedback (evaluate behavior).

• Emotion and affect influence thought and behavior:
– The kind of thoughts we have
• Mood congruency (e.g., positive moods triggers positive thought)
– The way we process information
• Broad vs. narrow look (
Goschke & Dreisbach ) (forest vs. trees)
• A lot vs. a little processing effort (
Scherer , Forgas )
–
What we think about things
• Emotion/mood as information (
Clore & Gasper ) (how does this feel?)
• Emotion as belief anchor (
Frijda & Mesquita
) (idea “fixation”)
– How we learn and adapt
• Emotion/affect as social reinforcement (emotion expression as signal)
• Emotion/affect as intrinsic reinforcement (use my own emotion as feedback)
• Emotion as “metaparameter” to control learning process (explore vs. exploit)

• Biological/Neurological (
Damasio , Panksepp , LeDoux , Rolls ):
– aim: find the necessary / sufficient brain areas and circuits involved in emotions /feelings / self-regulation / adaptation.
• Biological/Evolutionary ( Darwin , Ekman ):
– aim: why emotions exist in the first place, what’s the utility of emotion .
• Cognitive/Psychological ( Scherer , Frijda ):
– aim: understand relation cognition/emotion , why does event e results in emotion x while:
• the
same event e may result in a different emotion , and
• other events may result in the same emotion x.
• Social ( Ekman , and others ):
– aim: understand the role of emotion in communication.

•
Computing that relates to, arises from, or deliberately influences emotions ( Picard ).
• Different types of computer models :
– recognize emotions (perception, in ),
– interpret/elicit emotions (cognition/behavior, processing ),
– emotional influence on behavior (adaptation, attention),
– show emotions (expression, out ), and
– complex mixtures of these…
• An affective computing system is
– a system of computational processes that perceives, expresses, interprets, or uses emotions,
– e.g. a robot , a virtual character , a tutor agent, a fridge , etc…

• Entertainment: emotions are used to provide entertainment value .

• Social : Kismet, A framework, using a humanoid head expressing emotions, to study:
– effect of emotions on human-machine interaction .
– learning of social robot behaviors during human-robot play.
– joint attention .

• Cognitive : study the influence of artificial emotions on
– planning mechanism of virtual characters,
– training effect on trainees (emotion might enhance effect)

• Aibo (Sony, Japan)
Entertainment robot
• I-Cat (Philips, NL)
Robot assistant for elderly people
• Paro (Wada et al, Japan)
Robot companion for elderly
• Huggable (MIT, USA)
Robot companion for elderly

• From a theoretical point of view…
• Advance emotion theory
– simulated agents that elicit emotions to study possible psychological structure of emotions ( Scherer ),
– emotion as artificial motivator (e.g. a bored robot that explores )
( Canamero ).
• Understand intelligence and adaptation
– laws and rules are not sufficient for understanding or predicting human behavior and intelligence (e.g. how to sift thru many possible choices )
( Damasio , Picard )
– simulated learning agents influenced by emotions (e.g., emotion as reward ) ( Breazeal , Broekens ),
– Artificial Intelligence, Artificial Life?

• From a practical point of view…
• Facilitate Human-Computer (Robot) interaction
– use emotions in simulated-agent plans ( to simulate human reasoning )
( Gratch & Marsella ),
– communication and joint attention ( Breazeal , MIT)
– robot acceptation (
Heerink, Telin )
– Persuasive design (VR training, tutor agents (
Gratch & Marsella , Nijholt ))
– Interactive robot learning (second part of this lecture).
• Entertainment
– Computer games such as the Sims ( EA ), Aibo Robot ( Sony ) .

•
Affect has different meanings related to emotion.
– emotion : short term, high intensity, object directed,
– mood : unfocused, long term, low intensity,
– affect : sometimes seen as abstraction of emotion/mood in terms of,
• positiveness/negativeness and activation/deactivation .
• Affective computing is about computing with emotions
– “a system of computational processes that perceives, expresses, interprets, or uses
emotions”
•
Why?
– advance emotion theory,
– understand intelligence,
– facilitate Human-Computer (Robot) interaction,
– entertainment.

• Goals Human Robot Interaction:
– Develop more natural and effective ways of interacting between robots and humans.
– Allow non-expert interaction .
– Allow flexibility of robot behavior.
• How?
– Gesture recognition
– Cognitive robot architectures
– Machine learning (artificial adaptation)
– Affective computing
• Emotion recognition
• Emotion expression

• Interactive robot learning
– Learning by
Example
• Imitation learning (see
Breazeal & Scassellati )
• E.g., gesture repetition
– Learning by Guidance ( Thomaz & Breazeal )
• In general: future directed learning cues , such as
• Directing attention
• Communicating motivational intentions (e.g., anticipatory reward)
• Proposing actions
– Learning by Feedback
• Learn by trial (+feedback) and error (-feedback) and exploration.
• Additional human-based reinforcement signal ( Breazeal & Velasquez ; Isbell et al ; Mitsunaga et al ; Papudesi & Hubert )

• Robot perspective: Interactive robot learning
– Facilitate human-robot interaction
– Study learning and adaptation
– Future:
• enable robots to interactively cooperate in an efficient manner with humans in ways that are natural to humans .
• Psychological perspective: Understanding mechanisms
– How can affect influence learning?
– How can affect influence attention processes?
– Joint Human-Robot attention
– Study learner-teacher relations
– Etc..

EARL
• A framework to study the influence of E motion on A daptive behavior in the context of R einforcement L earning:
– emotion as
reinforcement (reward/punishment) ,
– emotion as influence on learning process,
– emotion as information ,
– artificial emotion resulting from learning process

• A typical EARL setup:
– Simulated robot in a
– Maze learning (navigation) tasks
– Reinforcement learning (RL) approach to robot learning
• Learn by trial (+feedback) and error (-feedback) and exploration.
– Robot has own model of emotion
– Robot head to express emotion
– And…
•
Webcam and emotion recognition to interpret emotions

• An experiment to study interactive robot learning
• Simulated Robot learns to find food in a maze.
–
Explore unknown environment (try actions).
– Reinforcement Learning
•
Feedback from environment based on taken actions
• + feedback=repeat
• - feedback=don’t repeat
– Learn which sequence of actions gives best +feedback.
• Robot also uses human facial expressions as +/feedback

• Affective signal as additional reinforcement
– Web cam
– Emotional expression analysis
–
Positive emotion (happy) = reward
– Negative emotion (sad) = punishment
- / +
Sad / happy
Robot behavior to screen
– So: in addition to feedback from environment emotional expression is used in learning as r human, observer a social reward coming from the human

Food (+)
Agent
Wall (-)
Feedback
Path
2 b human feedback path wall food

• Test learning difference between standard agent and social agents
– Standard agent uses rewards environment to update behavior.
– Social agent uses rewards from environment and reward from human emotional signal.
• Results:
– Social agent learns the path to the food quicker
( just believe me, so that I don’t have to explain the graphs
)
• Earl demo…

•
A special case of Human Robot Interaction
– Goal HRI: more efficient , flexible , personal , pleasant human robot interaction
– Interactive Robot Learning:
Imitation , Feedback , Guidance .
• Affective Interaction Feasible and useful
– can be used as guidance (MIT robot studies), and
– feedback (experiment shown).
• Why?
– Robot perspective
• Facilitate human-robot interaction
• Study learning and adaptation
– Psychological perspective
• How can affect influence learning?
• How can affect influence attention processes?
• Joint Human-Robot attention
• Study learner-teacher relations