The Affective
Loop
Kristina
Höök



Kristina Höök
Swedish Institute of Computer Science
www.sics.se
Professor in human-machine interaction
at the IT-university, Stockholm
University/KTH, www.dsv.su.se
Mostly hate?
Emotions do not hinder
rational thinking!
Emotions are the basis
for rational thinking
 Emotions and
reasoning is based both
in the brain but also in
our bodies

Gestures & tangible
interaction
Exertion:
Light
Flexible

Laban:
Space:
Fluent
– Analysis of dance, factory
Control:
Quick
Time: Sustained
work, human movement
– Certain body postures
Strong
and gestures ”encourage”
certain emotions

Ishii:
– Tangible interaction,
embodied
Direct
Bound
Affective computing
Make computers:
 Express emotions
 Influence user emotions
 Understand user emotions
 Allow users to express emotions
 ”Have” emotions?
Facial expressions
Affective computing
Make computers:
 Express emotions
 Influence user emotions
 Understand user emotions
 Allow users to express emotions
 ”Have” emotions?

AIBOs
Affective computing
Make computers:
 Express emotions
 Influence user emotions
 Understand user emotions
 Allow users to express emotions
 ”Have” emotions?
Recognize emotions
Affective computing
Make computers:
 Express emotions
 Influence user emotions
 Understand user emotions
 Allow users to express emotions
 ”Have” emotions?
SenToy
Affective computing
Make computers:
 Express emotions
 Influence user emotions
 Understand user emotions
 Allow users to express emotions
 ”Have” emotions?
Influencing Machine
Emotional in Florida?
John Thackara
A culture is surely in deep, if not terminal, crisis when engineers
start talking about the design of emotions. Now even Don
Norman is at it.
"In the past year I've been catching up on the scientific work in
emotion", he writes in the programme for an up-coming CHI
conference; "the result is a new theory of emotion, with
implications for the development of autonomous machines and
robots as well as to design".
Now Professor Norman is a great man, but this is both ridiculous
and....sad. To talk about putting emotions into products, in the
same way that Del Monte puts tomatoes into cans, is literally
meaningless. I will spell it out: emotion is a quality of
consciousness. Consciousness is something that people have,
and machines don't. It follows that the only "implication for
design" of emotion is: steer well clear of the subject, or risk
looking ridiculous.
The affective loop...



Inspired by Influencing Machine by
Phoebe Sengers & collegues and
SenToy by Ana Paiva & collegues
Emotions reside in and are
induced by both body and mind
Not recognise emotion as a onestep process – instead enter an
interaction? Needs strong narrative
(or context)
Influencing Machine
Phoebe Sengers & others




Intimate relationship between humans and
computational ‘beings’
Child-like drawings drawn in real time
project on the walls and music reflects the
agent’s emotional state
Users can send the agent postcards in a
mailbox
The postcard affects the agent’s emotional
state, which in turn changes the pictures
that are being drawn and music played
Affective input
Mailbox
Affective output:
childlike drawings &
music
Inspiration: kids’ drawings
Affective expressions:
• Complexity of shape
• Pressure of pen
• Colour
• Speed
• Shape
• Size
• ....
The point?
To provoke our cultural notions of
whether a machine can ”have”
emotions and act on its own
 (..but also to be charming)

The
enthusiastic quartet
”We might have influenced his
feelings, through holding up a
pink cards. Think of how a child
would react, sits there and
feels really sad and is thinking
about sad things, and then I
give him a pink card, and he
becomes happy, though he is
still somewhat cold, so I give
him a yellow card, so he
becomes warm.”
What they do
vs. what they say...
Video observation:
One of the women made an interesting
comment: she pointed at the computer
under the table with the table cloth, and
asked the man in the group whether this
computer was in fact connected to the
machine?
She meant that if it was connected, then
the Influencing Machine was just a
computer – not a machine in its own right.
It seems that to her a computer cannot be
what she perceives that the Influencing
Machine is. If it is a computer, it must be
predictable, not influenced by them.
Affective loop...?
No ”model” of users’ emotion
 Tangible input
 Interaction over a longer time span: 20
– 40 minutes

SAFIRA demo:
SenToy in FantasyA




A role-playing game
Agents in the game have emotional
states
To advance in the game, the user has
to understand the agents’ emotional
states, get magical tools, and fight
magical fights
The user has a plush toy, SenToy, with
sensors that s/he can use to direct
his/her character and express its
emotions
Affective input:
SenToy
...
Gloat
Sad
...
Happy
((
))
Fear
Angry
Surprise
User
experience
Affective loop...?





No ”model” of users’ emotion
Tangible input
Gestures – induce emotional state through
body posture?
Make narrative in the game, avatar
expressions, and gesture input into a
coherent whole that captures the mind &
body
Interaction over a longer time span 30
minutes – 2 hours
Affective loop?




The user expresses an emotion to a system
(intentionally, not true emotion)
The system computes an affective response making
sense in the application (be it a game, a narrative,
an e-commerce application, ...)
The response speaks to our senses and mind
(music or an activity in the game that induces a
state of stress or fear)
The user reacts affectively
New group at SU
Affective bandwidth
 Gestures
 Multimodal output
 MMS

Narrative intelligence
Narrative context for emotions
 Collaborative storytelling
 Interactive storytelling with synthetic
characters

MUD
Centre of the Cassiandora square marketplace. A small, ivy-covered
milestone at your feet reads “Mercinae”.
- goods
We have nothing to sell right now; please come back later.
– sigh
You sigh heavily.
– citizens
Allanon; Grim, the Alchemist; Mage Sparky, the Incandescent; Cinaed y
Llawen; Feanor.
– tell cinaed excuse me, milord, but I cannot find any clothes in the
city stalls, and I’m freezing. Can you help?
You tell Cinaed y Llawen, “Excuse me milord, but I cannot find any
clothes in the city stalls, and I’m freezing. Can you help?”.
–
Cinaed y Llawen tells you, “Hmm, let me check the city stores. One
moment please…”.
–
Cinaed y Llawen tells you, “I’m afraid the city is very low on cotton
and wool at the moment, but I have put a simple robe in the clothing
stall for you <smile>”.
– tell cinaed oh, that is all I need. My thanks for your aid, baron
You tell Cinaed y Llawen, “Oh, that is all I need. My thanks for your
aid, baron”.
– buy robe
You buy a grey robe for 40gp.
– wear robe
You are now wearing a grey robe. Cinaed y Llawen tells you, “Any time,
Feanor <smile>. Life be with you”.
Palace
Sam
Imagine a 3-D
animated character
who can pass multiple
toys back and forth
through the seemingly
immutable boundary
of a projection screen,
and thus tell stories
and share experiences
together with a child.
Pirates (Göteborg!)
Research questions
Artistic freedom versus proper models
 Models: appraisal versus discrete
emotions
 Understanding user emotions from
body sensors?
 New input/output modalities
 Emotions as social constructs
 Relationship narrative - character

Summary
Cool expressions:
 Affective computing
 Pervasive computing
 Synthetic characters
 Narrative intelligence
 Social capital and communities
Jakub Wejchert
”We live in a time in which objective
reality is given much more importance
than the subjective. It is in this context
that most technological and scientific
developments are carried out. The
danger is that the personal, the
subjective and the world of inner
experience is becoming reduced to an
insignificant by-product.”
kia@dsv.su.se
www.sics.se/~kia
www.sics.se/safira/