EMOTIONS IN
INTERACTION DESIGN
Lecture 13
History
• During the 1990ies - a wave of new research on the role of emotion in diverse areas such as
psychology, neurology, medicine, and sociology
•
Prior to this, emotions - a low-status topic of research
• researchers had mainly focused on how emotion got in the way of our rational thinking
(results - focused on issues like when getting really scared, pilots would get tunnel vision and
stop being able to notice important changes in the flight’s surroundings)
• emotions were seen the less valued pair in the dualistic pair rational – emotional, and
associated with body and female in the “mind – body”, “male – female” pairs
• In 90ies It became clear that emotions were the basis for behaving rationally.
• Without emotional processes we would not have survived (being hunted by a predator (or
enemy aircraft) requires focusing all our resources on escaping or attacking - Tunnel vision
makes sense in that situation
• Unless we can associate feelings of uneasiness with dangerous situations, as food we
should not be eating, or people that aim to hurt us, we would make the same mistakes over
and over
The Four Pleasures framework – Tiger
(1992)/Jordan(2000)
• categorizes the four broad types of pleasure enjoyed by
people
• The Four Pleasure model is a framework that can be used
to help evaluate how pleasurable a product will be use
and own.
• can also be used to identify and generate opportunities to
enhance a product
The Four Pleasures framework
• The Four Pleasures
• Physio-pleasure
• Psycho-pleasure
• Socio-pleasure
• Ideo-pleasure
Physio-pleasure
• Physio-pleasure is a sensual pleasure that is derived from
touching, smelling, hearing and tasting something.
• It also conveyed by an objects effectiveness in enabling
an action to be performed.
• Ex: When we close a car door and it makes a satisfying
clunk we experience a certain pleasure
Psycho-pleasure
• Psycho-pleasures are pleasures that are derived from
cognition, discovery, knowledge, and other things that
satisfy the intellect
• Games are enjoyable because they present challenges
that we need to figure out (finishing the Rubik’s cube, or
achieving checkmate in a few moves, there is a cognitiveemotional pleasure that is derived from such activities)
Socio-pleasure
• Socio-pleasures, as the name suggests, are concerned
with pleasures derived from social signifiers of belonging,
social-enablers and other social self-identification factors.
• Facebook is a tool that enables people to have a greater
sense of community and involvement with one another.
Ideo-pleasure
• Ideo-pleasures then are pleasures that are linked to our
ideals, aesthetically, culturally and otherwise.
• Aesthetic sensibilities are often closely linked to our
ideological or cultural identity and determine to a great
extent the pleasure a product may bring
Emotional Design
• Attractive things works better? (cheap wine – fancy glass,
washed car – it drives better ;))
• Products that make people feel good, work better
• Emotional design framework:
• Visceral – how things look, feel, sound – sensory inputs
• Behavioral – how things function – effectiveness and usability
• Reflective – self-image, personal satisfaction, memories – meaning
of things (influenced by knowledge, learning, culture)
Visceral
Behavioral
Reflective
Emotional Design
• Design & trust:
• Trust comes from experience – products that perform accordingly
to expectations
• Lack of trust comes from:
• Lack of understanding
• Lack of control
Design for Emotion
• Design for emotion “comprises studying the emotional
experiences of users with products, as well as the
emotional meanings assigned by users in relation to
experience and interaction with products, assessing how
emotions vary with different user characteristics and
integrating users’ emotional expectations into the product
development. It acknowledges the fact that the emotion is
not a feature of the design, but a subjective experience of
the user, owner or observer of the product.” ([Engage,
2005])
Design for Emotion
• Advances in our understanding of emotion and affect have implications for the
science of design.
• Affect changes the operating parameters of cognition:
• positive affect enhances creative, breadth-first thinking
• negative affect focuses cognition, enhancing depth-first processing and minimizing distractions.
• it is essential that products designed for use under stress follow good human-
centered design, for stress makes people less able to cope with difficulties and
less flexible in their approach to problem solving.
• Positive affect makes people more tolerant of minor difficulties and more flexible
and creative in finding solutions
• Products designed for more relaxed, pleasant occasions can enhance their
usability through pleasant, aesthetic design.
• Aesthetics matter: attractive things work better. (D. A. Norman, 2002)
Design for Emotion
• Product design that provides aesthetic appeal, pleasure and
satisfaction can greatly influence the success of a product.
• Traditional cognitive approaches to product usability have tended to
underestimate or fragment emotion from an understanding of the user
experience.
• Affect, which is inexplicable linked to attitudes, expectations and
motivations, plays a significant role in the cognition of product
interaction, and therefore can be usefully treated as a design aid.
• Emotion influences and mediates specific aspects of interaction
before, during and after the use of a product.
• These affective states regularly impact how a user manipulates and
explores a user interface in order to support a desired cognitive
state.” (Frank Spillers, 2007)
Ten Emotion Heuristics
• Frowning (...) can be a sign of a necessity to concentrate, displeasure
or of perceived lack of clarity.
• Brow Raising (...) should also be considered a negative expressive
reaction (...) is a sign of uncertainty, disbelief, surprise and
exasperation
• Gazing Away (...) from the screen may be perceived as a sign of
deception.
• Smiling (...) is a sign of satisfaction. The user may have encountered
an element of joy during the evaluation process.
• Compressing the Lip (...) should be perceived as a sign of frustration
and confusion (...) reflects anxious feelings, nervousness, and
emotional concerns.
Ten Emotion Heuristics
• Moving the Mouth (...) is associated with a sign of being lost and of
uncertainty.
• Expressing Vocally (...) as well as the volume of the expression, the tone
or quality of the expression may be signs of frustration or deception.
• Hand Touching the Face (...) is a sign of confusion and uncertainty,
generally a sign of the user being lost or tired.
• Drawing Back on the Chair (...) negative or refusing emotions. By
drawing back the chair, he / she [the user] may be showing a desire to
get away from the present situation.
• (...) Leaning forward and showing a sunken chest may be a sign of
depression and frustration with the task at hand (...) the user might be
encountering difficulties but instead of showing refusal, leaning forward is
a sign of attentiveness, of "getting closer".
Kansei engineering
• Kansei is a Japanese word and implies psychological feeling and
needs in mind
• Kansei is the instantaneous feeling and emotion that we experience
when we interact with things, such as products and services
• Kansei Engineering is a methodology for ensuring your product or
service evokes desirable emotional responses. The process allows
you to model customer’s instantaneous feelings and emotions and
subsequently translate them into design parameters.
• Kansei is similar to psychology – grasping the image that exists in
somebody’s mind
• Kansei engineering – transforming the image into something
measurable – multidisciplinary science
Kansei Engineering
• Before purchase of for example a passenger car one has images in mind of may
be “a powerful engine”, “easy operation”, “beautiful and premium exterior, “cool
and relaxed interior” and so on.
• These words express the kansei, and the consumers really want to have such
kind of a vehicle if the manufacturer succeeds in realizing a vehicle fitting to their
imaginations.
• Kansei engineering- a mechanism that technologically translates users Kansei
into design elements
• A good product is more appealing to its consumers in terms of price, functions,
colors, shape – represents users needs and has Kansei incorporated
• Kansei engineering – mix of feeling, emotion, and engineering
• First the product’s kansei is collected then the relationship to the product is
established
Emotions research directions in HCI
• In HCI, we understood the importance of considering users’ emotions explicitly in
our design and evaluation processes.
• the HCI research came to go in three different directions with three very different
theoretical perspectives on emotion and design.
1.
Rosalind Picard and her group at MIT - The cognitivistically inspired design
approach she named Affective Computing
2.
counter-reaction to Affective Computing -instead of starting from a more
traditional perspective on cognition and biology, the Affective
Interaction approach starts from a constructive, culturally-determined
perspective on emotion
3.
emotion as part of a larger whole of experiences we may design for – we can
name the movement Technology as Experience. In a sense, this is what
traditional designers and artists have always worked with (see e.g. Dewey
1934) – creating for interesting experiences where some particular emotion is a
cementing and congruous force that unites the different parts of the overall
system of art piece and viewer/artist
Affective computing
• basic idea: human rational thinking depends on emotional
processing
• it should be possible to create machines that relate to,
arise from, or deliberately influence emotion or other
affective phenomena
• the roots of affective computing came from neurology,
medicine, and psychology
• implements a biologistic perspective on emotion
processes in the brain, body, and interaction with others
and with machines
Affective computing
• Emotions, or affects, in users are seen as identifiable states or
at least identifiable processes.
• Based on the identified emotional state of the user, the aim is
to achieve an interaction as life-like or human-like as possible,
seamlessly adapting to the user’s emotional state and
influencing it through the use of various expressions
• Applications:
• affective learning - use an emotion model built on James A. Russell’s
model of affect relating phases of learning to emotions
• training autistic children to recognize emotional states in others and in
themselves and act accordingly.
Affective computing
• The most discussed and widespread approach in the design of
affective computing applications - to construct an individual cognitive
model of affect from what is often referred to as “first principles”- the
system generates its affective states and corresponding expressions
from a set of general principles rather than having a set of hardwired
signal-emotion pairs
• The model is combined with a model that attempts to recognize the
user’s emotional states through measuring the signs and signals we
emit in face, body, voice, skin, or what we say related to the emotional
processes going on
• facial expressions - portraying different emotions - can be
analyzed and classified in terms of muscular movements.
Affective computing
http://www.interaction-design.org/encyclopedia/affective_computing.html
Affective Computing
• Limitations:
• simplification of human emotion in order to model it,
• difficult approach into how to infer the end-users emotional states
through interpreting human behaviour through the signs and
signals we emit
• Pros
• provides for a very interesting way of exploring intelligence, both in
machines and in people.
Affective Computing
• Tools for:
• affective input
• facial recognition tools
• voice recognition
• body posture recognition
• bio-sensor models
• affective output:
• emotion expression for characters in the interface
• regulating robot behaviours
Affective Interaction
• sees emotions as constructed in interaction, whereas a
computer application supports people in understanding and
experiencing their own emotions
• will not aim to detect a singular account of the “right” or “true”
emotion of the user and tell them about it as in a prototypical
affective computing application, but rather make emotional
experiences available for reflection
• creates a representation that incorporates people’s everyday
experiences that they can reflect on
• tries to avoid reducing human experience to a set of
measurements or inferences made by the system to interpret
users’ emotional states.
Affective Interaction
• Approach
• recognizes affect as a social and cultural product
• relies on and supports interpretive flexibility
• avoids trying to formalize the unformalizable
• supports an expanded range of communication acts
• focuses on people using systems to experience and understand
emotions
• focuses on designing systems that stimulate reflection on and
awareness of affect
Affective Interaction
• Affector is a distorted video window connecting neighbouring
offices of two friends (and colleagues). A camera located under
the video screen captures video as well as 'filter' information
such as light levels, colour, and movement. This filter
information distorts the captured images of the friends that are
then projected in the window of the neighbouring office. The
friends determine amongst themselves what information is
used as a filter and various kinds of distortion in order to
convey a sense of each other's mood.
• eMoto is an extended SMS-service for the mobile phone that
lets users send text messages between mobile phones, but in
addition to text, the messages also have colorful and animated
shapes in the background
Affective Interaction
• Affective Diary - as a person starts her day, she puts on a body sensor
armband.
• During the day, the system collects time stamped sensor data picking up
movement and arousal.
• At the same time, the system logs various activities on the mobile phone:
text messages sent and received, photographs taken, and Bluetooth
presence of other devices nearby.
• Once the person is back at home, she can transfer the logged data into
her Affective Diary.
• The collected sensor data are presented as somewhat abstract,
ambiguously shaped, and coloured characters placed along a timeline.
• To help users reflect on their activities and physical reactions, the user
can scribble diary-notes onto the diary or manipulate the photographs
and other data
Affective Diary
Technology as experience
• a holistic approach to understanding emotion – emotions
should not be separated from other aspects of being in
the world
• Emotion processes are part of our social ways of being in
the world, they dye our dreams, hopes, and experiences
of the world.
• design for emotions - place emotions in the larger picture
of experiences, especially if we are going to address
aspects of aesthetic experiences in our design processes
Kansei engineering
• Kansei – difficult to measure
• Computer Aided Kansei Engineering
Kansei Engineering
• The traditional Kansei engineering method includes the
following steps (Nagamachi, 2008):
• Decision of product strategy (design domain + customer type)
• Collection of kansei expressions that relate to the domain. Usually
•
•
•
•
•
•
about 30-40 words are collected
A semantic differential scale is constructed
Samples that represent the domain are collected
Items/categories of the samples are identified, i.e. its "objective"
features are described
Subjects then evaluate each sample item with the constructed scale
Results are analyzed with standard multi-variate methods like factor
analysis, multiple regression or cluster analysis, or quantification
theory (a non-parametric method developed by Komazawa and
Hayashi)
Results, i.e. ratings of sampled items as well as kansei structures are
then interpreted, explained and mapped to the designer's sketches.
Affective computing
• 2 approaches:
• emotion is constructed in interaction – between people and
between people and machines > designing for emotion
• emotion is just one of the parameters we have to consider - instead
of placing emotion as the central topic in a design process, it is now
seen as one component contributing to the overall design goal.