1
Gentry Hanks
SOCY 921
Fall Term Paper
December 5, 2011
Affect, Emotion, and Technology
I will provide a brief overview and discussion of how emotion and affect are
theoretically conceptualized1, defined, and studied in affective computing,
information sciences, neurosciences, and sociology (and intersections thereof2) as
well as some examples of how this research is being applied. I will address how the
discipline of geography has contributed to the study emotion and affect and suggest
my future contributions to geography in affective, emotional studies. Lastly, I will
discuss how emotion and affect relate to my particular research interests in regard
to digital and material hoarding.
The topics of affect and emotion have seen a surge, as of late, in many
disciplines, particularly science and technology studies. While affect and emotion
have long been studied in psychology, cognitive and behavioral sciences, as well as
business and economics for the purpose of marketing and advertising, there seems
to have been a surge in interest in regard to technology. As new and different types
Given the long temporal attention to affect and emotion, I have only chosen a few
more recent studies to discuss and will opt to not address the deep pile of work
surrounding Freud, Derrida and Foucault on this matter in this paper.
2 Because of the collaborative efforts and overlapping work, it was difficult to
arrange some of the studies in neat categories of disciplines, so the arrangement
might seem circular at times.
1
2
of technology are generated, there is a push to connect what is perceived as artificial
and natural. While some biologists seek to clone ‘natural’ humans, other scientists,
in a sense, seek to (re)produce an artificial human through robotics. Of course this
becomes all the more complicated when we question, “what kinds of bodies are
being imagined, and what limits and possibilities does the robot embody in turn”
(Casatneda and Suchman, 2005, p. 2), particularly in regard to humans, machines,
and animals3. This has highlighted to a great extent the perceived divide between
mind and body and as Elizabeth Wilson (1998) notes, “there is very little written
about psychology and in a contemporary scientific context that does not in some
way invoke either the computer or the brain” (103). The very approaches taken to
bridge the divide between artificial and natural are based on the existence of the
separation of the two. Wilson (1998) also points out “emerging from the…field of
cognitive science, the computational approach has eschewed the idea of
neurological plausibility, aiming instead to produce a theory of cognition that exists
independently of biological constraints” (pp. 103-104). Mechanical bodies are
worked on as such to reproduce natural human movements, while other studies and
projects attempt to capture human cognitive processes and replicate them with
technology.
Neurosciences and Affective Computing
Many avenues of emotional and affectual research lead to an interest in
autism. Rosalind Picard (1997) explores and investigates human-computer
interactions so that computers might better assist autistic people through affective
3
See Panksepp (2004) for a discussion of animals and emotions.
3
computing (see Picard, 1997 and Lisetti, 1998). Picard (2009) notes the
multifariousness of inference, “when people infer an emotion such as ‘exuberance’,
they combine multiple channels of information in a complex way: smiles, shrieks,
upward bodily bouncing or arm gestures, and perhaps even tears of joy” (p. 3577).
Therefore Picard (2009) surmises, “measuring emotion is not as simple as
measuring a few bodily parameters” (p. 3577). The messiness of emotion is hard to
quantify and “no algorithms yet exist that describe how to precisely combine the
many contributing channels into a full space of emotions” (Picard, 2009, p. 3577).
This statement presupposes that indeed there exists a whole or complete emotion
space and denies that emotion might be wholly fractured and fragmented.
Locating affectual and emotive activity in particular parts of the brain is a
fastidious concern of neuroscientific research. According to Picard (2009), the
autonomic nervous system (ANS) is part of the peripheral nervous system, which is
responsible for functions at a pre-cognitive level. Picard (2009) describes the
autonomic nervous system as bimodal (see Gunes and Piccardi, 2007), comprised of
the sympathetic and parasympathetic portions (Picard, 2009). According to Picard
(2009), “the parasympathetic nervous system promotes restoration and
conservation of bodily energy, ‘rest and digest’, the sympathetic nervous system
stimulates increased metabolic output to deal with external challenges, the socalled, ‘flight or flight’” (p. 3577). This understanding compartmentalizes internal
and external processes of affect and emotion as dealt with separately. These
understandings are typically arrived at through observing the brain through brain
scans and measuring activity levels in different portions of the brain when
4
performing a spontaneous or given task. This becomes problematic when
heightened activity comes to stand for “location” of certain activities within the
brain (see Cordelia Fine, 2011), which may communicate a lack of activity in other
areas of the brain.
By geographically locating affect and emotion in the brain, neurologists are
able to label emotion and affect as cognitive or precognitive. Picard (1997) discusses
the limbic system, which she locates “between the brain stem and the two
hemispheres of the cortex” (p. 2) and asserts that it is responsible for “memory,
attention, and emotion” (p.2). Neuroscientist Antonio Damasio (1994) describes
two categories of locations of emotions—those occurring primarily and those as
secondary. For Damasio (1994), primary emotions occur at a pre- or meta-cognitive
level and secondary emotions occur at a cognitive level. Primary emotions relate
back to the ANS and the limbic system, whereas secondary emotions (and therefore
cognitive) are located within prefrontal somatosensory cortices (Damasio, 1994).
Once emotion and affect are located and locatable within the brain, many
potentialities and projections become possible.
The fields of affective computing, machine learning, and robotics have seen a
peaked interest in emotion and affect. Creating robots and machines (see Schroeder,
2010) that can harness natural speech patterns, natural body and facial movements
have come to be the golden goose of some robotics research. The emulation of
humans, yet not recognizable as such, serves as a large focus in technological
aspirations (although not all). These projects typically deal with the development of
prosthetics for amputees or to aid children and adults with autism (see Lacava et al.,
5
2007) or sensory perception disorders. For example, capturing emotion in the
human voice for natural speech patterns and storytelling was attempted through
machine learning to help children with autism experience and recognize emotion.
Machine learning has the potential to be applied in a variety of fields and
types of research. According to Ian Witten and Eibe Frank (2005), “machine
learning…is used to extract information from the raw data in databases” (p.xxiii)
and the extraction process “is one of abstraction: taking the data, warts and all, and
inferring whatever structure underlies it” (p. xxiii). For the purposes of quantifying
emotion and affect, machine learning is used to extract raw data (collected using any
number of quantitative methods) from facial expression recognition, neural activity,
prosody and pattern of voice and speech, body movements4, and eye movements.
Cecilia Alm, Don Roth, and Richard Sproat (2005), combining forces from
linguistics, computer science and electrical engineering, empirically explored “textbased emotion prediction” (p.579) through machine learning5. Their approach
involved classifying emotional contents of narrative in children’s fairytales using a
tripartite model that considered emotional valence, prosody, and natural language
processing (Alm, Roth, Sproat, 2005). They found that “despite the schematic
narrative plots of children’s stories, tales still differ in their overall affective
orientation, which increases the data complexity” (p. 583) and that “emotions are
not discrete objects; rather they have transitional nature, and blend and overlap
See Camurri et al (2000) and Zimmerman (2003) for a discussion on affect,
emotion, dance and music.
5 They used SNoW (Sparse network of Winnows) learning architecture, which is a
framework of linear functions commonly used in large-scale tasks in bioinformatics,
visual processing, and natural language processing (Alm, Roth, and Sproat, 2005).
4
6
along the temporal dimension” (p.585). This transitional and overlapping nature
make it all the more difficult to assess and estimate emotions (see Matsumoto et al.,
2007 and Kay and Loverock, 2008).
Neural Computational researchers Marian Bartlett et al. (2005) used
machine learning to recognize “spontaneous [facial] expressions in an interview
setting” (p.568) extracted from video. They relied on “objective coding standards”
(p. 570) known as FACS (Facial Action Coding System) to “objectively capture the
richness and complexity of facial expressions” (p.568). A limitation to the study
using FACS arises when there are head movements (ranging beyond the plane) and
movements of the mouth during speech (Bartlett et al., 2005). This is another
example of the dynamic and rich processes at work in emotion and affect that
complicate attempts to quantify and “objectively” understand emotion.
Within information studies, affect is defined by Ping Zhang and Na Li (2005)
as “a term that encompasses mood, emotions, and feelings” (p. 105) and is “a
fundamental aspect of human beings, one that influences reflex, perception,
cognition, and behavior (p. 105). Zhang and Li distinguish between affect and
affective quality, defining the latter as “the ability of an object or stimulus to cause
changes in one’s affect” (p. 105). Zhang and Li’s (2005) article “The Importance of
Affective Quality” opens with “Users aren’t always rational logical beings—emotion
plays an often overlooked role in user acceptance of technology,” which again places
rationality and logic as outside of emotion. Additionally, they define it as a
fundamental aspect of human beings. This definition marginalizes humans without
7
abilities to recognize, process or express emotion as well as conveying that emotion
is perhaps a uniquely human phenomenon.
Information studies researchers Ping Zhang and Na Li (2005) position their
empirical investigation of affective reaction to technology use within a sea of studies
that they claim “are based on the assumption that human beings are rational and
behave based on logical information-based thinking” (pp. 106-107). Zhang and Li
center their research on situating emotion, affect and mood as separate from logic
and reason. This approach allows them to assert that users of technology base their
perception and potential adoption of technology on perceived affective qualities and
behavioral intention in regard to a positive valence of perceived ease of use and
usefulness (2005). Again, their research relies on the notion that human
(pre)cognition compartmentalizes emotion and affect in geographically or spatially
different parts of the brain than logic and reason as well as viewing them as
mutually exclusive processes of (pre)cognition.
Social Sciences and Humanities: Geography, Affect and Emotion
Within the humanities and social sciences6, there has been what Patricia
Clough termed an “Affective Turn.” According to Clough (2007) there are “two
primary precursors to the affective turn…the focus on the body, which has been
most extensively advanced in feminist theory and the explorations of emotions,
conducted predominantly in queer theory” (p. x). The parameters of this turn are
potentially more limiting than liberating, because as sociologist Myra Hird (2003)
suggests, “when feminists study materiality, it tends to be in terms of how humans
6
See Beder (2005) for work done in social work regarding technology and emotion.
8
(such as scientists) interact with materiality, as though there is no outside of, or
beyond, the cultural context” (p. 448).
Clough (2007) poses, as many scholars dealing with affect do, that affective
labor “engages at once with rational intelligence and with the passions or feeling” (p.
xi). This presupposes that emotion and affect are dichotomous. Elizabeth Wilson
puts forth that this is a false dichotomy and Picard (2009) points out that “people
are quick to polarize thoughts and feelings as if they were opposites (p.2).
Davidson and Milligan review the literature dealing with emotion, organizing
and contextualizing it according to scale and sphere (e.g. domestic, public) within
geography and acknowledge parallel emotional, affectual research in sociology (see
Bendelow and Williams 1998). Davidson and Milligan’s (2004) approach to emotion
requires a reexamination of emotional embodiment and embodiment of emotion as
they suggest, “recognition of the inherently emotional nature of embodiment has,
thus, led many to the conclusion that we need to explore how we feel—as well as
think—through ‘the body’” (p. 523). I hope to explore thinking through the body in
a combination of bio/neurological ways as well as socio-cultural. Davidson and
Milligan’s (2004) suggestion that “emotions, then, might be seen as a form of
connective tissue that links experiential geographies of the human psyche and
physique with(in) broader social geographies of place” (p. 524) might be helpful in
addressing mind/body dualities and dichotomies.
Geographer Ben Anderson (2007) explores hope and anticipatory knowledge
in relation to affect, in particular “how the disclosure of the nanoscale as a place
subject to intervention, the act that is taken to define nanotechnology, can be
9
understood in the context of anticipatory knowledge practices that create futures”
(p 156). Anderson (2007) argues that “affect is transversal to both nanotechnoscience and anticipatory governance… that the ground that enables hope to
be placed in nanotechnology is the event that defines nano – to simultaneously
reduce ‘life’ to matter and to multiply ‘life’ into a limitless set of materialities.” (p.
156). I hope to expand these understandings to include unbounded bodies in
cyberspace and the reduction of hoarding (a matter of scale), yet further grounding
my work in materiality of bodies concurrently.
I will build upon geographical understanding of identity in regard to
extensions of the self, using geographer Paul Adams’ (2005) boundless self and
other work in geography, such as Davidson’s research in the Spatiality of Identity
(2003) by drawing from science and technology studies, particularly neuro and
cognitive sciences. Feminist geographies have renewed interests in bodies and
embodiment, yet still often neglect biological bodies, mainly focusing on
social/cultural experiences of embodiment. Feminist geographies, like other
feminist research is often centered around and carried out through critical cultural
discourse which, according to Wilson (1998) “misrepresents the complex relation
between neurology and its outside, but also, by locating malleability, politics, and
difference only in the domain of culture and environment, it abandons neurology to
the very biologism it claims to be contesting” (p.16). I aspire to in a way traverse
“feminism’s own naturalized antiessentialism” (Wilson, 1998, p. 15) in my work
because as Wilson astutely questions, “how can a critical habit nurtured on an
antibiologism produce anything but the most cursory and negating critique of
10
biology” (p. 16). Therefore, I will attempt to engage biological bodies in my future
work in addition to socio-cultural, discursive bodies7.
I aim to do a comparative study of spatial and emotional hoarding behaviors
in animals and humans and discuss how we, through neuroscience and technology
have come to understand the difference between hoarding (see Pertusa et al., 2008)
and collecting, particularly as is pertains to public (archives) and domestic spheres.
I want to delimit8 hoarding as a purely material physical amassing and extend it to
the virtual and digital realms of things. Additionally, I will acknowledge and discuss
the controversy of categorizing hoarding as a syndrome in and of itself as opposed
to its categorization as a symptom of obsessive-compulsive disorder (OCD). There
are implications of how hoarding is categorized through a social process within
cognitive and behavioral sciences for future clinical and neurological research.
Geography is situated in both physical and social sciences, which provides an
opportunity to study this cognitive and behavioral phenomenon within a social
context as well as social and behavioral within a scientific context.
I understand that this dichotomy is largely linguistic and not necessarily
ontological.
8 The term deconstruct is purposefully not used here because as Wilson (1998)
suggests, “deconstruction never offers the possibility of a move beyond or outside of
what it interrogates” (p.164). I want to expand and move beyond contemporary
understandings of psycho and cognitive theories surrounding hoarding, not merely
deconstruct them.
7
11
Adams, P. C. (2005). The Boundless Self: Communication In Physical And Virtual Spaces. Syracuse Univ Pr
(Sd).
Alaimo, S. (2008). Material feminisms. Indiana University Press.
Alm, C. O., Roth, D., & Sproat, R. (2005). Emotions from text: machine learning for text-based emotion
prediction. Proceedings of the conference on Human Language Technology and Empirical Methods in
Natural Language Processing, HLT ’05 (pp. 579–586). Stroudsburg, PA, USA: Association for
Computational Linguistics. doi:10.3115/1220575.1220648
Anderson, B. (2007). Hope for nanotechnology: anticipatory knowledge and the governance of affect. Area,
39(2), 156-165. doi:10.1111/j.1475-4762.2007.00743.x
Bartlett, M. S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., & Movellan, J. (2005). Recognizing Facial
Expression: Machine Learning and Application to Spontaneous Behavior. Computer Vision and Pattern
Recognition, IEEE Computer Society Conference on (Vol. 2, pp. 568-573). Los Alamitos, CA, USA: IEEE
Computer Society. doi:http://doi.ieeecomputersociety.org/10.1109/CVPR.2005.297
Beder, J. (2005). Cybersolace: Technology Built on Emotion. Social Work, 50(4), 355-358.
Bendelow, G., & Williams, S. J. (1998). Emotions in social life: critical themes and contemporary issues.
Routledge.
Camurri, Antonio, Hashimoto, S., Ricchetti, M., Ricci, A., Suzuki, K., Trocca, R., & Volpe, G. (2000). EyesWeb:
Toward Gesture and Affect Recognition in Interactive Dance and Music Systems. Computer Music
Journal, 24(1), 57-69.
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001).
Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, 18(1), 32-80.
doi:10.1109/79.911197
Damasio, A. R. (2000). The feeling of what happens: body and emotion in the making of consciousness.
Houghton Mifflin Harcourt.
Davidson, J. (2003). Phobic geographies: the phenomenology and spatiality of identity. Ashgate Publishing, Ltd.
Davidson, J., & Milligan, C. (2004). Embodying emotion sensing space: introducing emotional geographies.
Social & Cultural Geography, 5, 523-532. doi:10.1080/1464936042000317677
DeKoven, M. (2001). Feminist locations: global and local, theory and practice. Rutgers University Press.
Dennis, C., Newman, A., Michon, R., Josko Brakus, J., & Tiu Wright, L. (2010). The mediating effects of
perception and emotion: Digital signage in mall atmospherics. Journal of Retailing and Consumer Services,
17(3), 205-215. doi:10.1016/j.jretconser.2010.03.009
Devillers, L., Vidrascu, L., & Lamel, L. (2005). Challenges in real-life emotion annotation and machine learning
based detection. Neural Networks, 18(4), 407-422. doi:10.1016/j.neunet.2005.03.007
Fine, C. (2011). Delusions of Gender: How Our Minds, Society, and Neurosexism Create Difference (Reprint.).
W. W. Norton & Company.
Gunes, H., & Piccardi, M. (2007). Bi-modal emotion recognition from expressive face and body gestures.
Journal of Network and Computer Applications, 30(4), 1334-1345. doi:10.1016/j.jnca.2006.09.007
Hatano, G., Okada, N., & Tanabe, H. (2000). Affective minds. Elsevier Health Sciences.
Hird, M. J. (2003). New Feminist Sociological Directions. The Canadian Journal of Sociology / Cahiers
canadiens de sociologie, 28(4), 447-462. doi:10.2307/3341837
Kay, R. H., & Loverock, S. (2008). Assessing emotions related to learning new software: The computer emotion
scale. Comput. Hum. Behav., 24(4), 1605–1623. doi:10.1016/j.chb.2007.06.002
Lacava, P. G., Golan, O., Baron-Cohen, S., & Smith Myles, B. (2007). Using Assistive Technology to Teach
Emotion Recognition to Students With Asperger Syndrome. Remedial and Special Education, 28(3), 174 181. doi:10.1177/07419325070280030601
Lisetti, C. L. (1998). Affective computing. Pattern Analysis and Applications, 1, 71-73.
doi:10.1007/BF01238028
Marcus, A. (2003). The emotion commotion. interactions, 10(6), 28–34. doi:10.1145/947226.947239
Matsumoto, K., Ren, F., Kuroiwa, S., & Tsuchiya, S. (2007). Emotion estimation algorithm based on
interpersonal emotion included in emotional dialogue sentences. Proceedings of the artificial intelligence
6th Mexican international conference on Advances in artificial intelligence, MICAI’07 (pp. 1035–1045).
Berlin, Heidelberg: Springer-Verlag. Retrieved from http://dl.acm.org/citation.cfm?id=1775967.1776074
12
Nasoz, F., Alvarez, K., Lisetti, C. L., & Finkelstein, N. (2004). Emotion recognition from physiological signals
using wireless sensors for presence technologies. Cognition, Technology & Work, 6, 4-14.
doi:10.1007/s10111-003-0143-x
Panksepp, J. (2004). Affective neuroscience: the foundations of human and animal emotions. Oxford University
Press.
Pertusa, A., Fullana, M. A., Singh, S., Alonso, P., Menchon, J. M., & Mataix-Cols, D. (2008). Compulsive
Hoarding: OCD Symptom, Distinct Clinical Syndrome, or Both? American Journal of Psychiatry, 165,
1289-1298. doi:10.1176/appi.ajp.2008.07111730
Peter, C., & Herbon, A. (2006). Emotion representation and physiology assignments in digital systems.
Interacting with Computers, 18(2), 139-170. doi:10.1016/j.intcom.2005.10.006
Picard, R. W. (2000). Affective Computing. MIT Press.
Picard, R. W. (2009). Future affective technology for autism and emotion communication. Philosophical
Transactions of the Royal Society B: Biological Sciences, 364(1535), 3575 -3584.
doi:10.1098/rstb.2009.0143
Rachman, S., Elliott, C. M., Shafran, R., & Radomsky, A. S. (2009). Separating hoarding from OCD. Behaviour
Research and Therapy, 47(6), 520-522. doi:10.1016/j.brat.2009.02.014
Ramsey, W. (2011). Eliminative Materialism. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy
(2011st ed.). Retrieved from http://plato.stanford.edu/archives/spr2011/entries/materialism-eliminative/
Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110,
145-172. doi:10.1037/0033-295X.110.1.145
Schröder, M. (2010). The SEMAINE API: towards a standards-based framework for building emotion-oriented
systems. Adv. in Hum.-Comp. Int., 2010. doi:10.1155/2010/319406
Suzuki, K., Camurri, A., Ferrentino, P., & Hashimoto, S. (n.d.). Intelligent agent system for human-robot
interaction through artificial emotion (Vol. 2, pp. 1055-1060). IEEE. doi:10.1109/ICSMC.1998.727828
Sykes, J., & Brown, S. (2003). Affective gaming: measuring emotion through the gamepad. CHI ’03 extended
abstracts on Human factors in computing systems, CHI EA ’03 (pp. 732–733). New York, NY, USA:
ACM. doi:10.1145/765891.765957
Wajcman, J. (2010). Feminist theories of technology. Cambridge Journal of Economics, 34(1), 143 -152.
doi:10.1093/cje/ben057
Witten, I. H., & Frank, E. (2005). Data mining: practical machine learning tools and techniques. Morgan
Kaufmann.
Zhang, P., & Li, N. (2005). The importance of affective quality. Commun. ACM, 48(9), 105–108.
doi:10.1145/1081992.1081997
Zimmerman, J. (2003). Exploring the role of emotion in the interaction design of digital music players.
Proceedings of the 2003 international conference on Designing pleasurable products and interfaces,
DPPI ’03 (pp. 152–153). New York, NY, USA: ACM. doi:10.1145/782896.782939