CMS 386P
GESTURE
Dr. Jürgen Streeck
jstreeck@mail.utexas.edu
FALL 2011
WED 3:00 – 6:00 PM
CMA A3.130
Office hrs: TTH 2 - 3 and by appt.
CMA 7.136
DESCRIPTION
The class combines an appraisal of the history of thought and research on
gesture with training in the micro-analysis of gesture, embodied action, and
multimodal communication. We begin by reconstructing dominant conceptions of
gesture in ancient Roman rhetoric; medieval philosophy; Enlightenment ideas
about the origins of human communication and mind; 19th century evolutionism;
and 20th century anthropology, psychology, sociology, and semiotics. We then
investigate the roles and analysis of gesture in the visual and performing arts and
music. In the second half of the semester we look at video-recordings of
communication in diverse settings of work and social life to investigate gestures
and gesture-mediated action sequences in the context of multimodal interaction.
We use our findings to assess the merits of competing contemporary approaches
to gesture studies and interaction analysis. We conclude the course by
discussing new conceptions of the embodied mind that have been proposed in
the humanities and cognitive sciences.
ASSIGNMENTS
1. Mini presentation: 5 – 7 slides on an exemplary phenomenon/study/idea
related to historical and contemporary perspectives on gesture; give five
minute slide presentation to stimulate class discussion (10 pts; Sep. 28 –
Nov. 23; due date for theme: Sep 7)
2. 4 mini papers (300 words) in response to questions pertaining to these
themes/this literature (4 x 5 pts.):
1. hands and things (9/14)
2. gesture metaphors (9/28)
3. rhetorical action (10/5)
4. Noland on gesture and performance (11/23)
3. Research paper (70 pts): including
1. Choice and confirmation of research topic (9/21, 5 pts)
2. research proposal (500 words, incl. example) (10/21, 5 pts)
3. 15 – 20 video segments showing varieties of a gesture ‘phenomenon’
(10/26, 10 pts.)
1
4. 3 two-hour small group research meetings (‘data sessions’),
documented by ‘minutes’ (10/14 – 11/23, 10 pts)
5. 5 min. presentation and 5 min. discussion of questions and findings
(last week of classes, evening date TBA) (10 pts)
6. paper: 15 – 20 pp. (30 pts), incl.
7. Appendix: CD or DVD of data/presentation
Research paper: Themes
Choose one of the following topics for your research paper; discuss details with
JS before due date of 3.1. If you want to work on a different topic, discuss with
JS before that date.
• ritual practices
• gestures in an instructional context
• gesture and narrative
• children’s gestures
• functions and styles of gestures in political communication
• gestures at work
• pointing and direction giving
• gesture and language acquisition
Research paper: Data
You may use existing data or collect new video data for your research project.
‘Data’ means that they show ‘phenomena’ in their integrity, i.e. that you can
actually observe the behaviors that you analyze in their entire dynamic ‘gestalt’.
You are expected to have about 15 – 20 ‘moments’, i.e. interaction sequences of
typically ten seconds or less, sometimes shorter than a second. These segments
will be edited (copied and pasted) using a simple digital video editing program.
2
PROGRAM
Aug 24
Introduction: How to understand meanings made by hands
EMPIRICAL GROUNDING: ANALYZING EVERYDAY GESTURES
Aug 31
How gestures are coordinated with action, gaze, and talk
Sep 7
The varied functions of gesture
Sep 14
Hands and things
Sep 21
How gestures mediate interaction and relationship
HISTORICAL PERSPECTIVES
Sep 28
Humanity’s original language
Oct 5
Rhetorical action
Oct 12
Embodied culture
CONTEMPORARY PERSPECTIVES
Oct 19
Gesture as communicative craft: Praxeology
Oct 26
Window onto mind: Cognitive science
Nov 2
Learning (by) gestures
ROLES AND REPRESENTATIONS OF GESTURE IN THE ARTS
Nov 9
Painting gestures, painted gestures
Nov 16
Dance and the musicality of gesture
Nov 23
Gesture as performance, habitus, and style
SYNTHESIS
Nov 30
Intercorporeality: Bodies and minds in interaction
3
Bibliography
Books:
Kendon, A. Gesture—Visible Action as Utterance. Cambridge University Press,
2004 (‘Visible Action’)
Streeck, J. Gesturecraft. The Manu-facture of Meaning. Paperback edition.
Benjamins, 2011 (‘G-craft’)
Sections from:
Allert, T. (2008). The Hitler Salute. On the Meaning of a Gesture. New York:
Henry Holt & Company.
Andrén, M. (2010). Children's Gestures from 18 to 30 Months (Travaux de
l'Institut de Linguistique de Lund ed. Vol. 50). Lund: Lund University.
Bateson, G., & Mead, M. (1942). Balinese Character. A Photographic Analysis.
New York: New York Academy of Sciences.
Burke, P. (1992). The language of gesture in early modern Italy. In J. Bremmer &
H. Roodenburg (Eds.), A Cultural History of Gesture (pp. 71-83). Ithaca,
NY: Cornell University Press.
Calbris, G. (2008). From left to right...: Coverbal gestures and their symbolic use
of space. In A. Cienki & C. Müller (Eds.), Metaphor and Gesture.
Amsterdam: Benjamins.
Cienki, A. (2008). Why study metaphor and gesture? In A. Cienki & C. Müller
(Eds.), Metaphor and Gesture (pp. 5-26). Amsterdam: Benjamins.
Condillac, E. (1746). An essay on the origin of human knowledge, being a
supplement to Mr. Locke's essay on the human understanding. London: J.
Noursse.
Costall, A. (1995). Socializing affordances. Theory and Psychology, 5(4), 467481.
de Jorio, A. (2000 (1832)). Gesture in Naples and Gesture in Classical Antiquity.
Bloomington: Indiana University Press.
Efron, D. (1972 (1941)). Gesture, race and culture. The Hague: Mouton.
Goldin-Meadow, S. (2003). Hearing gesture: how our hands help us think.
Cambridge, Mass.: Belknap Press of Harvard University Press.
Goodwin, C. (2000). Gesture, Aphasia and Interaction. In D. McNeill (Ed.),
Language and Gesture (pp. 84-98). Cambridge: Cambridge University
Press.
Goodwin, C. (2003). Pointing as situated practice. In S. Kita (Ed.), Pointing.
Where Language, Culture and Cognition Meet (pp. 217-242). Mahwah,
N.J.: Lawrence Erlbaum.
Goodwin, C. (2007). Environmentally coupled gestures. In S. D. Duncan, J.
Cassell & E. T. Levy (Eds.), Gesture and the dynamic dimension of
language: Essays in honor of David McNeill (pp. 195-212). Philadelphia:
Benjamins B.V.
Goodwin, C. (2011). Contextures of action. In J. Streeck, C. Goodwin & C.
4
LeBaron (Eds.), Embodied Interaction. Language and Body in the Material
World (pp. 182-193). New York: Cambridge University Press.
Goodwin, C., & Goodwin, M. H. (1986). Gesture and coparticipation in the activity
of searching for a word. Semiotica, 62(1-2), 51-75.
Graf, F. (1991). Gestures and conventions: the gestures of Roman actors and
orators. In J. Bremmer (Ed.), Roodenburg, H. (pp. 36-58). Ithaca, N.Y.:
Cornell University Press.
Gullberg, M. (2011). Multilingual multimodality: Communicative difficulties and
their solutions in second-language use. In J. Streeck, C. Goodwin & C.
LeBaron (Eds.), Embodied Interaction. Language and Body in the Material
World (pp. 137-151). New York: Cambridge University Press.
Hawhee, D. (2004). Bodily arts: rhetoric and athletics in ancient Greece. Austin,
TX: University of Texas Press.
Heath, C. (1986). Body Movement and Speech in Medical Interaction.
Cambridge: Cambridge University Press.
Hibbitts, B. J. (1992). "Coming to our senses": Communication and legal
expression in performance cultures. Emory Law Journal, 41(4), 973-960.
Hutchins, E., & Nomura, S. (2011). Collaborative construction of multimodal
utterances. In J. Streeck, C. Goodwin & C. LeBaron (Eds.), Embodied
Interaction. Language and Body in the Material World (pp. 289-304). New
York: Cambridge University Press.
Kendon, A. (1988). How gestures can become like words. In F. Poyatos (Ed.),
Corsscultural Perspectives in Nonverbal Communication (pp. 131-141).
Toronto: C.J. Hogrefe.
Kendon, A. (2003). On the origins of modern gesture studies. In S. Duncan, J.
Cassell & E. T. Levy (Eds.), Gesture and the Dynamic Dimension of
Language (pp. 13-28). Amsterdam: Benjamins.
Kendon, A. (2004). Gesture: Visible Action as Utterance. Cambridge: Cambridge
University Press.
Mallery, G. (1978). The gesture speech of man. In D. J. Umiker-Sebeok & T. A.
Sebeok (Eds.), Aboriginal Sign Languages of the Americas and Australia
(pp. 407-437). New York: Plenum Press.
McNeill, D. (1992). Hand and Mind. What Gestures Reveal about Thought.
Chicago: The University of Chicago Press.
Mead, G. H. (1934). Mind, Self and Society. Chicago: University of Chicago
Press.
Mirivel, J. 2011). Embodied Arguments: Verbal Claims + Bodily Evidence. In
J.Streeck, C.Goodwin & C LeBaron (Eds.). Embodied Interaction.
Language and Body in the Material World. Cambridge: Cambridge
University Press.
Noland, C. (2009). Agency and Embodiment. Performing Gestures/Producing
Culture. Cambridge MA: Harvard University Press.
Noland, C., & Ness, S. A. (2008). Migrations of gesture. Minneapolis: University
of Minnesota Press.
Quintilianus, M. F. (1922 (100)). The Institutio Oratoria of Quintilian (H. E. Butler,
Trans. Vol. IV). London: Heinemann.
5
Sauer, B. (2003). The Rhetoric of Risk. Technical Documentation in Hazardous
Environments. Mahwah, N.J.: Lawrence Erlbaum.
Schmitt, J.-C. (1991). The rationale of gestures in the West: third to thirteenth
centuries. In J. Bremmer (Ed.), Roodenburg, H. (pp. 59-70). Ithaca, N.Y.:
Cornell University Press.
Steinberg, L. (2001). Leonardo's Incessant Last Supper. New York: Zone Books.
Streeck, J. (2002). Grammars, words, and embodied meanings. On the evolution
and uses of so and like. Journal of Communication, 52(3), 581-596.
Streeck, J. (2003). A moment and its gestures. Leonardo's Incessant Last
Supper by Leo Steinberg. Gesture, 3(2), 213-236.
Streeck, J. (2008). Gesture in Political Communication: A Case Study of the
Democratic Presidential Candidates During the 2004 Primary Campaign.
Research on Language & Social Interaction, 41(2), 154 - 186.
Streeck, J. (2009). Forward-gesturing. Discourse Processes, 45(3/4), 161-179.
Streeck, J. (2009). Depicting gestures. The representation of body motion in the
visual arts of the West. Gesture, 9(1), 1-34.
Wilkins, D. (2003). Why pointing with the index finger is not a universal (in
sociocultural and semiotic terms). In S. Kita (Ed.), Pointing : where
language, culture, and cognition meet (pp. 171-216). Mahwah, N.J.: L.
Erlbaum Associates.
6
PROGRAM
Aug 24
How we understand meanings made by hands
We discover that we make sense—different kinds of sense—from hand gestures
depending on the ecology of the context in which they are being made: some we
understand because of the way the hand links up with the things at hand, others
because we know how to look away from the gesture. Some gestures are
elements of human rituals, others tell us how to “take” the next bit of talk.
“Gesture” really is a term for a diverse family of practices by which we make
sense of the world in interaction with others.
Discussion of course program, research, and assignments.
7
EMPIRICAL GROUNDING:
ANALYZING EVERYDAY GESTURES
In order to rapidly provide a foundation of research strategies, we begin with a
series of meetings devoted to the micro-analysis of gesture sequences in
everyday conversation and workplace interaction. The strategies, units and terms
of analysis, and findings are applicable to individual research projects.
Aug 31
How gestures are coordinated with action,
gaze, talk, and the world
G-Craft 1, 5
Goodwin & Goodwin 1985
Streeck 2002
We observe how hand gestures are delicately coordinated with talk (cf. words
like ‘like’ or ‘be like’) and how the gesturer’s gaze directs the listener’s gaze to
them. Hand gestures are coordinated with talk in a variety of distinguishable
ways; depending on where in relation to emerging bits of talk they are made they
make different ‘projections’, i.e. enable listeners to anticipate what will be
done/said next.
8
Sep 7
The varied functions of gesture
G-Craft 4, 6
Visual Action 6 - 10
Efron 1942
Goodwin 2003
Hand gestures do a number of different “jobs” in social interaction. In this
meeting we will review examples of different “genres” of gestural communication,
as well as typological grids in terms of which they have been sorted. We will
notice how in everyday interaction the function of the gesturing hand can change
fluidly and frequently, and how dynamic micro-processes such as altering motion
speed can convey subtle changes in meaning.
9
Sep 14
Hands and things
G-Craft 4, 7, 9
Costall 1995
Goodwin 2007
Mirivel 2011
We explore how gesturing of the hands is related to the hands’ practical
engagements with the material world. We think about gesture in terms of its
corporeal foundations, as a ‘language’ that is abstracted from practical, ‘worldly’
experience.
10
Sep 21
How gestures mediate interaction and
relationship
G-Craft 8
Visible Action 12, 13
Allert 2008
de Jorio 2000
Hibbitts 1992
The great majority of gestures made in ordinary conversation—and most
definitely the greatest number of conventionalized gestures—are of the
“pragmatic” kind: they communicate something about the nature of the
communication, including the speech act that is currently being performed, and
various of its social implications. Overwhelmingly, pragmatic gestures are
culturally standardized, and often they are manual metaphors for communicative
action. We discuss examples of shrugs, presentation gestures, precision grips,
and setting-aside gestures.
11
HISTORICAL PERSPECTIVES
We bundle the many extremely interesting ‘discourses’ on gesture in the
Occident into three broad themes, thus excluding many relevant dimensions.
More broadly-based reading on the history of gesture studies is encouraged.
Sep 28
Humanity’s original language
Visible Action 3-5
Calbris 2008
Cienki 2008
Condillac 1746
Mallery 1978
Quintilian 1922
That gesture is the original, universal language of all of humankind is an idea
entertained at least since Roman times, and it was corroborated by the
experience of explorers anywhere who found themselves able to communicate
with the natives by gestures of the hand. While we no longer believe in the
universality of gesture (it has a varied and long history, just like any cultural
medium of communication), the idea that gesture encodes universal bodily
experiences in currently much discussed in the context of cognitive science. We
compare cognitive science-accounts of gesture as metaphor with historical
sources on conventional gesture.
12
Oct 5
Rhetorical action
Graf 1993
Qunitilian 1922
Hawhee 2004
Sauer 2003
Streeck 2008
We examine gesture in the context of oratory and oratorical training, from
Quintilian to contemporary political campaigns. Although gesture never gained
central importance in the Western rhetorical tradition, rhetoric is nevertheless one
rare discipline that has consistently paid attention to gesture and bodily
expression. We will juxtapose classical treatments to contemporary research on
rhetorical strategies in everyday gesture (illustrated by miners re-enacting
accidents). We discuss issues of “orality” in gesture and verbal communication.
13
Oct 12
Embodied culture
Bateson & Mead 1942
Burke 1993
Efron 1942
Schmitt 1993
That the expressive body is a product of its culture is a widely shared belief. We
learn about the cultural patterning of gesture in the European middle ages and
the early modern period; and in New York City and Bali (Indonesia) in the mid20th century. We discuss what it means that human bodies are enculturated and
in what situations bodies acquire cultural skills.
14
CONTEMPORARY PERSPECTIVES
Oct 19
Practice and craft: Praxeology
G-Craft 2
Goodwin 2011
Bourdieu (TBA)
From this perspective, exemplified by the instructor’s research, gesture is
conceived as an open-ended set or family of manual sense-making practices:
methodical, routine ways of moving the hands to make sense in and of a
situation. Gesture comprises heterogeneous semiotic modes, including pointing
(individuating, focusing), highlighting, depicting, con-ceiving. Everyday gesturing
is skilled practice that comprises both standard (culturally shared) and
individuated (improvised, idiosyncratic) forms and methods.
15
Oct 26
Window onto mind: Cognitive science
McNeill 1992
Duncan & McNeill 2000
Goldin-Meadow 2003
Hutchins 2005, 2011
Cognitive scientists regard gesture as a separate window onto the working of the
human mind, in addition to speech. Gesture is seen as comprising the globalsynthetic dimension of thought, in comparison to the analytic-‘digital’ dimension
of linguistic cateorization. Gesture externalizes the mind. Another approach,
subtly different, treats gestures as thought’s tools, i.e. as human-made
(fabricated), material structures that anchor symbolic thought and other cognitive
processes.
16
Nov 2
Learning (by) gesture
Goldin-Meadow 2003
Andrén 2010
Gullberg 2011
A great deal of research has been devoted to the role of gesture as precursor
and support of the acquisition of language, as well as its role in other forms of
symbolically mediated learning (mathematics). We concentrate on two research
endeavors: an in-depth study of gesture in connection with early language
development and Goldin-Meadow’s well-known experiments on gesture and
concept learning in schools.
17
ROLES AND REPRESENTATIONS OF GESTURE IN THE
ARTS
A great deal can be learned by investigating gesture in light of neighboring artforms or in those forms which in one way or another include representations of
gesture. Only three forms of artistic expression are included here—painting,
music performance, and dance/performance art. Other art-forms can be included
if there is enough interest.
Nov 9
Painting gestures, painted gestures
Noland 2009
Steinberg 2001
Streeck 2003, 2009
How painters have analyzed the motions of gesture; indexical art: brush-strokes
as gestures.
18
Nov 16
Dance and the musicality of gesture
Noland 2009
Haviland 2011
Gesture mediating musical performance; dance, intercorporeality, and
interaction.
19
Nov 23
Bodily performance and embodiment style
Style and self-referentiality. Performance styles in contemporary societies: hiphop; armies of one; self-phones.
20
SYNTHESIS
Nov 30
Intercorporeality:
Bodies and minds in interaction, inhabiting a
shared world
21