Spontaneous speech,
interaction
& large databases
for prosodic research
Roxane Bertrand & Cristel Portes
Université de Provence
Laboratoire Parole et Langage,
UMR 6057 CNRS
France
Prosody &
pragmatics/discourse analysis
Grammar approach
Information flow tradition
Monologue
Constructed data
Naturally occuring data
Contextualization tradition
Dialogue
Spontaneous speech in large databases
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Prosody &
pragmatics/discourse analysis

Beyond the sentence, 4 dimensions

1. Using sentences:


2. Combining sentences:


discourse typology
4. Coordinating with the interlocutor(s):

S2S
04/20/2009
coherence/cohesion, discourse analysis
3. Adapting to context (situation):


speech acts, pragmatics
interaction, conversation analysis
Spontaneous speech, interaction
& large databases for prosodic research
Prosody and illocution
Contours
Non Final
(without discourse value)
L- H* / L- HL*
Final
(with discourse value)
Falling
‘No revision anticipated’
H- L* (T%)
Marandin (2004, 2006)
Contours as constructions
Rising
‘Speaker not ready
to revise’
Simple Rise
L- H* (T%)
S2S
04/20/2009
Non-falling
‘Revision anticipated’
Falling from penultimate peak
‘Speaker ready to revise’
L- H+L* (T%)
Rising-falling
L- HL* (T%)
Spontaneous speech, interaction
& large databases for prosodic research
Prosody and illocution
Contour
Non stylized
Non-falling
Rising
Falling
A
Simple rise
B
Stylized
Fall from peak
Rising-Falling
D
C
Marandin (2004, 2006) Contours as constructions
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Prosody & Illocution
Portes, Bertrand & Espesser 2007
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
A discourse prosodic unit:
the paratone


Discourse topic = what a piece of discourse is ‘about’
Paratone = a group of utterances corresponding to a
discourse topic and prosodically marked:

Beginning phrases (vs intermediate vs final phrases)







higher and wider (resetting)
Delayed first pitch peak
Louder
Slower
=
downtrend
between
intonational
phrases (IP)
Final lowering
Very long pause
Brown & Yule (1983), Grosz & Hirschberg (1992), Swerts
(1994), Wichmann (2000)
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Phonological units are preserved
in spontaneous data despite…


Disfluencies
Orthogonal prosodic variation

Tempo



Pitch range



Faster
slower
Register level
Register span
Both have discursive and interactional specific
functions …
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Disfluency inside
the Accentual phrase
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Tempo & register variation
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Slower tempo
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Register
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Interactional Linguistics Perspective (1)
(Couper-Kuhlen 2001; Couper-Kuhlen & Selting, 1996)

Background
Interaction in langage?


Linguistics in Interaction?
How linguistic structures are shaped by interaction?
How, simultaneously, linguistic structures influence
interaction?
Attempt of elaboration of an Interaction Grammar
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Interactional Linguistics Perspective (2)

Take into account verbal activities in which speakers are
involved to reach some objectives or specific tasks in talkin-interaction


Locate interactional activities (such as narratives, requests, reported
speech, humor, etc.)
Characterize these activities at various linguistic levels

What type of resources available?
Role of prosodic cues?

Observables, tools and methods from the Conversation
Analysis (Sacks et al. 1974)

The Interactional Phonetics (Local, Ogden, etc.)
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Basic notions of an approach of talk-ininteraction (1)

Naturally-occurring data


More often
BUT : elicited conversations, task-oriented corpus, etc.
enable to analyze in a systematic way the whole resources availabe

avoid to reject some phenomena (overlaps among others)
(see Bertrand et al. 2008, The CID: Corpus of Interaction Data)


Activity collectively fulfilled


Take into account all the partners
Analysis unit based on their relevance in the interaction for the
participants
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Basic notions of an approach of talkin-interaction (2)

Co-ordination, alignment and negociation according
to the shared knowledge

Specific units or phenomena to make this understandable:

adjacency pair (request~answer for example, extract from
the CID)
Sp1
Sp2
Sp1

S2S
04/20/2009
mais les euh les nanas du foyer elles étaient pas au courant
but girls of residence they did not know
non non
no no
ah…
Localisation in specific points (TCU & TRP)
Spontaneous speech, interaction
& large databases for prosodic research
Overview of the turn-taking system

Mechanism for the organization of turntaking:

relies on 2 components related to the
construction and the allocation of the turn


S2S
04/20/2009
A turn constructional component = turnconstructional unit (TCU)
A turn-allocation component = transition
relevance place (TRP)
Spontaneous speech, interaction
& large databases for prosodic research
TCU and TRP



How do speakers build and recognize
TCU and TRP?
What type of resources or cues are used?
Turn-Constructional Unit (TCU): the
smallest interactionnally relevant complete
linguistic unit (Selting 1998: 40)

A TCU is a point of completeness (Ford &
Thompson, 1996)




Syntactic (clause)
Prosodic (intonative unit achieved in
a terminal rising)
Pragmatic (complete action of
request and answer)
Transition-Relevance Place (TRP)

Completion points which make a transition
relevant but not necessarily accomplished
(Schegloff 1996: 55)
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Crucial notion of Projection (1)

Large projects (Selting 2000)

Consists of more than 1 TCU = multi-unit turns (explanation,
narratives, description, etc.)

Need to be projected by the SPEAKER

Typical prosodic features of turn-design are used locally to
project more-to-come (Ogden 2005, Bertrand et al. 2007, Kern 2007)


pitch contours (extracted from the CID): the rising contour (H*H%)
Turn-holding device
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Crucial Notion of Projection (2)

What about the RECIPIENT?

Each TRP occurs in a point of potential
achievement built from the different linguistic
criteria which are used by recipients as
predictable cues.
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Backchannel signals (1)

BCs are short utterances produced by the recipient


to co-construct discourse by orienting it in one or another way
different functions






Continuer
Acknowledgement
Assesment
Attitude statement, etc.
They provide information on interlocutor’s listening but also
comprehension processes of discourse (Fox Tree 1999)
Prosodic role in the projection of this kind of response?
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Backchannel signals (2)

Context of production of BCs


Different BC’s function according to a terminal rising vs a continuation
rising (Bertrand et al. 2007)





function as continuer for continuative rising
function as assesment for terminal rising
Confirmation of the « more-to-come » intonation function
Prosodic cues organize the floor by making specific recipient response
relevant.
Multimodal analysis (Bertrand et al. 2007): gestural resources also play a role in
the context of production of BC



Higher occurrences of BC after terminal rising contours (continuation rise)
More gestural BC after continuative rising than vocal BC
less intrusive
Confirm that BC highlight some steps in the elaboration of discourse, and
more precisely in the construction of different steps of the larger projects
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Direct Reported Speech in conversation


Who is speaking and for what?
Reported speech is not only used to report words but also

to convey their assessment of the utterance while reproducing it
(Holt 2000)


to increase one’s standing or saying something without really
assuming it (Bertrand 2003)
to typify a character on which members of the same community
shared knowledge and typical representations (Klewitz & CouperKuhlen 1999; Bertrand & Priego-Valverde, 2009)

Using of a specific prosodic delivery to make the another voice hearable
and understable (Couper-Kuhlen 1996, Klewitz & Couper-Kuhlen 1999
Bertrand & Espesser 2002)



Melodic Shift in the beginning of DRS
But an absence of such a shift is yet a relevant cue
Specific prosodic design of each voice in a reported dialog
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Reported dialog in conversation
oh puis tu sais le fait de suite de dire
non mais je vais aller le voir moi le prof et tout je vais aller lui dire que c’est
oh eh j’ai dit béa attend quand même eh
ReLe
ReSp
Pauses



(
(
N
N
(
)
(
)
[722ms]
Rai
Exp
(
)
(
)
[331ms]
N
Com
)
)
Re(gister)Span/Re(gister)Le(vel) N(ormal) = direct speech
Rai(sed) and Exp(ansion) of span = reported speech (other figure)
N(ormal)/Com(pressed) = reported speech (self-quotation)
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Prosodic orientation (Szczepek-Reed 2006)

Belongs to the general frame of the interactional
orientation

throughout the course of a conversation (…)
speakers display in their sequentially “next” turns an
understanding of what the “prior” turn was about’
(Hutchby & Wooffitt 1998: 15).

Several cases of prosodic orientation



complementation of a prior turn
continuation of a previous unfinished prosodic pattern
copy of a previous prosodic pattern
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Prosodic mapping in humor
(Bertrand & Priego-Valverde 2009)
he works at I.R.A.
and he blows up uh
and he put the detonators uh
yeah he is watchmaker at I.R.A
4 TCUs with the same syntactic and prosodic form:
little variation in f0 curve, slightly falling, final lengthening + filler (euh)
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
In Sum

Prosody is a crucial resource in the
management of turn-taking system and in
the structuring of various activities
displayed in everyday conversations




Construction of turn
Projection of points of completion
Projection of more-to-come
Step by step constitution of the shared
knowledge
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
Conclusion

Of course, naturally occurring data present to the observation multiple
sources of information mixed together so that they may be hidden at first
sight.

However, phonological (grammatical) units are recoverable in spontaneous
speech. The confrontation of grammatical formalization with attested data
and the reverse appear to be very fruitful.

In order to do so, we need a separate model of disfluencies (which are not
grammatical but play important interactional roles).

We also need to treat orthogonal dimensions of prosody (tempo, pitch
range, intensity) separately because of their specificity (gradience).

Finally, we need to dissociate the speaker from the addressee, and to study
the way each of them takes its specific part in the co-construction of
discourse.
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research
References
Bertrand R. & Espesser R. (2002) Voice diversity in conversation : a case study, in Bel B. & Marlien I (éds), Speech Prosody 2002, Aix-en-Provence,
171-174.
Bertrand R., Portes C., Sabio F. (2007) Distribution syntaxique, discursive et interactionnelle des contours intonatifs du
français dans un corpus de conversation Travaux neuchâtelois de linguistique, 47, 59-77
Bertrand R., Ferré G., Blache P., Espesser R. & Rauzy S. (2007) Backchannels revisited from a multimodal perspective' Proceedings of Auditoryvisual Speech Processing. Hilvarenbeek, Cederom.
Bertrand, R., Blache, P., Espesser, R., & al., (2008) « Le CID - Corpus of Interactional Data -Annotation et Exploitation Multimodale de Parole
Conversationnelle », in Traitement Automatique des Langues, 49 : 3 (in press)
Bertrand, R. & Priego-Valverde B. Does prosody play a specific role in conversational humor?, Pragmatics and Cognition (accepted)
Couper-Kuhlen E. & Selting M (1996), Prosody in conversation. Interactional studies, Cambridge University Press,.
Couper-Kuhlen E. (2001) The Handbook of Discourse analysis.
Couper-Kuhlen E. & Ford C.E. (2004) Sound Patterns in Interaction, John Benjamins Publishing Company.
Couper-Kuhlen E. (1999) Coherent voicing: On prosody in conversational reported speech, In Wolfram Bublitz & Uta Lenk (eds.) Coherence in
Spoken and Written Discourse: How to create it and how to describe it, Amsterdam: Benjamins, 11-32.
Ford C. E. & Thompson S. A. (1996), Interactional Units in Conversation : syntactic, intonational and pragmatic resources for the management of
turns. In Interaction and Grammar, E. Ochs, E. A. Schegloff & S. A. Thompson (eds), 134-184, Cambridge UP.
Fox Tree J.E. (1999) Listening in on Monologues and Dialogues", Discourse Processes 27(1): 35-53.
Hirschberg J. (2002) The pragmatics of Intonational meaning, in Bel B. & Marlien I (éds), Speech Prosody 2002, Aix-en-Provence, 65-68.
Hutchby I and Wooffitt R. (1998) Conversation Analysis. Principles, Practices and Applications (Cambridge: Polity Press)
Jefferson G. (1978) Sequential aspects of storytelling in conversation, Studies in the organization of conversational interaction. New York (Academic
Press), 219-248.
Kern F. (2007) Prosody as a resource in children’s game explanations: Some aspects of turn construction and recipiency, Journal of Pragmatics 39,
111-133
Klewitz G. & Couper-Kuhlen E. (1999) Quote-Unquote? The Role of Prosody in the Contextualization of Reported Speech Sequences, Pragmatics
9, 459-485.
Labov W. & Waletzky J. (1966) Narrative analysis: oral versions of personal experience. In: J. Helm (ed.), Essays on the verbal and visual arts:
Proceedings of the 1966 Annual Spring Meeting of the American Ethnological Society. Seattle (University of Washington Press), 12-44.
Local J. (2007) Phonetic detail in talk-in-interaction: on the deployment and interplay of sequential context and phonetic resources, Nouveaux
cahiers de linguistique française 28, 67-86, Université de Genève.
Ogden R. & Routarine S. (2005) The communicative functions of Final Rises in Finnish Intonation, Phonetica, 62, 160-175.
Sacks H.,Schegloff E. A., Jefferson G. (1974) A Simplest Systematics for the Organization of Turn-Taking for Conversation, Language, 50, 4, 696735.
SchegloffF E. A. (1996), Turn organization: One intersection of grammar and interaction. In Interaction and Grammar, E. Ochs, E. A. Schegloff & S.
S2S
Spontaneous
speech, interaction
A. Thompson
(Eds), Cambridge: Cambridge University
Press, 52-133.
04/20/2009
&
large
databases
for prosodic
research
Selting, M., (2000) The construction of 'units' in conversational talk", Language
in Society
29: 477-517.
Szczepek Reed B. (2006) Prosodic orientation in English Conversation, Palgrave Macmillan
Prosodic cues in turn-taking
S2S
04/20/2009
Spontaneous speech, interaction
& large databases for prosodic research