Audible smiles and frowns during spoken sentence comprehension
Anne van Leeuwen
We often smile (and frown) while we talk. Listeners to such affective speech have
to integrate the affective and the linguistic cues in the speech signal. Following up
on earlier work (see Quené et al., 2012), I investigated when and how listeners
integrate perceived smiles/frowns with unfolding sentence meaning.
I also
investigated if and how perspective modulates this integration
I explored this by presenting phonetically and semantically manipulated spoken
Dutch sentences to listeners while collecting behavioral
and neural
(ERP)
measurements. Materials consisted of utterances that contained a positive or
negative content word. Additionally, perspective was taken into account so that
sentences were in first person (‘ik’) or in third person (‘hij’ or ‘ze’). Utterances were
phonetically manipulated to obtain a frowning and a smiling version. This resulted
in consistent realizations (positive–smiling, negative–frowning), or inconsistent
realizations (negative–smiling, positive–frowning).
The general predictions were that inconsistent sentences were responded to more
slowly and that they would elicit a different ERP response than congruent sentences.
We were especially interested in the effect of perspective: would listening to first
person perspective sentences result in a qualitative and/or quantitative different
response
than
the
responses
to
third
person
perspective
sentences?
We
hypothesized that perspective modifies the different ERP responses because first
person sentences convey direct information about the affective state of the speaker
(through facial expression and linguistic content), while in the third person
perspective sentences this link between speaker expression and linguistic meaning
is less clear.
I will present the results of the EEG study and discuss how listeners integrate
perceived smiles and frowns with the unfolding sentence meaning. I will also talk
about how perspective of the speaker modulates this integration.
References
(1) Quené, H., Semin, G. R., & Foroni, F. (2012). Audible smiles and frowns affect
speech comprehension. Speech Communication.