Speech Unit(e)s
The multisensory-motor unity of speech
Understanding how speech unites the sensory and motor streams,
And how speech units emerge from perceptuo-motor interactions
ERC Advanced Grant, Jean-Luc Schwartz, GIPSA-Lab, Grenoble, France
Proposal for a post-doc position beginning in September 2014: “Joint
development of perception, action and phonology”
Context
The Speech Unit(e)s project is focused on the speech unification process associating the
auditory, visual and motor streams in the human brain, in an interdisciplinary approach
combining cognitive psychology, neurosciences, phonetics (both descriptive and developmental)
and computational models. The framework is provided by the “Perception-for-Action-Control
Theory (PACT)” developed by the PI (Schwartz et al., 2012).
PACT is a perceptuo-motor theory of speech communication, which connects in a principled way
perceptual shaping and motor procedural knowledge in speech multisensory processing. The
communication unit in PACT is neither a sound nor a gesture but a perceptually shaped gesture,
that is a perceptuo-motor unit. It is characterized by both articulatory coherence – provided by
its gestural nature – and perceptual value – necessary for being functional. PACT considers two
roles for the perceptuo-motor link in speech perception: online unification of the sensory and
motor streams through audio-visuo-motor binding, and offline joint emergence of the perceptual
and motor repertoires in speech development.
Objectives of the post-doc position
The general objective here is to gather phonetic data on the joint development of perception,
action and phonology, to assess how perceptuo-motor speech units emerge and evolve in the
course of acquisition, reacquisition, evolution or learning of a given phonological system.
Among the considerable amount of studies about the development of speech production and
speech perception in the first years of life, very few consider the link between perception and
action. Apart from historical disciplinary reasons – perceptual development is mainly studied by
psycholinguists and production development by phoneticians – there is also a theoretical cause
to this puzzling separation. Since it is now well known that phonetic perceptual abilities in
infants are well in advance of production skills, it has become implicitly obvious for most
researchers that perception was largely independent from production. The advance of
perceptual on production skills has actually been considered as a major weakness of motor
theories, and generally a decisive argument in favor of auditory theories of speech perception.
The view in PACT is different. While it is known that the construction of perceptual prototypes
begins earlier than the development of motor prototypes, we assume in PACT that the later
development of speech production intervenes in modifying the perceptual repertoire. This will
be studied in this post-doc position through two series of experiments, aiming at illustrating two
possible mechanisms for motor-driven evolution of perceptual phonetic categories.
1) Reconfiguration of perceptual categories: coupled auditory and motor idiosyncrasies – The
development of motor prototypes during acquisition should induce reconfiguration of
perceptual categories. Therefore, motor idiosyncrasies – specific choices of individual
ways to produce a target inside the space defined by native phonology – should result in
perceptual idiosyncrasies: if a subject chooses to produce a specific phonological contrast
in her language in a specific way, this should be mirrored in the way the subject perceives
the same contrast. This hypothesis will be studied in various phonetic dimensions.
2) Creation of new categories by perceptuo-motor coupling – According to PACT, the
acquisition of new motor knowledge could enable the infant to create a new category. A
typical example concerns plosive place of articulation, for which acoustic
characterisation remains controversial (e.g. Stevens & Blumstein, 1981; Sussman et al.,
1998), while articulatory characterization is straightforward. The relationship between
the emergence of such categories in perception and in production is seldom considered.
We assume that once an infant knows how to produce syllables with different places of
articulation, she can realise that they are associated with specific articulatory gestures
and hence form a new perceptual category related to articulatory properties. We will test
this assumption in infants before and after the onset of canonical babbling (3 to 12
months), to possibly relate the emergence of perceptual categories with the onset of
articulatory abilities. The same kind of studies will be possibly undertaken on pre-lingual
children after cochlear implantation.
The work will be realized within a speech team in GIPSA-Lab Grenoble (partners in the project:
Jean-Luc Schwartz, Anne Vilain), in close collaboration with a developmental and cognitive
psychology team in LPNC Grenoble (partners in the project: Hélène Loevenbruck, Olivier
Pascalis, David Meary).
Practical information
The post-doc position is open for a two-year period, with a possible third-year prolongation. The
position is open from September 2014, or slightly later if necessary.
Candidates should have an experience in phonetics, cognitive and neurocognitive psychology,
and developmental psychology, or at least two of these three domains.
Candidates should send as soon as possible a short email to Jean-Luc Schwartz (JeanLuc.Schwartz@gipsa-lab.grenoble-inp.fr) to declare their intention to submit a full
proposal.
Then they must send a full application file in the next weeks. This application file will include an
extended CV and a list of publications, together with a letter explaining why they are interested
in the project, what could be their specific interests, possibly proposing other experiments
related to the general question of the joint development of perception, action and phonology;
and also how this position would fit into their future plans for the development of their own
career. They should also provide two names (with email addresses) for recommendations about
their applications. Preselected candidates will be interviewed.
Final selection should occur before mid-June.