ABRALIN
22-FEB-05
Language Perception
Eva M. Fernández
Queens College & Graduate Center
CUNY
Language Is…
PERCEPTION
SIGNAL
grammar
&
lexicon
MEANING
PRODUCTION
logic
knowledge
about
the real world
2
Production
•
•
•
•
We have a lot of churches in our minister.
They roasted a cook.
If you give the nipple an infant…
You ordered up ending.
• I’ll give you my undevoted
attention!
• You’ll earn her eternal
grapefruit.
• This restaurant hasn’t
been awake very long.
• Put the oven on at a very
low speed.
SIGNAL
MEANING
Phonological
Encoding
• phonological fool
• a glear plue sky
• spattergrain
Structural
Assignment
Lexical
Retrieval
3
Perception
Phonological
Decoding
Lexical
Access
Structure
Building
SIGNAL
MEANING
Phonological
Encoding
Structural
Assignment
Lexical
Retrieval
4
Not Present in (Speech) Signal:
phonemes
word boundaries
clause boundaries
location of empty categories
intended attachments for locally or globally
ambiguous strings
hidden intents of the speaker!
SO HOW COME WE’RE SO GOOD AT
DECODING?
5
Visual Illusions
when the experiences people report don’t
correspond to physical properties of the stimulus
very cool…
but also very informative about the way the
visual / perception system works
(which is: modularly)
The Hermann Grid Illusion
How many grey dots do you see at the “cross-roads”?
Source: http://dragon.uml.edu/psych/illusion.html
A great page to visit for many more visual illusions.
7
A Face Can’t Be Hollow!
A face is always perceived as convex… not concave.
http://www.kyb.tuebingen.mpg.de/de/bu/demo/
Max Planck Institut für Biologische Kybernetik
8
Perceptual Illusions
also very cool…
and also very informative about the way the
language perception system works

its modularity ensures its speed and accuracy,
which are both in turn compromised
when the signal is
AMBIGUOUS
McGurk Effect
by Arnt Maasø, of the University of Oslo:
http://www.media.uio.no/personer/arntm/McGurk_english.html
10
Perceptual Displacement and
Phonemic Restoration
“The state governors met with their respective
legislatures convening in the capital city.”
11
“legislatures”, with “cough!” (~ 145 msec) spliced in
12
“legislatures”, intact --- [s] ~ 145 msec
13
Bottom  ~ Top 
study with Broca’s patients (Pollack & Picket, 1964)
The apple the boy is eating is red.
The girl the boy is chasing is tall.
bait, date, gate study (Garnes & Bond, 1976)
Here’s the fishing gear and the ___.
Check the time and the ___.
Paint the fence and the ___.
14
Structure Building: The Parser
its input is a string of lexical items
its job is to build syntactic structure
its output is sent to a mechanism that
decodes meaning
it probably has limited access to information
that’s not in the grammar or in the lexicon
it probably operates following a very small set of
strategies, grounded on limitations imposed by
working memory
15
RSVP Paradigm
Center-screen, word-by-word display
Timing: N ms per word (here: N = 500 ms)
Sentence-recall task

beautiful
colorful
chased
black
The
ball
the
cat
.
The beautiful black cat chased the colorful ball.
17

beautiful
colorful
chased
Black
ball
the
cat
.
Black colorful the ball chased cat beautiful the.
18
The Garden Path Sentence
The soldiers marched into the desert surprised the Persian forces.
Since Joel always jogs a mile seems like a short distance to him.
Carmela put the candy on the table in her mouth.
Konstantin understood the problem had no solution.
Everybody at the party knew Ann’s date was a total fool.
Local ambiguity
Disambiguation downstream,
which goes against parser’s preferences
Reanalysis… or meltdown!
19
The Garden Path Theory
Lyn Frazier & Janet Fodor, late 1970s
Minimal Attachment: build the simplest tree
Late Closure: attach locally
Minimal Chain Principle / Active Filler Strategy:
posit shortest possible chain / posit gaps for fillers
ASAP
(the parser is lazy)
20
Minimal Attachment
building complex structure = processing cost
The soldiers marched into the desert surprised the Persian forces.
Since Joel always jogs a mile seems like a short distance to him.
Carmela put the candy on the table in her mouth.
Konstantin understood the problem had no solution.
Everybody at the party knew Ann’s date was a total fool.
21
Late Closure
attaching non-locally = processing cost
John said Mary will arrive last night.
Physicists are thrilled to explain what they are doing to people.
Under the glistening tree there was a gift for a boy in a box.
Professor Humperdinck artfully avoided looking at
the exams of the students that were sitting in his office ungraded.
Two sisters reunited after 18 years in check-out counter!
22
Late Closure
attaching non-locally = processing cost
Mary
Mary saw
saw a
a gift
gift for
for a
a boy…
boy in a box.
NP
NP
a gift
PP
P
NP
PP
for
a boy
in a box
23
Late Closure
attaching non-locally = processing cost
John said Mary will arrive last night.
Physicists are thrilled to explain what they are doing to people.
Under the glistening tree there was a gift for a boy in a box.
Professor Humperdinck artfully avoided looking at
the exams of the students that were sitting in his office ungraded.
Two sisters reunited after 18 years in check-out counter!
24
The RC Attachment Ambiguity
N1
N2
The plot concerns the guardian of the prince
who was exiled from the country for decades
RC
La trama es sobre el guardián del príncipe
que fue exiliado del país por décadas
25
Cross-Linguistic Differences
N1 attachment rates (%), in studies using
questionnaire instruments where:
• RC was long
• N1/N2 were equal in animacy
• Complex NP was in canonical object position for the language
EN
42
67
46
62
63
55
43
47
48
Hemforth et
al., submittd
63
Fernández,
2000/2003
SP
(US)
40
(UK)
Ehrlich et al.,
1999
Carreiras,
1992
Bradley et
al., 2003
Cuetos &
Mitchell,
1988
26
Cross-Linguistic Differences
As in previous table, for languages other than English & Spanish,
listed (for lack of a better strategy!) in alphabetical order:
94!!
(Fra)
SP-like 55
EN-like
64 66 62
60
(Can)
62
44
65 66
48
61 57
32
82
42
28
(FS)
SWEDISH
RUSSIAN
ROMANIAN
PORT. (EUROPE)
PORT. (BRAZIL)
NORWEGIAN
JAPANESE
ITALIAN
HEBREW
GERMAN
FRENCH
DUTCH
CROATIAN
BULGARIAN
ARABIC
AFRIKAANS
27
Cross-Linguistic Differences
… could be driven by …
genetic relationship?
syntactic properties?
existence of unambiguous alternatives?
distribution of unambiguous strings in input?
prosody?
PROSODY
(phonology)
PARSER
PRAGMATICS
28
Pragmatics?
Grice’s Cooperative Principle
“Make your conversational contribution such as is
required, at the stage at which it occurs, by the
accepted purpose or direction of the talk exchange in
which you are engaged”
The plot concerns
the guardian of the prince who was exiled
the prince’s guardian who was exiled
* the prince’s guardians who was exiled
29
Long RCs are Informationally Heavy
The plot concerns the guardian of the prince
… who was exiled.
… who was exiled from the country for decades.
N1 INTERP
MORE
LIKELY
Long RC has more lexical content,
so it’s more informative.
Does informativeness influence attachment?
RC length effect, confirmed in:
English, Spanish; Arabic, Bulgarian, Croatian,
French, German, Hebrew, Hindi, Portuguese…
30
Long RCs are Prosodically Heavy
The plot concerns the guardian of the prince who was
exiled.
The novel’s plot concerns the guardian of the prince who
was exiled.
The plot concerns the guardian of the prince who was
exiled from the country for decades.
The novel’s plot concerns the guardian of the prince who
was exiled from the country for decades.
31
Elicited Production
Fernández, Bradley & Taylor, in prep
N = 8 native US English speakers — 5F, 3M
N = 6  4 sentences,
RC Length  Matrix-Subject Weight
RC = 1 versus 3 prosodic words
…who was exiled ( from the country for decades )
MX = 1 versus 2 prosodic words
The ( unusual ) plot…
32
1
The unusual plot concerns the guardian of the prince.
The prince was exiled from the country for decades.
33
Fernández, Bradley & Taylor, in prep
Acoustic Analysis: Regions
Duration: Uniform acoustic signature of phrasal break
Wt
S
V
N1
The plot concerns the
( unusual )
guardian
of
N2
RC1
the who was
prince
exiled
RC3
( from the country
for decades )
34
Fernández, Bradley & Taylor, in prep
Acoustic Analysis: Regions
S][V
V ] [ N1
N1 ] [ N2
N2 ] [ RC
35
Fernández, Bradley & Taylor, in prep
1200
MX1, RC1
MX2, RC1
MX1, RC3
MX2, RC3
Mean Duration (msec)
1000
800
600
400
N2] [RC
200
0
Wt
S
V
N1
N2
RC1
RC3
36
Fernández, Bradley & Taylor, in prep
Region = N2
RC3
RC1
450
MX2
MX1
500
550
600
650
700
750
Mean Duration (msec)
37
Fernández, Bradley & Taylor, in prep
Elicited Production: Summary
MX:
F1(1,7) = 2.80, p=.138
F2(1,5) = 2.07, p=.209
RC: F1(1,7) = 11.46, p<.02
F2(1,5) = 9.96, p<.05
Interaction MX x RC: F1 < 1, F2 (1,5) = 1.62, p > .25
… N2 ] [ RC — and nowhere else
Likelihood of break grades with RC length and
matrix weight, additively, i.e., with sentence
length
38
Fernández, Bradley & Taylor, in prep
Questionnaire Procedure
 “Reading comprehension test”
 36 targets, 108 fillers (1:3 ratio)
 Comprehension question after each sentence
Example of target
The plot concerns the guardian of the prince who was exiled from the country
for decades.
Who was exiled?
the guardian
the prince
Example of filler
The sneaky burglars took all the stereo equipment but overlooked
the computer system.
What was stolen?
the stereo
the computer
39
Fernández, Bradley & Taylor, in prep
Questionnaire Participants
N = 44, Queens College students
US English speakers
Language-history questionnaire, non-native
speakers excluded/replaced
Rejected/replaced for errors > 15% in fillers
40
Fernández, Bradley & Taylor, in prep
Questionnaire Results
70
Relative Clause Length
F1(1,40) = 24.95, p<.001
F2(1,32) = 30.12, p<.001
% N1 Attachment
MX, 2 PWds
60
MX, 1 PWd
Matrix Subject Weight
F1(1,40) = 5.51, p<.05
F2(1,32) = 9.43, p<.01
50
Interaction
F1 < 1
F2 < 1
40
30
1 PWd
3 PWds
Relative Clause Length
41
The Implicit Prosody Hypothesis
(IPH)
“In silent reading, a default prosodic contour
is projected onto the stimulus, and it may
influence syntactic ambiguity resolution”
(Fodor 1998, 2002)
the brother of the bridegroom who snores
the brother of the bridegroom ][ who snores
42
Selkirk, 1986
Prosody and Syntax Align
the brother of the bridegroom
][ who often unknowingly snores
the brother of the bridegroom who snores
prosodic discontinuity
el hermano del novio
][ que a menudo inconscientemente roncaba
el hermano del novio
][ que roncaba
NP
NP
syntactic discontinuity
N1
N1
PP
P
NP
N2
RC
PP
P
NP
N2
RC
43
Empirical Support for the IPH
Behavioral evidence on how RCs
are interpreted during silent reading
existing dataset: Hemforth et al. (submitted)
Evidence on how the N-of-N-RC construction
is produced in discourse-neutral speech
elicited production experiment
Do the patterns in the two datasets match up?
44
Hemforth et al. (submitted)
Behavioral Evidence
Materials in English and Spanish:
with short and long RCs
N1-N2-RC placed post- and pre-verbally
The guest impressed X.
X impressed the guest.
El invitado impresionó a X.
X impresionó al invitado.
X = the brother of the bridegroom
who (often unknowingly) snores
el hermano del novio
que (a menudo inconscientemente) roncaba
45
Hemforth et al. (submitted)
Behavioral Evidence
Pre-Verbal Subjects:
RC length effect
reduced
Cross-linguistic
difference reduced
60
% High Attachment
Post-Verbal Objects:
Cross-linguistic
difference
RC length effect
Who snores?
The brother (N1)
English
Spanish
50
40
30
20
Short RC Long RC Short RC Long RC
Post-Verbal
Objects
Pre-Verbal
Subjects
46
ENGLISH
SPANISH
N2][RC
N2][RC
RC.]
The guest impressed the brother of the bridegroom
who often unknowingly snores.
El invitado impresionó al hermano del novio
que a menudo inconscientemente roncaba.
N2][RC
N2][RC
RC.]
RC][V
RC][V
The brother of the bridegroom who often unknowingly snores
impressed the guest.
El hermano del novio que a menudo inconscientemente roncaba
impresionó al invitado.
47
Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004
Experiment: Elicited Production
Participants, N = 8 per language
English  New York
Spanish  Madrid
Materials, N = 8  4 per language
(selected from Hemforth et al.’s 32  4)
Post- and pre-verbal of identical length
RC’s right boundary with same lexical content,
whether short or long
The guest impressed X.
X=
X impressed the guest.
the brother of the bridegroom
who (often unknowingly) snores
48
Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004
Analyses: N2 & RC’s Verb
Duration: Presence of Boundary
Pitch movement: Type of Boundary
The guest
impressed
the brother of the bridegroom ][ who … snores.]
N2][RC
RC.]
The brother of the bridegroom ][ who … snores ][
impressed
N2][RC
RC][V
.
the
49
Monolinguals: N2 Durations
Long RC
Short RC
100 ms
Placement × Length Interaction
F1(1,14) = 5.77, p < .05, F2(1,14) = 12.37, p < .005
RC-Length  =
123 ms Post-Verbal
68 ms Pre-Verbal
ENGLISH
SPANISH
Post-Verbal
Objects
Pre-Verbal
Subjects
Mean duration (ms)
Mean duration (ms)
50
Long RC
Monolinguals: RC Vb Durations
Short RC
100 ms
Placement × Length Interaction
F1(1,14) = 6.38, p < .025; F2(1,14) = 5.90, p < .05
RC-Length  = –10 ms Post-Verbal
35 ms Pre-Verbal
ENGLISH
SPANISH
Post-Verbal
Objects
Pre-Verbal
Subjects
Mean duration (ms)
Mean duration (ms)
51
Monolinguals: N2 Pitch
Long RC
Short RC
Post Pre
Placement × Language Interaction
F1(1,14) = 16.56, p < .002, F2(1,14) = 14.43, p < .002
Mean rise (Hz) per 200 ms
Placement  = 0.4 Hz/200 ms English
23.6 Hz/200 ms Spanish
ENGLISH
SPANISH
52
Monolinguals: RCVb Pitch
Long RC
Short RC
Post Pre
Interaction: Placement × Language
F1(1,14) = 6.05, < .05, F2(1,14) = 14.72, < .002
8.7 Hz/200 ms English
38.6 Hz/200 ms Spanish
Mean rise (Hz) per 200 ms
Placement  =
ENGLISH
SPANISH
53
Duration & Pitch: Monolinguals
ENGLISH
Post-Verbal
Objects
Pre-Verbal
Subjects
N2][RC
RC.]
SPANISH
][RC
N2
Post-Verbal, Short
Post-Verbal, Short
Post-Verbal, Long
Post-Verbal, Long
N2][RC
RC][V
N2][RC
Pre-Verbal, Short
Pre-Verbal, Short
Pre-Verbal, Long
Pre-Verbal, Long
RC.]
RC][V
54
Fernández, Bradley, Igoa & Teira, 2003; Fernández & Bradley, 2004
Summary of Data Outcomes
Pitch Movements: Type of Boundary
and Cross-Linguistic Differences
Spanish: N2 falls pre-verbally, rises post-verbally
English: N2 uniformly falls, pre- and post-verbally
Duration: Presence of Boundary
and Cross-Linguistic Similarities
In both languages: Likelihood of breaks
before RC is modulated by position
55
Conclusions and Speculations
Behavioral similarities and differences are
indexed in the prosodic patterns of
Spanish and English
But what is the source for the contrasting
sentence-medial tunes in Spanish?
Are such patterns projected entirely
within the syntax-prosody interface?
Or are such patterns the result of an interplay
of syntax, prosody, and information structure?
56