Kim & Osterhout (2005) JML
The independence of combinatory
semantic processing: Evidence from
event-related potentials
10/13/10
Psyc / Ling / Comm 525 Fall 10
Background
• Everyone believes
– N400 reflects semantic processing difficulty
– P600 reflects form-related processing difficulty
• However, 2 studies have found P600 when they expected
to find N400
– Kolk et al. (2003) found P600 at joeg
• De vos die op de stropers joeg …
• The fox that the poachers hunted …
• = The fox that hunted the poachers …
– Kuperberg et al. (2003) also found P600 at eat
• For breakfast, the eggs would only eat …
• In both cases, there was a noun that was plausible in
some role of the verb’s event, just not the role its
position indicated it had
10/13/10
Psyc / Ling / Comm 525 Fall 10
Experiment 1 - Stimuli
• Active Control
– The hungry boy was DEVOURING the cookies.
• Passive Control
– The hearty meal was DEVOURED by the kids.
• Anomaly
– The hearty meal was DEVOURING the kids.
10/13/10
Psyc / Ling / Comm 525 Fall 10
Procedures/Design
• Sentences presented word-by-word centrally
• SOA = 650 msec (slow!)
• End-of-sentence acceptability judgments
• Expt 1
– N = 24
– 96 sets of 3 sentence versions (32)
– 107 distractors, some sem anom, some ungramm
• Numbers varied across lists to make acceptability ~50/50
– 96 + 107 – 203 trials (58% acceptable, 42% unacceptable)
• Expt 2
– N = 29
– 96 sets of 3 sentence versions (32)
– 112 distractors, some sem anom, some ungramm
• Numbers varied across list to make acceptability ~50/50
– 96 + 112 = 208 (50% acceptable, 50% unacceptable)
10/13/10
Psyc / Ling / Comm 525 Fall 10
Experiment 1 - Results
P600
P600
10/13/10
Psyc / Ling / Comm 525 Fall 10
Experiment 1 - Discussion
• Why does “The hearty meal was devouring…” evoke
P600 rather than N400?
– Because hearty meal can play SOME thematic role in a
devouring event?
– Maybe the fact that hearty meal fits so well with
devouring makes the processing system think there’s a
grammatical error, like the wrong inflection on
devouring, rather than a semantic anomaly
10/13/10
Psyc / Ling / Comm 525 Fall 10
Experiment 2 - Stimuli
• Passive Control
– The hearty meal was DEVOURED …
• No-attraction Violation
– The dusty tabletops were DEVOURING …
• Attraction Violation
– The hearty meal was DEVOURING …
10/13/10
Psyc / Ling / Comm 525 Fall 10
Experiment 2 - Results
N400
10/13/10
Psyc / Ling / Comm 525 Fall 10
Experiment 2 - Discussion
• Argue that these results show that
– Semantic processing can “drive” sentence
comprehension
– Rather than always having to wait for structural
processing to give the relationships among words
before their semantic combination can proceed
10/13/10
Psyc / Ling / Comm 525 Fall 10
The Role of Prosody
• Embedded Clause / Direct Object sentences can
be disambiguated with prosodic phrasing
• Acoustic correlates:
–
–
–
–
–
10/13/10
Pause
Pre-boundary lengthening
Pitch contour
Pitch reset
…
Psyc / Ling / Comm 525 Fall 10
How does this kind of prosodic
boundary marking influence sentence
interpretation?
• Example of DO Prosody
– The basketball star accepted the contract…
…because it paid so well.
Example of Clause Prosody
- The basketball star accepted …
… the contract requires him to play every game.
10/13/10
Psyc / Ling / Comm 525 Fall 10
Do speakers actually produce different
prosody in DO and Clause structures?
Gahl & Garnsey (2004)
10/13/10
Psyc / Ling / Comm 525 Fall 10
Boundaries marked more strongly when
Structure not consistent with Verb Bias
10/13/10
Psyc / Ling / Comm 525 Fall 10
Using ERPs to Study Prosody
• On-line nature of ERPs especially good for
investigating immediate effects of prosody
during spoken sentence processing
• Steinhauer, Alter, & Friederici (1999)
– Discovered a positive ERP component at prosodic
boundaries
– “Closure Positive Shift” (CPS)
10/13/10
Psyc / Ling / Comm 525 Fall 10
Closure Positive Shift (CPS)
10/13/10
Psyc / Ling / Comm 525 Fall 10
Steinhauer, Alter & Friederici (1999)
CPS
• Demonstrated CPS with
–
–
–
–
–
–
Delexicalized speech (Steinhauer & Friederici 2001)
Jabberwocky sentences (Pannekamp et al. 2005)
Pseudosentences (Pannekamp et al. 2005)
Hummed sentences (Pannekamp et al. 2005)
Musical phrases (Knösche et al. 2005)
Commas in orthographic stimuli (Steinhauer 2003; described in
Frazier, Carlson, & Clifton 2006)
• But only for people with good knowledge of comma rules
10/13/10
Psyc / Ling / Comm 525 Fall 10
CPS Studies from Other Labs
• Kerkhofs et al. (2007)
– Embedded stimuli in discourses
– Found smaller CPS when discourse made a boundary
highly predictable
– Casts some doubt on CPS as “pure prosody”
processing
10/13/10
Psyc / Ling / Comm 525 Fall 10
ERP Prosody Study
(Jackson, Patel, & Garnsey, 2010)
• Structure of sentence completion
– Beginnings ambiguous
The basketball star accepted the contract…
– Direct Object (DO) ending
…because it paid so well.
– Embedded Clause (Clause) ending
…requires him to play every game.
• All Verbs DO-Bias & Critical Nouns plausible as DO
– To maximize garden-pathing, to have the best possible
chance to see whether prosody can prevent it
10/13/10
Psyc / Ling / Comm 525 Fall 10
Stimuli
• Prosodic Phrasing
– Direct Object (DO) phrasing
[ The basketball star accepted the contract ]
[ because it paid so well. ]
– Embedded Clause (EC) phrasing
[ The basketball star accepted ]
[ the contract requires him to play every game. ]
10/13/10
Psyc / Ling / Comm 525 Fall 10
More Stimuli & Design
• Fully crossed
• Matching conditions:
– DO ending + DO prosody
[ The basketball star accepted the contract ]
[ because it paid so well. ]
– Clause ending + Clause prosody
[ The basketball star accepted ]
[ the contract requires him to play every game. ]
10/13/10
Psyc / Ling / Comm 525 Fall 10
More Stimuli & Design
• Mismatching conditions:
– Clause ending + DO prosody
[ The basketball star accepted the contract ]
[ requires him to play every game. ]
– DO ending + Clause prosody
[ The basketball star accepted ]
[ the contract because it paid so well. ]
10/13/10
Psyc / Ling / Comm 525 Fall 10
Materials Construction
• “Natural” recordings made in all four conditions.
• From these, spliced:
– One beginning per prosody condition
– One ending per structure condition
• Splice locations counterbalanced to ensure
equivalent (un)naturalness across conditions
10/13/10
Psyc / Ling / Comm 525 Fall 10
Splicing example
• Beginnings
– DO prosody
– Clause prosody
• Endings
– DO ending
– Clause ending
• Results
–
–
–
–
10/13/10
DO prosody, DO ending
Clause prosody, DO ending
DO prosody, Clause ending
Clause prosody, Clause ending
Psyc / Ling / Comm 525 Fall 10
ERP Data Analysis
Predictions:
– CPS when there’s a boundary compared to no boundary
– No P600 when prosody could prevent garden-pathing
Limitation on data analysis:
Can’t directly compare across prosodic conditions at the
critical disambiguating word (requires or because)
– Because in DO Prosody, CPS directly precedes it,
but not in SC Prosody (CPS earlier, after main verb accepted)
– So compare difference waves across Structures within Prosody
• ERP starting at end of pre-boundary word minus ERP starting at
end of same word without boundary shows CPS
10/13/10
Psyc / Ling / Comm 525 Fall 10
Structure Matches Boundary Location
(collapsed over Clause & DO-Structure)
Difference Waves:
Boundary minus
No-Boundary
0 msec = end of
“pre-boundary” word
10/13/10
Psyc / Ling / Comm 525 Fall 10
Structure MISmatches Boundary Location
(collapsed over Clause & DO-Structure)
Difference Waves:
Boundary minus
No-Boundary
0 msec = end of
“pre-boundary” word
10/13/10
Psyc / Ling / Comm 525 Fall 10
When Structure Matches
Boundary Location
10/13/10
Psyc / Ling / Comm 525 Fall 10
When Structure MISmatches
Boundary Location
10/13/10
Psyc / Ling / Comm 525 Fall 10
Next Steps
• Manipulate Verb Bias
• Manipulate Plausibility
– Would a Boundary before a Noun that’s implausible as
a DO prevent N400 effects on Noun?
• The referees warned // the game would probably go
into overtime.
10/13/10
Psyc / Ling / Comm 525 Fall 10
Summing Up
• Multiple sources of information constrain
sentence interpretation
– Lexical bias, plausibility, prosody …
• The sources interact
– BUT some provide stronger constraints
– And/or are available more rapidly
– And thus drive the interaction
• And this all happens really fast!
10/13/10
Psyc / Ling / Comm 525 Fall 10