EEG, semantics/pragmatics, and
processing questions
A fairy tale…
SYNTAX
A fairy tale…
SYNTAX
ERP response 1
ERP response 2
A fairy tale
• How to isolate the syntax and semantics brain
responses?
– Show people sentences with grammatical
violations and compare to controls
– Show people sentences with semantic violations
and compare to controls
Hagoort, Brown &
Groothusen 1993
Hagoort, Brown &
Groothusen 1993
A fairy tale…
SYNTAX
…on shaky foundations
• But what would it mean for there to be an ERP
response for ‘semantics’ vs. ‘syntax’, given that
these are not processes?
• Is a grammatical violation a good way to probe
‘syntax’ processes?
• What are these ‘semantic violations’ anyway?
Empirical Challenges
• N400 is more sensitive to lexical-semantic
predictability than to semantic violation
N400: Predictability
Kutas & Hillyard, 1984
Van Petten, 1991; Federmeier & Kutas, 1999; Federmeier et al., 2007
13
Empirical Challenges
• N400 is more sensitive to lexical-semantic
predictability than to semantic violation
dainty nose
runny nose
innocent bag
yellow bag
Lau, Namyst, Fogel &
Delgado, submitted
Empirical Challenges
• P600 is observed to lots of different violations:
plausibility violations, spelling violations, etc.
• Also observed for syntactic ambiguity and
formation of wh-dependencies
Empirical Challenges
• P600
– P600 observed for formation of wh-dependencies
(Kaan et al. 2000, Phillips et al. 2006)
– P600 observed for syntactic ambiguity (e.g.
Osterhout & Holcomb, 1992)
– P600 often larger for syntactic violations when
acceptability judgment is required
Empirical Challenges
• ELAN - ~150ms
– Continues to be utilized as an index of phrase
structure violation by some (e.g. Batterink &
Neville)
– Some work assumes the reality of the effect but
investigates the underlying process in more detail
(Lau et al. 2006, Dikker et al. 2009)
– Other work suggests that the ELAN is an artifact,
in part because much work fails to replicate it
Bottom line
• Based on current evidence, N400 is not a
reliable indicator of semantic or pragmatic
anomaly/implausibility
• Based on current evidence, P600 appears to
be a fairly reliable indicator of any kind of
problem/reanalysis in sentence, but not
syntactic in particular
Let’s start over…
• What would we like to know about the
timecourse of parsing and interpretation?
(processing question)
Today
• What would we like to know about the
timecourse of interpretation?
– How fast do we access lexical semantics?
– How fast do we build the sentence-level and then
discourse-level interpretation?
– Do anaphoric elements invoke a stage at which
the antecedent lexical semantic information is
accessed prior to the discourse entry?
–…
• Measurement
– Changing current associated with changing neural
activity will change electrical and magnetic fields
outside the head
– These changing fields give rise to the EEG and MEG
DOG
Time 0: baseline activity
Time 200ms: ‘language’ activity
• Key idea behind EEG & MEG:
– Arrange detectors outside the head to measure
changes in the electrical activity inside the head
due to brain function
• What kind of activity?
Single neuron
firing?
Thousands or
millions of aligned
neurons firing
together
• History—EEG
• Hans Berger records the first human
electroencephalogram in 1924
• Berger reports it in 1929
• Finding not accepted by establishment until
1937, after Nobel winner Edgar Adrian
replicates Berger’s results
• What is the underlying quantity we will be
trying to measure with EEG and MEG?
ELECTRICAL CURRENT
• What is current?
MOVING CHARGED PARTICLES
-
-
• Measured in units of amperes
• Why current?
Remember what happens in
the classic neural signaling
picture:
• Firing ‘presynaptic’ neurons
release neurotransmitters near
the dendrites of the neuron,
causing positive charges to
move into the dendrite, and a
net negative charge outside the
dendrite
 charges moving = current
• Remember that to be measurable outside of head,
activity of many neurons must be synchronous
Action potentials unlikely to be
measurable outside of head
because neurons would have to
fire at exactly the same instant in
order to sum together
Postsynaptic potentials more likely
to be measurable outside of head
because even slightly shifted
curves will sum together
• Measuring current outside the head
– Postsynaptic potentials
Fortunately because many
neurons are lined up in
parallel–perpendicular to the
cortex—then if they all receive
input at about the same time,
the currents will sum together
to be big enough to measure
outside the head
(in other words, the CHARGES
are MOVING parallel to each
other)
+
+
+
ERP Analysis Basics
Time-course focused measures
• Relative to behavioral measures of
timecourse?
• Topoplots
Let’s start over…
• What would we like to know about the
timecourse of interpretation?
N400
• Since sentence level interpretation influences
semantic access, we can actually investigate
both of these processes with the same ERP
component: the N400
N400
• What happens when you see the word ‘cat’ in
isolation?
– Activate lexical entry
– Activate conceptual information
– Enter this into episodic memory, working
memory?
• Current guess is that these basic processes
occur 300-500ms after word is presented and
generate N400 ‘bump’
N400
N400: lexical facilitation
Kutas & Federmeier, 2000
48
Predictive Processing
• Previous work suggests that readers/listeners
routinely generate predictions about
upcoming words
– Reaction times faster to predictable words
– Neural responses are reduced in amplitude to
predictable words
Prediction or integration difficulty?
• Perhaps more predictable events are also
those which are more ‘strongly’ represented
in semantic memory (knowledge of the world)
and so ‘integrating’ the new word is easier
Prediction or passive priming?
kite
It was windy, so the boy went out to fly his
kite
• Actively predicting the next word
– Analysis-by-synthesis models of perception:
having an initial guess about the identity of the
input makes perception more efficient
Prediction or passive priming?
kite
It was windy, so the boy went out to fly his
kite
53
• N400 effect seems to reflect
lexical/conceptual facilitation due to priming
or prediction
• Now that we understand the N400 effect a
little better, what kinds of questions can we go
after?
• Questions have to be targeted!
• How quickly do anaphors like pronouns
reactivate their antecedent?
Lago, Namyst, & Lau, 2015 CUNY
Bachelors are often set in their ways.
And in general, they love their freedom.
Bachelors are often set in their ways.
?? And in general, they love their wives.
Susanna consulted the farmer before the harvest.
She was sad to hear that ...
Prono
un
his patio / crops had been damaged in the
storm.
• If his reactivates farmer, then we also
expect a reduced N400 for crops.
• The pronoun and repeated NP conditions
should pattern alike.
Susanna consulted the farmer before the harvest.
She was sad to hear that ...
Pronoun
control
her crops had been damaged in the
storm.
• This condition controls for context priming.
• If semantic reactivation is specifically due to
coreference, no N400 reduction for crops
when it is preceded by her.
Susanna consulted the farmer before the harvest.
She was sad to hear that her crops ...
No semantic relatedness effect if pronoun does not
refer to “farmer”: her crops = his patio
[μV]
her crops
his patio / crops
[ms]
• How quickly do anaphors like pronouns
reactivate their antecedent?
– Within at least 800ms of encountering the
pronoun (500ms pronoun + 300ms noun)
Lago, Namyst, & Lau, 2015 CUNY
• How quickly is negation interpreted?
With proper equipment, scuba diving is very …
With proper equipment, scuba diving is very …
safe
With proper equipment, scuba diving isn’t very…
With proper equipment, scuba diving isn’t very…
dangerous
• How quickly is negation interpreted?
– Very rapidly (at least, if the negation is
pragmatically licensed)
• Does ‘discourse’ information about an
entity rapidly override lexical-semantic
information in the generation of
predictions?
• Does ‘discourse’ information about an entity
rapidly override lexical-semantic information
in the generation of predictions?
– Yes: when predicate immediately follows
anaphoric definite NP, the predicate consistent
with the discourse information seems to be
predicted, NOT the predicate consistent with the
lexical-semantic information
• Does information about the speaker drive
online predictions for upcoming linguistic
input?
Every night I drink some …
• Does information about the speaker drive
online predictions for upcoming linguistic
input?
– YES!
Van Berkum et al. 2008
• Do speakers rapidly make inferences that
inform upcoming predictions?
– YES!
Ditman & Kuperberg, 2007
• Delong: comprehenders seem to make lexical
predictions about nouns that impact processing
of preceding determiner form (a/an)
• But do comprehenders make ‘referential’
predictions about whether a previously
referenced entity will be mentioned again, and
can we see this in predictions for definite vs.
indefinite determiner (the/a)?
Anna wore a necklace to her friend’s party. She
told everyone that ___
Kirsten et al., 2014
Schlueter, Williams & Lau, 2015 CUNY
Anna wore a necklace to her friend’s party. She
told everyone that ___
, - . +/- . 0 1#23 ' 4#24567' #8%1' *#
$%&'( ( &&) *#+, - .#/01/#0/*- 2#
! "#
, - . +/- . 0 1#23 ' 4#24567' #8%1' *#
! "#
! "#
2
.5
! /" &
2
1
.5
1
0
.5
0
#&
−
0
.5
−
1
(&
(&
! " #$ %&
' " #$ %&
−
1
.5
−
2
0! /" &
−
2
.5
(&
(&
! " #$ %&
' " #$ %&
$%&' ( %)*' #+#&' ( %)*' #
/01/#
'&
! " #$ %&
( ) *+,-) &
,+( ) *+,-) &
) *+, -.*&
) *+, -.*&
-, ) *+, -.*&
'&
. / ##$ %&
$%&' ( %)*' #+#&
3/4- 5/6
Kirsten et al., 2014
Schlueter, Williams & Lau, 2015 CUNY
Antje noticed that ____
• All of this shows that interpretation and
corresponding predictions are very FAST
• Is there any aspect of these mechanisms that
is slow?
• Slow?
– Using determiner information to update
predictions
• If you encounter the indefinite determiner, it’s
much less likely that the previous noun will be
repeated, but N400 doesn’t seem to be
sensitive to this
" #$%&$' ( ) *+$, - . /00. . 1 2$
#! $%%to
#&'her( )friend’s
*#+*#*,party.
( #! "She
-.
Anna wore!a "
necklace
told everyone that the/a necklace…
#
! " #! $%%#&' ( ) *#+*#*, ( #! " - . #
./
2
! ""#$""%&'
$%%&' %%( )#*+ , -#. / 0. #/ . ), 1#
2
.5
2
.5
1
.5
. /0%
2
! "#
1
$%%&' %%( )#*+ , -#. / 0. #/ . ), 1#
1
.5
! "#
1
0
.5
0
.5
"%
0
0
"
−
0
.5
−
1
'%
. / 0. #
−
0
.5
'%
! " "# $%&"" "# $%
−
1
.5
−
1
−
2
'%
. / 0. #
'%
1. /0%
( ) *+,-) %
! " " # $%&" " " # $%
,+( ) *+,-) %
−
2
.5
−
1
.5
2. 3, 4. 51, #&#3, 4. 51, #
−
2
Kirsten et al., 2014
( ) *+,-) %
/ 0+. 12+3( 0+4( #&567#+*#( 8( ) *0" 1( #69#+. 1#*, ( #*" : " 40+: , ;) #1;7*0;<- =" . #" >#
Schlueter,
& Lau,
#+B( 0#. " - . #" . Williams
7( *C#
,+( ) *+,-) % &56#( ' ( ) *7#;. #*, ( #?%%2@%%A 7#;. *( 03+8
1.
2015 CUNY
−
2
.5
• Slow?
– Using thematic role information to update
predictions
• No N400 reduction for canonical sentences vs.
role-reversed sentences using SOV order
The cop the thief arrested
The thief the cop arrested
(canonical/predictable)
(role-reversed/unpredictable)
Van Herten et al. 2003, Kuperberg et al.
2003, Kim & Osterhout, 2005
• No N400 reduction for canonical sentences vs.
role-reversed sentences using SOV order
– Why? TAKES TIME to assign roles and generate
predictions
• Slow?
– Using thematic role information to update
predictions—takes TIME
– Recently replicated in Japanese (Momma et al.)
Beyond the N400…
• When native speakers hear accented speech,
do they change their predictions?
Question
• How much exposure is needed before the ERP
response to a word in the second language
looks like the response to a word in the first
language?
Paradigm
• Three test sessions while L2 is being acquired
in French101:
– 14 hours of exposure
– 63 hours of exposure
– 138 hours of exposure
• Prime-target pairs presented
– Targets related, unrelated or pseudowords
• Task is lexical decision on target
What would native speakers do?
• Pseudowords elicit a bigger N400 than words
(in your native language)
• Related words elicit a smaller N400 than
unrelated words
Holcomb, 1993
N400 effect: brain basis
• We don’t need to know about the brain area
generating the N400 effect to use it effectively to
answer these processing questions
• But since we know a lot about the properties of
the N400 effect (since ERP studies are pretty
cheap to run), knowing where it is generated
might tell us something useful about that area
• Also would allow us to link ERP and fMRI
literatures
N400 effect: brain basis
• Lau et al. 2008 review of prior literature
– posterior MTG (fMRI/MEG data); anterior
temporal cortex (intracranial recordings)
N400 effect: brain basis
Lau et al., 2013
N400 effect: brain basis
• fMRI confirmation: more precise! Mid-anterior
STS/STG
Lau et al., 2013
Referential Ambiguity
ERPology
• Does the Nref indicate waiting/anticipation for
disambiguation by a post-noun modifier?
– …the girl that was on the phone
• Does the Nref indicate waiting/anticipation for
disambiguation by a post-noun modifier?
– …the girl that was on the phone
• NO
• Does the Nref index ‘deep’ discourse model
ambiguity, or does it index simple ‘resonance’
with lexical items that have the same features
as the anaphor?
• Does the Nref index ‘deep’ discourse model
ambiguity, or does it index simple ‘resonance’
with lexical items that have the same features
as the anaphor?
• DEEP DISCOURSE MODEL AMBIGUITY