Aug 10 Outline
• Spoken word recognition
– Evidence for top-down feedback
• TRACE theory
• Cohort theory
• Windmann Presentation
– But isn’t word recognition automatic?
• Differences between spoken & written word
recognition
Evidence for Top-Down influence
on speech perception
• Phoneme Restoration Effect (Warren,
1970)
• Lexical bias in categorical perception task,
e.g. dype vs. type (Clifton & Connine,
1987)
• Errors made by close shadowers
(Marslen-Wilson, 1973)
What kinds of Top-Down
knowledge can we use for Speech
Perception?
• Lexical
• Syntactic and Semantic
– Right-context comes too late, but Left-context
might be useful IF our syntactic and semantic
processing keeps pace with speech
perception.
• The driver turned the *eel.
• She saw his/him duck.
Marslen-Wilson (1973)
Speech Shadowing Task
– While listening to continuous speech, repeat it back
as rapidly as possible.
• For isolated words or nonsense syllables, RT is about 150 –
250 ms.
• For continuous prose, shadowing latency is about 500 –
1500 ms.
– Why different? Maybe because of syntactic and
semantic processing for sentences, which requires
larger units of processing (e.g. phrase or clause).
• If so, people shadowing at very short latencies should make
errors that ignore syntactic and semantic constraints of
sentence.
• Only “distant shadowers” will make errors that respect
syntactic and semantic constraints
Marslen-Wilson (1973)
• Ran 65 participants in shadowing task and
measured average latency.
– 7 participants were “close shadowers” < 350 ms.
– Remaining participants had latencies of 500 -800 ms.
• Test passage presented over headphones to 7
close & 7 distant shadowers
– 300 words @ 160 words/min.
– average syllable = 200 ms
– Original passage & shadowing performance recorded
on separate tracks of tape recorder.
– 4 closest shadowers had 254-287 ms latencies &
made 1.7% - 6.6% errors
Marslen-Wilson (1973)
• Were close shadowers comprehending the input
more superficially than distant shadowers?
• No.
– Memory test on 600 word passage showed no
reliable correlation between shadowing latency &
memory score
• But this could reflect additional processes that lag behind
shadowing performance.
– Do close shadowers make different types of errors in
their shadowing performance itself?
Marslen-Wilson (1973)
• There were 111 constructive errors, in which
participants added a real word or changed a
word into another real word.
– All but 3 were grammatical & semantically
appropriate.
– No qualitative difference between close & distant
shadowers; sometimes they made the exact same
error:
• It was beginning to be light enough so THAT I could see.
– Especially for close shadowers, constructive errors
tended to occur at very short latencies, perhaps
relying more on predictive top-down cues than
bottom-up information.
Marslen-Wilson (1973)
Summary
• Syntactic and semantic information (higher order
structure) was available to both close and distant
shadowers
– When shadowers made errors, they were
syntactically and semantically well-formed
• Language Comprehension is Incremental—DTC
cannot be correct
• Syntactic and Semantic processing keep pace
with speech perception (within a syllable or so)
– Potential source of top-down cues to guide speech
perception & spoken word recognition.
Connine & Clifton (1987)
• Lexical Bias effect is enhanced by sentential
context.
– At her birthday, she received a valuable *ift.
– * is ambiguous between /g/ & /t/
• Such top-down effects are clearly consistent with
interactive (though underspecified) models like
TRACE.
• How would an autonomous (modular) account of
speech perception handle this finding?
TRACE (McClelland & Elman, 1986)
•
•
•
•
At each level, individual
nodes (corresponding to
features, phonemes, or
words) compete for
activation.
Facilitatory activation
from bottom-up & topdown sources
Inhibition from bottom-up,
top-down, & lateral
sources
Recognition occurs when
network settles into
stable state with a clear
winner.
Word-level competition in TRACE
• “bald”
Activation
Cycles (time)
Visual Trace
Example
Cohort (Marslen-Wilson)
Theory combines initial autonomous stage with
secondary interactive stage.
• Word-initial cohort formed solely on the basis of
bottom-up acoustic input
– All cohort members are actual words
– Lexical access of candidates
• Words in the cohort are removed on the basis
of…
– Inconsistency with further acoustic input
– Inconsistency with context
– Word recognition = only one candidate remains
Cohort Example: stand
Use Gating to find Recognition Pt
Gating study from Zwitserlood (1989)
– People heard successively longer fragments of critical words
– In 3 kinds context
• Carrier phrase: The next word is kapitein.
• Neutral context: They mourned the loss of their kapitein.
• Biasing context: With dampened spirits the men stood around the
grave. They
mourned the loss of their kapitein.
– Guessed what the word was
– Recognition point = Point in word where everyone identifies it as the
critical word
• Often earlier than uniqueness point
• How much earlier typically depends on degree of contextual
constraint
– Get to see what competitors are produced before recognition point
Zwitserlood (1989)
Evidence for parallel lexical activation of cohort
members
• Present participants with /kaept/, which is ambiguous
between captain and captive
– Experiments were conducted in Dutch, so modified here slightly
to work in English
• Then present a word related to either of those
continuations – like ship and guard
• Both “ship” & “guard” recognized fast, compared with
unrelated control word. Indicates access to semantics for
both cohort members
– True, even in biasing sentence context, so top-down context did
not prevent lexical access of cohort candidates!
– Example of semantic priming
Priming paradigm
Name (or make LDT to) red stimulus (i.e.,
target).
prime word: CAT
target word: CAT
Repetition Priming: Faster to name/LDT target after
same-word prime than after any other kind of prime.
Semantic Priming: CAT is faster after related prime
(DOG) compared to unrelated prime (DOT)
Implications of Cohort
• Special role for word-onset
• Recognition point can precede end of
word
• An infelicitous word might not be
accurately recognized
– I mailed the letter w/o a STEAK.
• Can account for most top-down effects
– But not word-initial phoneme restoration
TRACE vs. Cohort
• Cohort focus specifically on word level, whereas TRACE
models feature and letter/phoneme identification as well.
– A later, connectionist version of Cohort incorporates speech
perception & addresses shortcomings of original cohort model
(Gaskell & Marslen-Wilson, 1997)
• Both theories allow for top-down effects on spoken word
recognition
• TRACE is fully interactive; Cohort has an initial
autonomous stage
• Cohort depends upon clear phonological input at word
onset, for activation of cohort.
– TRACE allows for graded activation based on shared features
• “ba…” activates “papa” as well as /b/ words.
– TRACE allows for activation of rhyming words
• “ball” partially activates “fall” and “call”
Windmann Presentation
• Sandy & Joanne
Take-Home Points
• Speech perception is fast and many aspects of it
seem to be automatic and feed-forward.
• Yet when bottom-up input is ambiguous, noisy,
or conflicted, top-down knowledge can influence
final percept, and perhaps the initial percept.
• Sentence-level Syntactic and Semantic
Processing keeps pace with speech perception,
lagging by no more than a syllable or two.
– Unit of syntactic analysis during comprehension is
word, not sentence; build parse tree incrementally.
Is lexical access Automatic &
Modular?
Automatic Processes
– Fast
– Do not require attention
– Feed-forward (can’t be guided, controlled,
or stopped midstream)
– Not subject to top-down feedback
(informational encapsulation)
Priming paradigm
Name (or make LDT to) red stimulus (i.e.,
target).
prime word: CAT
target word: CAT
Repetition Priming: Faster to name/LDT target after
same-word prime than after any other kind of prime.
Subliminal Priming: Even if prime is presented too
quickly for conscious awareness
Stroop Effect
Name font color
RED
GREEN
BLUE
YELLOW
GREEN
What happens if you have to name word?
Stroop Effect
• When font color conflicts with word itself, we are
slower and less accurate to name the font color.
– Recognition of word interferes with naming color of
letters.
• No such interference from font color if task is to
name the word.
• Word recognition is fast & feed-forward; we can’t
stop recognizing the word, even when doing so
is detrimental to task performance.
Is lexical access sensitive to topdown context?
• Maybe not.
• Zwitserlood (1989) found that cohort
members were activated, even if they
were inconsistent with the semantic
context.
– Context did have an effect, but it was after the
initial bottom-up activation of cohort members.
A Puzzle
• Lexical Access seems like an automatic, feedforward, bottom-up process.
• Speech perception seems quite sensitive to topdown context effects.
• Can both of these be true?
• Is lexical access really more interactive than it
appears?
• Is speech perception really more bottom-up than
it appears?
Word Recognition Across
Modalities
Production
Spoken vs. Written
Lexical Access in Language
Production
Levels of Processing
• Concept selection
• Word selection
• Phonological & phonetic encoding
• Construction of motor plan
• Articulation
 Is this bottom-up or top-down processing?
 Describe the Stroop effect in terms of these levels of
processing.
 Describe Ashcroft’s deficit in terms of these levels of
processing.
Differences between spoken and
written word recognition
• For relatively short words, letters in a
written word are processed in parallel
– Eye movement data
– Word superiority effect
– Letter-Search Task
• Spoken word unfolds across time
– Can recognize some words before they are
completely pronounced.
Eye Tracking
Word Superiority Effect
(Cattell, 1886; Reicher, 1969)
Present stimulus for brief (near threshold)
interval on T-scope. Is the (final) letter a D
or a K?
It is easier to
recognize a
letter when it
is in a word,
compared to a
non-word or
isolation.
OWRK
***
• OWRK
***
K
•K
WORK
***
• WORK
Is the word easier, due to guessing?
Visual Trace
Example
Equal bottom-up
support for R & K,
but R wins due to
top-down support
from word level.
How many instances of the
letter “t” in the first sentence?
Which letter “t”s do people
miss?
Implications:
• Word Superiority effect
• Letter Search Task
– Do we recognize a word by recognizing each
of the letters?
– Does word recognition facilitate letter
recognition?
– What is the role of top-down and bottom-up
processing in these tasks?
Letter Recognition in Words
• Just like for phoneme perception in
spoken words, there is a great deal of
evidence that word & letter perception are
intertwined in visual word recognition.
• We may recognize the word faster than we
can recognize each of the letters,
providing the opportunity for top-down
processing from word to letter.
A Psycholinguistic Hoax
Aoccdrnig to rscheearch at Cmabrigde Uinervtisy, it deosn't
mttaer in waht oredr the ltteers in a wrod are, the olny
iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit
pclae.
The rset can be a total mses and you can sitll raed it wouthit a
porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey
lteter by istlef, but the wrod as a wlohe.
amzanig huh?
•Can we take this at face value? Is the order of intermediate
letters really irrelevant? Do the number and identity of
intermediate letters matter?
•How do we notice typos such as transposed letters?
•How do we realize we’re reading novel words?
•How do we distinguish “skates” from “steaks”?
Tasks for studying Word
Recognition
Words in Isolation
• Naming
• LDT
Words in Context
• Eye-tracking during reading
• Priming (often cross-modal)
Some Basic Findings about
Word Recognition
• Frequency influences RT in naming and LDT,
and gaze duration in eye-tracking
• LDT slow for wordy non-words
• Priming (Repetition, Semantic, etc.)
– Subliminal priming demonstrates that WR doesn’t
require attention
• High-level context effects???
– Faster to recognize word in congruent context?
– Slower in incongruent context?
Experimental Design
Balota et al. (2004)
• Factorial designs
– Very common
– Many important findings
– Limitations
• (large-scale) Regression studies
– Increasingly popular in word recognition lit
Experimental Design
• Factorial
– Item factors manipulated categorically
• E.g. frequency or contextual bias split into high and
low conditions
– ANOVA
• Main effects, interactions
– If there is a main effect (e.g. of frequency on
naming latency), it suggests that that factor
(frequency) impacts lexical access
Example Experiment: Factorial LDT
• Hypothesis: High frequency words will be
recognized faster than low frequency words.
• Null Hypothesis: No effect of frequency on word
recognition
• Dependent Measure: time to say “yes”,
measured from onset of visually presented word.
• Participants: 24 college students who are native
speakers, with normal vision and no reading
problems.
Stimuli
• Critical Trials: 2 levels of Frequency
– 20 high frequency words, ranging from 75 to 300
tokens per million words
• 5-8 letters in length
– 20 low frequency words, ranging from 1 to 15 tokens
per million words.
• 5-8 letters in length
• Filler Trials
– 20 words
• 5-8 letters in length
– 60 nonwords
• 5-8 letters in length
• All are pronounceable and word-like
Analysis of Variance
• For each participant,
measure average latency
on high frequency trials &
average latency on low
frequency trials.
• Is there a main effect of
frequency?
• F ratio = variance between
conditions/variance within
conditions
• p = probability that an effect
of size F is significant, given
degrees of freedom in your
study
Partici
pant
1
2
3
4
5
6
7
8
High
low
560
603
491
621
555
602
597
533
590
721
496
621
631
674
711
698
2 by 2 Factorial Design
• Hypothesis: Frequency effect is larger for
long words than for short words
• Stimuli (4 critical conditions, 2 factors)
– Short, High freq words
– Short, Low freq words
– Long, High freq words
– Long, Low freq words
• Predicting an interaction between our 2
factors
Limitations of Factorial Designs
• Hard to manipulate one factor while holding all
other variables constant
– E.g., length, regularity, imagability, and age of
acquisition are all correlated with frequency
• If we don’t control for imagability, it could be confounded with
frequency. If so, our “frequency effect” might really be an
imagability effect.
• Words are not randomly selected
– Though this assumptions is implicit in ANOVA
– Researchers may be using intuitions to select subsets
of words that are recognized fast/slow due to
variability on dimensions not intentionally
manipulated.
Limitations of Factorial Designs
• Unwanted list-context effects
– Related to non-random sampling
– Experimental stimuli may lead participants to
expect certain types of words
• Categorizing continuous variables
decreases statistical power (sensitivity)
• More informative to know how much a
factor influences word recognition rather
than simply that the factor has an impact
Balota et al. Regression Study
Goals
• What is the best way to measure frequency?
• What is the independent contribution of
theoretically interesting predictor variables?
– how much variance can each explain?
• Does importance of predictor variable differ for
naming and lexical decision?
• Does it differ for younger (mean age = 20) and
older (74) adults?
– 50+ more years of practice
– Cognitive declines in late adulthood
Balota et al. Stimuli
• Critical Stimuli: All monosyllabic,
monomorphemic words from million-word,
balanced corpus (Kucera & Francis, 1967).
– 2,428 words with high accuracy in analyses
– Each word coded for various types of frequency,
length, and many other variables.
• LDT version has an equal number of nonwords
created by changing 1-3 letters of real words
Naming RT was not very predictive
of LDT RT
Naming RT was not very predictive
of LDT RT
Young RT predicts Old RT fairly
well
Young RT predicts Old RT fairly
well
Interim Summary
• For a given word, RT in naming is not a very
good predictor of RT in LDT
– Suggests that some predictor variables contribute
more to naming, and some contribute more to LDT
• For a given word, RT by young adults is a pretty
good predictor of RT by older adults, regardless
of tasks
– Older adults are slower, but performance may be
influenced by same predictor variables as young
adults
Mean
SD
Young adults were faster and less variable.
Overall, predictor variables accounted for
about 50% of variance in young, 40% of
variance in old.
Frequency explains more variance in LDT than in
Naming
They like Zeno
(17 million)
corpus, which
does pretty
well. Other
large-scale
studies have
found that
spoken lg
corpora do
better than
text corpora
(all these are
text).
Most common
measure does
poorly (1 million
words)
Regression Analysis
• Surface Predictors
– Phonetic features of onset phoneme coded as
1 (present) or 0 (absent)
• Bilabial, dental, fricative, voiced…
• Important for naming, because of voice key
sensitivity
• Lexical Predictors
• Semantic Predictors
Regression Analysis
• Surface Predictors
• Lexical Predictors
– Length in letters (2-8)
– Neighborhood size (# of other words that differ only
by 1 letter)
– Objective frequency (Zeno norms)
– Subjective frequency (Balota familiarity ratings)
– Consistency (4 types of spelling-sound
correspondence)
• Semantic Predictors
Regression Analysis
• Surface Predictors [step 1]
• Lexical Predictors [step 2]
• Semantic Predictors [step 3]
– Nelson’s set size: # of associates in free association
task
– Imagability rating
– Wordnet connectivity (Miller)
Many of predictor variables are related to one
another (e.g., longer words have smaller
neighborhoods), so analysis must partial out
shared variance.
I will focus on RT-by items analysis (ignoring
Accuracy & subject-level analyses)
Regression Analysis
Phono onset
matters a lot for
Naming,
especially for
young.
Length also
matters more for
Naming
Freq matters
more for LDT
Cost of length higher for low
frequency words
Interaction
Implications of Length Effects
• Coltheart et al. (2001) predict a length by
lexicality interaction, with non-words
showing a greater effect of length.
• They also predict the length by frequency
interaction observed by Balota et al.
• Motivated by the Dual Route Model of
Visual Word Recognition
Dual Route Model
Two pathways for lexical access:
• ‘lexical’ route: proceeds directly from
orthography to lexicon
– Available to well-known words
– Preferred for irregularly spelled words
– May dominate in LGs with little ortho-graphemic
consistency
• ‘sublexical’ route: graphemic form converted
to phonological representation BEFORE
lexical access (each grapheme is assigned a
pronunciation by mapping to a phoneme)
Some semantic effects (esp LDT) after
partialling out phono & lexical effects
Meaning probably
plays a stronger
role in the LDT
compared with
Naming
Summary of Balota et al.
• Large-scale regression study replicated many
effects established by factorial studies
– PLUS…power to detect many small effects, such as
the influence of imagability on naming RT
– while over-coming limitations associated with small
item sets
• Clarified unique contributions & interactions of
specific variables
• Allowed careful examination of
– task differences between naming & LDT
– age differences between younger & older adults
Bilingual Word Recognition
Assumptions about the Lexicon
• Storehouse of knowledge about all the
words you know
• Organized phonologically
– Word-initial cohort together
• Distinct from Semantic Memory
Are bilinguals any different from
monolinguals?
How is bilingual memory
organized?
General agreement on the separation of
lexicon(s) and semantic memory.
– Dog and chien access same semantic
network, because both prime cat in FrenchEnglish bilinguals.
Whether there is a distinct lexicon for each
language is controversial
Why study the bilingual lexicon?
• Not really a special case, worldwide
• Mapping between words and meanings
• Mapping between phonology and
words/meanings
– Are words from multiple languages in wordinitial cohort?
– If not, can we also limit by topic domain—e.g.,
no physics words in history class?
Two Opposite Hypotheses
1. Bilinguals have 2 distinct
lexicons; tri-linguals have 3, &
so on.
2. Everyone has a single lexicon
– How keep lgs straight?
Possible Evidence for
Separate Lexicons
• Lack of repetition priming across
languages: chien doesn’t prime dog like
dog primes dog.
– But couch doesn’t prime sofa like sofa primes
sofa either
Possible Evidence for
Separate Lexicons
• Release from PI
– In a single language
• It is difficult to recall an item that occurs late on a list when it
is preceded by a lot of similar items. The earlier items cause
proactive (as opposed to retroactive) interference.
apple, pear, peach, orange, pineapple…
As the list increases in length, likelihood of remembering a
late-occurring item decreases, unless it is from a new
semantic category
apple, pear, peach, orange, fireman…
– The same release from PI occurs w/a language change
pear, peach, orange, pineapple, manzana…
Possible Evidence for Combined
Lexicon
• In comprehension, word-initial cohort
includes candidates from both languages
• In production, code-switching midsentence
– Just use the best word, regardless of
language?
– But only if your listener knows both
languages too!
• Lexical access vs. lexical selection
What about Cog-Neuro evidence?
• Patterns of aphasia in bilingual and
multi-lingual speakers
• Pre-operative brain stimulation
• Imaging (PET, fMRI) and ERP studies
Patterns of Recovery in Aphasia
Fabbro (1999)
• 40% L1 and L2 recover in
parallel
• 32% L1 > L2
• 28% L2 > L1
Pre-op
electrical
stimulation
(Ojemann)
Imaging Dutch-French-English trilinguals in Belgium (Vingerhoets et al., 2003)
•Picture naming, word
fluency and paragraph
comprehension tasks
•All tasks revealed
“predominantly
overlapping” regions
for the 3 languages
•L2’s show activation
in more areas and
“more extensive
recruitment” of areas
activated by L1
(Dutch)
Word Fluency Task: Covertly generate as many words
as possible beginning with a specified letter.
Lexical Access in Speaking
There is currently enthusiasm for single-store
models or partially-overlapping lexicons.
When preparing an utterance for output, how then
does the bilingual activate only words from a
single language?
*automatic, parallel access [concept – words]
*deliberate selection mechanism [best word]
Some insights from Bilinguals
• More evidence for separation of semantic
memory & lexicon
• More evidence for automaticity of lexical access
in both comprehension & production
• Distinction between lexical access & lexical
selection
– So we may activate physics words in history class,
and then filter them out
Brain potential and functional
MRI evidence for how to
handle two languages with
one brain
Rodriguez-Fornells et al