Abstract Neuron
output y
y {1 if net > 0
0 otherwise
n
net   wiii
i 0
w0
i0=1
w1
i1
w2
i2
wn
...
input i
in
Computing with Abstract Neurons

McCollough-Pitts Neurons were initially
used to model

pattern classification


linking classified patterns to behavior



size = small AND shape = round AND color = green AND Location =
on_tree => Unripe_fruit
size = large OR motion = approaching => move_away
size = small AND location = above => move_above
McCollough-Pitts Neurons can compute
logical functions.

AND, NOT, OR
Computing logical functions: the OR
function
i1
i2
b=1
w01
w02
w0b
x0
f
y0
i1
i2
y0
0
0
0
0
1
1
1
0
1
1
1
1
• Assume a binary threshold activation function.
• What should you set w01, w02 and w0b to be so that
you can get the right answers for y0?
Many answers would work
y = f (w01i1 + w02i2 + w0bb)
i2
recall the threshold function
the separation happens when
w01i1 + w02i2 + w0bb = 0
i1
move things around and you get
i2 = - (w01/w02)i1 - (w0bb/w02)
Decision Hyperplane
The two classes are therefore separated
by the `decision' line which is defined by
putting the activation equal to the
threshold.
 It turns out that it is possible to generalise
this result to Threshold Units with n inputs.
 In 3-D the two classes are separated by a
decision-plane.
 In n-D this becomes a decision
Linearly separable patterns
Linearly Separable Patterns
PERCEPTRON is an architecture which can
solve this type of decision boundary problem.
An "on" response in the output node
represents one class, and an "off" response
represents the other.
The XOR function
i1
i2
y
0
0
0
1
0
1
1
1
0
1
1
0
The Input Pattern Space
The Decision planes
Multiple Layers
y
0.5
1
-1
0.5
1.5
1
1
1
I1
1
I2
Multiple Layers
y
0.5
1
-1
0.5
1.5
1
1
1
1
I1
I2
0
1
Multiple Layers
y
0.5
1
-1
0.5
1.5
1
1
1
1
I1
I2
1
1
Computing other relations
The 2/3 node is a useful function that
activates its outputs (3) if any (2) of its 3
inputs are active
 Such a node is also called a triangle node
and will be useful for lots of
representations.

Triangle nodes and
McCullough-Pitts Neurons?
Relation (A)
Object (B) Value (C)
A
B
C
“They all rose”
triangle nodes:
when two of the
abstract
neurons fire,
the third also
fires
model of
spreading
Basic Ideas behind the model
Parallel activation streams.
 Top down and bottom up activation
combine to determine the best matching
structure.
 Triangle nodes bind features of objects to
values
 Mutual inhibition and competition between
structures
 Mental connections are active neural

5 levels of Neural Theory of
Language
Pyscholinguistic
experiments
Cognition and Language
abstraction
Computation
Structured Connectionism
Triangle Nodes
Neural Net and
learning
Computational Neurobiology
Biology
Neural
Development
Quiz
Midterm
Finals
Psychological Studies
Eva Mok
CS182/CogSci110/Ling109
Spring 2006
Read the list
ORANGE
BROWN
GREEN
YELLOW
BLUE
RED
Name the print color
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
XXXXX
Name the print color
RED
GREEN
BLUE
BROWN
ORANGE
YELLOW
The Stroop Test
Form and meaning interact in
comprehension, production and learning
Top down and bottom up information

Bottom-up: stimulus driving processing

Top-down: knowledge and context driving
processing

When are these information integrated?

Modular view: Staged serial processing

Interaction view: Information is used as soon as
available
Tanenhaus et al. (1979) [also Swinney, 1979]
Word / non-word forced choice
Modeling the task with triangle nodes
Reaction times in milliseconds after:
“They all rose”
flower
0 delay
200ms. delay
685
659
(facilitation)
stood
677
(facilitation)
desk
(control)
711
(no facilitation)
623
(facilitation)
652
When is context integrated?

Prime: spoken sentences ending in homophones
They all rose
vs. They bought a rose

Probe: stood and flower

No offset: primes both stood and flower

200 ms offset: only primes appropriate sense

Modularity? Or weak contextual constraints?
Allopenna, Magnuson & Tanenhaus (1998)
Eye camera
Eye tracking
computer
Scene camera
“Pick up the beaker”
Adapted from Jim Magnuson, “Interaction in language processing: Pragmatic constraints on lexical access”
Do rhymes compete?

Cohort (Marlsen-Wilson): onset similarity is primary because of the
incremental (serial) nature of speech
 Cat activates cap, cast, cattle, camera, etc.
 Rhymes won’t compete

NAM (Neighborhood Activation Model; Luce): global similarity is
primary
 Cat activates bat, rat, cot, cast, etc.
 Rhymes among set of strong competitors

TRACE (McClelland & Elman): global similarity constrained by
incremental nature of speech
 Cohorts and rhymes compete, but with different time course
Adapted from Jim Magnuson, “Interaction in language processing: Pragmatic constraints on lexical access”
TRACE predicts different time course for
cohorts and rhymes
Adapted from Jim Magnuson, “Interaction in language processing: Pragmatic constraints on lexical access”
TRACE predictions match eye-tracking data
Adapted from Jim Magnuson, “Interaction in language processing: Pragmatic constraints on lexical access”
Natural contexts are used continuously

Conclusion from this and other eye-tracking studies:

When constraints from natural contexts are extremely
predictive, they are integrated as quickly as we can
measure
Suggests rapid, continuous interaction among


Linguistic levels

Nonlinguistic context

Even for processes assumed to be low-level and
automatic

Constrains processing theories, also has implications
for, e.g., learnability
Adapted from Jim Magnuson, “Interaction in language processing: Pragmatic constraints on lexical access”
Eye movement paradigm

More sensitive than conventional paradigms

More naturalistic

Simultaneous measures of multiple items

Transparently linkable to computational model
Adapted from Jim Magnuson, “Interaction in language processing: Pragmatic constraints on lexical access”
Eye-tracker without headsets
http://www.bcs.rochester.edu/infanteyetrack/eyetrack.html
Recap: Goals of psycholinguistic studies

Direct goal: finding out what affect sentence
processing

Indirect goal: getting at how words, syntax,
concepts are represented in the brain

Modeling: testing out these hypotheses with
computational models
Areas studied in psycholinguistics

Lexical access / lexical structure

Syntactic structure

Referent selection

The role of working memory

Disfluencies
Disfluencies and new information

Disfluencies: pause, repetition, restart

Often just seen as production / comprehension
difficulties

Arnold, Fagnano, and Tanenhaus (2003)

How are disfluent references interpreted?

Componenets to referent selection

lexical meaning

discourse constraints
Candle, camel, grapes, salt shaker
a. DISCOURSE-OLD CONTEXT:
DISCOURSE-NEW CONTEXT:
b. FLUENT:
DISFLUENT:
Put the grapes below the candle.
Put the grapes below the camel.
Now put the candle below the salt shaker.
Now put theee, uh, candle below the salt shaker.
Predictions on 4 conditions: (Target = candle)


Disfluent/New, Fluent/Given: Target

Put the grapes below the camel.
Now put theee, uh, candle below the salt shaker.

Put the grapes below the candle.
Now put the candle below the salt shaker.
Disfluent/Given, Fluent/New: Competitor

Put the grapes below the candle.
Now put theee, uh, candle below the salt shaker.

Put the grapes below the camel.
Now put the candle below the salt shaker.
Disfluencies affect what we look at
Percentage of fixations on all new objects from 200 to 500 ms after the
onset of “the”/“theee uh” (i.e. before the onset of the head noun)
Target is preferred in two conditions
Percentage of target fixations minus percentage competitor fixations in each
condition. Fixations cover 200–500 ms after the onset of the head noun.
A lot of information is integrated in
sentence processing!

Stroop test [i.e. color words]:
form, meaning

Tanenhaus et al (1997) [i.e. “they all rose”]:
phonology, meaning, syntactic category

Allopena et al (1998) [i.e. cohorts & rhymes]:
phonology, visual context

Arnold et al (2003) [i.e. “theee, uh, candle”]:
discourse information, visual context
Producing words from
pictures or from other words
A comparison of aphasic lexical access from two different
input modalities
Gary Dell
with
Myrna Schwartz, Dan Foygel, Nadine Martin, Eleanor Saffran, Deborah Gagnon, Rick
Hanley, Janice Kay, Susanne Gahl, Rachel Baron, Stefanie Abel, Walter Huber
A 2-step Interactive Model of Lexical Access
in Production
Semantic Features
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
1. Lemma Access:
Activate semantic features of CAT
Semantic Features
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
1. Lemma Access:
Activation spreads through network
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
1. Lemma Access:
Most active word from proper category is
selected and linked to syntactic frame
NP
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
N
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
2. Phonological Access:
Jolt of activation is sent to selected word
NP
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
N
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
2. Phonological Access:
Activation spreads through network
NP
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
N
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
2. Phonological Access:
Most activated phonemes are selected
Syl
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
On Vo Co
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
Modeling lexical access errors

Semantic error

Formal error (i.e. errors related by form)

Mixed error (semantic + formal)

Phonological access error
Semantic error: Shared features
activate semantic neighbors
NP
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
N
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
Formal error: Phoneme-word feedback
activates formal neighbors
NP
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
N
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
Mixed error: neighbors activated by
both top-down & bottom-up sources
NP
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
N
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
Phonological access error: Selection of
incorrect phonemes
Syl
FOG
f
r
d
Onsets
DOG
k
CAT
m
ae
RAT
MAT
o
t
Vowels
On Vo Co
g
Codas
Adapted from Gary Dell, “Producing words from pictures or from other words”
I’ve shown you...

Behavioral experiments, and

A connectionist model

with the goal of understanding how language is
represented and processed in the brain

Next time:
Lisa will talk about imaging experiments