The ICSI/Berkeley Neural Theory of Language Project
ECG
Learning early
constructions
(Chang, Mok)
Connectionist Model of
Word Recognition (Rumelhart
and McClelland)
Constraints on Connectionist Models
100 Step Rule
Human reaction times ~ 100 milliseconds
Neural signaling time ~ 1 millisecond
Simple messages between neurons
Long connections are rare
No new connections during learning
Developmentally plausible
Can we formalize/model these intuitions
What is a neurally plausible computational
model of spreading activation that
captures these features.
 What does semantics mean in neurally
embodied terms

 What
are the neural substrates of concepts
that underlie verbs, nouns, spatial predicates?
Abstract Neuron
output y
y {1 if net > 0
0 otherwise
n
net   wiii
i 0
w0
I0 = 1
w1
i1
w2
i2
wn
...
input i
in
Computing with Abstract Neurons

McCollough-Pitts Neurons were initially used to
model
 pattern classification
 size = small AND shape = round AND color = green AND
location = on_tree => unripe
 linking classified patterns to behavior
 size = large OR motion = approaching => move_away
 size = small AND direction = above => move_above

McCollough-Pitts Neurons can compute logical
functions.
 AND,
NOT, OR
Distributed vs Localist Rep’n
John
1
1
0
0
John
1
0
0
0
Paul
0
1
1
0
Paul
0
1
0
0
George
0
0
1
1
George
0
0
1
0
Ringo
1
0
0
1
Ringo
0
0
0
1
What are the drawbacks of each
representation?
Distributed vs Localist Rep’n


John
1
1
0
0
John
1
0
0
0
Paul
0
1
1
0
Paul
0
1
0
0
George
0
0
1
1
George
0
0
1
0
Ringo
1
0
0
1
Ringo
0
0
0
1
What happens if you want to
represent a group?
How many persons can you
represent with n bits? 2^n


What happens if one neuron
dies?
How many persons can you
represent with n bits? n
Sparse Distributed Representation
Visual System


1000 x 1000 visual map
For each location,
encode:
orientation
direction
…
of motion
speed
size
color
depth

Blows up
combinatorically!
…
Coarse Coding
info you can encode with one fine resolution unit =
info you can encode with a few coarse resolution
units
Now as long as we need fewer coarse units total,
we’re good
Coarse-Fine Coding
Coarse in
F2,
Fine in F1
Feature 1
e.g.
Orientation
Y-Orientation
Y
G
X-Orientation
G
X
Y-Dir
X-Dir
Coarse in
F1,
Fine in F2
but we can
run into
ghost
“images”
Feature 2
e.g. Direction of Motion
Connectionist Models in Cognitive Science
Structured
PDP
Hybrid
Neural
Conceptual
Existence
Data Fitting
Computing other relations
The 2/3 node is a useful function that
activates its outputs (3) if any (2) of its 3
inputs are active
 Such a node is also called a triangle node
and will be useful for lots of
representations.

Triangle nodes and
McCullough-Pitts Neurons?
A
B
C
A
B
C
“They all rose”
triangle nodes:
when two of the
neurons fire, the
third also fires
model of
spreading
activation
Spreading activation and feature
structures





Parallel activation streams.
Top down and bottom up activation combine to
determine the best matching structure.
Triangle nodes bind features of objects to values
Mutual inhibition and competition between
structures
Mental connections are active neural
connections
Representing concepts using
triangle nodes
Feature Structures in Four Domains
Barrett
Ham
Container
Push
dept~CS
Color ~pink
Inside ~region
Schema ~slide
sid~001
Taste ~salty
Outside ~region
Posture ~palm
Bdy. ~curve
Dir. ~ away
emp~GSI
Schneider
Pea
Purchase
Stroll
dept~Ling
Color ~green
Buyer ~person
Schema ~walk
sid~002
Taste ~sweet
Seller ~person
Speed ~slow
Cost ~money
Dir. ~ ANY
emp~Gra
Goods ~ thing
Categories and conceptsintroduction
CS182/Ling109/CogSci110
Spring 2008
Lecture Outline

Categories
 Basic
Level
 Prototype Effects
 Neural Evidence for Category Structure


Aspects of a Neural Theory of concepts
Image Schemas
 Description
and types
 Behavioral Experiment on Image Schemas

Event Structure and Motor Schemas
The WCS Color Chips

Basic color terms:
 Single
word (not blue-green)
 Frequently used (not mauve)
 Refers primarily to colors (not lime)
 Applies to any object (not blonde)
Concepts

What Concepts Are: Basic Constraints
 Concepts
are the elements of reason, and
 constitute the meanings of words and
linguistic expressions.
Concepts Are:
•Universal: they characterize all particular
instances; e.g., the concept of grasping is the
same no matter who the agent is or what the
patient is or how it is done.
•Stable.
•Internally structured.
•Compositional.
•Inferential. They interact to give rise to inferences.
•Relational. They may be related by hyponymy,
antonymy, etc.
•Meaningful.
•Not tied to the specific word forms used to express
them.
Concepts: Traditional Theory

The Traditional Theory
 Reason
and language are what distinguish human
beings from other animals.
 Concepts therefore use only human-specific brain
mechanisms.
 Reason is separate from perception and action, and
does not make direct use of the sensory-motor
system.
 Concepts must be “disembodied” in this sense.
The neural theory
Human concepts are embodied. Many
concepts make direct use of sensorymotor, emotional, and social cognition
capacities of our body-brain system.
 Many of these capacities are also present
in non-human primates.
 Continuity Principle of Am. Pragmatists
Classical vs prototype model of
categorization

Classical model
 Category membership determined on basis of
essential features
 Categories have clear boundaries
 Category features are binary

Prototype model
 Features that frequently co-occur lead to
establishment of category
 Categories are formed through experience with
exemplars
Prototype theory
1.
2.
3.
4.
5.
6.
Certain members of a category are prototypical – or
instantiate the prototype
Categories form around prototypes; new members
added on basis of resemblance to prototype
No requirement that a property or set of properties be
shared by all members
Features/attributes generally gradable
Category membership a matter of degree
Categories do not have clear boundaries
Prototype theory
1.
Certain members of a category are prototypical – or
instantiate the prototype
Category members are not all equal
a robin is a prototypical bird, but we may not want to say it is the
prototype, rather it instantiates (manifests) the prototype or ideal -it exhibits many of the features that the abstract prototype does
“It is conceivable that the prototype for dog will be unspecified for
sex; yet each exemplar is necessarily either male or female.”
(Taylor)
Prototype theory
3.
No requirement that a property or set of properties be
shared by all members -- no criterial attributes


Category where a set of necessary and sufficient attributes can
be found is the exception rather than the rule
Labov household dishes experiment


Necessary that cups be containers, not sufficient since many
things are containers
Cups can’t be defined by material used, shape, presence of
handles or function
Prototype theory
 Wittgenstein’s
examination of game
Generally necessary that all games be amusing,
not sufficient since many things are amusing
 Board games, ball games, card games, etc. have
different objectives, call on different skills and
motor routines

- categories normally not definable in terms
of necessary and sufficient features
Prototype theory

What about mathematical categories like odd or even
numbers? Aren’t these sharply defined?

(Armstrong et al.) Subjects asked to assign numbers a degree
of membership to the categories odd number or even number
 3 had a high degree of membership, 447 and 91 had a lower
degree (all were rated at least ‘moderately good’)
Categories - who decides?
Embodied theory of meaning- categories
are not pre-formed and waiting for us to
behold them. Our need for categories
drives what categories we will have
 Basic level categories - not all categories
have equal status. The basic level
category has demonstrably greater
psychological significance.

Basic-level categories
furniture
Superordinate
chair
desk chair
easy chair
rocking chair
lamp
desk lamp
floor lamp
table
dining room table
coffee table
Basic
Subordinate
Categories & Prototypes:
Overview
Superordinate
Furniture
Sofa
leather
sofa

fabric
sofa
Basic-Level Category
Desk
L-shaped
desk
Reception
disk
Subordinate
Three ways of examining the categories we form:



relations between categories (e.g. basic-level category)
internal category structure (e.g. radial category)
instances of category members (e.g. prototypes)
Basic-level -- Criteria

Perception –
 overall
perceived shape
 single mental image
 fast identification
Basic-level -- Criteria
Perception
 Function – motor program for interaction

Basic-level -- Criteria
Perception
 Function
 Words –


shortest
 first learned by children
 first to enter lexicon
Basic-level -- Criteria
Perception
 Function
 Communication
 Knowledge organization –


most attributes are stored at this level
Basic-Level Category
What constitutes a basic-level category?

Perception:



similar overall
perceived shape
 single mental image
 (gestalt perception)
 fast identification





Function:

general motor
program
Communication:

shortest
most commonly used
contextually neutral
first to be learned by
children
first to enter the lexicon
Knowledge Organization:

most attributes of category
members stored at this level
Other Basic-level categories
Objects
 Colors
 Motor-routines

Concepts are not categorical
Mother

The birth model
The person who gives birth is the mother

The genetic model
The female who contributes the genetic material is the mother

The nurturance model
The female adult who nurtures and raises a child is the mother of the
child

The marital model
The wife of the father is the mother

The genealogical model
The closest female ancestor is the mother
(WFDT Ch.4, p.74, p.83)
Radial Structure of Mother
Genetic
mother
Stepmother
Unwed
mother
Surrogate
mother
Biological
mother
Adoptive
mother
Central
Case
Foster
mother
Birth
mother
Natural
mother
The radial structure of this category is defined
with respect to the different models
Marriage

What is a marriage?

What are the frames (or models) that go into
defining a marriage?

What are prototypes of marriage?

What metaphors do we use to talk about
marriages?

Why is this a contested concept right now?
Concepts and radial categories
Concepts can get to be the "prototype" of their category in
various ways.
 Central subcategory (others relate to this)



Essential (meets a folk definition: birds have feathers,
beaks, lay eggs)




Move involves change of location.
Typical case (most are like this: "sparrow")


Amble and swagger relate to WALK
Shove relates to PUSH
Going to a conference involves air travel.
Ideal/anti-ideal case (positive social standard: "parent");
anti-ideal case (negative social standard: "terrorist")
Stereotype (set of attributes assumed in a culture:
"Arab")
Salient exemplar (individual chosen as example)
Category Structure

Classical Category:


necessary and sufficient conditions
Radial Category:

a central member branching out to less-central and non-central cases
 degrees of membership, with extendable boundary

Family Resemblance:

every family member looks like some other family member(s)
 there is no one property common across all members (e.g. polysemy)


Prototype-Based Category
Essentially-Contested Category (Gallie, 1956) (e.g.
democracy)

Ad-hoc Category (e.g. things you can fit inside a shopping bag)
Prototype

Cognitive reference point




standards of comparison

Social stereotypes


snap judgments
 defines cultural expectations
 challengeable




an individual member that
exhibits the ideal
Salient examples


e.g. ideal vacation
can be abstract
may be neither typical nor
stereotypical
Paragons / Anti-paragons

Typical case prototypes
default expectation
 often used unconsciously in
reasoning
Ideal case / Nightmare case
e.g. 9/11 – terrorism act
Generators

central member + rules
 e.g. natural number = singledigit numbers + arithmetic
Neural Evidence for category
structure

Are there specific regions in the brain to
recognize/reason with specific categories?
Category Naming and Deficits
People with brain injury have selective
deficits in their knowledge of categories.
 Some patients are unable to identify or
name man made objects and others may
not be able to identify or name natural
kinds (like animals)

A PET Study on categories (Nature
1996)
Study

16 adults (8M, 8F) participated in a PET (positron
emission tomography) study.




Involves injecting subject with a positron emitting radioactive
substance (dye)
Regions with more metabolic activity will absorb more of the
substance and thus emit more positrons
Positron-electron collisions yield gamma rays, which are
detected
Increased rCBF (regional changes in cerebral blood
flow) was measured

When subjects viewed line drawings of animals and tools.
The experiment





Subjects looked at pictures of animals and tools
and named them silently.
They also looked at noise patterns (baseline 1)
And novel nonsense objects (baseline 2)
Each stimulus was presented for 180ms
followed by a fixation cross of 1820 ms.
Drawings were controlled for name frequency
and category typicality
medial
lateral
Left middle temporal gyrus
ACC
Premotor
Calcarine Sulcus
Conclusions




Both animal and tool naming activate the ventral
temporal lobe region.
Tools differentially activate the ACC, pre-motor
and left middle temporal region (known to be
related to processing action words).
Naming animals differentially activated left
medial occipital lobe (early visual processing)
The object categories appear to be in a
distributed circuit that involves activating
different salient aspects of the category.