Unified Cognitive Science
• Neurobiology
• Psychology
• Computer Science
• Linguistics
• Philosophy
• Social Sciences
• Experience
Take all the Findings and Constraints Seriously
What are schemas?
•
•
•
Regularities in our perceptual, motor and
cognitive systems
Structure our experiences and interactions
with the world.
May be grounded in a specific cognitive
system, but are not situation-specific in their
application (can apply to many domains of
experience)
Basis of Image
schemas
• Perceptual systems
• Motor routines
• Social Cognition
• Image Schema properties depend on
• Neural circuits
• Interactions with the world
Image schemas
•
Trajector / Landmark (asymmetric)
• The bike is near the house
• ? The house is near the bike
•
Boundary / Bounded Region
•
Topological Relations
•
Orientation
• a bounded region has a closed boundary
TR
LM
boundary
bounded region
• Separation, Contact, Overlap, Inclusion, Surround
• Vertical (up/down), Horizontal (left/right, front/back)
• Absolute (E, S, W, N)
Similarity:
• Perceptual and motor systems
• Basic functional interactions with the world
• Environment
Variation:
Cross-linguistic variation in how schemas are
used.
Cross-linguistic
Variations
English
Japanese
English
AROUND
ON
OVER
IN
Bowerman & Pederson
Dutch
OP
OM
AAN
BOVEN
IN
Bowerman & Pederson
Chinese
ZHOU
LI
SHANG
Bowerman & Pederson
Spatial schemas
• TR/LM relation
• Boundaries, bounded region
• Topological relations
• Orientational Axes
• Proximal/Distal
Trajector/Landmark
Schema
• Roles:
Trajector (TR) – object being located
Landmark (LM) – reference object
TR and LM may share a location (at)
TR/LM -- asymmetry
• The cup is on the table
• ?The table is under the cup.
• The skateboard is next to the post.
• ?The post is next to the skateboard.
Boundary Schema
Region A
Region B
Boundary
Roles:
Boundary
Region A
Region B
Topological Relations
• Separation
Topological Relations
• Separation
• Contact
Topological Relations
• Separation
• Contact
• Coincidence:
Topological Relations
• Separation
• Contact
• Coincidence:
- Overlap
Topological Relations
• Separation
• Contact
• Coincidence:
- Overlap
- Inclusion
Topological Relations
• Separation
• Contact
• Coincidence:
- Overlap
- Inclusion
- Encircle/surround
Orientation
• Vertical axis -- up/down
up
above
upright
below
down
Orientation
Horizontal plane – Two axes:
Language and Frames
of Reference
• There seem to be three prototypical
frames of reference in language
(Levinson)
• Intrinsic
• Relative
• Absolute
Intrinsic frame of
reference
left
back
front
right
Relative frame of
reference
right??
back
front
left??
Absolute frame of
reference
west
south
north
east
TR/LM and Verticality Schemas
• The book is under the table.
up
down
under
Proximal/Distal
Schema
.
Simple vs. Complex
Schemas
Container Schema
• Roles:
• Interior: bounded region
• Exterior
• Boundary
C
TR/LM + Container
out
in
TR
C
TR C
Container Schema
Elaborated
• Complexities –more
roles/specifications:
• Boundary properties
• Strength
• Porosity
• Portals
Source-Path-Goal
Constraints:
initial = TR at Source
central = TR on Path
final
Source
= TR at Goal
Path
Goal
SPG -- simple example
She drove from the store to the gas station.
TR = she
Source = the store
Goal = the gas station
Source
Path
Goal
SPG and Container
She ran into the room.
SPG. Source ↔ Container.Exterior
SPG.Path ↔ Container.Portal
SPG. Goal ↔ Container.Interior
PATH landmarks
past
across
along
LM
LM
LM
Part-Whole Schema
Part
Whole
Representing image
schemas
semantic schema Source-Path-Goal
roles:
source
path
goal
trajector
semantic schema Container
roles:
interior
exterior
portal
boundary
Boundary
Source
Trajector
Goal
Interior
Portal
Path
Exterior
These are abstractions over sensorimotor experiences.
Language and Spatial Schemas
•
•
•
•
People say that they look up to some people, but look down on
others because those we deem worthy of respect are somehow
“above” us, and those we deem unworthy are somehow “beneath”
us.
But why does respect run along a vertical axis (or any spatial axis,
for that matter)? Much of our language is rich with such spatial talk.
Concrete actions such as a push or a lift clearly imply a vertical or
horizontal motion, but so too can more abstract concepts.
Metaphors: Arguments can go “back and forth,” and hopes can get
“too high.”
Regier Model Lecture
Jerome A. Feldman
February 27, 2007
With help from Matt Gedigian
Neural Theory of Language
Language Development in Children
• 0-3 mo: prefers sounds in native language
• 3-6 mo: imitation of vowel sounds only
• 6-8 mo: babbling in consonant-vowel segments
• 8-10 mo: word comprehension, starts to lose sensitivity to
consonants outside native language
• 12-13 mo: word production (naming)
• 16-20 mo: word combinations, relational words (verbs, adj.)
• 24-36 mo: grammaticization, inflectional morphology
• 3 years – adulthood: vocab. growth, sentence-level
grammar for discourse purposes
Trajector/Landmark
Schema
• Roles:
Trajector (TR) – object being located
Landmark (LM) – reference object
TR and LM may share a location (at)
TR/LM -- asymmetry
• The cup is on the table
• ?The table is under the cup.
• The skateboard is next to the post.
• ?The post is next to the skateboard.
Language and Frames of
Reference
• There seem to be three prototypical frames of
reference in language (Levinson)
• Intrinsic
• Relative
• Absolute
Basis of Image Schemas
• Perceptual systems
• Motor routines
• Social Cognition
• Image Schema properties depend on
• Neural circuits
• Interactions with the world
Image schemas
•
•
Trajector / Landmark (asymmetric)
• The bike is near the house
• ? The house is near the bike
Boundary / Bounded Region
•
TR
LM
boundary
bounded region
bounded region has a closed boundary
•
Topological Relations
•
Orientation
• Separation, Contact, Overlap, Inclusion, Surround
• Vertical (up/down), Horizontal
• Absolute (E, S, W, N)
Spatial schemas
• TR/LM relation
• Boundaries, bounded region
• Topological relations
• Orientational Axes
• Proximal/Distal
Regier’s Model
above below
left
right
in
out
on
off
Learning System
TR
Input:
above
LM
• Training input: configuration of TR/LM and the correct
spatial relation term
• Learned behavior: input TR/LM, output spatial relation
Issue #1: Implicit Negatives
•
•
•
•
•
Children usually do not get explicit negatives
But we won’t know when to stop generalizing if we don’t
have negative evidence
Yet spatial relation terms aren’t entirely mutually
exclusive
The same scene can often be described with two or
more spatial relation terms (e.g. above and outside)
How can we make the learning problem realistic yet
learnable?
Dealing with Implicit Negatives
•
•
•
Explicit positive for above
Implicit negatives for below, left, right, etc
in Regier:
E = ½ ∑i,p (( ti,p – oi,p) * βi,p )2,
where i is the node, p is the pattern,
βi,p = 1 if explicit positive,
βi,p < 1 if implicit negative
Learning
System
dynamic relations
(e.g. into)
structured connectionist
network (based on
visual system)
Topological Relations
• Separation
• Contact
• Coincidence:
- Overlap
- Inclusion
- Encircle/surround
Issue #2: Shift Invariance
•
•
•
Backprop cannot handle shift invariance (it cannot
generalize from 0011, 0110 to 1100)
But the cup is on the table whether you see it right
in the center or from the corner of your eyes (i.e. in
different areas of the retina map)
What structure can we utilize to make the input
shift-invariant?
Limitations
• Scale
• Uniqueness/Plausibility
• Grammar
• Abstract Concepts
• Inference
• Representation
Language and
Thought
Language
•
•
Thought
cognitive processes
•
•
We know thought (our cognitive
processes) constrains the way
we learn and use language
Does language also influence
thought?
Benjamin Whorf argues yes
Psycholinguistics experiments
have shown that linguistics
categories influence thinking
even in non-linguistics task