CS 182
Sections 103 - 104
Eva Mok (emok@icsi.berkeley.edu)
March 3, 2004
Some languages use an absolute orientation system, e.g.
1. Guugu Yimithirr (Cape York, Queensland, Australia)
2. Hai//om (Khoisan, Kalahari, Namibia, Africa)
3. Tzeltal (Mayan, Chiapas, Mexico)
They describe all relations with East, South, West, North
(e.g. the knife is south of the fork)
Imagine using this system:
The bathroom is ___________ of this classroom.
I am standing ___________ of you guys.
Announcements
• a4 is due on Friday.
• Midterm in class this Tuesday, March 7th
• Be there on time!
• Format:
– closed books, closed notes
– short answers, no blue books
– up to March 2th lecture
• Review Session: Monday, March 6th
Time TBA (check the website), Location TBA
Quick Recap
• Last Week
– Image Schemas and related experiments
• This Week
– Regier’s Model of learning spatial relation terms
• Coming up
– Motor control
Quiz!
1. What are image schemas? What are they useful for?
2. How many senses of the English on can you think of?
How would you distinguish them?
3. How is Regier’s model capable of learning without
explicit negative examples? How does it learn spatial
relation terms in different languages?
4. What mechanism do we use to link concepts
together? How does it work? What are other models
of learning?
Quiz!
1. What are image schemas? What are they useful for?
2. How many senses of the English on can you think of?
How would you distinguish them?
3. How is Regier’s model capable of learning without
explicit negative examples? How does it learn spatial
relation terms in different languages?
4. What mechanism do we use to link concepts
together? How does it work? What are other models
of learning?
Why image schemas?
• Different languages make different distinctions in
their use of spatial relation terms
• The weaker claim:
Image schemas give us a ‘vocabulary’ to talk about
the different dimensions of spatial structure that
languages care about
• The stronger claim:
These dimensions are embodied -- our bodies
constrain the way we observe and interact with the
world. Therefore these schemas are universal.
English
AROUND
ON
OVER
IN
Bowerman & Pederson
Dutch
OP
OM
ANN
BOVEN
IN
Bowerman & Pederson
Chinese
ZHOU
LI
SHANG
Bowerman & Pederson
The collection of image schemas
• Trajector / Landmark (asymmetric)
– The bike is near the house
–
?
The house is near the bike
TR
• Boundary / Bounded Region
– a bounded region has a closed boundary
LM
boundary
bounded region
• Topological Relations
– Separation, Contact, Overlap, Inclusion, Surround
• Orientation
– Vertical (up/down), Horizontal (left/right, front/back)
– Absolute (E, S, W, N)
More image schemas
• Proximal / Distal
– distance from center (near/far)
• Part / Whole
– top of the hill, cover of the magazine
• Container
– interior, exterior, boundary, portal
• Source-Path-Goal
– source, path, goal, trajector
• Force-Dynamics
– support, force
S
TR
P
G
Quiz!
1. What are image schemas? What are they useful for?
2. How many senses of the English on can you think of?
How would you distinguish them?
3. How is Regier’s model capable of learning without
explicit negative examples? How does it learn spatial
relation terms in different languages?
4. What mechanism do we use to link concepts
together? How does it work? What are other models
of learning?
The English ‘on’
1. The computer is on the desk
2. The picture is on the wall
3. The projector is on the ceiling
UP
TR
LM
DN
TR/LM, verticality,
contact, support
TR
LM
TR
LM
TR/LM, contact,
attaching force
TR/LM, contact,
attaching force
Writing out the on construction
SCHEMA TR-LM
ROLES
tr: trajector
lm: landmark
SCHEMA Contact
ROLES
r1: bounded-region
r2: region
SCHEMA Support
ROLES
supporter: bounded-region
supportee: bounded-region
The first sense of on
CONSTRUCTION On
FORM: Word
self.f.orth  “on”
MEANING
EVOKES TR-LM as trlm
EVOKES Contact as c
EVOKES Support as s
trlm.tr  c.r1
trlm.lm  c.r2
trlm.tr  s.supportee
trlm.lm  s.supporter
Quiz!
1. What are image schemas? What are they useful for?
2. How many senses of the English on can you think of?
How would you distinguish them?
3. How is Regier’s model capable of learning without
explicit negative examples? How does it learn spatial
relation terms in different languages?
4. What mechanism do we use to link concepts
together? How does it work? What are other models
of learning?
Language Development in Children
• 0-3 mo: prefers sounds in native language
• 3-6 mo: imitation of vowel sounds only
• 6-8 mo: babbling in consonant-vowel segments
• 8-10 mo: word comprehension, starts to lose sensitivity to
consonants outside native language
• 12-13 mo: word production (naming)
• 16-20 mo: word combinations, relational words (verbs, adj.)
• 24-36 mo: grammaticization, inflectional morphology
• 3 years – adulthood: vocab. growth, sentence-level grammar
for discourse purposes
Regier’s Model
above below
left
right
in
out
on
off
Learning System
TR
Input:
above
LM
• Training input: configuration of TR/LM and the correct
spatial relation term
• Learned behavior: input TR/LM, output spatial relation
Issue #1: Implicit Negatives
• Children usually do not get explicit negatives
• But we won’t know when to stop generalizing if we don’t
have negative evidence
• Yet spatial relation terms aren’t entirely mutually exclusive
• The same scene can often be described with two or more
spatial relation terms (e.g. above and outside)
• How can we make the learning problem realistic yet
learnable?
Dealing with Implicit Negatives
• Explicit positive for above
• Implicit negatives for below, left, right, etc
• in Regier:
E = ½ ∑i,p (( ti,p – oi,p) * βi,p )2,
where i is the node, p is the pattern,
βi,p = 1 if explicit positive,
βi,p < 1 if implicit negative
Issue #2: Shift Invariance
• Backprop cannot handle shift invariance (it cannot
generalize from 0011, 0110 to 1100)
• But the cup is on the table whether you see it right in the
center or from the corner of your eyes (i.e. in different
areas of the retina map)
• What structure can we utilize to make the input shiftinvariant?
Learning System
dynamic relations
(e.g. into)
structured connectionist
network (based on
visual system)
We’ll look at the
details next lecture
Quiz!
1. What are image schemas? What are they useful for?
2. How many senses of the English on can you think of?
How would you distinguish them?
3. How is Regier’s model capable of learning without
explicit negative examples? How does it learn spatial
relation terms in different languages?
4. What mechanism do we use to link concepts
together? How does it work? What are other models
of learning?
Models of Learning
• Hebbian – coincidence
• Reinforcement – delayed reward
• Recruitment – one-trial
• Supervised – correction (backprop)
• Unsupervised – similarity
Elman Nets & Jordan Nets
Output
1
Output
1
Hidden
Context
Input
α
Hidden
Context
Input
• Basically the same recurrent nets we mentioned a few weeks ago
• Updating the context as we receive input
• In Jordan nets we model “forgetting” as well
• The recurrent connections have fixed weights
• You can train these networks using good ol’ backprop
Recurrent Backprop
w2
a
w4
b
w1
c
w3
unrolling
3 iterations
a
b
c
a
b
c
a
b
c
w1
a
w2 w3
b
w4
c
• we’ll pretend to step through the network one iteration
at a time
• backprop as usual, but average equivalent weights (e.g.
all 3 highlighted edges on the right are equivalent)
The Idea of Recruitment Learning
K
Y
• Suppose we want to
link up concept X to
concept Y
• The idea is to pick the
two nodes in the
middle to link them up
X
N
B
F = B/N
Pno link  (1  F )
• Can we be sure that we
can find a path to get
from X to Y?
BK
the point is, with a fan-out of 1000,
if we allow 2 intermediate layers,
we can almost always find a path
Remember triangle nodes?
• Let’s say we are trying to remember a green circle
• currently weak connections between concepts (dotted lines)
has-color
blue
has-shape
green
round
oval
Strengthen these connections
• and you end up with this picture
has-color
has-shape
Green
circle
blue
green
round
oval
Demo of the
Regier System
on the English above
above – positive examples
above – negative examples
above – after training
above – test examples