Nature requires Nurture

Initial wiring is genetically controlled
 Sperry

Experiment
But environmental input critical in early
development

Occular dominance columns

Hubel and Wiesel experiment
Innervation of the Optic tectum

Ganglion Cells in the frog retina (maps particular regions in space) map
systematically to cells in the optic tectum (movement in specific directions).



The retinal cells “see” in a direction and map to tectal cells which cause the frog
to move in that direction
The image of the external stimulus is inverted in the retina and the mapping
from the retina to the optic tectum reverts to the original image.
The Nasal ganglion cells of the retina map to the posterior region of the
Optic tectum and the temporal ganglion cells map to the anterior region of
the tectum
Sperry’s experiment



Sperry took advantage of the fact that in
amphibians, the optic nerve will regrow after it
has been interrupted
Sperry cut the optic nerve and simultaneously
rotated the eye 180 degrees in the eye socket.
In 'learning’ movements to catch prey, the part of
the retina now looking forward (backward)
should connect to the part of the brain which
causes forward (backward) movement.
Sperry’s findings
After regeneration,
 his animals responded to prey items in
front by turning around and
 to prey items behind by moving forward.
and
 kept doing this even though they never
succeeded in reaching the prey.

Conclusion from experiment
The conclusion from this (and some
supporting experiments) is
 that the pattern of connections between
retina and tectum, and the movement
information represented is not based on
experience.
 It is innate based on the initial distribution
of chemical markers in the brain.

Lecture Overview
Summary Overview
 Development from embryo
 Initial wiring
 Activity dependent fine tuning

The role of the environment

The development of ocular dominance columns


Retinal input connects to the LGN (Thalamus)


Cat and later monkey (Hubel and Wiesel)
LGN is composed of layers. Each layer receives input (axons)
from a single eye
LGN connects to layer IV of the visual cortex


The visual cortex develops ocular dominance columns
Cells that are connected to similar layers in the LGN get stacked
together in columns forming stripes.
http://neuro.med.harvard.edu/site/dh/
LGN
VISUAL CORTEX
Monocular deprivation critical
period



Hubel and Wiesel deprived one of the eyes of
the cat (later macaque monkey) at various times
1 week – 12 weeks (in the monkey case) 4
weeks – 4 months (for the cat).
The found that the ocular dominance cell
formation was most severely degraded if
deprivation occurred at 1 – 9 weeks after birth.
Deprivation after the plastic period had no longterm effect.
Cat Striate Cortex Layer IV
CLOSED
EYE
C
OPEN
EYE
C
I
I
2
1
C
I
C
3
I
4
Monkey Striate Cortex Area 17 (V1) Layer IV
C
C
I
I
6
5
C
I
7
Critical Periods in Development
There are critical periods in development
(pre and post-natal) where stimulation is
essential for fine tuning of brain
connections.
 Other examples of columns

 Orientation
columns
Pre-Natal Tuning: Internally
generated tuning signals

But in the womb, what provides the feedback to establish which
neural circuits are the right ones to strengthen?

Not a problem for motor circuits - the infant moves its limbs to refine the
feedback and control networks.
 But there is no vision in the womb.



--Systematic moving patterns of activity are spontaneously generated prenatally in the retina.
A predictable pattern, changing over time, provides excellent training data for
tuning the connections between visual maps.
The pre-natal development of the auditory system

Research indicates that infants, immediately after birth, preferentially
recognize the sounds of their native language over others. The
assumption is that similar activity-dependent tuning mechanisms work
with speech signals perceived in the womb.
Post-natal environmental tuning

The pre-natal tuning of neural connections using
simulated activity can work quite well –
a
newborn colt or calf is essentially functional at birth.


This is necessary because the herd is always on the move.
For many animals, including people, experience is
absolutely necessary for normal development (as in
the kitten experiment).

For a similar reason, if a human child has one weak eye, the
doctor will sometimes place a patch over the stronger one,
forcing the weaker eye to gain experience.
Adult Plasticity and Regeneration
The brain has an amazing ability to reorganize itself
through new pathways and connections rapidly.
• Through Practice:
• London cab drivers, motor regions for the skilled
• After damage or injury
• Undamaged neurons make new connections and take
over functionality or establish new functions
• But requires stimulation (phantom limb sensations)
• Stimulation standard technique for stroke victim
rehabilitation
When nerve stimulation changes, as with amputation, the brain
reorganizes. In one theory, signals from a finger and thumb of an
uninjured person travel independantly to separate regions in the brain's
thalamus (left). After amputation, however, neurons that formerly
responded to signals from the finger respond to signals from the thumb
(right).
Possible explanation for the
recovery mechanism




The initial pruning of connections leaves some
redundant connections that are inhibited by the
more active neural tissue.
When there is damage to an area, the lateral
inhibition is removed and the redundant
connections become active
The then can undergo activity based tuning
based on stimulation.
Great area for research.
Summary

Both genetic factors and activity dependent
factors play a role in developing the brain
architecture and circuitry.
 There
are critical developmental periods where
nurture is essential, but there is also a great ability for
the adult brain to regenerate.


Next: What computational models satisfy some
of the biological constraints.
Question: What is the relevance of development
and learning in language and thought?
Connectionist
Models: Basics
Srini Narayanan
CS182/CogSci110/Ling109
Spring 2006
Neural networks abstract from
the details of real neurons
Conductivity delays are neglected
 An output signal is either discrete (e.g.,
0 or 1) or it is a real-valued number
(e.g., between 0 and 1)
 Net input is calculated as the weighted
sum of the input signals
 Net input is transformed into an output
signal via a simple function (e.g., a
threshold function)

The McCullough-Pitts Neuron
yj
wij
xi
f
yi
ti : target
xi = ∑j wij yj
yi = f(xi – qi)
Threshold
yj: output from unit j
Wij: weight on connection from j to i
xi: weighted sum of input to unit i
Mapping from neuron
Nervous System
Computational Abstraction
Neuron
Node
Dendrites
Input link and propagation
Cell Body
Axon
Combination function,
threshold, activation function
Output link
Spike rate
Output
Synaptic strength
Connection strength/weight
Simple Threshold Linear Unit
Simple Neuron Model
1
A Simple Example
a = x1w1+x2w2+x3w3... +xnwn
.
a= 1*x1 + 0.5*x2 +0.1*x3
x1 =0, x2 = 1, x3 =0
Net(input) = f = 0.5
Threshold bias = 1
Net(input) – threshold bias< 0
Output = 0
Simple Neuron Model
1
1
1
1
Simple Neuron Model
1
1
1
1
1
Simple Neuron Model
0
1
1
1
Simple Neuron Model
0
1
1
1
0
Different Activation Functions
BIAS UNIT
With X0 = 1
Threshold Activation Function (step)
 Piecewise Linear Activation Function
 Sigmoid Activation Funtion
 Gaussian Activation Function

 Radial
Basis Function
Types of Activation functions
The Sigmoid Function
y=a
x=neti
The Sigmoid Function
Output=1
y=a
Output=0
x=neti
The Sigmoid Function
Output=1
Sensitivity to input
y=a
Output=0
x=neti
Changing the exponent k(neti)
K >1
K<1
Radial Basis Function
f ( x)  e
 ax 2
Stochastic units

Replace the binary threshold units by binary
stochastic units that make biased random
decisions.
 The
“temperature” controls the amount of
noise
p( si 1)

1 e
1
  s j wij
j
T
temperature
Types of Neuron parameters




The form of the function - e.g. linear, sigma-pi,
cubic.
The activation-output relation - linear, hardlimiter, or sigmoidal.
The nature of the signals used to communicate
between nodes - analogue or boolean.
The dynamics of the node - deterministic or
stochastic.
Computing other functions

McCollough-Pitts Neurons can compute
logical functions.
 AND,
NOT, OR
Computing other functions: the OR function
i1
i2
b=1
w01
w02
w0b
x0
f
y0
i1
i2
y0
0
0
0
0
1
1
1
0
1
1
1
1
• Assume a binary threshold activation function.
• What should you set w01, w02 and w0b to be so that
you can get the right answers for y0?
Many answers would work
y = f (w01i1 + w02i2 + w0bb)
i2
recall the threshold function
the separation happens when
w01i1 + w02i2 + w0bb = 0
i1
move things around and you get
i2 = - (w01/w02)i1 - (w0bb/w02)
Decision Hyperplane




The two classes are therefore separated by the
`decision' line which is defined by putting the
activation equal to the threshold.
It turns out that it is possible to generalise this
result to TLUs with n inputs.
In 3-D the two classes are separated by a
decision-plane.
In n-D this becomes a decision-hyperplane.
Linearly separable patterns
Linearly Separable Patterns
PERCEPTRON is an architecture which can
solve this type of decision boundary problem.
An "on" response in the output node
represents one class, and an "off" response
represents the other.
The XOR Function
X1/X2
X2 = 0
X2 = 1
X1= 0
0
1
X1 = 1
1
0
The Input Pattern Space
The Decision planes
Computing other relations
The 2/3 node is a useful function that
activates its outputs (3) if any (2) of its 3
inputs are active
 Such a node is also called a triangle node
and will be useful for lots of
representations.

Triangle nodes and
McCullough-Pitts Neurons?
A
B
C
A
B
C
Representing concepts using
triangle
triangle nodes
nodes:
when two
of the
neurons
fire, the
third also
fires
“They all rose”
triangle nodes:
when two of the
neurons fire, the
third also fires
model of
spreading
activation
Link to Vision: The Necker Cube
Basic Ideas behind the model





Parallel activation streams.
Top down and bottom up activation combine to
determine the best matching structure.
Triangle nodes bind features of objects to values
Mutual inhibition and competition between
structures
Mental connections are active neural
connections
5 levels of Neural Theory of
Language
Spatial
Relation
Motor
Control
Metaphor Grammar
Cognition and Language
abstraction
Computation
Structured Connectionism
Neural Net
Triangle Nodes
SHRUTI
Computational Neurobiology
Biology
Neural
Development
Quiz
Midterm
Finals
Can we formalize/model these
intuitions
What is a neurally plausible computational
model of spreading activation that
captures these features.
 What does semantics mean in neurally
embodied terms

 What
are the neural substrates of concepts
that underlie verbs, nouns, spatial predicates?