Tutorial: Plasticity Revisited Motivating New Algorithms Based On
Recent Neuroscience Research
Approximate Outline and
References for Tutorial
Tsvi Achler MD/PhD
Department of Computer Science
University of Illinois at Urbana-Champaign, Urbana, IL 61801, U.S.A.
Plasticity:
Outline
Intrinsic
Synaptic
Homeostatic
‘Systems’
1. Plasticity is observed in many forms. We
review experiments and controversies.
•
•
•
•
Intrinsic ‘membrane plasticity’
Synaptic
Homeostatic ‘feedback plasticity’
System: in combination membrane and
feedback can imitate synaptic
2. What does this mean for NN algorithms?
Algorithms:
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
Outline: NN Algorithms
Common computational Issues
• Explosion in connectivity
• Explosion in training
• How can nature solve these problems with
the plasticity mechanisms outlined?
1. Plasticity
Intrinsic
Intrinsic ‘Membrane’ Plasticity
• Ion channels responsible for activity, spikes
• ‘Plastic’ ion channels found in membrane
• Voltage sensitive channel types:
– (Ca++, Na+, K+)
• Plasticity independent of synapse plasticity
Review:
G. Daoudal, D, Debanne, Long-Term Plasticity of Intrinsic Excitability:
Learning Rules and Mechanisms, Learn. Mem. 2003 10: 456-465
Synaptic
Synaptic Plasticity Hypothesis
•
•
•
•
Bulk of studies
Synapse changes with activation
Motivated by Hebb 1949
Supported by Long Term Potentiation /
Depression (LTP/LTD) experiments
Review:
Malenka, R. C. and M. F. Bear (2004). "LTP and LTD: an embarrassment
of riches." Neuron 44(1): 5-21.
Synaptic
LTP/LTD Experiment Protocol
Pre-synaptic electrode
A50
50%
?
Post-synaptic
electrode
Brain
• Establish ‘pre-synaptic’ cell
• Establish ‘post-synaptic’ cell
• Raise pre-synaptic activity to amplitude to
A50 where post-synaptic cell fires “50%”
• Induction: high frequency high voltage
spike train on both pre & post electrodes
• Plasticity: any changes when A50 is applied
Synaptic
Plasticity: change in post with A50
• LTP : increased activity with A50
• LTD : decreased activity with A50
• Can last minutes hours days
– Limited by how long recording is viable
Synaptic
Strongest Evidence
Systems w/minimal feedback:
• Motor, Musculature & tetanic stimulation
• Sensory/muscle junction of Aplesia Gill
Siphon Reflex
• Early Development: Retina → Ocular
Dominance Columns
Synaptic
Variable Evidence
Cortex, Thalamus, Sensory Systems &
Hippocampus
• Basic mechanisms still controversial
60 years and 13,000 papers in pubmed
• It is difficult to establish/control when LTP
or LTD occurs
Synaptic
LTP vs LTD Criteria is Variable
• Pre-Post spike timing:
(Bi & Poo 1998; Markram et al. 1997)
– Pre-synaptic spike before post  LTP
– Post-synaptic spike before pre  LTD:
• First spike in burst most important
• Last spike most important
• Frequency most important:  Freq  LTP
(Froemke & Dan 2002)
(Wang et al. 2005)
(Sjöström et al. 2001; Tzounopoulos et al. 2004).
• Spikes are not necessary
(Golding et al. 2002; Lisman & Spruston 2005)
Synaptic
Many factors affect LTP & LTD
•
•
•
•
•
Voltage sensitive channels ie. NMDA
Cell signaling channels ie via Ca++
Protein dependent components
Fast/slow
Synaptic tagging
Review:
Malenka, R. C. and M. F. Bear (2004). "LTP and LTD: an embarrassment
of riches." Neuron 44(1): 5-21.
Synaptic
Studies of Morphology Unclear
Synapse Morphology and density studies:
• Spine changes ≠ Function changes
• Many other causes of changes in spines:
– Estrus, Exercise, Hibernation, Epilepsy,
Irradiation
Review:
Yuste, R. and T. Bonhoeffer (2001). "Morphological changes in dendritic
spines associated with long-term synaptic plasticity." Annu Rev
Neurosci 24: 1071-89.
Synaptic
Many Components & Variability
• Indicates a system is complex
– involving more than just the recorded presynaptic and postsynaptic cells
• Means NN learning algorithms are difficult
to justify
• But the system regulates itself
Review of LTP & LTD variability:
Froemke, Tsay, Raad, Long, Dan, Yet al. (2006) J Neurophysiol 95(3):
1620-9.
Homeostatic
Homeostatic Plasticity
Self-Regulating Plasticity
Networks Adapt to:
Channel Blockers
Genetic Expression of Channels
Homeostatic
Adaptation to Blockers
Pre-synaptic
electrode
Post-synaptic
electrode
Pre-Synaptic Cell Post-Synaptic Cell
Culture Dish
• Establish baseline recording
• Bathe culture in channel blocker (2 types)
– Either ↑ or ↓ Firing Frequency
• Observe System changes after ~1 day
• Washing out blocker causes reverse
phenomena
Homeostatic Adaptation to Blockers
Pre-Synaptic Cell Post-Synaptic Cell
Displays
Feedback
Inhibition
Response
↑ Frequency → ↓ Synaptic Strength
↓ Frequency → ↑ Synaptic Strength
Frequency x Strength = Baseline
Turrigiano & Nelson (2004)
Homeostatic
Homeostatic Adaptation to Expression
Cell
Channels Involved
1
2
3
Marder & Goaillard (2006)
Cells with different numbers & types of
channels show same electrical properties
Homeostatic
Homeostatic Summary
• Adapts networks to a homeostatic baseline
• Utilizes feedback-inhibition (regulation)
Homeostatic
Feedback Inhibition
Pre-Synaptic Cell
Post-Synaptic Cell
Feedback Ubiquitously Throughout Brain
Feedback Throughout Brain
Homeostatic
Thalamus
& Cortex
LaBerge, D. (1997) "Attention, Awareness, and the Triangular Circuit". Consciousness and Cognition, 6, 149-181
Homeostatic
Overwhelming Amount of
Feedback Inhibition
•
•
•
•
Feedback loops
Tri-synaptic connections
Antidromic Activation
NO (nitric oxide)
• Homeostatic Plasticity
Regulatory Mechanisms Suggest
Modified from Chen, Xiong & Shepherd (2000).
Pre-Synaptic Feedback
Figure from Aroniadou-Anderjaska, Zhou, Priest, Ennis & Shipley 2000
Homeostatic
Summary
• Homeostatic Plasticity requires and
maintains Feedback Inhibition
‘Systems’
‘Systems’ Plasticity
Feedback Inhibition combined with
Intrinsic Plasticity
Can be Indistinguishable from Synaptic
Plasticity
‘Systems’
Many cells are always present in
plasticity experiments
Pre & Post synaptic cells are never in isolation
Pre-synaptic
electrode
Post-synaptic
electrode
Studies:
• In Vivo
Culture Dish
• Brain slices
• Cultures: only viable with 1000’s of cells
Changes in neuron resting activity is tolerated
‘Systems’
Feedback Inhibition Network
∆↓
∆↓
∆↓
∆↑
∆↓
∆↓
Then learning is
Increase pre-synaptic cell
Induction
can
Immeasurable
Only
the
two
recorded
With
Pre-Synaptic
recorded
activity
until
induced artificially
affect
all
connected
changes
of
all
cells Inhibition
and the synapse
postsynaptic cell fires 50%
by
activating
both
post-synaptic
cells
connected
neurons
between them are
Pre-synaptic
cells connect tobut
many
this post-synaptic
is rarely
considered
cells
neurons
together
considered
Causes
big
change
in
the
LTP protocol: find pre-synaptic and post-synaptic cells
recorded neuron
‘Systems’
Simulation: Up to 26 Cell Interaction
Normalized Activity Scale (0-1)
LTP
1
0.9
0.8
LTD
Immeasurable changes
of all connected neurons
Resting ∆
Value
All Neurons 0.01
0.7
0.6
0.5
Causes big change in
the recorded neuron
0.4
Baseline
0.3
0.2
0.1
0
‘Systems’
Significance
Experiments can not distinguish between
synaptic plasticity and feedback inhibition
• Membrane voltage Vm allowed Δ ~6mV
• 0.01 = ~∆Vm of 0.3 mV
• Thus not likely to see membrane affects
• Presynaptic cells connect to >> 26 cells
– Effect much more pronounced in real networks
‘Systems’
Regulatory Feedback Plasticity
• Feedback Inhibition + Intrinsic Plasticity
are indistinguishable in current experiments
from Synaptic Plasticity theory
• Why have ‘apparent’ synaptic plasticity?
• Feedback Inhibition is important for
processing simultaneous patterns
Break
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
2. Algorithms
Challenges In Neural Network
Understanding
Limited Cognitive Intuition
Large Network Problems
lw13
Y1
w11
lw12
Y2
lw23
Y3
w21 w31
w
w12 w22 w32 33 w43
w13 w23
w42
w41
x1
x2
x3
x4
Neural Networks
Lateral Connections:
connectivity explosion
Y1
Lateral Connectivity
x1
Y2
x2
x3
Y3
x4
Millions of representations possible
Can lead
to an implausible
of
Every
representation
can notnumber
be connected
to all
-> a connection required to logically relate between representations
others connections
in the brain and variables
Combinatorial Explosion in Connectivity
0.8
What would a weight
variable between them
mean?

?

Challenges In Neural Network
Understanding
Large Network Problems
lw13
Y1
lw12
Y2
lw23
Y3
w21w31
w33 w43
w
32
ww1221ww2231w w w
33
32
43
w12 w22
w
w
w2323
w1111 w
w1313 w
w4242
ww4141
x1
x2
x3
x4
Neural Networks
Lateral Connections:
connectivity explosion
Weights:
combinatorial training
Y1
Weights: Training Difficulty
Given Simultaneous Patterns
• Teach A B C … Z separately
• Test multiple simultaneous letters
Y2
Y3
w21 w31
w12 w22 w32 w33 w43
w11 w13 w23 w w42
41
x1
x2
x3
x4
A
D
B
D
AA
ECB
G E
Not Taught with simultaneous patterns:
Will not recognize simultaneous patterns
Teaching simultaneous patterns is a
combinatorial problem
Y1
Weights: Training Difficulty
Given Simultaneous Patterns
• Teach A B C … Z separately
• Test multiple simultaneous letters
Y2
Y3
w21 w31
w12 w22 w32 w33 w43
w11 w13 w23 w w42
41
x1
x2
x3
x4
A D
G E
‘Superposition Catastrophe’ (Rosenblatt 1962)
Can try to avoid by this segmenting each pattern
individually but it often requires recognition or
not possible
Composites Common
•
•
•
•
Natural Scenarios (cluttered rainforest)
Scenes
Noisy ‘Cocktail Party’ Conversations
Odorant or Taste Mixes
Segmentation not trivial (requires recognition?)
Segmentation is not possible in most modalities
Y1
41
x1
Frog
Feature Space
Feature Space
Chick
1
0
0
1
0
1
0
1
Y3
w21 w31
w12 w22 w32 w33 w43
w11 w13 w23 w w42
Segmenting Composites
Learn:
Y2
x2
x3
x4
New Scenario:
If can’t segment image
must interpret composite
1
0
0
1
Chick & Frog
Simultaneously
+
0
1
0
1
=
1
1
0
2
Challenges In Neural Network
Understanding
Large Network Problems
Y1
Y2
Y3
w21w31
w33 w43
w
32
w12w22
w11 w13 w23 w42
w41
x1
x2
x3
x4
Neural Networks
Lateral Connections:
connectivity explosion
Weights:
combinatorial training
Feedback Inhibition:
avoids combinatorial issues
interprets composites
Feedback Inhibition
Feedback Inhibition
Control Theory
Perspective
Every output inhibits only its own inputs
• Gain control mech for each input
• Massive feedback to inputs
• Iteratively evaluates input use
Output
Input
• Avoids optimized weight
parameters
ya
yb
Output
Network
I1
x1
I2
x2
Input
Neuroscience
Perspective
Feedback Inhibition
Equations Used
Xb
ya
yb
Xb Raw Input Activity
I1
x1
I2
x2
Feedback Inhibition
Equations
Xb
Ib =
Qb
ya
yb
I1
x1
Xb Raw Input Activity
Ib Input after feedback
Qb Feedback
I2
x2
Equations
Output
Feedback Inhibition
Ya (t )
Ya (t + D t ) =
Ii

n a iYa
Inhibition
Xb
Ib =
Qb
Feedback
Qb =
ya
Q1=ya+yb =
Y
j X b
j
(t )
yb
I1
x1
= Q2=yb
I2
x2
Ya Output Activity
Xb Raw Input Activity
W I Input after feedback
b
Qb Feedback
na # connections of Ya
Equations
Output
Feedback Inhibition
Ya (t )
Ya (t + D t ) =
Ii

n a i Ya
Inhibition
Xb
Ib =
Qb
Feedback
Qb =
ya
x1
Q1=ya+yb
=
Y
j Xb
j
(t )
yb
I1
=
I2
x2
Q2=yb
x1
x2
NoRepeat
Oscillations
No Chaos
Feedback Inhibition
Simple Connectivity
Y1
Y2
Y3
Y4
Output Nodes
W
I1
x1
I2
x2
I3
x3
I4
Input Nodes
x4
Source
ofconnects
Training
Problems
All
links
have
same to
strength
NewSource
node
only
its
inputs
ofhave
Connectivity
Problems
Inputs
positive
real
values indicating intensity
Feedback Inhibition
Outputs
Y1
Inputs
I1
‘R’
1
0
Y2
I1
I2
Features
‘P’
Features
Allows Modular Combinations
1
1
Algorithm
Interprets Composite Patterns
Network Configuration
y1
y2
(-)
Outputs
Inputs
x1
x2
BehavesSupports
as if there is an
Non-Binary
Inputs
inhibitory
connection
yet there
is no direct supporting
Inputs
simultaneously
connectionboth
between
x 2 & y1
outputs
Steady State
Inputs
Outputs
x1 , x2
1, 0
y1 , y2
1, 0
1, 1
‘R’
2,2
0, 1
0,2
2,1
1,1
P&R
Solution
x1≥ x2
x1≤ x2
‘P’
2Rs
y1
y2
x1–x2
x2
0
(x1+x2)/2
How it Works
Feedback Inhibition Algorithm
Iterative Evaluation
Outputs
Inputs
Y1
Y2
I1
I2
x1
x2
How it Works
Feedback Inhibition Algorithm
Back
Outputs
Inputs
Y1
Y2
I1
I2
x1
x2
How it Works
Feedback Inhibition Algorithm
Outputs
Inputs
Forward
Y1
Y2
I1
I2
x1
x2
How it Works
Feedback Inhibition Algorithm
Outputs
Inputs
Back
Y1
Y2
I1
I2
x1
x2
How it Works
Feedback Inhibition Algorithm
Outputs
Y1
Y2
Active (1)
Inactive (0)
I1
I2
=
Features
Inputs
1
1
How it Works
Feedback Inhibition Algorithm
Initially both outputs become active
Outputs
C2
Active (1)
Inactive (0)
Inputs
I1
I2
How it Works
Feedback Inhibition Algorithm
I1 gets twice as much inhibition as I2
Outputs
C2
Active (1)
Inactive (0)
Inputs
I2
How it Works
Feedback Inhibition Algorithm
I1 gets twice as much inhibition as I2
Outputs
C2
Active (1)
Inactive (0)
Inputs
I2
How it Works
Feedback Inhibition Algorithm
Outputs
Active (1)
Inactive (0)
Inputs
How it Works
Feedback Inhibition Algorithm
This affects Y1 more than Y2
Outputs
Active (1)
Inactive (0)
Inputs
How it Works
Feedback Inhibition Algorithm
This separation continues iteratively
Outputs
Active (1)
Inactive (0)
Inputs
I2
How it Works
Feedback Inhibition Algorithm
This separation continues iteratively
Outputs
Active (1)
Inactive (0)
Inputs
How it Works
Steady State
Until the most encompassing
representation predominates
Y1
1
0
Y2
1
‘R’
Y1
Y2
Activity
Outputs
1
1
Graph of Dynamics
0
I1
I2
=
Features
Inputs
1
1
0
1
2
3
4 5
Simulation Time (T)
Demonstration
Demonstration: Appling Learned
Information to New Scenarios
• Nonlinear: mathematical analysis difficult
– demonstrated via examples
• Teach patterns separately
• Test novel pattern combinations
• Requires decomposition of composite
• Letter patterns are used for intuition
Demonstration
Teach single patterns only
B
ED
A
C
• Learn A B C … Z separately
Nodes
…….
Feature Space
Features
A
0
1
0
0
1
.
.
.
.
B
1
1
0
0
0
.
.
.
.
C
0
1
0
1
1
.
.
.
.
D
1
0
1
0
1
.
.
.
.
E
1
1
0
1
1
.
.
.
.
Modular
Combination
…….
26
Nodes
Demonstration
This Defines Network
Nothing is changed or re-learned further
Comparison networks are trained & tested
with the same patterns
– Neural Networks (NN)*
Representing synaptic plasticity
– Lateral Inhibition
(Winner-take-all with ranking of winners)
* Waikato Environment for Knowledge Analysis (WEKA)
repository tool for most recent and best algorithms
Demonstration
Tests: Increasingly Complex
• 26 patterns presented one at a time
– All methods recognize 100%
• Choose 2 letters, present simultaneously
– Either: union logical-‘or’ features
325 Combinations
A+B
A|B
– add features
B
A
•
1
1
0
Choose 4 letters, present
simultaneously
21
To
1
1
=
+or 0
0
– Either: add or ‘or’0features
14,950 Combinations
Networks
0
0
0
– Include repeats in 1add case0 (ie ‘A+A+A+A’)
1
.
.
.
.
.
.
.
.
.
456,976.. Combinations
.
Two Patterns Simultaneously A B
% of combinations
100
90
80
70
60
50
40
30
20
10
0
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
• Train 26 nodes
• Test w/2 patterns
• Do 2 top nodes
match?
325
Combinations
0/2
1/2
2/2
Letters Correctly Classified
Demonstration
Simultaneous Patterns
Four pattern union
Feature Space
A D
C E
A
0
1
0
0
1
.
.
.
.
or
C
0
1
0
1
1
.
.
.
.
or
D
1
0
1
01
.
.
.
.
or
E
1
1
0
1
1
.
.
.
.
=
A|C|D|E
1
1
1
1
1
.
.
.
.
To
Network
Union of Four Patterns :
% of combinations
100
90
80
70
60
50
40
30
20
10
0
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
A B
C D
• Same 26 nodes
• Test w/4 patterns
• Do 4 top nodes
match?
14,950
Combinations
0/4
1/4
2/4
3/4
Letters Correctly Classified
4/4
Union of Five Patterns:
% of combinations
100
90
80
70
60
50
40
30
20
10
0
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
A B
CDE
• Same 26 nodes
• Test w/5 patterns
• Do 5 top nodes
match?
65,780
Combinations
0/5
1/5
2/5
3/5
4/5
Letters Correctly Classified
5/5
Demonstration
Pattern Addition
Feature Space
Improves feedback inhibition
performance further
A
0
1
0
0
1
.
.
.
.
+
C
0
1
0
1
1
.
.
.
.
+
D
1
0
1
0
1
.
.
.
.
+
E
1
1
0
1
1
.
.
.
.
=
A D
C E
A+C+D+E
2
3
1
2
4
.
.
.
.
To
Network
Addition of Four Patterns :
% of combinations
100
90
80
70
60
50
40
30
20
10
0
A
K
S
X B
C
C
V
OA D
M
Synaptic Plasticity
Same 26 nodes
Lateral Inhibition
Test w/4 patterns
Pre-Synaptic Inhibition •Do 4 top nodes
match?
14,950
Combinations
0/4
1/4
2/4
3/4
Letters Correctly Classified
4/4
Addition of Eight Patterns:
% of combinations
100
90
80
70
60
50
40
30
20
10
0
A G B L
CD X E
• Same 26 nodes
• Test w/8 patterns
• Do 8 top nodes
match?
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
1,562,275
Combinations
0/8
1/8
2/8
3/8
4/8
5/8
6/8
Letters Correctly Classified
7/8
8/8
Demonstration
With Addition Feedback
Algorithm Can Count
Feature Space
• Repeated patterns reflected by
value of corresponding nodes
A
0
1
0
0
1
.
.
.
.
+
B
1
1
0
0
0
.
.
.
.
+
B
1
1
0
0
0
.
.
.
.
+
C
0
1
0
1
1
.
.
.
.
A B
B C
A+B+B+C
2
4
=
0
1
2
.
.
.
.
Nodes:
A=1
B=2
C=1
D→Z=0
100%
456,976 Combinations
Demonstration
Tested on Random Patterns
•
•
•
•
•
50 randomly generated patterns
From 512 features
4 presented at a time
6,250,000 combinations (including repeats)
100% correct including count
Computer starts getting slow
Insight
What if Conventional Algorithms
are Trained for this Task?
A+B
1
2
0
0
1
This vector
is ‘A’ & ‘B’
together
A+C+D+E
2
This vector
3
is ‘A’ ‘C’
1
‘D’ & ‘E’
2
4
together
Insight
Y1
Training is not practical
• Teach pairs: 325 combinations
Y3
w21 w31
w12 w22 w32 w33 w43
w11 w13 w23 w w42
41
x1
26 letters
Y2
x2
x3
x4
KA
MS
A
E
A DC
A
B
A
P V
L
• Teach triples: 2600 combinations
• Quadruples: 14,950. Furthermore ABCD can
be misinterpreted as AB & CD, or ABC & D
• Training complexity increases combinatorialy
Insight
Training Difficulty Given
Simultaneous Patterns
Feedback inhibition inference
seems to avoid this problem
Y1
Y2
Y3
w21 w31
w12 w22 w32 w33 w43
w11 w13 w23 w w42
41
x1
x2
x3
x4
A D
G E
Known as: ‘Superposition Catastrophe’
(Rosenblatt 1962; Rachkovskij & Kussul 2001)
Binding problem
Simultaneous Representations
Chunking features:
Computer Algorithms
similar problems with
simpler representations
Simultaneous Representations Cause
The Binding Problem
‘Barbell’
y2
‘Wheels’
y1
Outputs
‘Car Chassis’
y3
Inputs
x1
Given:
x1
x2
x2
x3
all are patterns matched unless
the network is explicitly trained otherwise.
However it is a binding error to call this a barbell.
y1 y2
Binding Comparison
x1
y3
x2
x3
Vector Activity
1
Synaptic Plasticity
Lateral Inhibition
Feedback Inhibition
0.8
0.6
0.4
0.2
0
y1
y2
y3
‘Wheels’
‘Barbell’
‘Car Chassis’
Binding: Network-Wide Solution
y1
y2
y3
Outputs
Inputs
x1
x2
Inputs
Outputs
x1, x2, x3
1, 0, 0
1, 1, 0
1, 1, 1
y1, y2, y3
1, 0, 0
0, 1, 0
1, 0, 1
x3
Wheels
Barbell
Car Barbell
Network Under Dynamic Control
Recognition inseparable from attention
Feedback: an automatic way to access inputs
‘Symbolic’ control via bias
Symbolic Effect of Bias
y1
y2
y3
Outputs
Inputs
x1
x2
x3
Inputs
Outputs
x1, x2, x3 y1, y2, y3
1, 1, 1 0.02, 0.98, 0.71
Is barbell present?
Barbell
Bias y2 = 0.15
Summary
• Feedback inhibition combined with intrinsic
plasticity generates a ‘systems’ plasticity
that looks like synaptic plasticity
• Feedback inhibition gives algorithms more
flexibility with simultaneous patterns
• Brain processing and learning is still
unclear: likely a paradigm shift is needed
Acknowledgements
Eyal Amir
Cyrus Omar, Dervis Vural, Vivek
Srikumar
Intelligence Community Postdoc Program &
National Geospatial-Intelligence Agency
HM1582-06--BAA-0001