PSY105 Neural Networks 4/5
4. “Traces in time”
Assignment note: you don't need to read the full book to
answer the first half of the question. You should be able to
answer it based on chapter 1 and the lecture notes.
Lecture 1 recap
• We can describe patterns at one level of
description that emerge due to rules followed
at a lower level of description.
• Neural network modellers hope that we can
understand behaviour by creating models of
networks of artificial neurons.
Lecture 2 recap
• Simple model neurons
– Transmit a signal of (or between) 0 and 1
– Receive information from other neurons
– Weight this information
• Can be used to perform any computation
Lecture 3 recap
• Classical conditioning is a simple form of
learning which can be understood as an
increase in the weight (‘associative strength’)
between two stimuli (one of which is
associated with an ‘unconditioned response’)
Nota Bene
• Our discussion of classical conditioning has
involved
– A behaviour: learning to associate a response with
a stimulus
– A mechanism: neurons which transmit signals
– These are related by…a rule or algorithm
Learning Rules
“When an axon of cell A is near enough to excite
a cell B and repeatedly or persistently takes
part in firing it, some growth process or
metabolic change takes place in one or both
cells such that A's efficiency, as one of the
cells firing B, is increased.”
Hebb, D.O. (1949), The organization of behavior,
New York: Wiley
Operationalising the Hebb Rule
• Turn ….“When an axon of cell A is near enough
to excite a cell B and repeatedly or persistently
takes part in firing it, some growth process or
metabolic change takes place in one or both
cells such that A's efficiency, as one of the
cells firing B, is increased.”
• ….Into a simple equation which is a rule for
changing weights according to inputs and
outputs
A Hebb Rule
• Δ weight = activity A x activity B x learning rate
constant
• In words: increase the weight in proportion to the
activity of neuron A multiplied by the activity of
neuron B
Stimulus Off
Stimulus On
Time
Implications of this rule
CS1
Stimulus 1
?
Stimulus 2
UCS
Implications of this rule
CS1
Stimulus 1
?
Stimulus 2
UCS
Implications of this rule
CS1
Stimulus 1
?
Stimulus 2
weight
UCS
Implications of this rule
Activity A
CS1
Stimulus 1
?
weight
Activity B
Stimulus 2
UCS
Implications of this rule
Activity A
CS1
Stimulus 1
?
weight
Activity B
Stimulus 2
=
x
UCS
x 0.1
The most successful model of Classical
Conditioning is the Rescorla Wagnar
model
Accounts for the effects of combinations of stimuli in
learning S-S links
Based on the discrepancy between what is expected to
happen and what happens
But…
Deals with discrete trials…ie has no model of time
Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and
nonreinforcement. In: Classical Conditioning II: Current Research and Theory (Eds Black AH, Prokasy WF) New York:
Appleton Century Crofts, pp. 64-99, 1972
Robert Rescorla (2008) Rescorla-Wagner model. Scholarpedia, 3(3):2237.
The problem of continuous time
Stimulus 1
Stimulus 2
The problem of continuous time
Stimulus 1
Stimulus 2
The problem of continuous time
Stimulus 1
Stimulus 2
The problem of continuous time
Activity A
Stimulus 1
Activity B = 0
Stimulus 2
The problem of continuous time
Activity A
Stimulus 1
Activity B = 0
Stimulus 2
The problem of continuous time
Activity A = 0
Stimulus 1
Activity B
Stimulus 2
• We need to add something to our model to
deal with a learning mechanism that is always
“on”
Traces
Stimulus 1
Traces
Stimulus 1
Traces
Stimulus 1
Stimulus 2
Traces
Activity A
Stimulus 1
Activity B
Stimulus 2
Traces
Stimulus 1
Stimulus 2
Consequences of this implementation
•
•
•
•
•
•
Size of CS stimulus
Duration of CS stimulus
Size of UCS stimulus
Duration of UCS stimulus
Separation in time of CS and UCS
The order in which the CS and UCS occur
– (cf. Rescola-Wagner discrete time model)