Negative
Reinforcement Escape
Negative reinforcement - Avoidance
Avoidance: Experimental Paradigm
Light = CS
Light  Shock
Shuttling stops
shock
The shuttle box
Two-Process Theory of Avoidance
Light  Shock
( = Pavlovian Conditioning)
-Light elicits fear
Shuttling  Reduction of Fear (= negative reinforcement)
Now, what happens with continued training?
Learned Helplessness Paradigm
“Triadic” Design
Phase 1
Group A: Escapable Shock
Group B: Yoked Inescapable Shock
Phase 2
Escape/Avoidance
training
Group C: Exposure to apparatus only
(For Group A shock can be terminated by rotating a wheel.)
Phase 2 Results
Inescapable shock
Inescapable shock
Possible Explanations
• Learned Helplessness: Organisms learn that their
behavior is ineffectual
• Poverty of activity: inescapable shock reduces the
variability in behavior that is so crucial for operant
conditioning
• Inattention: animals stop attending to their own
behavior
LH in the Spinal Cord
• Recall: many reflexes are mediated within the
spinal cord.
• Operant conditioning can occur within SC
(escape/avoidance of leg shock after SC
transection)
• Grau: Experience with inescapable legshock will
prevent subsequent avoidance learning.
LH in Humans
LH produced by…
insoluble logic problems
living in a crowded dorm
LH in the Spinal Cord
• Recall: many reflexes are mediated within the
spinal cord.
• Operant conditioning can occur within SC
(escape/avoidance of leg shock after SC
transection)
• Grau: Experience with inescapable legshock will
prevent subsequent avoidance learning.
Extinction
Session 1
Extinction
Session 2
Session 3
Spontaneous recovery occurs as a function of time
8-day break
no break
no CS
Theories of Extinction
• Forgetting?
• Associative loss? (= “reverse acquisition”)
Extinction  Associative Loss
“Renewal”
Train
Tone  Shock
Extinguish
Test
Tone -
Tone: CR
Tone -
Tone: CR
Context A
Context B
Bouton & King (1983)
In contrast, acquisition is not context-specific
Train
Tone  Shock
Test
Tone: CR
Tone: CR
Context A
Context B
Extinction  Associative Loss
“Reinstatement”
Train
Extinguish
Reinstatement
Shock alone
ToneShock
Test
Tone: CR
Tone --
Tone: CR
Extinction  Associative Loss
Post-extinction sensitivity to outcome devaluation
Rescorla 1996
So, what is learned in extinction?
An inhibitory SR association?
Context
S
R
Inhibitory SR Associations Theory
• In extinction, the context effectively
becomes a conditioned inhibitor.
• Why? Just like in normal CI, there is the
violation of expectations of reinforcement
• But is this true?
Inhibitory SR Associations
Does extinction produce them?
• Summation test
Train
Ext
Test
A+/B+
A-
Test: AB
Does A
inhibit
responding
to B?
• Retardation test
Train
A+
Ext
A-
Train
AB+
Does A inhibit
acquisition to B?
So, what is learned in extinction?
An inhibitory SR association?
Context
S
R
Paradoxical Effects of Reward
• Overtraining extinction effect: more training
leads to faster extinction
• Reinforcement magnitude effect: Big
rewards lead to faster extinction
• And, of course, the partial reinforcement
extinction effect (PREE)
Paradoxical effects of reward: Why?
• Frustration hypothesis (Amsel): animals learn to
make response as a reaction to nonreward.
• Discrimination hypothesis: Nonreinforcement is
easier to detect after CRF than PRF.
• Sequential theory (Capaldi): The memory of
nonreinforcement becomes a cue that elicits
responding.