Extinction of Conditioned
Behaviour
Extinction
•
•
•
•
•
•
CS without US
Response without outcome
Not simply a reversal of acquisition
Not the same as forgetting
New learning
An “inhibitory relationship”
Real Life
• Common (and necessary!) occurrence
• Adaptation to changed conditions
• E.g., Stop calling a friend after they stop
returning calls
Effects of Extinction
•
•
•
•
Continue original behaviour for a time
Increase behaviour
Vary behaviour
E.g., Call friend more; wait for friend
after work.
Neuringer et al. (2001)
•
•
•
•
•
•
•
Rats, operant chamber
Two levers and a key
Three responses in a row to receive food
Group 1: had to vary response pattern
Group 2: no variation required (yoked)
Acquisition phase
Put on extinction
Results
• Variation in
behaviour
• Change in response
rate
Emotional Effects
• Frustration
• Emotional reaction induced by
withdrawal of expected reinforcer
• Intensifies behaviour
• Aggression
Tomie et al. (1993)
•
•
•
•
•
•
Rats
Water deprived
3 min. VT-30sec delivery of water
3 min. no water (ext.), signaled by tone (S-)
Target bite bar (plexiglass wrapped in tape)
Target biting a sign of frustration in rats;
readily produced by delivering aversive
stimulus (e.g., Azrin et al. 1968)
Results
Azrin et al. (1966)
• Pigeons
• Conditioned to peck key under
alternating periods of food
reinforcement and extinction
• Restrained pigeon or stuffed pigeon
model in chamber attacked during
extinction
Design
Attack Behaviour
• Cumulative records
of 3 pigeons
• Pen stepped up for
each 1 sec. of attack
Length of Attacks
• Average duration of
attack post
termination of food
reinforcement for
different pigeons
Stuffed Model
• Remarkably similar
behaviour
Number of Food Deliveries
• Number of food reinforcements before
extinction implemented
• 0, 1, 3, 5, 10, or 30 food deliveries
• Stuffed pigeon
• Positive correlation between amount of
food and attack duration
Data
Implications for Therapy
• Extinction acts as aversive
• Aversives create frustration
• Frustration can produce aggressive
behaviour
• Directed against: therapist, anyone in
proximity, self
Extinction and Original
Learning
• Not a reversal
• Forgetting a different underlying
process
• Actually a new acquisition of learning
Disinhibition
• Fully condition CS with US
• Impose extinction protocol
• Present novel stimulus along with
extinguished CS
• CR will reoccur
Why “Disinhibition”?
• Pavlov’s terminology
• Excitatory conditioning
– Increase in excitatory strength
• Extinction
– Inhibition of excitatory conditioning
– Net sum effect
– Full extinction = excitation + inhibition = 0
• Disinhibition inhibits the “extinction
inhibition”
• Temporarily reduces strength of inhibition
• Excitation + (inhibition + disinhibition) > 0
• A temporary effect
Parallels Dishabituation
• Temporary return of habituated response
without rest period
• Three ways
– New stimulus presented with habituated stimulus
(e.g., Graves & Thompson, 1970)
– Change habituated stimulus (e.g., Fisher, 1962
with the Coolidge effect)
– Change context of habituation (e.g., Schein &
Hale, 1974)
• Temporary sensitization process
superimposed over habituation process
Spontaneous Recovery:
Classical
•
•
•
•
•
Fully condition CS-US
Fully extinguish CS
Let some time pass
Present CS
Will get return of CR
Rescorla (1997)
• Goal tracking for two different CSs
• Full conditioning for all groups followed
by full extinction
• CS-Rest: test session 8 days after
extinction
• CS-No Rest: test session immediately
after extinction
• Pre-CS: control group
Results
Spontaneous Recovery:
Operant
•
•
•
•
Condition three term contingency
Extinguish response
Allow time to pass
Response will return in presence of SD
Rescorla (1996)
• Rats
• Responses (lever press or nose poke)
acquired, then extinguished
• R-Rest: tested 7 days post-extinction
• R-No rest: tested shortly after extinction
Results
Renewal
• Recovery of acquisition of performance
when context cues present during
extinction are changed
• If extinction is learning another three
term contingency, then changing the
cues eliminates the SD for extinction
• Think of this in terms of stimulus control
Bouton & King (1983)
•
•
•
•
Rats press lever for food
Tone (CS) paired with footshock
Training in two chambers
Post training, 20 extinction trials
– Group 1 in original (A) chamber
– Group 2 in novel (B) chamber
– Group 3 had no extinction (control)
• All groups tested for response in chamber A
Results
Renewal Also In:
•
•
•
•
Classical appetitive conditioning
Conditioned inhibition
Instrumental conditioning
Physiological states, such as from drugs
can also act as the SD for extinction that
can be renewed (e.g., Bouton et al.,
1990)
Reinstatement
• Recovery of the excitatory responding
to an extinguished stimulus produced by
exposure to the US
• Example: fear of flying
– Extinguish fear through therapy
– Have one frightening flying experience
– Phobia re-established to high level
Bouton (1984)
• Conditioned suppression in rats
• US = shock
Phase 1
Phase 2
Reinstatement Test
CS --> weak US
(weak CR)
No treatment US, same
CS-->
(weak CR)
context as test weak CR
CS --> weak US
(weak CR)
No treatment US, different
CS -->
(weak CR)
context as test weak CR
CS --> strong US
(strong CR)
Extinction
(weak CR)
US, same
CS -->
context as test strong CR
CS --> strong US
(strong CR)
Extinction
(weak CR)
US, different
CS -->
context as test weak CR
• Boulton suggests reinstatement may be
subset of renewal (US activates context cues)
Sensitivity to US Devaluation
• Utilize US devaluation to determine if
CS-US association persists through
extinction
• Show S-R and R-O association
maintained post-extinction
Study Design
•
•
•
•
Two CS (light, tone), two US (food, sucrose)
Counterbalanced across subjects
US devaluation via LiCl
Note: need to recondition extinguished CSs
to different US to get measurable CR in test
Phase 1
Phase 2
Phase 3
Test
L --> food
T --> sucrose
L extinguished
T extinguished
food devalued
L…?
T…?
food devalued
L…?
T…?
L --> food
T --> sucrose
Results
NotDev
CR Strength
NotDev
Dev
Light
Tone
Dev
Extinguished
Not extinguished
• Extinguished group shows weaker CR
• Within groups, devalued stimulus shows even weaker
CR; pre-extinguished CS-US association still affected
Study Design
• Different responses, different outcomes
• Note: recondition R1 & R2 to new O predevaluation
Phase 1
Phase 2
Outcome
Devaluation
Test
R1 --> O1
R1 Ext
O1 --> LiCl
R1 vs. R2 &
R3 vs. R4
R2 --> O2
R2 Ext
O1 --> LiCl
R1 vs. R2 &
R3 vs. R4
R3 --> O1
O1 --> LiCl
R1 vs. R2 &
R3 vs. R4
R4 --> O2
O1 --> LiCl
R1 vs. R2 &
R3 vs. R4
Results
NotDev.
Resposnes
NotDev.
O2
Dev.
Dev.
Extinguished
Not Extinguished
O1
Enhancing Extinction
• So extinction doesn’t actually eliminate
prior learning
• Sometimes extinguished response
comes back
• Techniques to minimize return of
extinguished learning
Number & Timing
• More extinction trials!
• Space extinction trials closer together
(massed) rather than spread out
(spaced)
– Works with aversive conditioning; don’t
really know about appetitive conditioning
yet
Reducing Spontaneous
Recovery
• Repeat periods of rest and testing
– Less recovery with each successive cycle
• Manipulating interval between acquisition and
extinction
– Fear conditioning study found less spontaneous
recovery with shorter interval
– Appetitive conditioning study found the opposite
• Present cues associated with extinction
– Reactivates extinction performance
Reducing Renewal
• Conduct extinction in multiple settings
– Increases stimulus generalization
• Present SD for extinction during renewal
Compound Extinction Stimuli
• Present two stimuli undergoing
extinction simultaneously
Rescorla (2006)
• Rats
• Three stimlui: Light, Noise, Tone
• Acquisition of lever pressing (VI30 sec.)
in presence of stimuli
• Extinguish each of the stimuli
• Compound extinction phase
– Light with one of auditory stimuli; other
auditory alone
Results
Response Rate
Elevated responding
(summation of
subthreshold responding
remaining to L & A1)
No recovery of A1;
compound extinction
increased A1’s
extinction
Extinction
Light
Substantial
spontaneous
recovery of A2
Auditory 1
Compound
extinction
Auditory 2
Test
(6 days later)
Light & Auditory 1
What is Learned in Extinction
• S-O and R-O associations not eliminated
• Current research suggests an inhibitory S-R
association
• Extinction effects will be highly specific to the
context in which the response was
extinguished
– E.g., if you never got birthday presents on your
birthday as a kid, you won’t be disappointed if you
don’t get presents as an adult
Rescorla (1993)
• 1. Discrimination training (nose poke --> food)
whenever Light or Noise present
• 2. Lever press & chain pull (R1 & R2) --> food
– No S-R association b/t L or N with R1 or R2
• 3. Extinction of N:R1 and L:R2
– Establishes inhibitory S-R associations
• 4. Test
– N: R1 vs. R2… more R2 responding
– L: R1 vs. R2… more R1 responding
• Can’t be due to S-O or R-O effects; has to be
S-R
S-R
• Think back to our discussion of Central
Emotional States
• Decline in responding in extinction
linked to frustration due to not getting
what you expected
• Leads to some seemingly odd effects
Overtraining Extinction Effect
• The more acquisition trials, the greater
the expectancy of reward, hence the
greater the frustration when extinction
introduced
• Produces more rapid extinction
• Odd, because you’d expect that more
training results in a stronger response
that is more resistant to extinction
Magnitude Reinforcement
Extinction Effect
• More rapid extinction if trained with
larger rather than smaller magnitude
reinforcer
• Expectancy of greater reward produces
greater frustration when it is not
forthcoming
Partial Reinforcement
Extinction Effect
• Continuous reinforcement (FR-1)
• Intermittent reinforcement (everything
else)
• Extinction faster on CRF
• The less likely each response is of
being reinforced, the more difficult it is
to extinguish
Discrimination Hypothesis
• Easier to notice start of extinction if on FR1
than e.g., FR10
• But PREE may not be due to difficulty noticing
start of extinction
• Jenkins (1962)
– Two groups of pigeons, 1st on CRF, 2nd on PRF
– Both put on CRF, then immediately on extinction
– 2nd group took longer to extinguish
• Effects of PRF long-lasting; “don’t give up in
face of failure”?
Frustration Theory
• Intermittent reinforcement has rewarded and nonrewarded responses
• Rewarded responses motivating
• Non-rewarded responses frustrating
• Typically some variation in when you get reinforced;
sometimes what you expect will be a non-reinforced
response produces a reinforcer
• Hence, frustrated responses lead to future
expectation of reward
• On CRF can’t learn to respond when expecting nonreward
Sequential Theory
• Memory concepts
• You can remember whether or not a response
was reinforced in recent past
• In PRF training, non-reinforced responses
become an “SD” for performing the response
• Depends on sequencing
– E.g., R N N R R N R
• With experience, learn to respond when
remembering not having been reinforced for
recent responses
Behavioural Momentum
• Analogy with Newtonian physics
– Momentum = mass*speed
– “A body in motion tends to stay in motion.”
• Behaviour that has a lot of momentum will be
hard to disrupt through manipulation
• Studied using multiple schedules of
reinforcement
– Two or more components, each with its own SD
and reinforcement schedule
– Add disruption (e.g., extra food between
components, extinction, novel salient stimuli, etc.)
Behavioural Momentum
• Response rate
– Often (but not always) unrelated to behavioural
momentum
• Rate of reinforcement
– Higher produces more momentum; less
susceptible to disruption
– Seems to be most important element
• Implies S-O associations regulate
behavioural momentum
– Adding non-contingent reinforcers to one schedule
increases its behavioural momentum
Mace et al. (1990)
VI 60… more
reinforcement…
more momentum