Chapter 9 – Extinction of Conditioned Behavior
• Outline
– Effects of Extinction Procedures
• Decreased responding
• Increased variability in responding
– Extinction of Original Learning
• Spontaneous Recovery
• Renewal of Original Excitatory Conditioning
• Reinstatement of Conditioned excitation
– Enhancing Extinction
•
•
•
•
Number and Spacing of Extinction Trials
Reducing Spontaneous Recovery
Reducing Renewal
Compounding Extinction Stimuli
– What is learned in Extinction
• Paradoxical Reward Effects
• Mechanisms of the Partial-Reinforcement Extinction Effect
• So far we have focused on acquisition effects.
– What happens when a stimulus predicts the presence of
some outcome
• This chapter focuses on what happens when that
outcome is later omitted
• Pavlovian
– Acquisition
• CS-US
– Extinction
• CS alone
• Instrumental
– Acquisition
• S+
– Barpress/keypeck  RF
– Extinction
• Barpress/keypeck  no longer RF
• Extinction is a hot area of research
– Particularly relevant to therapy
• Exposure therapies for phobias
– Extinguish fear
• Also for drug addictions
– Extinguish cues for drug taking behavior
• Effects of Extinction Procedures
– Domjan’s key example
• Key no longer works for door
– What do you do?
» Try several times
» Try it in a variety of ways
» Jiggle it
» Eventually quit
• Effects of Extinction
– Decrease in responding
– Increase in variability of responding
• Empirical Evidence
– Neuringer, Kornell, and Olufs (2001)
– Right lever(R), left lever (L), key (K)
• Three responses required
• Group Var
– Not allowed to repeat
• Yoked control
– No variability requirement but RF was yoked to Group
Var
Train
Extinction
Var
Variability
control
control
Var
Responding
• Extinction also can cause a strong
emotional response
– Frustration (possible aggression)
• Car won’t start
• Vending machine doesn’t work
• Pigeon’s out of food
• Pigeon’s key pecking with restrained
partner
– Extinction = attack
• Extinction and Original Learning
– Does extinction erase original learning
• Evidence says no
– From several lines
» Spontaneous Recovery
» Renewal
» Reinstatement
» Retention of knowledge of the Reinforcer
• Spontaneous Recovery
– Phase 1
• acquisition
– Train CS-US
– Phase 2
• extinction
– CS alone
– Phase 3
• Time off
– Phase 4
• Extinction
• What does Spontaneous recovery tell us about
extinction learning?
– Original learning remains
• A little time off?
– Responding returns
• Renewal
– A shift in context can renew extinguished learning
• bring back responding
– Demonstrated by Bouton and King (1983)
• used the conditioned suppression procedure.
• Phase 1
– Train all rats to bar press
• Phase 2
– Train conditioned emotional response to a CS
(tone)
• CS (tone)  US (shock)
• Phase 3 (Extinction)
– 3 Groups
• Group Ext A)
– Extinguish CER same context as phase 1 and 2
» Present CS alone in same context
• Group Ext B)
– Extinguish CER different context
» Present CS alone in different context
• Group NE
– No extinction
• Test
– Conditioned suppression to tone in original context
• Result
– Ext A?
• No Fear
– Ext B?
• Fear
– NE?
• Most Fear
• Which group demonstrated Renewal?
• What does this say about Extinction?
– 1) Initial learning is not forgotten
– 2) Extinction is at least somewhat context
specific
• Has implications for therapists attempting
to extinguish unwanted behaviors
– Extinction of phobia or drug taking behavior
may be specific to the therapists office
– Extinction in multiple contexts?
• Reinstatement
– Exposure to the US serves as a reminder
• reinstates an extinguished response.
• Train
– CS (tone)  US (shock)
– tone elicits fear.
• Extinction
– CS alone.
• Reinstatement
– US alone
• Test
– Fear tone?
• Yes = reinstatement
• Like Renewal, Reinstatement is context
specific
– US exposure works best if in the same context
• Reinstatement is also an issue for therapists
– Worry that extinguished fears/behaviors will
return if exposed to certain reminder stimuli
• Patient has intimacy issues because of abusive parents
– Treated with therapy
• Abusive encounter later in life
– Reinstates intimacy issues?
• Enhancing Extinction
– Because Extinction can be so useful
therapeutically,efforts have been made to
enhance it
• Number and Spacing of Extinction of Trials
– More extinction is more effective
• Makes sense
– New learning after all
– Massed trials are more effective than spaced
trials
• This effect seems to be temporary
– Within session effect
» Large spontaneous recovery
• Reducing Spontaneous Recovery
– Repeated spontaneous recovery sessions
reduces the effect
– If there are cues that are specific to extinction,
those cues can be effective in reducing
spontaneous recovery
• Requires special extinction cues
• Reducing Renewal
– Providing extinction training in multiple
contexts can reduce renewal effects
– Extinction cues can reduce renewal
• Same as for spontaneous recovery
• patients asked to recall the context of extinction
training (therapists office) showed reduced anxiety
in novel locations
• Prompted some therapists to specifically train
“portable” extinction cues
– Memorize a specific “relaxation phrase” or carry a
“relaxation card”
• One view of extinction is that it is due to an
increase in frustration that interferes with
normal responding
• Paradoxical Reward Effects
– support the acquired frustration view of extinction
• Whenever expectancy of reward is greatest following
training, extinction occurs fastest
• The Paradox?
– Better acquisition causes faster extinction
• we might expect that the better learning would slow
extinction
– Not the case
• Overtraining extinction effect
– More training = faster extinction
• animal is more sure that reward is forthcoming
• causes increased frustration when the reward does
not come.
• Magnitude of reinforcement effect
– Bigger reinforcers = faster extinction
– Animals are far more frustrated when they miss
out on big reinforcers
• Leads to faster extinction
– Would you behave the same?
• If you expect a big bonus at work and don’t get it?
– Might slow you down quite a bit
– May not affect you much if you miss out on a small bonus
• Partial reinforcement extinction effect
(PREE)
– Partial reinforcement slows extinction.
– Much faster extinction following continuous
reinforcement (CRF)
– An animal that expects a reinforcer after every
response quickly becomes frustrated.
• PREE has received a lot of research
attention
– Has real world relevance
• Gambling behavior
– Widely dispersed pay offs make responding (gambling)
very resistant to extinction
» Keep playing despite a lot of loss
• Parenting
– Giving in after repeated (annoying) requests for
candy/toys ensures that future annoying requests will
persist.
• Mechanisms of the Partial-Reinforcement
Extinction Effect
– Three hypotheses
• Discrimination hypothesis
• Frustration Theory
• Sequential Theory
• Discrimination hypothesis
– Extinction is easier to detect following continuous reinforcement
• When reinforcement only occurs every so often maybe you don’t notice
extinction for a while.
– Turns out this isn’t it.
– Jenkins (1962) Theios (1962)
• Phase 1
– Train animals
» Group 1
» Partial RF
» Group 2
» CRF
• Phase 2
– Put all animals on CRF for a while
• Test
– Extinction
» Still more responding in Partial RF group
– Extinction should have been equally discriminable for both
groups.
– Seems animals learn something longer lasting
• Perhaps it teaches them not to give up?
• Frustration Theory (I consider this a molar theory)
– Amsel
– Persistence in extinction occurs because the animals have
learned to make responses even when they expect nonreward
(or are frustrated).
– Breaking down frustration theory
– Partial RF
• Sometimes animals get rewarded when they don’t expect it
– Early reward on a variable ratio schedule (after a few responses) = surprised
• They also sometimes get rewarded when they are frustrated
– Late rewards on a variable ratio schedule (after many responses)
– Thus animals on partial RF schedules learn to respond even
when they don’t expect reward and even if frustrated
– Animals on CRF never learn this
• In addition frustration can be viewed as a kind of drive in Amsel’s
theory
– It energizes behavior
• Respond in order to reduce frustration
• Daly (1969)
– Trained rats to expect food in a goal box
• Then stopped feeding them there
– frustrated rats.
– Allowed them to jump a hurdle to escape the goal box
• rats learned to jump the hurdle
– No other RF was available
– Presumably jumped to escape frustration
» Negative RF
– Shows that frustration reduction can motivate behavior.
• We previously discussed the differential outcomes procedure
– Matching to sample
• One choice was reinforced with no food
• Animals still learn the correct response
– Could be to avoid the frustration of repeating the trial
• Keep in mind, that continued responding in
the face of frustration is a human
characteristic as well
• We often value it
– Songwriter continues to work in the face of
multiple rejections
• Sometimes it can be a negative
– Gambling in the face of continued losses
• Sequential Theory (I consider this a molecular theory)
– Capaldi
• Much of Capaldi’s work is done in a straight alleyway
– To examine PREE sometimes there would be food at the
end (R) and sometimes not (N)
• Capaldi examines behavior across specific sequences
of trials
– RNNRRNR
• The underlined trials represent rewarded trials that were
preceded by nonrewarded trials
• Thus the rat will have a recent memory of an N trial that is
essentially reinforced.
– Leading to persistence in extinction
– The longer the strings of Ns that the animal has
experienced that eventually lead to RF
• The more resistance to extinction
• Both theories are likely correct.
– Frustration theory seems to do a better job of
explaining PREE when trials are spaced out
• Long ITIs
– Sequential theory does better when trials are
close to one another (easy to remember)
• Short ITIs