Extinction
How do animals “know” when a
schedule is on extinction?
Remember
• Operant conditioning extinction differs from
classical conditioning extinction
• Responds decreases to near zero for both
• Operant conditioning:
– Transient increase
– Extinction induced aggression
Partial Reinforcement
Extinction Effect: PREE
• Extinction occurs at different rates depending on the schedule:
– Continuous reinforcement: FAST extinction
– Partial reinforcement schedules: SLOWER extinction
– Variable schedules show slower extinction than fixed (rate or time)
schedules.
• PREE used to describe greater persistence in instrumental
responding during extinction after partial (or intermittent)
reinforcement training
– Faster extinction after continuous reinforcement training.
• Partial reinforcement schedules show RESISTANCE TO EXTINCTION
Other Extinction Effects
• magnitude reinforcement extinction effect
– Less persistence of instrumental behavior in extinction
following training with a large reinforcer
– More persistance of responding with a small or moderate
reinforcer.
– Effect is most prominent with continuous reinforcement.
• overtraining extinction effect
– Less persistence of instrumental behavior in extinction following
extensive training with reinforcement (overtraining)
– Faster extinction following moderate levels of reinforcement
training.
– Again, effect most prominent with continuous reinforcement
Other Extinction Effects
• reinstatement
– Recovery of responding to an extinguished stimulus
– produced by exposures to unconditioned stimulus or
reinforcer
• renewal
– Recovery of excitatory responding to an extinguished
stimulus
– produced by shift away from the contextual cues that
were present during extinction.
Why do we get these effects?
• Behavioral momentum and Sequential theory
• Trouble with discrimination
• Frustration
Behavioral Momentum
• Suggests that PREE occurs because the animal has a high momentum of
responding and it is more difficult to stop this momentum
• Timberlake and Lucas 1985:
–
–
–
–
Ball bearing studies
Rolled ball bearing across cage; rats had to let it go past to receive reinforcer
Played with the ball bearing, slowing reinforcement
During extinction (ball bearing but no food): played with ball bearings MORE
• Does suggest that animals show strong patterns of behavior that may
interfere and thus slow the extinction process
• But not a complete explanation
Discrimination and Frustration
• Discrimination hypothesis:
– Mowrer and Jones 1945
– In order for subjects’ behavior to change during extinction, the
subject must be able to discriminate the change in
reinforcement contingencies
• With CRF: This is immediately noticeable
• With PRF: not immediately noticeable
– More discriminative on fixed schedules
– Less discriminative on variable schedules
• Evidence does not completely support this
Generalization Decrement Hypothesis
• Capaldi, 1966
• Generalization decrement: decreased responding observed
in generalization test when test stimuli become less and
less similar to training stimulus
• Responding during extinction is weak if the stimuli present
during extinction are different from those during the
reinforcement phase
• Responding during extinction is STRONG if the stimuli
present during extinction are very similar to those during
reinforcement phase.
Generalization Decrement Hypothesis
• Large generalization decrement when schedule moves
from CRF to EXT
– Subject never experienced situation in which some of its
responses are not reinforced
– Not been taught to keep responding in absence of a
reinforcer
• Small generalization decrement when schedule moves
from PRF to EXT
– Subject has experience in situation where some of its
responses are not reinforced
– HAS been taught to keep responding in absence of a
reinforcer
Sequential Theory
• sequential theory: memory of reward vs. non reward
– Cognitive theory
• Fast extinction after CRF
– Extinction occurs quickly because the instrumental
response has NOT been conditioned to the memory of
nonreward
• Slow extinction during PREE
– extinction is slowed after partial reinforcement because
the instrumental response becomes conditioned to the
memory of nonreward.
Bottom Line
• Rate of extinction is affected by:
– Discriminability of the situation
– Previous experience with non-reinforced responding
– Possibly, memory of non-reinforced responding
• Why frustration?
– Not getting what expected
• Because of contingency between response and reinforcer:
– Subject “controls” responding, and thus reward
– During EXT: this contingency is disrupted
– Subject becomes “frustrated” in that
•
•
•
•
Tries other responses
Increases magnitude of responding
Increases intensity of responding
Pushed into fear/flight/fight sequence: aggression is elicited
Avoidance:
Avoidance Tests
•
Negative reinforcement = removing a stimulus to INCREASE a behavior
•
Negative reinforcement =
– escape: a response removes something
– avoidance: a response prevents some event
•
Procedure for studying negative reinforcement and avoidance: Discriminated
avoidance:
– a response CANCELS a shock
– Organism is responding for food reinforcers
– When light comes on, must press another lever to AVOID the shock
•
•
•
if the response does not occur during the S+ the stimulus is followed by a shock
if the response does occur during the S+, the shock is cancelled
–
–
thus: signal or sD for shock
if this were an escape: response could also occur DURING the shock to shut off shock
Two Avoidance Procedures:
• Sidman Avoidance:
–
–
–
–
–
the response POSTPONES or DELAYS the shock
thus: only temporary solution
must keep responding to keep delaying the shock
results in lots of responding
again: some signal may be used to signal when must respond
• Herrnstein and Hineline Procedure:
–
–
–
–
the response reduces the rate of the shock
note: note delay or cancel, just slows down rate of delivery
the response switches the schedule of shock to a lower rate
Note: cannot entirely AVOID shock in this procedure:
• once animal receives shock on lowered schedule, reverts back to original
schedule
• animal must respond again to switch schedule again
Characteristics of Avoidance Behavior
• extremely variable:
– from subject to subject
– from session to session with SAME subject
– procedure to procedure
• choice of response is important
– determines how quickly will learn contingency
– how well learning is maintained
• 1-way vs 2-way shuttle avoidance tests:
– 1-way shuttle: run to other end w/sD
– 2-way shuttle: run to opposite end w/sD
– rat will learn to run to other end of shuttle box when sD comes on to avoid
shock
•
difficult time learning 2-way shuttle avoidance
Characteristics of Avoidance Behavior
• Species specific Defense reactions:
– behaviors which animal does naturally in time of
danger
– includes: freezing, fleeing, fighting
• Why?
– animal has innate behaviors does when avoiding
noxious stimulus– can't make it go against its nature
Characteristics of Avoidance Behavior
• Negatively reinforced behavior is difficult to
extinguish:
– escape behaviors take long time to go away
– e.g.: rat in 1-way shuttle still runs when light
comes on-even after hundreds of EXT trials
• BUT: will extinguish quickly if animal can
detect change from conditioning to EXT
situation
Negative Reinforcement in Humans
• most often "reinforcement" technique used in
real world
• often used because is cheaper, easier, more
natural
• produces "bad" side effects: avoidance
responses to sD = boss, principal, spouse, etc.
Theories of Avoidance: Two Factor
theory
•
Two things happen during avoidance conditioning:
– animal learns to fear S+ via class. cond'ing
•
•
CS (light)---> US (shock): UR (fear)
animal learns to fear light via pairing with shock
– animal will then learn a response to AVOID shock and thus remove/lessen their fear
•
thus: not getting shocked reduces fear that was signaled by the CS
•
experimental evidence:
– on initial training trials:
– light/CS produces physiological symptoms of fear
– escape response results in decrease in these physiological symptoms
•
on later trials:
– little or no evidence of physiological fear with CS presentation
– suggests fear has been reduced/replaced by the escape response
•
in sense: forms a negative feedback loop
Two Factor theory in Humans
• many ineffective and/or irrational fears
– often involve avoidance responses due to original fear
– maintained by decrease in fear
– e.g., banging two sticks to keep the tigers away
• Symptoms of obsessive/compulsive disorders:
– compulsions = repeated, stereotyped, ritualized actions
• individual feels compelled to engage in them
– obsessions = compulsive thoughts (no actual actions)
– many, many examples of this
– can begin to interfere in life
Problems with 2-factor theory:
• Signs of fear dissipate w/time:
– as animal gets "better" at avoidance response
– thus: no fear to be avoided
• the CS is not as important in avoidance learning
as 2-factor theory states:
• animals can learn to avoid in a discriminated
avoidance situation long before there is any sign
that they are responding to/detecting the CS
Herrnstein and Hineline:
Test of 2-factor theory
•
Herrnstein and Hineline procedure:
– not use any CS, but the animal still learns to lessen/avoid the shock
• Test of the theory:
– two groups of rats used
– Group 1: can turn off light, but still shock
– Group 2: can turn off shock, light still on
•
2-factor theory would predict that Group 1 should respond more,
because this would be cancelling the CS that produces fear
• results: group 2 responds much more accurately, faster
Alternative: One-Factor Theory
• responses occur whenever they reduce the rate
at which aversive events occur
• when a CS is present: only providing information
about the effectiveness of a response
• fear may be a by-product of avoidance training,
but not crucial to learning/ maintaining an
avoidance response
Evidence for One-Factor theory
• Almost postulating a "cognitive" theory of avoidance:
• Seligman and Jonston (1973) did postulate cognitive
theory:
– like Rescorla Wagner theory in that deals with predictability
• Basic premise:
– learning occurs only when discrepancy between observation
and expectation
– subjects' behavior will change in avoidance task whenever
there is a discrepancy between expectancy and observation
Evidence for One-Factor theory
• Two important expectations in avoidance task:
– expectation about consequences of a response
– .expectation about consequences of not responding
• Data support One-factor theory
– on trial 1: no expectations
– On trial 2 (and more): expectation about what will happen
• no shock will occur if response is made
– shock will occur if no response is made
– animal prefers no shock to shock- so responds
• Contingency is what is important in avoidance, fear is by-product!
Evidence, con’t.
• as long as animal continues to respond- no shock
– not know when extinction occurs- no sampling
– only stop when learn situation has changed
• Thus: to EXT responding:
–
–
–
–
Must use response blocking or flooding:
present sD, but prevent R from occurring
thus animal learns that shock no longer comes
animal stops responding in presence of sD
Flooding
• To extinguish an inappropriate response: must make
contact with "changed reinforcement or punishment"
situation
• sometimes used as alternative to systematic desensitization
• flood with presentation of fear-provoking stimulus
– Again, no actual consequence occurs
– Continue presentations until the response is extinguished
• Problem: may "scare the patient to death"