Aversive Control: Avoidance
and Punishment
Avoidance/Escape
• Escape: getting away from an aversive
stimulus in progress
• Avoidance: preventing the delivery of an
aversive stimulus
• Negative contingency between
response and aversive stimulus
• Increase in operant responding
Brogden et al. (1938)
• Guinea pigs
• CS = tone, US = shock,
UR = pain, CR =
running
• Classical conditioning
group
– CS followed by US
• Avoidance group
– CS -- CR --> no US
– CS -- no CR --> US
Discriminative Avoidance
• Stimulus signals onset of aversive US
Avoidance
Escape
CS
CS
US
US
R
R
Shuttle Box
• Standard experimental paradigm
Escape
•
•
•
•
In presence of aversive stimulus
Make response
Aversive terminated
Negative reinforcement
Avoidance “Paradox”
• Make response before aversive
delivered
• Behaviour clearly increases, so
reinforcer
• But what is taken away (or delivered)?
• Mowrer & Lamoreaux (1942)
– “…not getting something can hardly, in and
of itself, qualify as rewarding.”
Two-Process Theory
• Two mechanisms: classical and
instrumental
– 1. Classical conditioning process activated
by CS when avoidance not made; CR of
fear produced
– 2. Negative reinforcement: successful
avoidance removes fear caused by CS
• Classical and instrumental conditioning
processes are independent
• Avoidance = escape from fear, not
prevention of shock
Acquired Drive Experiment
• Phase 1: condition fear to CS through
classical conditioning procedure
• Phase 2: let subject make operant
response to terminate CS
– No shock
• Drive to avoid learned through classical
conditioning
Brown and Jacobs (1949)
•
•
•
•
Rats in shuttle box
Experimental and control groups
Phase 1: light/tone CS --> shock
Phase 2: CS --> no shock; turn CS off by
crossing barrier
• Measure: time to change sides
• Supports two-process theory
• Termination of fear CS drives operant
response
Rescorla & LoLordo (1965)
• Dog in shuttlebox
– No signal
– Response gives “safe time”
• Pair tone with shock
– Tone increases rate of response
• CS+ can amplify avoidance
• CS- can reduce avoidance
Problems for Theory
• Fear a necessary component
• Fear reduction with experience
Kamin, Brimer & Black (1963)
• Rats
• Lever press in operant
chamber for food
• Auditory CS+ for shock;
avoidance in shuttle box
until: 1, 3, 9, 27
avoidances in a row
• CS+ in operant chamber;
check for suppression of
lever press
Alternation of Behaviour (Yo-yo)
• Every successful avoidance puts CS on
extinction
• With extinction, fear drops, so
motivation to avoid decreases
• Resulting in more shocks, strengthening
CR again and increasing avoidance
response
• But… we don’t really see this
Persistence of Avoidance
• Sometimes a
problem
• Phobias
• Need to extinguish
avoidance
• Flooding, response
prevention
Sidman Free-Operant
• Can avoidance be learned without warning
CS?
• Shocks at random intervals
• Response gives safe time
• Extensive training, but rats learn avoidance
(errors, high variability across subjects)
Hernstein & Hineline (1966)
• Rapid and slow shock rate schedules
• Response switches from rapid to slow
• Shift back to rapid random so no time
signal
• Response produces shock reduction
Reduction of Shock Frequency
• Molar account
• Response reduces in amount of shocks
over long run
• Negative reinforcement
– Overall shocks taken away, behaviour
increases
Safety Signals
• Molecular account
• Positive reinforcement
• Context cues associated with “safety”
– Either SD or CS-
• Making response gives safety
• Giving explicit stimuli makes avoidance
learning easier
SSDRs
• Species-specific defense reactions
• Innate responses; evolved
• SSDRs predominate in initial stages of
avoidance
• Hierarchy
– If first SSDR works, keep it
– If not, try next, etc.
• Aversive outcome (punishment) is the
selector of appropriate avoidance response
SSDRs
• Fight, flight, freeze
• Also thigmotaxis, defensive burying, light
avoidance, etc.
• Environmental content influences selected
SSDR
– E.g., freezing not useful if predator right in front of
you…
• Some responses easier to learn than others
– E.g., rats: wheel run --> avoid shock (easy)
– E.g., rats: rear --> avoid shock (hard)
Predatory Imminence
• Different innate defensive behaviours at
different danger levels
Differences from SSDR
• 1. Behaviours in anticipation, not
response
• 2. Predatory imminence, not
environmental cues leads to response
• 3. Not selected via punishment
Punishment
• Positive punishment
– Delivery of stimulus --> reduction in
behaviour
• Negative punishment
– Removal of stimulus --> reduction in
behaviour
• Time out
• Overcorrection
Introduction of Punisher
•
•
•
•
Effective use of punishment
Tolerance
Start with high(er) intensity
Can then reduce and behaviour will
remain suppressed
Response-Contingent vs.
Response-Independent
• Does your response cause the aversive outcome?
• More behavioural suppression if aversive stimulus
produced by operant response
Phase 1: train on VI-60 sec
FR-3
punishment
light
response-independent
punishment
Yoked
Suppression ratio
Phase 2: tone
light
tone
Trials
Delay
• Interval between response and delivery
of aversive
• Longer the delay, less suppression of
behaviour
Punishment Schedule
• Continuous or intermittent schedules
• Azrin (1963)
– Different FR punishment schedules; responding
maintained with VI reinforcement
Cumulative responses
no punishment
FR 1000
FR 500
FR 100
FR 5
Time
Positive Reinforcement
Schedules and Punishment
• Without some positive reinforcement,
behaviour generally stops quickly
– As in previous study, responding maintained with
appetitive outcome on VI schedule
• Interval
– Overall decrease
– VI: suppressed but stable
– FI: scalloping
• Ratio
– Increases post-reinforcement pauses
Alternative Sources of
Reinforcement
• Options
• Availability of
reinforceable alternatives
increases suppression of
punished response
no punishment
Punishment,
no alternative
response
available
Cumulative responses
– No alternatives but
punished behaviour
– Alternative behaviours (e.g.,
differential reinforcement
schedules; DRA, DRI, etc.)
Punishment,
alternative
response
available
Time
SD for Punishment
• Suppression limited
to presence of SD
• E.g., garden owl
• E.g., cardboard
“cops” and “kids”
Punishment as SD for
Availability of Pos. Reinf.
• Sometimes punishment seeking
behaviour
• Punisher becomes S+ for positive
reinforcement
• E.g., masochism, children seeking
attention
CER Theory of Punishment
• Estes (1944)
• Conditioned suppression
– E.g., freeze prevents lever press
• CER incompatible with making response
• Punishment suppresses behaviour through
same mechanism
• In real world, no explicit CS
– Stimuli immediately before punished response
serve this function
• Estes (1969): incompatible motivational state
Avoidance Theory of
Punishment
• Tied to two-process theory
• Engage in incompatible behaivour
– Prevents making punished behaviour
• Strengthening of competing avoidance
response
– Not weakening of punished response
• Same theoretical problems of avoidance
Negative Law of Effect
• Thorndike (1911)
– Positive reinforcement and punishment are
symmetrical opposites
• Similar to Premack Principle
– Low probability behaviours reduce high
probability behaviours
– Forced to engage in low-valued behaviour
after doing high probability behaviour