Operant Conditioning
Complex Learning
 Why do we learn new behaviors?
 Classical conditioning only deals with reflex
responses that we already possess.
 Most of our behaviors are voluntary.
Volitional. Stimulated by something in our
environment.
Operant Conditioning
 Defined as - the form of learning concerned
with changes in emitted responses as a
function of their consequences.
Origins of Operant
Conditioning
 Edward Thorndike
 Instrumental
Conditioning
 “Law of Effect”


Satisfying outcome
Unsatisfactory outcome
Outcomes of Thorndike’s Work
How Long - the length
it declined
ofAs
time
it tooklearning
the cat to
was taking
escape
fromplace.
the puzzle
This change in
box.
performance
represented a change
in behavior from
experience.
Learning
Question
 In Thorndike’s terms, what sort of things
give you satisfaction? What things produce
dissatisfaction? Why?
Edward Thorndike
 His research provided a foundation for the
study of “non-reflexive” learning.
 He drew a connection between action and its
outcomes.
B. F. Skinner
 Skinner coined the
term “operant”.
 Disagreed with the
“soft” concepts of
Thorndike’s
“satisfying” and
“unsatisfactory”
outcome(s)
B. F. Skinner
 Operant Conditioning
replaced Thorndike’s term
“instrumental learning”
 Emitted behavior is now
called “operant responses”
 Classical conditioning is
now called ‘respondent
conditioning.
The Skinner Box or “autoenvironmental chamber”
Skinner Box in Action
Zack Florin '99 using a Skinner box to shape a rat's behavior
Reinforcment
 Primary reinforcers - food, water, shelter.
Those innate biological needs.
 Secondary reinforcers (Conditioned
reinforcers) - something that will provide a
primary reinforcer. (money, poker chips etc.)
Primary vs. Secondary
 Which of the following are secondary
reinforcers:





quarters spilling from a slot machine,
a winner’s blue ribbon,
a piece of candy,
an A on an exam,
frequent-flyer miles.
Reinforcement
 Negative Reinforcer - an aversive stimulus
which serves to decrease the probability of
the response in the future.
 Positive Reinforcer - a stimulus which when
applied increases the probability of the
response in the future.
Contingencies of
Reinforcement
 According to Skinner the relationship
between a response and a reinforcer is a
contingency.
 One type of contingency is “reinforcement”
Desired change in behavior
Type of reinforcer
Increases response
Decrease response
Positive reinforcer
POSITIVE
REINFORCEMENT
OMISSION
NEGATIVE
REINFORCEMENT
PUNISHMENT
Negative reinforcer
(escape, avoidance)
(withholding positive
reinforcer)
Shaping
 Some learning does not
occur in a single event.
 A series of successive
steps leads to a learned
behavior.
 Playing the piano,
swimming etc.
Applying the Principles
 When asked choose the best alternative and
explain why.

You want your 2-year-old to ask for water with a
word instead of a grunt. Should you give him
water when he says “wa-wa” or wait until his
pronunciation improves.
Applying the Principles
 When asked choose the best alternative and
explain why.

Your roommate keeps interrupting your studying
even though you have asked her/him to stop.
Should you ignore her/him completely or
occasionally respond for the sake of good
manners?.
Applying the Principles
 When asked choose the best alternative and
explain why.

Your father, who rarely writes to you, has finally
sent a letter. Should you reply quickly or wait a
while so he will know how it feels to be
ignored?.
Extinction
 What happens when
the reinforcement
stops.
 Extinction - in operant
conditioning, a drop I
responding when
reinforcement is
discontinued.
Schedules of Reinforcement
 Continuous reinforcement - every response is
followed by a reinforcer. (FR1 schedule)
 Partial reinforcement - a contingency of
reinforcement in which every response does
not get a reinforcer.
Fixed Interval Schedule
 Referred to as FI x reinforcement
contingency defined by
the amount of time that
must pass since the
previous reinforcer.
 Based on time.
 Example: pay checks
Fixed Ratio Schedule
 Referred to as FR x reinforcement
contingency defined by
the number of
responses the organism
must make in order to
get a reinforcer.
 Example: piece work.
Variable Interval Schedule
 Referred to as VI x - a
reinforcement
contingency defined by
the average time
interval which must
elapse since the last
reinforcer.
 Example: Quality
Control
Variable Ratio Schedule
 Referred to as VR x - a
reinforcement
contingency defined in
terms of the average
number of responses
required to receive a
reinforcer.
 Example: Slot Machine
Non-Contingent Reinforcement
 Random
“reinforcement”
 Development of what
Skinner called
‘superstition’ in the
pigeon.
Applying Conditioning
 We must always keep in mind that all this is
done to match the goals of psychology.
 Behavior Modification.
 Mary Cover Jones - the mother of behavior
therapy
Aversive
Positive
 Controls
Punishment
 Most used and most
misunderstood
 Occurs after the
‘offense’ has taken
place.
 Requires “contiguity”
 Encourages avoidance
behaviors.
Negative Reinforcement
Autonomic Conditioning
 Neal Miller and Leo
DiCara
 ‘proprioceptive
feedback
Biological Constraints
 Some unanswered
questions:





Equipotentiality
premise
Ethology
Species-specific
behavior
Critical period
Preparedness
• The
The
premise
that
principles
Behaviors
which
are
A
study
period
of
during
the
behavior
of
A
concept
developed
by
ofdevelopment
conditioning
willmembers
apply
characteristic
of
all
animals
in
their
natural
where
there
Martin
Seligman
to
toare
any
response
and any
of
a
particular
species.
environment.
optimal
periods
for
describe
how
species.
(instincts)
learning.
physiological
structure
influences the
occurrence of behavior
Biological Constraints
Degree of biological preparedness
Prepared
Species-specific
behavior
Unprepared
Bait Shyness
Classical
and operant
conditioning
Contraprepared
Unlearnable
Associations