Chapter 9
Adjusting to Schedules of Partial
Reinforcement
Schedules of Reinforcement


Specify the conditions the individual has
to satisfy in order to obtain the
reinforcing event
These conditions can include…



How many times the individual must
perform the behavior
When the behavior has to be performed
Performing a certain pattern of behaviors
Classifying Schedules of
Reinforcement


Continuous Reinforcement = provides the
consequence for each occurrence of the target
behavior
Schedule of Partial (Intermittent) Reinforcement =
specifies which instances of the target behavior are
followed by the consequence



It is the type of schedule established when not every instance of the
target behavior is followed by the consequence
Most of these schedules require that the individual either perform
the target behavior a certain number of times, perform that
behavior after a certain time interval has elapsed or some combo of
both
e.g., The Lottery takes advantage of the fact that some individuals
will continue to play even though they win only occasionally
3 Most Common Classes of
Schedules



1) Ratio Schedules = based on the number of
individual actions required to obtain a
reinforcer
2) Interval Schedules = behaving after a
certain time interval has elapsed in order to
obtain a reinforcer
3) Interresponse Time Schedules = spacing
individual actions a certain time apart in order
to be rewarded
Ratio Schedules






Specify the number of times an individual has to do
something in order to obtain a certain consequence
That is the only requirement
The consequence occurs each time the requirement
is satisfied
Direct Variables = formally imposed requirements of
a schedule
Indirect Variables = unstated requirements
These variables affect how the individual adjusts to it

e.g., in all ratio schedules, the faster the individual gets
through the ratio, the faster he or she earns the
consequence
Ratio-Schedules Cont.


Ratio schedules tie the rate of reinforcement
directly to the rate of behavior
Molar Feedback Functions = define the relationship
b/w the rate of performance of the target behavior
and the rate at which the consequence is earned



These are called molar functions b/c they relate the overall rate of
reinforcement to the overall rate of behavior
Molar feedback functions for ratio schedules encourage high rates of
behavior b/c pausing & dawdling delay earning the next reward
Molecular Feedback = involves the moment-tomoment relationship b/w behaviors and
consequences
Ratio Schedules Cont.


Ratio schedules are more likely to reinforce
short interresponse times (IRTs) rather than
long ones b/c the IRT that is followed
immediately by reinforcement is likely to be a
short one, and short IRTs are synonymous
with rapid behavior
Most common types of ration schedules:


1) Fixed Ratio
2) Variable Ratio
Fixed-Ratio Schedules (FR n)


The number of times the target behavior must
occur is the same across successive reinforcers
Symbolized as FR n where n specifies the required
number of times the target behavior must occur in
order to obtain the consequence


e.g., An FR 5 schedule means that the individual must
always perform the required action 5 times in order to
obtain a certain consequence
Continuous Reinforcement – is a special case of a
fixed-ratio schedule in the which the ratio
requirement is 1 (FR 1)
Typical Performance on FixedRatio Schedules



“Break and Run” Pattern: the “break” refers to a
pause for a period of time after the presentation of
each consequence
The length of this pause called a PostReinforcement Pause (PRP) is typically an
increasing function of the fixed-ratio requirement:
the longer the required ratio, the longer the pause
Following the PRP (the “break”), the individual
typically makes an abrupt transition into the “run,”
a high steady rate of behavior that ends with the
delivery of reinforcement
Why there is a post-reinforcement
pause on fixed-ratio schedules


The time b/w the receipt of successive
reinforcers called the Interreinforcer Interval
depends on how long the individual takes to
complete each action and the number of
actions required by the schedule
This interval affects the length of the postreinforcement pause (PRP): the longer the
interreinforcer interval, the longer the PRP
Examples of Fixed-Ratio
Schedules in Everyday Life


Working on Commission, being paid for the
number of tasks or objects completed (piece
work), or receiving a bonus for each item or
task above the minimum number
In all these 3 cases, the amount of pay
earned depends on the number of items sold
(or made) based on a pay schedule which
specifies how much is earned per sale or item
produced
Variable-Ratio Schedules (VR n)



Require the individual to perform the target
behavior a certain number of times to obtain
the consequence, but that number varies
around a certain average (n)
VR schedules generate high rates of behavior
e.g., a VR 5 schedule means that the
individual will obtain the consequence after
performing the target behavior an average of
5 times such that sometimes a single instance
is sufficient, sometimes 3, sometimes 10, and
other times perhaps 15 times
Examples of Variable-Ratio
Schedules in Everyday Life



All forms of Gambling – winning depends on
betting, and the more times you bet, the more
likely you are to win
In Basketball, not every shot is successful, the
more you shoot, the more likely you are to get a
score
These examples are different from other
schedules b/c the individual continues to work on
the schedule after receiving the reward; in all the
other schedules, once the reward is earned, the
individuals typically go on to perform other tasks
FI vs. FR Schedule
Interval Schedules




Require the individual to perform the target behavior after a
certain time has elapsed; it does not matter what the
individual does before that time
Note: it is not merely the passage of time that produces the
consequence – the individual has to do something after that
interval has elapsed in order for the consequence to occur
As long as the individual is performing the target behavior
some minimum number of times over the course of the
experimental session, he/she will earn the maximum
number of reinforcers possible, but more rapid behavior will
not earn any more.
Therefore, unlike ratio schedules, after a certain point, the
rate of reinforcement is independent of the rate of behavior
Interval Schedules Cont.


These schedules encourage lower rates of
behavior than do ratio schedules b/c pausing
and low rates of behavior do not appreciably
delay the consequence
The most common ways to program the
interval schedule requirement are:


1) To require the same interval for each reinforcer
(Fixed Interval)
2) To unsystematically vary the interval across
successive possibilities (Variable Interval)
Fixed-Interval Schedule (FI t)



The first occurrence of the target behavior
after a specified interval of time has elapsed
will produce the consequence; that interval is
the same across successive reinforcers
Symbolized as FI t, where t specifies the time
interval in whatever units are appropriate
(seconds, minutes, and so on)
e.g., an FI 1 minute schedule programs the
consequence following the first instance of
the target behavior that occurs after 1 minute
has elapsed since the last consequence
Typical Performance on FixedInterval Schedules




Performance on FI schedules is characterized by a
post-reinforcement pause followed by either a
progressively increasing rate of behavior or an
abrupt transition to the terminal rate of behavior
(break and run)
These schedules usually generate higher rates of
behavior closer to the time of reinforcement
e.g., delaying studying until just before a test
e.g., waiting for something to cook (You are
rewarded for checking your food when it is done; if
you check it before it is done, you have wasted
some effort)
Variable-Interval Schedules (VI
t)



Requires the individual to perform the target
behavior after a certain time has elapsed, but that
time is not the same for each reinforcer
These schedules generate slow, steady rates of
behavior with little or no post-reinforcement
pauses
e.g., a VI 1 minute schedule, the individual can
obtain the consequence for the first occurrence of
the target behavior after a certain time has
elapsed, but the required time will vary across
successive reinforcers with an average
interreinforcer time of 1 minute
Everyday Examples of VI
Schedules

If you are trying to call a friend on the
telephone and the line is busy (b/c that
person does not have call waiting), you
tend to wait for some time and then try
again. Repeatedly pressing the redial
button does not speed up your getting
through, but if you wait too long, that
person may hang up and take another
call
VI vs VR Schedule
Schedules of Reinforcement
The Effects of Prior Experience




How an individual adjusts to a schedule of partial
reinforcement is usually affected by his or her prior
experiences
Individuals working under one schedule of reinforcement
and then shifted to a second schedule may perform
differently on the second schedule than individuals who did
not have that prior experience with the first schedule
Repeated experience with change makes individuals more
sensitive to change so that they more readily adapt to it
These carryover effects provide another perspective on how
individuals adjust to changing circumstances
Behavioral Persistence


The tendency to continue to perform a behavior
when that behavior no longer produces a
reinforcing event
The conventional measure is called Resistance to
Extinction


how long it takes an individual to cease performing a
behavior when that behavior no longer produces the
reinforcing event (the procedure of extinction)
the longer it takes the individual to cease performing a
behavior when that behavior no longer produces the
reinforcing event, the greater the resistance to extinction
3 Factors that Influence
Behavioral Persistence

1) How much success the individual has experienced
obtaining the reinforcing event by performing the target
behavior: the greater the amount of success, the more
persistent the behavior

2) How discriminable the change in procedure is to the
individual - i.e., how quickly the individual can detect that
reinforcement is no longer available


Discriminability is determined by how similar the 2 situations are:
with respect to persistence, the more similar (less discriminable)
extinction is to training, the greater the persistence (the more
resistance to extinction)
3) Having had prior experience in which persistence has
been rewarded: experience with partial reinforcement
produces resistance to extinction
Behavioral Momentum



The tendency for a behavior to persist when
the reinforcement contingency is changed
Behaviors that are more resistant to change
have more momentum
From this perspective, amount of training
increases persistence (resistance to
extinction) b/c successful behaviors acquire
more momentum and thus become more
resistant to change when extinction is
introduced
Discriminability and
Persistence

Persistence is a function of how discriminable the
conditions of reinforcement are from the conditions
of extinction: extinction occurs faster (there is less
persistence) when these two are more easily
discriminated


Discriminability is related to Salience and is
determined in part by the physical attributes of the
stimulus and in part by what the individual brings
to the situation (sensory capacities, knowledge)
Two events are discriminable when the individual
can tell them apart
Persistence and Partial
Reinforcement

Behaviors maintained by partial reinforcement
persist longer in extinction than behaviors
maintained by continuous reinforcement


This is called the Partial Reinforcement Extinction Effect
(PREE)
The PREE is due in part to the fact that it is easier
for individuals to discriminate the difference
between continuous reinforcement and extinction
than b/w partial reinforcement and extinction, and
in part to the fact that experience with partial
reinforcement teaches individuals to tolerate
frustration
Persistence & Partial
Reinforcement Cont.

Experience with partial reinforcement
increases persistence: individuals who
experience partial reinforcement, even if their
most recent experience is with continuous
reinforcement, show more resistance to
extinction (persistence) than individuals who
only experience continuous reinforcement of
that behavior

This occurs because partial reinforcement
increases frustration tolerance