Schedules of Reinforcement
CH 17,18,19
Divers of Nassau
• Diving for coins
• Success does not follow every attempt
• Success means reinforcement
Intermittent Reinforcement
• A reinforcer follows the response only once
in a while.
Continuous Reinforcement
(CRF)
• A reinforcer follows every response.
• Best used for shaping or maintaining
difficult behavior.
Shaping with Reinforcement
Reinforcement
Before:
Andrew
has no gum
Behavior Initial:
Andrew moves lips
After:
Intermed: Andrew
makes croaking sound
Andrew
receives
gum
Terminal: Says words
clearly
Behavior Initial: NA
Intermed: Moves lips
only
Terminal: Says words
unclearly
Extinction
After:
Andrew
receives no
gum
Schedule of Reinforcement
• The way reinforcement occurs because of
the number of responses, time between
responses, and stimulus conditions.
Schedules of Reinforcement
• Ratio
– Fixed Ratio (FR)
– Variable Ratio (VR)
• Interval
– Fixed Interval (FI)
– Variable Interval (VI)
Fixed Ratio (FR) Schedule of
Reinforcement
• A reinforcer follows a fixed number of
responses.
FR Responding
• After a response is reinforced, no
responding occurs for a period of time, then
responding occurs at a high, steady rate
until the next reinforcer is delivered.
Reinforcers
Reinforcement
always immediately
follows the 5th
response!!!!
This is an FR5
schedule of
reinforcement.
Pauses
Postreinforcement Pause (PRP)
• The name for the pause after the
consumption of the reinforcer and before
the next ratio of responses begins.
FR & PRP
• Length of the pause is proportional to the
size of the ratio.
• Small ratio – pausing is short
• Large ratio – pausing is long
General Rule for establishing
intermittently reinforced
behavior:
First use CRF and gradually
increase the intermittency of
reinforcement as responding
stabilizes at a high rate.
FR120
• What are the response requirements?
• When will reinforcement be delivered?
• What will the pattern of responding look
like?
Cumulative Number of Responses
Cumulative Graph for FR
140
120
100
80
Post
reinforcement
pauses
60
40
Reinforcers
20
0
Time
Variable Ratio (VR) Schedule
• A reinforcer follows after a variable number
of responses
VR Responding
• VR schedules produce a high, constant rate
of responding, with almost no
postreinforcement pausing.
VR 50
180
Cumulative Number of Responses
160
140
120
100
80
60
40
20
0
Time
VR 50
• Schedule notation
• Number designates the average number of
responses required for reinforcement
How do typical schedules of
reinforcement differ from
gambling?
• See pages 290-291
Skinner Box vs. 1 Armed Bandit
Typical VR
Many interspersed
learned reinforcers
No
Gambling
Schedule
Yes
Amount of reinforcer
varies from ratio to ratio
No
Yes
Small ratio
No
Yes
Emotional reinforcers
No
Yes
Time Dependent Schedules
• Fixed Interval (FI)
• Variable Interval (VI)
FI Schedule of Reinforcement
• A reinforcer is contingent on the first
response after a fixed interval of time since
the last opportunity for reinforcement.
Cumulative Number of
Responses
Fixed Interval 10s Schedule of Reinforcement
200
150
100
Reinforcer
50
0
0
10
20
Seconds
Example: Fixed Interval (FI) Schedule of
Reinforcement
FI 200-s
Fixed Interval 200-s Schedule of Reinforcement
Cumulative Number of Responses
200
180
160
140
120
100
80
Reinforcer
60
40
20
0
Seconds
200 seconds
400 seconds
600 seconds
Fixed-Interval Scallop
• A FI schedule often produces a scallop – a
gradual increase in the rate of responding
with responding occurring at a high rate just
before reinforcement is available. No
responding occurs for some time after
reinforcement (Post Reinforcement Pause.
Example: Fixed Interval (FI) Schedule of
Reinforcement
FI 200-s
Fixed Interval 200-s Schedule of Reinforcement
Cumulative Number of Responses
200
PRP
180
160
140
Scallop
120
100
80
Reinforcer
60
40
20
0
Seconds
200 seconds
400 seconds
600 seconds
Lever Pressing of Mice after Training
Fixed Interval 200 s
A
Relative Frequency
0.3
+/+
+/rl
rl/rl
0.2
0.1
0
20
40
60
80
100 120 140 160 180 200
Time (sec)
Term paper vs. FI
• Term paper does not have a deadline.
FI vs. Term Paper
FI
Term Paper
Does early responding
effect anything?
No
Yes
Do you get more if you
work harder?
No
Yes
Is the relevant response
class clear?
Yes
No
Are there calendars and
clocks?
No
Yes
Is there a deadline?
No
Yes
Is the reinforcer too
delayed?
No
Yes
Fixed Time & Superstitious
Behavior
• Fixed time schedules of reinforcement
– A reinforcer is delivered after the passage of a
fixed period of time, independently of the
response.
• Superstitious behavior
– Behaving as if the response causes some
specific outcome when it really does not.
Superstitious behavior of the
pigeon:
Experiment by Skinner
Fixed Time Schedule
Variable Interval (VI) Schedule
of Reinforcement
• A reinforcer is contingent on the first
response after a variable interval of time
since the last opportunity for reinforcement.
VI Responding
• VI schedules produce a moderate rate of
responding with almost no
postreinforcement pausing.
Comparing/Contrasting Ratio and
Interval Schedules
• Pg. 305: table
• Pg. 305: cumulative records
Continuous vs. Intermittent
• CRF – every response is reinforced
• Intermittent reinforcement – only some
responses produce a reinforcer.
Intermittent Reinforcement
• Makes the response more resistant to
extinction than does continuous
reinforcement.
Resistance to Extinction
• The number of responses or the amount of
time before a response extinguishes
Resistant to Extinction and
Intermittent Reinforcement
• Intermittent reinforcement makes the response
more resistant to extinction than does continuous
reinforcement.
• Why? Hard for the rat to tell the difference
between only an occasional reinforcement and no
reinforcement
• CRF followed by EXT: rats quickly stop
responding: easy discrimination between
reinforcement & extinction contingencies
4 Types of Concurrent
Contingencies
1. Two compatible responses
2. Compatible contingencies for a single
response
3. Incompatible contingencies for a single
response
4. Two incompatible responses
Concurrent Contingencies
• Contingencies that are available at the same
time.
• More than one contingency of
reinforcement or punishment is available at
the same time.
Concurrent Contingencies
• Two levers in chamber
– VI 300 s on Left lever
– VI 30 s on Right lever
• Will animal allocate more responses to the
left or the right lever?
• Matching Law (pg. 330)
% of Left lever presses = % Left-lever reinforcers
Matching Law
• Relative Frequency of responding on two
concurrent schedules of reinforcement
equals the relative value of reinforcement
on those two schedules