Operant vs. Classical
Operant responses understood by
comprehending the consequences they
produce
vs.
Pavlovian responses may have
environmental effects, but this is not what
controls them
Edward Thorndike
Acquisition of Goal-directed responses.
Law of Effect - behavior occurs in a
random, trial-and-error fashion.
Consequences of a behavior can
increase or decrease the future
probability of that behavior.
1874-1949
Escape Time
Cat Puzzle Box
Trials
Law of Effect
Responses followed by a satisfying outcome will strengthen the
association between the situation and the response.
Responses followed by a dissatisfying outcome will weaken the
association between the situation and the response.
Law of Effect
Time 1:
Press lever  Escape from box
Strengthens box-lever press association
Time 2:
Placed in box  higher probability of
pressing lever
Time 1:
Pull string  Remain in box
Weakens box-string pulling association
Time 2:
Placed in box  lower probability of
pulling string
Discrete-Trial Procedures
W. S. Small (1900) – used the maze as a
tool to study operant learning in rats.
“complex” maze; measures = running speed, latency to
reach goal.
Hampton Court Palace Maze
T-Maze
Food
No Food
Start
Straight Alley “Maze”
Start
Food
Operant (Instrumental) Conditioning
The Skinner Box:
2 Stages of training: 1. Magazine training
2. Shaping = reinforcement of
successive approximations
Lever
Lever
Lever
Lever
Lever
Lever
Free-Operant Procedures
Discrete-trial procedures provide animal with limited
opportunities to respond.
Free-operant procedures allow the animal to
perform the instrumental response repeatedly
without constraint.
Discrete-Trial Operant Procedure
Lever
Light = Discriminative stimulus (SD)
Acquisition and Extinction of
Lever Press Response
Interval
Fixed
Variable
Ratio
Schedules of Reinforcement
FI scallop
Post-reinforcement pause
time
Size of post-reinforcement
pause determined by
response-reinforcement ratio
PREE
What can be an operant?
Practically any behavior or behavioral parameter!
rate of response
time of response
variability of response
pushes and pulls
posture
study habits
athletic performance
arts and crafts
creativity
bad habits and behavioral disorders
Allen et al. (1964)
• The case of Ann, a nursery school student
• Problems: asocial behavior, complaining,
weird habits.
• Solution: Stop paying attention to her when
she does these things.
Extinction of temper tantrums
Reinforcement without
awareness
•Conditioning in amesics – eyeblink in HM
“Memento”
•Radio static study
Subjects told it was an experiment on stress.
Twitching of very small thumb muscle actually
terminated the aversive stimulus (harsh noise)
•Students condition professor
Pigeon in a Pelican
Ginger
Does anyone know how google
works?
google
Marian Breland Bailey – How to train a
chicken
The famous dancing chicken
Limits on Operant Conditioning
• Instinctive drift – “misbehavior.”
• Belongingness effects – reinforcer/behavior
specificity e.g., behavior systems effects.
• Unintended consequences –Doing one thing
precludes doing other things.
• Some responses cannot be conditioned.
– E.g., yawning or sneezing.
Factors that affect O.C.
• The response
– Maze running vs. lever pressing
– Running vs. throwing football
• Temporal contiguity
Delayed reinforcement
Overcoming the effects of delay
• Secondary reinforcers
• “Marking” procedure
Factors that affect O.C.
• The response
– Maze running vs. lever pressing
– Running vs. throwing football
• Temporal contiguity
• Contingency
“Superstitious Behavior”
• Suggested that temporal contiguity more
important than contingency
• 15-s FT, no response requirement
• “adventitious reinforcement”
“In 6 out of 8 cases the resulting responses were so
clearly defined that two observers could agree
perfectly in counting instances. One bird was
conditioned to turn counter-clockwise about the cage,
making 2 or 3 turns between reinforcements. Another
repeatedly thrust its head into one of the upper corners
of the cage….”
Orienting
toward feeder
Pecking
near feeder
Moving
along wall
¼ turn
Degraded Contingency Effect
= bar press
= food
= light CS
Response
Perfect
contingency
Strong
Degraded
contingency
Weak
Signal extra
food
Strong
Factors that affect O.C.
• The response
– Maze running vs. lever pressing
– Running vs. throwing football
• Temporal contiguity
• Contingency
• The reinforcer
– Magnitude, quality
– Upshifts and downshifts
Anticipatory Contrast - Crespi (1942)
Running Speed (ft/sec)
Rats run down maze to find food pellets in goal arm.
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
256-16 Pellets
16-16 Pellets
1 - 16 Pellets
2 4 6 8 10 12 14 16 18 20 2 4 6 8
Trials
Reinforcement of Variability