Outline
•
B.F. Skinner
–
–
Biography
Theoretical notions
>
>
>
>
>
>
>
Respondent and Operant Behaviour
Operant Conditioning Principles
The Skinner Box
Shaping and Extinction
Superstitious behavior
Discriminative responding
Secondary reinforcers
“Stimulus, response! Stimulus, response! Don’t
you ever think?”
B. F. Skinner
• Biography
B. F. Skinner (1904-1990)
– Wanted to be writer
>B.A. in English Literature (1926)
– Entered graduate school at Havard (1928)
>Mentored by the Chair of Physiology (W. Crozier)
 Who studied the “animal as a whole” without
appealing to internal processes.
>Obtained his Ph.D. in 1931
– Taught at University of Minnesota (1936 - 1945)
>Published “The Behaviour of Organisms” (1938)
– Affiliated with Havard until he died (1990)
B. F. Skinner
• Biography
– Inventions
> The air-crib
B. F. Skinner (1904-1990)
 Easily-cleaned, temperature and humidity-controlled
 Somewhat controversial (but effective)
 Commercially manufactured
>Project pigeon
 Received a 25K grant to develop a cruise missile
• Guided by trained pigeons
• U.S. Navy passed on it
(but retested the idea
in the 1980s)
B. F. Skinner
• Biography
– Inventions
B. F. Skinner (1904-1990)
> The Skinner Box (AKA, operant chamber)
 Animal can respond multiple times
 Operant response: Bar pressing
 Operant conditioning: Increased bar pressing when food is
delivered following the response.
B. F. Skinner
• Biography
– Inventions
B. F. Skinner (1904-1990)
> Cumulative recorder
 Keeps track of the animal’s responding
• Time is recorded on the ‘X’ axis
• Total number of responses is recorded on the ‘Y’ axis
• Faster rates of responding lead to steeper slopes
B. F. Skinner
• Major Theoretical Notions
– Respondent and operant behaviour
B. F. Skinner (1904-1990)
>Respondent behaviour - Behaviour elicited by a
known stimulus
 E.g., Unconditioned responses
• Elicited by unconditioned stimuli
• Reflexive
>Operant behaviour - Behaviour not elicited by a known
stimulus
 E.g., Most of our everyday behaviour
• Occurs spontaneously
B. F. Skinner
• Major Theoretical Notions
– Type S and Type R conditioning
> Two kinds of conditioning
B. F. Skinner (1904-1990)
 Type S (respondent conditioning) - classical
(Pavlovian) conditioning
• ‘S’ to emphasize the role of the eliciting Stimulus
• Strength is measured by the magnitude of the CR
 Type R (operant conditioning) - learning that
involves operant behaviour
• ‘R’ to emphasize Response
• Strength is measured by the response rate
B. F. Skinner
• Major Theoretical Notions
B. F. Skinner (1904-1990)
– Type S and Type R conditioning
> Comparison with Thorndike’s approach
 Thorndike’s puzzle box
• Dependent variable was time-to-solution
-> I.e., how long it took to learn a (single) response
 Skinner
• Dependent variable was rate of responding
-> I.e., What variables affect the rate
of responding
Edward Lee Thorndike (1874 - 1949)
B. F. Skinner
• Major Theoretical Notions
– Operant conditioning principles
B. F. Skinner (1904-1990)
>Two general principles
 Any response that is followed by a reinforcing
stimulus tends to be repeated
 A reinforcing stimulus is anything that
increases the rate with which an operant
response occurs
• I.e., anything that increases the probability
of a response’s re-occurring
B. F. Skinner
• Major Theoretical Notions
– Operant conditioning principles
B. F. Skinner (1904-1990)
>Contingent reinforcement
 Emphasis on behaviour and its consequences
• Gaining reinforcement depends (i.e., is contigent)
on making the appropriate response
 Culture as a set of reinforcement contigencies
• Different cultures reinforce different behaviour
patterns
 Controlling reinforcement -> controls behaviour
• E.g, child rearing
B. F. Skinner
• Major Theoretical Notions
B. F. Skinner (1904-1990)
– Conditioning the lever-pressing response
>Three steps
1. Deprivation
• Food/water deprived for 23 hours per day
• Animal is held at 80% of its free-feeding body
weight
2. Magazine training
• Food pellets are delivered by the experimenter
• Animal learns to associate the sound of the
delivery mechanism with food
B. F. Skinner
• Major Theoretical Notions
B. F. Skinner (1904-1990)
– Conditioning the lever-pressing response
>Three steps
3. Lever pressing
• Animal is placed in the box
• Eventually hits the lever (operant response)
• Delivery of food pellet reinforces the response.
B. F. Skinner
• Major Theoretical Notions
B. F. Skinner (1904-1990)
– Conditioning the lever-pressing response
>Shaping
 A faster method of teaching the rat to lever press
 Two components
1. Differential reinforcement
-> Some responses are reinforced, others are not
2. Successive approximations
-> Only reinforce responses that become
progressively closer to the desired response
B. F. Skinner
• Major Theoretical Notions
– Extinction and spontaneous recovery
B. F. Skinner (1904-1990)
>Extinction - removing the reinforcer removes the
operant response
>Spontaneous recovery - The reoccurrence of a
Rest
Cumulative responses
response that had been extinguished, with no additional
training.
Extinction
Time
Spontaneous
recovery
B. F. Skinner
• Major Theoretical Notions
– Superstitious behavior
B. F. Skinner (1904-1990)
>What if we delivered pellets noncontingently?
 Random behaviour would get reinforced
• E.g., Dog running in circles while waiting to get fed
 Humans are susceptible to similar conditioning
• E.g., Athletes/ coaches’ game rituals
B. F. Skinner
• Major Theoretical Notions
– Discriminative operant
B. F. Skinner (1904-1990)
>Skinner box is set so that reinforcement is only
available when the light is on.
 The light is the discriminative stimulus
• I.e., indicates that reinforcement is available
 SD = light on, SD = light off, SR = reinforcing stimulus
• A discriminative operant is symbolized as:
SD -> R -> SR
• Skinner was interested in the SD -> R association
-> Cf. respondent conditioning
 Stimulus control of behaviour
B. F. Skinner
• Major Theoretical Notions
– Secondary reinforcement
B. F. Skinner (1904-1990)
>A neutral stimulus paired with a reinforcer can
take on reinforcing qualities of its own
>To test this notion
 Lever press -> light -> food
 Extinguish the response
• Neither light nor food is delivered
 Allow lever - press to deliver light (not food)
• Response rate increases
B. F. Skinner
• Major Theoretical Notions
– Secondary reinforcement
B. F. Skinner (1904-1990)
>A secondary reinforcer can be used to reinforce
other responses
 Clicker training for dogs
• Warning: secondary reinforcers can be
extinguished!
 Money for humans
• Generalized reinforcer