SNIFFY THE RAT
1
Sniffy the Virtual Rat
by
Graeme Hodgkinson
201005996
A laboratory report
presented to Jenna Wright
in Psychology 210
Learning
Department of Psychology
St. Francis Xavier University
April 7, 2014
SNIFFY THE RAT
2
Sniffy the Virtual Rat
The purpose of this experiment is to compare extinction times of numerous different
types of reinforcement schedules. Continuous reinforcement schedules reinforce for every good
behavior, whereas partial reinforcement schedules reinforce good behavior every now and then.
Partial reinforcement can be either fixed, requiring the same amount of bar presses or same
amount of time to pass per reinforcement, or variable, where the reward is given after slightly
changing amounts, which hover around an average value. Partial reinforcement can also be either
based on a certain amount of bar presses, or a certain amount of time. Extinction is hypothesized
to occur earlier the faster Sniffy pulls the bar. Therefore, the reinforcement schedules are likely
to exhibit extinction in this order, from quickest to slowest: Continuous, Fixed Ratio, Variable
Ratio, Fixed Interval, Variable Interval. Reinforcement schedules which cause rapid responding
condition the subject to expect much more reinforcement than slower schedules, and thus
experience high levels of frustration over a short duration when extinction mode is activated and
they constantly pressing the bar and expecting rewards. Slower schedules do not create the
expectation of reinforcement from every bar pull, and thus experience less frustration.
Method
Subjects
The subject used in the experiment is a virtual rat, Sniffy. Programmed to act similar to a
real life rat, Sniffy can learn and have its behavior modified through conditioning.
Apparatus
SNIFFY THE RAT
3
Sniffy the Virtual Rat is a software program which features Sniffy, a two dimensional
cartoon white rat with red eyes. When opened, the program creates two windows stacked on top
of each other, the top window with Sniffy in his cage, and the bottom window displaying a
cumulative record of Sniffy’s training. These two windows appear in the top left of the computer
screen, covering approximately a quarter of the area of the monitor. Sniffy’s cage has a white
floor, and grey walls. The back wall has a water spout on the left side, as well as the pellet
magazine in located in the center. Above the magazine is a bar. Pressing the spacebar on the
computer keyboard dispenses a food pellet for Sniffy. A clickable menu is at the top of the
window of the cage. In this menu is the option ‘Parameters’, which allows the user to manipulate
Sniffy’s training schedule, and also send him for a timeout. There are four types of training
schedule available: fixed, variable, continuous, and extinction. For the fixed and variable
schedules, the user also chooses whether to use seconds or responses, and fills in a numerical
value. Each window of the cumulative record depicts Sniffy's bar-pressing performance over a
period of approximately two hours of program time. As time elapses a line is drawn horizontally
across the page. Each time Sniffy presses the bar the line moves up a notch and continues along
horizontally. When a bar press is reinforced a back-slash marker appears on the line of the
cumulative record to mark the event.
The other materials used in the experiment were a personal computer running Windows
XP operating system, a computer mouse, a computer keyboard, and a stopwatch program.
Procedure
Sniffy the Virtual Rat is installed on all St. Francis Xavier University lab computers. It
was opened by double-clicking the WebFX Lab Applications shortcut on the computer’s
SNIFFY THE RAT
4
desktop, then double-clicking the Sniffy the Virtual Rat folder, then double-clicking the shortcut
for the software. The first stage of the experiment was to simply observe Sniffy’s baseline
behavior for 10 minutes, recording his bar presses per minute.
The second stage of the experiment was magazine training. Waiting until Sniffy was a
reasonable distance from the magazine, a pellet was dropped. When Sniffy ate the pellet, another
was instantly dropped. This process was done 15 times total. To test whether Sniffy was properly
trained, the experimenters waited until he was in a back corner to drop a pellet. When Sniffy
goes immediately to eat a pellet when dropped like this, he is trained. Recorded the length of
time required to complete magazine training.
The third stage of the experiment was to train Sniffy to press the bar via shaping by
successive approximation. Behaviors approximate to pressing the bar were rewarded with a
pellet. As the training progressed, Sniffy would have to elicit behavior closer and closer to
actually pressing the bar in order to get a food pellet. When Sniffy presses the bar five or more
times within one minute, the next stage begins, in which Sniffy stopped receiving reinforcement
pellets for pressing the bar. When the cumulative record reaches to top of the window, it will fall
immediately back to the bottom. After this occurred for the second time, Sniffy’s behavior and
rate of bar pressing were observed and recorded over a period of two minutes. Progress was then
saved by clicking File in the top of window menu, then Save As, then selecting w :\\alonso\ from
the dropdown Drives menu. Named CRF.cdf
The fourth stage of the experiment was to observe Sniffy’s baseline extinction rate. The
program was restarted, and the progress from stage three was opened by clicking File in the top
of window menu, then Open, then selecting CRF.cdf. Sniffy was then allowed to run free for one
SNIFFY THE RAT
5
minute under the same conditions as stage three. After one minute, extinction mode was enabled
by clicking Parameters in the top of window menu, then Training Schedules, then selecting
Extinction and clicking OK. The number of non-reinforced bar presses in the first two minutes,
as well as the total time until extinction was recorded. Sniffy is considered extinguished when he
presses the bar less than three times in five minutes. Progress was then saved by clicking File in
the top of window menu, then Save As, then selecting w :\\alonso\ from the dropdown Drives
menu. Named Ext.cdf.
The fifth stage of the experiment was to observe Sniffy’s spontaneous recovery.
Immediately following extinction, Sniffy was removed for a timeout by clicking Parameters in
the top of window menu, then clicking Remove Sniffy for Time-Out, then clicking OK. The
number of non-reinforced bar presses in the first two minutes, as well as the total time until
extinction was recorded. Progress was then saved by clicking File in the top of window menu,
then Save As, then selecting w :\\alonso\ from the dropdown Drives menu. Named Recover.cdf.
The sixth stage of the experiment was to train Sniffy with a partial reinforcement
schedule and observe extinction rate. The program was restarted, and the progress from stage
three was opened by clicking File in the top of window menu, then Open, then selecting
CRF.cdf. Sniffy was then allowed to run free for one minute under the same conditions as stage
three. After one minute, partial reinforcement was activated by Parameters in the top of window
menu, then Training Schedules, then selecting Fixed, then selecting Responses, and then
inputting five into the open field. When Sniffy maintained a steady bar pressing rate for 15
pellets in a row, the number of Responses was increased to 10. When Sniffy maintained a steady
bar pressing rate for 15 pellets in a row, the number of Responses was increased to 15. When
Sniffy maintains a steady bar pressing rate for 15 pellets in a row, the number of Responses was
SNIFFY THE RAT
6
increased to 20. When Sniffy maintains a steady bar pressing rate for 15 pellets in a row, he is
considered trained. Progress was then saved by clicking File in the top of window menu, then
Save As, then selecting w :\\alonso\ from the dropdown Drives menu. Named FR20.cdf. The
program was restarted, and the progress from stage six was opened by clicking File in the top of
window menu, then Open, then selecting FR20.cdf. Sniffy was then allowed to run free for one
minute under the same conditions as stage six. After one minute, extinction mode was enabled
by clicking Parameters in the top of window menu, then Training Schedules, then selecting
Extinction and clicking OK. The number of non-reinforced bar presses in the first two minutes,
as well as the total time until extinction was recorded. Progress was then saved by clicking File
in the top of window menu, then Save As, then selecting w :\\alonso\ from the dropdown Drives
menu. Named FR20Ext.cdf.
The seventh stage of the experiment was to train Sniffy with a partial reinforcement
schedule and observe extinction rate. The program was restarted, and the progress from stage
three was opened by clicking File in the top of window menu, then Open, then selecting
CRF.cdf. Sniffy was then allowed to run free for one minute under the same conditions as stage
three. After one minute, partial reinforcement was activated by Parameters in the top of window
menu, then Training Schedules, then selecting Variable, then selecting Responses, and then
inputting fives into the open field. When Sniffy maintained a steady bar pressing rate for 15
pellets in a row, the number of Responses was increased to 10. When Sniffy maintained a steady
bar pressing rate for 15 pellets in a row, the number of Responses was increased to 15. When
Sniffy maintains a steady bar pressing rate for 15 pellets in a row, the number of Responses was
increased to 20. When Sniffy maintains a steady bar pressing rate for 15 pellets in a row, he is
considered trained. Progress was then saved by clicking File in the top of window menu, then
SNIFFY THE RAT
7
Save As, then selecting w :\\alonso\ from the dropdown Drives menu. Named VR20.cdf. The
program was restarted, and the progress from stage six was opened by clicking File in the top of
window menu, then Open, then selecting VR20.cdf. Sniffy was then allowed to run free for one
minute under the same conditions as stage seven. After one minute, extinction mode was enabled
by clicking Parameters in the top of window menu, then Training Schedules, then selecting
Extinction and clicking OK. The number of non-reinforced bar presses in the first two minutes,
as well as the total time until extinction was recorded. Progress was then saved by clicking File
in the top of window menu, then Save As, then selecting w :\\alonso\ from the dropdown Drives
menu. Named VR20Ext.cdf.
The eighth stage of the experiment was to train Sniffy with a partial reinforcement
schedule and observe extinction rate. The program was restarted, and the progress from stage
three was opened by clicking File in the top of window menu, then Open, then selecting
CRF.cdf. Sniffy was then allowed to run free for one minute under the same conditions as stage
three. After one minute, partial reinforcement was activated by Parameters in the top of window
menu, then Training Schedules, then selecting Fixed, then selecting Interval, and then inputting
five into the open field. When Sniffy maintained a steady bar pressing rate for 15 pellets in a
row, the value of the Interval was increased to 10. When Sniffy maintained a steady bar pressing
rate for 15 pellets in a row, the value of the Interval was increased to 15. When Sniffy maintains
a steady bar pressing rate for 15 pellets in a row, the value of the Interval was increased to 20.
When Sniffy maintains a steady bar pressing rate for 15 pellets in a row, he is considered trained.
Progress was then saved by clicking File in the top of window menu, then Save As, then
selecting w :\\alonso\ from the dropdown Drives menu. Named FI20.cdf. The program was
restarted, and the progress from stage eight was opened by clicking File in the top of window
SNIFFY THE RAT
8
menu, then Open, then selecting FI20.cdf. Sniffy was then allowed to run free for one minute
under the same conditions as stage eight. After one minute, extinction mode was enabled by
clicking Parameters in the top of window menu, then Training Schedules, then selecting
Extinction and clicking OK. The number of non-reinforced bar presses in the first two minutes,
as well as the total time until extinction was recorded. Progress was then saved by clicking File
in the top of window menu, then Save As, then selecting w :\\alonso\ from the dropdown Drives
menu. Named FI20Ext.cdf.
The ninth stage of the experiment was to train Sniffy with a partial reinforcement
schedule and observe extinction rate. The program was restarted, and the progress from stage
three was opened by clicking File in the top of window menu, then Open, then selecting
CRF.cdf. Sniffy was then allowed to run free for one minute under the same conditions as stage
three. After one minute, partial reinforcement was activated by Parameters in the top of window
menu, then Training Schedules, then selecting Variable, then selecting Interval, and then
inputting five into the open field. When Sniffy maintained a steady bar pressing rate for 15
pellets in a row, the value of the Interval was increased to 10. When Sniffy maintained a steady
bar pressing rate for 15 pellets in a row, the value of the Interval was increased to 15. When
Sniffy maintains a steady bar pressing rate for 15 pellets in a row, the value of the Interval was
increased to 20. When Sniffy maintains a steady bar pressing rate for 15 pellets in a row, he is
considered trained. Progress was then saved by clicking File in the top of window menu, then
Save As, then selecting w :\\alonso\ from the dropdown Drives menu. Named VI20.cdf. The
program was restarted, and the progress from stage eight was opened by clicking File in the top
of window menu, then Open, then selecting VI20.cdf. Sniffy was then allowed to run free for one
minute under the same conditions as stage eight. After one minute, extinction mode was enabled
SNIFFY THE RAT
by clicking Parameters in the top of window menu, then Training Schedules, then selecting
Extinction and clicking OK. The number of non-reinforced bar presses in the first two minutes,
as well as the total time until extinction was recorded. Progress was then saved by clicking File
in the top of window menu, then Save As, then selecting w :\\alonso\ from the dropdown Drives
menu. Named VI20Ext.cdf.
Results
During the initial ten minutes of observation, Sniffy pressed the bar zero times. He
exhibited behavior such as drinking from the water spout, scratching himself, cleaning himself,
and rearing up onto his back legs.
After baseline observation was complete, Sniffy began magazine training. This process
took five minutes and 11 seconds to complete.
Following magazine training was Sniffy’s shaping via successive approximation. In the
two minute period following the second reset of the cumulative record, Sniffy pulled the bar 27
times, staying in close proximity of the magazine the entire time.
In the two minutes after extinction mode was enabled, Sniffy pressed the bar 17 times. It
took a total of six minutes and 27 seconds for the behavior to become extinct.
After being sent for a Time-Out, Sniffy returned to the cage and pressed to bar 12 times
in two minutes. It took a total of six minutes and eight seconds for the behavior to become
extinct.
The extinction time for the FR20 schedule was 10 minutes and 43 seconds. Sniffy
pressed to bar 53 times in the first two minutes.
9
SNIFFY THE RAT
10
The extinction time for the VR20 schedule was 22 minutes and nine seconds. Sniffy
pressed to bar nine times in the first two minutes.
The extinction time for the FI20 schedule was 22 minutes and 51 seconds. Sniffy pressed
to bar 17 times in the first two minutes.
The extinction time for the VI20 schedule was 29 minutes and 53 seconds. Sniffy pressed
to bar six times in the first two minutes.
Discussion
The hypothesis is partially supported by the data. The time required for each schedule to
go extinct, from shortest to longest, was correctly guessed as Continuous, Fixed Ratio, Variable
Ratio, Fixed Interval, Fixed Ratio. However, the data collected on frequency of bar presses in the
first two minutes of extinction indicates the assumption that a higher frequency of bar pressing
does not cause faster extinction. The FI20 schedule resulted in 17 bar presses, nearly double that
of the VR20 schedule, and yet took 42 more seconds to become extinct. It should be noted that
the data from the FI20 schedule does not align with the rest of the experiment. The hypothesis
would be entirely supported by the data if FI20 was excluded. This indicates there could have
been an error in the procedure.
Despite using the same extinction procedures, the extinction time for Spontaneous
Recovery was shorter than the CRF schedule because of the time Sniffy spent in Time-Out.
During even this short time, newly learned information will being to interfere with the recall of
the desired behavior. Spontaneous Recovery occurs at a time when non-reinforced secondary
information begins to fade from short-term memory and the reinforced behavior that has been
solidified into long-term memory becomes stronger than the fading competitive responses.
SNIFFY THE RAT
11
The best explanation for the success of the VI20 schedule could be attributed to the fact
that by using a variable system, the subject becomes more accustomed to not receiving
reinforcement for a bar press, versus a fixed system where reinforcement is guaranteed under
certain conditions, and therefore undergoes less frustration from the lack of reinforcement during
extinction. Another factor is the interval system, because it does less to promote excessive
unreinforced bar pressing, which leads to faster extinction rates.
Each schedule is unique, offering its own set of advantages and disadvantages.
Continuous reinforcement creates a high level of responding over a short period of time, but can
be time consuming and produce satiation. Fixed interval produces and even higher response rate,
but it can be highly irregular. Variable interval has the longest time to extinction, but the fewest
responses during that time.
SNIFFY THE RAT
12
References
Olson, M. H., & Hergenhahn, B. R. (2013). Burrhus Frederic Skinner. An Introduction to
Theories of Learning (Ninth ed., pp. 71-114). Upper Saddle River, NJ: Pearson.