A Quantitative Analysis of the Behavior Maintained by Delayed Reinforcers
A. Charles Catania
University of Maryland, Baltimore County (UMBC)
Mark P. Reilly, Dennis Hand
Central Michigan University
Lara Kowalsky Kehle, Leanne Valentine, Eliot Shimoff
University of Maryland, Baltimore County (UMBC)
Suggested Running Head: Behavior Maintained by Delayed Reinforcers
Corresponding Author
A. Charles Catania
10545 Rivulet Row
Columbia, MD 21044
Phone: 410-730-6949
FAX: Same as phone, but requires advance notice
Email: catania@umbc.edu
Catania et al., Page 2
Abstract
Random-interval reinforcement contingencies were arranged for a sequence of pigeon pecks
on one key followed by a sequence on a second key. Whether the second sequence required
specified numbers of pecks, minimum times preceding reinforced pecks, or both, first-key
pecking was separated from reinforcers by delays that included number of pecks and time. Firstkey response rates decreased as number of responses or time separating them from reinforced
second-key responses increased. These functions, delay gradients, were obtained in one
experiment in which reinforced sequences consisted of M first-key pecks followed by N secondkey pecks, where M+N equaled 16, in a second in which the number of first-key pecks was held
constant at 8, and in a third that varied the minimum delay between most recent first-key pecks
and reinforced second-key pecks. In each case, gradients were equally well fit by exponential,
hyperbolic and logarithmic functions. Performances were insensitive to reinforcer duration, and
functions were consistent across varied random-interval values. In one more experiment,
differential reinforcement of rate of second-key responding during delay intervals independently
varied time and number. Forms of the delay gradients depended primarily on time rather than on
number of second-key responses separating first-key responding from the reinforcer.
(199 words)
Key words: Delay of reinforcement, delay gradient, time versus number, random-interval
schedule, topographical tagging, key pecks, pigeons
Catania et al., Page 3
Delay of reinforcement has a long history in research on learning, whether it has been
evoked in explicit terms (e.g., Hamilton, 1929; Perin, 1943; Renner, 1964; Seward, 1942;
Watson, 1917; Wolfe, 1934) or only implicitly (e.g, the goal gradients of Hull, 1943).
Throughout much of its history, delayed reinforcement has been regarded mainly as an
impediment to the acquisition and maintenance of behavior rather than as a ubiquitous property
of environments that can organize extended behavior sequences. Other perennial issues are what
fills the time between a reinforced response and its later reinforcing consequence and the
functional properties of what fills it (e.g., Keller & Schoenfeld, 1950; Laties, Weiss, Clark, &
Reynolds, 1965). The idea that intervening responding matters has a long history, as in the
following early example: “It may be that time in delay of reward is important only in so far as it
is roughly proportional to the amount of activity or the number of responses which occur
between the response to be strengthened and the occurrence of the reinforcement” (Grice, 1942,
p. 488). But we will find that elapsed time is a more important determinant of the effects of
delayed reinforcers than quantity of intervening behavior.
If two events are simultaneous, one of course cannot have been the cause of the other, so
responses are necessarily followed by their consequences only after some delay. Considerations
of how to deal with the gap have received substantial attention, but while we acknowledge that
behavior is necessarily extended in time, what bridges the gap will not be our main concern here.
The issue is not about molar versus molecular analyses; even so-called molecular measures such
as interresponse times (IRT) are based on distributions obtained over extended time periods (cf.
Baum, 2004; Hineline, 2001). We will also have little to say about delay as it arises in
respondent procedures (Pavlov, 1927). Our main objective is to characterize the function relating
the rate of a response to the delay that separates it from a subsequent reinforcer, to establish the
Catania et al., Page 4
appropriate dimensions for such a functional relation, and to determine some of the variables that
may serve as its important parameters (cf. Lattal, 1984, 2010).
A milestone in the analysis of delay came with the work of Dews (1960, 1962, 1966, 1970),
especially with his discovery that the fixed-interval (FI) scallop, the increasing response rate
within each interval, survives repeated interruptions. The phenomenon is illustrated in Figure 1.
Because this pattern could not plausibly be accounted for in terms of discriminative effects of
current behavior or other appeals to properties of responding, Dews concluded that the FI scallop
emerges because the reinforcer at the end of the interval has effects that depend on its relation to
all of the preceding responses and not just the one that produced it. Contingent on the time
elapsed since the start of the interval, that reinforcer strengthens responding based on its relation
to each response in the interval. FI scallop as the joint product of a gradient of temporal
discrimination, in that the organism’s different rates at different times in the interval imply its
discrimination of elapsed time, and a gradient of delayed reinforcement, in that responses at
various locations in the interval are consistently followed by the reinforcer at particular delays
(e.g., in FI 500 s, responding at 400 s is typically followed not less than 100 s later by a
reinforced peck). The two gradients must exist together, but it is important to recognize that
responding at any time in an interval depends on what happened after that time in previous
intervals; it cannot depend on the reinforcer at the end of the current interval, because that
reinforcer has not been delivered yet (see also Church & Gibbon, 1982; Neuringer, 1969).
<Insert Figure 1 about here>
In the early days of reinforcement schedules, schedule effects were discussed as partial
reinforcement, and it was seen as a paradox that more behavior could be generated by reinforcing
some fraction of the total responses than by reinforcing every response. But looking at schedules
Catania et al., Page 5
in terms of the delayed reinforcement of all responses that precede the reinforced response
suggests that intermittent or partial reinforcement works as it does because it allows each
reinforcer to reinforce many responses instead of just one. Figure 2 makes the point
schematically. Both 1 and 2 show a time sample including an early response, b, and a later
reinforced response, a. In 1, no other responses intervene; in 2, responses c through j occur
between them. One way to interpret these two cases is to say that in 1 the second of two
responses was reinforced (1/2) whereas in 2 only the last of ten was reinforced (1/10). But that
interpretation assumes that the effect of the reinforcer was limited to a even though it followed
the other responses too.
<Insert Figure 2 about here>
An alternative, following from Dews’ insight, is to assume that the effect of the reinforcer
depends on its relation to all of the responses that preceded it, some after longer delays than
others. From that perspective, only two responses were reinforced in 1 whereas ten were
reinforced in 2. In this view, the partial reinforcement effect need not be regarded as paradoxical.
Instead, schedules allow each reinforcer to act on several responses at a time instead of just the
most recent one.
Although the earlier responses in a sequence that ends with a reinforcer contribute to future
responding based on their relation to the reinforcer, they contribute less than the later ones
because of the longer delays that separate them from the reinforcer (Dews, 1962). This means
that in interpreting effects of schedules, it would help to know the form of the delay gradient.
The delay gradient has entered successfully into mathematical models of operant behavior (e.g.,
Berg & McDowell, 2011; Catania, 2005e, 2011b; Killeen, 1994), but technical problems
complicate its experimental determination. For example, as illustrated in Figure 3, if we arrange
Catania et al., Page 6
delayed reinforcers only some fixed time after a response, we must either allow additional
responses to occur in the interim, in which case the actual time from the last response to the
reinforcer will typically be shorter than the one we scheduled, or we can reset the delay with
each subsequent response, in which case the resetting of the timer will differentially reinforce
pauses at least as long as the delay interval, and that differentiation will be confounded with the
delayed effects of the reinforcer (e.g., Catania & Keller, 1981; Katz & Catania, 2005). We
cannot avoid these problems by presenting a stimulus during which the delay operates, because
that simply substitutes an immediate conditional reinforcer for the delayed one.
<Insert Figure 3 about here>
Delays can be discussed in terms of time elapsed since a response, but they can also be
interpreted in terms of decaying traces of events correlated with responses (e.g., Killeen, 1994,
2001, 2011; Killeen & Sitomer, 2003). Accounts in terms of delay between a response and a
reinforcer and decay of memory for a response can be made quantitatively equivalent, but until
the time at which decay can be identified with physiological processes, we will here favor a
vocabulary in terms of dimensions that can be manipulated directly in experimental procedures
(cf. Wixted & Gaitan, 2002). Time between a response and a reinforcer is one such dimension
(cf. Catania, 1970, 1991).
Figure 4 shows hypothetical gradients that increase as the time to the reinforcer decreases.
In the top example, only one response occurs, just before the reinforcer, and it makes a
contribution to subsequent responding proportional to the height of the gradient at that single
point. In the bottom example, eight responses occur and, based upon Dews’ arguments, we will
assume that each response contributes to subsequent responding proportional to the height of the
gradient at the time at which it occurs. In other words, consistent with what has been well-
Catania et al., Page 7
established in the literature on delay of reinforcement, the shorter the time to the reinforcer, the
bigger its effect on subsequent responding.
<Insert Figure 4 about here>
One method for studying delay gradients is topographical tagging, arranging contingencies
so that responses followed by particular delays are distinguished topographically, as, for
example, by location (Catania, 1971). The procedure is illustrated in Figures 5 and 6. In a singlekey procedure, as shown in Figure 5, the response sequence ba produces a reinforcer. In a twokey procedure, the ba sequence again produces a reinforcer, but this time a contingency is
arranged so that b occurs on Key B rather than Key A. It is reasonable to assume that subsequent
responding on Keys A and B depends on the respective contributions of the relation of response
a to the reinforcer and that of response b to the reinforcer. Figure 6 shows a sequence of
responding in which contingencies have produced a sequence of responses on one key (top)
followed by a sequence on a second key that ends with a reinforced response (bottom). If future
responding on the first key is determined by the summed delay gradient heights at the time of
each response to the left of the dashed line, the total output on the two keys would be expected to
be about the same as the output if all responses had occurred on the second key, because the
reinforcer would have provided the same cumulative reinforcing effect in each case.
<Insert Figures 5 and 6 about here>
When the separation of a Key-B response from the reinforcer was increased by requiring
longer sequences of pecks on Key A (e.g., baa, baaa, baaaa, etc.), the rate of Key-B responding
decreased, as illustrated in Figure 7, which plots the Key-B response rate against the average
time (sum of interresponse times or IRTs) separating the Key-B response from the reinforcer
(derived from data presented in Catania, 1971). In this procedure, all reinforcers are produced by
Catania et al., Page 8
a Key-A peck at the end of the sequence, so relative Key-A and Key-B rates can be determined
only by relative differences in delay and not by relative reinforcement.
<Insert Figure 7 about here>
Proposals for the form of the delay function have most prominently included exponential
functions, logarithmic functions and variations on hyperbolic functions (Catania, 2005e; Chung,
1965; Killeen, 2011; Mazur, 1987, 1995). Choosing among them raises several problems, not
least of which is the measure of responding. For example. we may choose response rate or
resistance to change (Nevin & Grace, 2000; Podlesnik, Jimenez-Gomez, Ward, & Shahan, 2006;
Shahan & Lattal, 2005), and if we study response rate, as in the present research, procedures may
dictate the options. Figure 8 provides an illustration. A procedure that estimates indifference
points with different delays pitted against each other, as in Mazur (1987, 1995), yields the height
of the function at a given delay (top gradient in Figure 8); one that examines responding over a
range of time, as in topographical tagging, yields an area under the function (bottom gradient of
Figure 8). But if the former is hyperbolic then the latter should approach a logarithmic function
(Killeen, 1994). Furthermore, if the parameters of exponential functions are themselves variable,
they can yield data best fit by power functions, of which one is the hyperbolic (Murre & Chessa,
2011). Although we will favor exponential fits to our data, the differences in variances accounted
for by these and other functions are typically too small to favor one strongly over another.
<Insert Figure 8 about here>
One more preliminary remains before we can proceed to the research proper. In dealing with
delay gradients, we must determine how far back from the reinforcer we should extend them. A
body of evidence strongly implies that for any given reinforcer its effect is limited to responses
that occur during the time that separates it from the previous reinforcer (Catania, Sagvolden, &
Catania et al., Page 9
Keller, 1988), as illustrated in Figure 9. The top gradient shows the hypothetical effect of a
reinforcer, rfrx, produced by a response at the end of some time period, where the effect of the
reinforcer on a response is given by the height of the gradient at the time at which it occurs. The
bottom gradients show the effect of an earlier reinforcer, rfry, on the rfrx gradient; rfry generates
its own gradient, but it truncates the tail of the rfrx gradient. The implication is that the
introduction of rfry reduces the responding maintained by rfrx, and this is of course consistent
with what we know about the interacting effects of reinforcers (e.g., Catania, 1969, 1973).
<Insert Figure 9 about here>
The effect in the figure can be thought of as a blocking effect, and it is not clear whether it
would be attenuated with varied instead of constant reinforcers (e.g., Steinman, 1968), or if some
reinforcers were delivered independently of responding (cf. the concept of coupling, as in
Killeen & Sitomer, 2003). This feature of delay gradients was an essential component of a
mathematical model that assumed that, as in Skinner’s Reflex Reserve (Killeen, 1988; Skinner,
1938, 1940), all of the responses impacted by the delay gradient contributed to a reserve of
response potential that was depleted as subsequent responses were emitted (Catania, 2005e).
General Method
The experiments that follow are variations on topographical tagging procedures conducted
with pigeons either in the pigeon laboratories of the Department of Psychology at the University
of Maryland, Baltimore County (UMBC) or those of the Department of Psychology at Central
Michigan University (CMU). We begin with some features common to all or most procedures,
with details to follow later.
Catania et al., Page 10
Subjects
White Carneaux or Silver King pigeons with histories in various operant procedures served
in the research conducted at UMBC. Homing Pigeons provided with delayed-reinforcement
histories in preliminary sessions served in the research conducted at CMU. All pigeons were
housed in individual cages with free access to water in a room with a 12-hr light/dark cycle, were
maintained at approximately 80% of free-feeding weights, and were provided supplemental
feeding to maintain body weights after each session. Experimental sessions were conducted in
the light portion of the light/day cycle. All pigeons were treated in accordance with the
respective UMBC and CMU Institutional Animal Care and Use Committee guidelines, and
procedures were conducted in accordance with protocols approved by those committees. Upon
the closing of the UMBC laboratories (see Catania, 2005c), all pigeons housed there were placed
with a local pigeon fancier.
Apparatus
Daily sessions were conducted in pigeon chambers that included two active keys, a feeder,
and a houselight. In the UMBC pigeon laboratory, all chambers included two standard keys
separated 6.4 cm center-to-center 28 cm above the floor. Below the keys was a 10-cm-square
opening to a standard Gerbrands feeder that provided fixed durations of access to Purina
Nutrabrand pellets as reinforcers. Reinforcer durations were adjusted for each pigeon so as to
minimize postsession feeding, typically within the range from 3 to 5 s; they remained constant
throughout an experiment for each pigeon, except for Experiment 5 on reinforcer duration
Chambers were controlled by Alpha-Bus interfaces operated by an assembly language
routine nested within FORTH programs (see Appendix I in Catania, 2005e) and running on PS/2
or other IBM computers with MS-DOS operating systems. These arrangements did not allow for
Catania et al., Page 11
the collection of individual responses throughout sessions, but a variety of detailed measures
summated over individual sessions were recorded for each pigeon, some of which are provided
in Appendixes 1 and 2.
Sessions at CMU were conducted in four equally equipped Med-Associates (St. Albans,
VT) pigeon chambers, each 27 cm wide by 28 cm tall by 25 cm deep and each containing three
2.5-cm-diameter pigeon keys 21 cm above the floor and 8 cm apart. Floors consisted of steel
rods spaced approximately 1 cm apart suspended over a waste tray. The keys were located above
a 5.7 cm tall by 5.2 cm wide opening behind which was mounted a solenoid-operated Coulbourn
Instruments (Allentown, PA) feeder, which provided white millet and was illuminated by a 27-v
white light during reinforcer deliveries. The right key and two 4.5-cm-wide levers mounted
approximately 1 cm above the floor and directly below the right and left response keys were not
used. Only the left and center keys were used, both lit red in the main procedure, but for
consistency with discussions of data from two-key chambers, they will subsequently be referred
to as the left and right keys. A 25-v houselight was on the opposite wall. Computer control
allowed all responses to be recorded, so that data analysis was not limited to summary measures
decided upon in advance.
Hall-Effect Pigeon Keys. Instead of the electrical make-break contacts of standard
Gerbrands pigeon keys, the keys in the UMBC experiments relied on the Hall effect, the
production of current by the movement of a magnet relative to a sensor, and therefore obviated
concerns about changes in conductivity due to arcing at key contacts. As shown in Fig. 10, a
magnet attached to the moving portion of the key rested against the Hall-effect switch
(Multiplexed two-wire Hall Effect sensor IC, Model UGN3055U, from Allegro Microsystems,
Worcester, MA). Pecks moved the magnet away from the switch and generated an output
Catania et al., Page 12
processed at the Alpha-Bus interface. Switch reset as the magnet returned to its resting position
involved sufficient hysteresis that key bounce or chatter was not a significant problem. Key
tension, set to record pecks of about 1.5 N or greater, was adjusted in the usual way and
maintenance consisted mainly of minor cleaning so that pigeon dust or other debris did not build
up between the magnet and the switch. We found the keys to be highly reliable, and calibration
by Shimoff via the assembly portion of the program showed them capable of following response
rates up to roughly 20 responses/s.
<Insert Figure 10 about here>
Procedure
In all procedures, random-interval (RI) contingencies were arranged for sequences of pecks
on each of two pigeon keys. In one arrangement, for example, with an RI schedule of 60 s,
reinforcers became available (were set up) on the average of once a minute for the last peck of a
sequence of exactly M left-key pecks followed by exactly N right-key pecks. Figure 10
illustrates the procedure schematically and provides some nomenclature.
<Insert Figure 11 about here>
Data Analysis. The figure shows a hypothetical sequence of responses on left (L) and right
(R) keys as they might occur between two reinforcer deliveries (RFR) in a well-established
performance in which the sequence to be reinforced is exactly eight left pecks followed by
exactly eight right pecks. Pecks occur as runs of responses on one key uninterrupted by
responses on the other key (L RUNs and R RUNs) and runs are separated by changeovers from
left to right (LR) or from right to left (RL). A sequence consisting of an L RUN, an LR
changeover, an R RUN and an RL changeover constitutes a CYCLE.
Catania et al., Page 13
To insure homogeneity of the data entering into computations of response rates, some
portions of the time between two successive reinforcers were excluded from the analysis. These
included post-reinforcer pauses (PRPs) and all cycles that ended with a reinforced right-key
peck. These data-analysis boundaries are represented by the dashed verticals at a and b in the
figure. Post-reinforcer pauses (PRP) were excluded because they were typically longer than IRTs
and changeovers. In a history of pilot sessions with these procedures, all pigeons came to
respond almost exclusively on the left key after each PRP, and this pattern remained in place
throughout all subsequent sessions, so no provision was made for the very rare right-key
responses that occurred after a PRP. Also, for the purposes of data analysis the time to the first
peck of the session was treated as equivalent to a PRP. These exclusions were for the purposes of
data analysis only: the clocks for overall session duration and for the scheduling of RI setups
stopped only during operations of the pigeon feeder.
Correcting for Selective Stopping. Cycles ending with a reinforcer were excluded because
they constituted the selective stopping of response sequences, with the potential consequence
that the required L and R pecks might be overrepresented in derived measures such as L and R
response rates. Consider sequences of three responses, each l or r, with responding terminated
only with the reinforcement of two successive r responses. If the responses occur with roughly
equal frequencies, the possible sequences are lll, llb, lrl, lrr, rll, rlr, rrl and rrr, but if two
successive reinforced r responses terminate a sequence, the behavior recorded will consist of lll,
llr, lrl, lrr, rll, rlr, rr and rr and the sequences ending with a reinforcer will consist of lrr, rr and
rr, which include only a single l among seven responses. If the sequences not ending with a
reinforcer continue, they again end only when they include two successive r responses, and so
on, thereby maintaining the bias toward higher r than l response rates produced by the selective
Catania et al., Page 14
stopping. Overall response rates for a given key were therefore based upon total responses over
the cumulated time from the end of the PRP to the start of but excluding the reinforced cycle (a
to b in Figure 10). These overall rates are to be distinguished from local rates, estimated by
dividing mean responses per run by mean run duration.
Random-Interval Scheduling. Contemporary arrangements usually program the variety of
variable-interval schedule called random interval (RI), in which the repeated sampling of a
probability gate produces reinforcement setups. For example, producing a setup with a
probability (p) of .02 at the end of every second (t) arranges a mean interval of t/p, or 50 s. In the
days of electromechanical equipment, intervals were instead scheduled by a motor-driven loop of
tape with holes punched in it, with minimum intervals dictated by the minimum spacing that
allowed a switch to throw completely as its sensor cleared one hole and then fell into the next
(e.g., Catania & Reynolds, 1968; Ferster & Skinner, 1957). This led to the practice of stopping
the programming tape each time a punched hole was sensed. The tape started again only after the
set up reinforcer was delivered. Without this arrangement, the next hole in the tape could move
past the sensor without detection, thereby losing reinforcers that should have been scheduled.
But stopping the tape also added time to intervals, so obtained rates of reinforcement were
typically lower than scheduled ones.
. Contemporary interval programs usually preserve the procedural artifact, though it arose
from the limitations of electromechanical equipment. To stop probability sampling once a
reinforcer has been set up in an RI schedule is functionally equivalent to stopping the tape when
setups are arranged electromechanically. But in computer programs setups need not be treated as
yes-no states; instead, their value can be incremented with each setup and decremented with each
reinforcer delivery, and probability sampling can continue even if reinforcers already set up have
Catania et al., Page 15
not yet been collected. With this arrangement, for example, if three setups have occurred but
responding has briefly ceased for some reason, the next three responses in a row will be
reinforced once responding resumes (cf. Catania, 2002, 2011b).
Figure 12 compares traditional RI scheduling and this alternative, which was used for the RI
schedules in the present research. Because reinforcer durations were excluded from calculations
of session duration, reinforcers have been represented as essentially instantaneous. The top
example illustrates standard practice: Scheduling stops with each setup until a response produces
the current reinforcer. No setups are lost, but the time from setup to reinforced response adds to
each scheduled interval. In the example only 16 of the 18 reinforcers that could have been
available are actually delivered; two that would have been delivered if timing had not stopped
during setups have been pushed beyond the available time..
<Insert Figure 12 about here>
In the bottom example, timing continues after each setup and setups accumulate. Thus,
when setups 7 and 8 occur during a single IRT, the next two responses each produce a reinforcer.
After four setups during a later long pause, four responses in a row produce reinforcers. By
session end, all 18 scheduled reinforcers have been delivered. Occasional setups can be lost if a
session ends without a response just after they are arranged. Even so, these contingencies
produce much closer correspondences between scheduled and obtained reinforcers than those in
the traditional example. (This might not be the case with schedules in which the effect of the
reinforcer may oppose differential reinforcement, as in tandem RI differential-reinforcement-oflow-rate or drl schedules).
Furthermore, the number of uncollected setups remaining at the end of a session is an
index of control by the scheduled contingencies, especially where these involve the differential
Catania et al., Page 16
reinforcement of complex response sequences. If very few setups remain uncollected at the end
of a session, this is evidence that responding has conformed to the schedule contingencies, so
this number should rarely be larger than zero or one in steady-state performance. In our
procedures, therefore, the number of uncollected setups at the end of a session was used as a
measure of stability in our decisions about whether to advance to a new experimental condition.
The topographical tagging schedule. Unless otherwise noted, an RI 60-s schedule operated
to set up reinforcers for the last of a sequence of exactly M left-key pecks followed by exactly N
right-key pecks, daily sessions were arranged for a maximum of 60 min or 60 reinforcers,
whichever came first, and reinforcer duration was 4 s, with reinforcer time excluded from the
timing of session duration. Arranging a reinforcer for the last peck of the M+N sequence meant
that a sequence that had already been initiated could produce a reinforcer if a setup occurred
during its emission. Without this contingency, interreinforcer intervals would always have been
lengthened by the time it took for the sequence to be completed, so that procedures that required
longer sequences would also have produced lower reinforcement rates. It is tempting to describe
the contingencies as a tandem schedule that includes the RI along with the two FR components,
but the setup contingency makes it more appropriate instead to describe it as RI 60 s in which the
higher-order reinforced operant consists of the tandem sequence FR M (left key) FR N (right
key), or, in other words, as an RI 60-s (tandem FR M FR N) schedule. We will often refer to M
and N as the left-key ratio and the right-key ratio.
Experiment 1: Varying Delay by Varying Number of Intervening Responses (Part I)
The procedures that provided the data in Figure 8 were obtained while M was held constant
at 1 and N varied (Catania, 1971). The variations in N effectively added a varying fixed-ratio
Catania et al., Page 17
contingency to right-key responding. The present experiment changed the delayed reinforcement
of left-key responding by varying N also, but to avoid varying the overall ratio contingency it
held the sum of M plus N constant at sixteen. Another assumption was that responses that were
seventeen or more responses back from the reinforcer were far enough removed that their
cumulative effects were small compared to those for the sixteen for which contingencies were
arranged. But if delays are implicated even in the early parts of long FI schedules, then certainly
they can operate beyond the times occupied by the required response sequences, so this is an
issue that will probably needed to be revisited. This was beyond the reach of the present analyses
because most data were collected by computers with insufficient capacity to record entire
sessions in detail. Overall response rates may allow the proportions of lefts and rights in such
cases to be estimated but we have not attempted to do so.
Method
The subjects and apparatus were as described in General Method. All pigeons were White
Carneaux; Pigeons 73, 74 and 76 were males and Pigeon 78 was a female. The sessions were
preceded by several weeks of pilot sessions during which equipment and programs were
calibrated and tested, the pigeons were provided with a history of different M and N values, and
their steady-state performances and transitions to new parameter values were observed to insure
that each was a suitable preparation in terms of key-peck topography and variability for the
purposes of the study (see Methodological Note in Catania, de Souza, & Ono, 2004; see also
Staddon, 1965, on metastability).
Conditions included a range of values from M=1 and N=15 to M=15 and N=1, as outlined in
Appendix 1; data are means across the last five sessions of each condition. Most conditions
lasted fourteen daily sessions and were extended if performances showed two or more
Catania et al., Page 18
uncollected setups at the end of a session; this criterion was supplemented by visual inspection of
the data, where sessions were also extended if session-to-session variability was apparent even if
the setup criterion was met. Stability was judged independently for each pigeon. Because of staff
availability on weekends and holidays, experimenter error, apparatus problems or other issues,
on a few occasions fewer than fourteen sessions were arranged for a condition or a condition
continued even though stability criteria had been met. Subsequent data inspection showed that
except at the extremes of parameter values pigeons’ behavior adjusted to new conditions quickly
after transitions, usually within just one or two sessions (cf. Reilly & Lattal, 2004).
Results
Figure 13 shows response rates on the left key for each pigeon as a function of the mean
time to a reinforcer, i.e., the time separating the last left-key peck from the next reinforced rightkey peck, with exponential fits to the data for each bird (see Experiment 3, Discussion, for the
details of these and other fits). One assumption in topographical tagging is that the procedure
involves the movement of responses attributed to certain ranges of delays from one key to
another without affecting total output. Figure 14 shows the sum of the response rates across the
two keys. The linear trend-lines for each bird show overall rates decreasing with required left
responses for Bird 73 and increasing for Bird 76, with the slope essentially flat for the other two.
Given the opposite directions of effect for Birds 73 and 76, the overall data are therefore
reasonably consistent with the assumption, especially given that the different proportions of ratio
contingencies across the two keys at different parameter values might have been expected to
matter.
<Insert Figures 13 and 14 about here>
Catania et al., Page 19
Figure 15 shows that the responses per run for both the left key and the right key closely
followed the required responses on those keys (M and N). Lines show linear fits, with the leftkey slopes -1.00, -1.15, -0.81, and -0.99 and the right-key slopes 1.19, 1.21, 1.29 and 1.32 for
Pigeons 73, 74, 76 and 78 respectively. A corresponding match to the contingencies did not
appear in the delays produced by the different values of N (required right-key pecks). Figure 16
shows the mean time from the last left peck to the next reinforced right peck as a function of N.
The relation is nonlinear and is shown with an exponential fit.
<Insert Figures 15 and 16 about here>
Other measures of performance are shown in Figure 17. Changeovers from left to right were
characteristically short and did not vary systematically with parameter values across pigeons.
Consistent with the ratio contingencies created by the M and N requirements, changeovers from
right to left were of substantial duration, as were postreinforcement pauses except for those of
Pigeon 76. Again, changes with parameter values were inconsistent across pigeons, as shown by
the slopes of the linear fits for changeovers (in the aid of clarity, no fit is shown for
postreinforcer pauses, unfilled circles).
<Insert Figure 17 about here>
The overall performance is summarized in Figure 18, which includes both left and right
response rates as a function of the delay created by N, the right-key requirement. The left-key
rates are well-fitted by exponential functions, but not the right-key rates, for which logarithmic
functions have been fitted instead. Consistent with the contingencies, the rates on the two keys
are inversely related, i.e., left-key responding increases as right-key responding decreases, and
vice versa.
<Insert Figure 18 about here>
Catania et al., Page 20
Discussion
The main question, of course, is whether it is appropriate to regard the left-key data as
representing a delay gradient. Clearly left-key rates decrease as the separation between left-key
rates and later reinforcers grows. Presumably part of the decrease occurs because that separation
is also correlated with shorter required sequences on the left key. A brief exploratory study with
Pigeons 74 and 78 that examined some parts of the M and N contingency space provided some
relevant data. With other variables maintained as in Experiment 1, N was held constant at 8
while M was set at 8, 10, 12, 4 and 2 for 7, 7, 7, 14 and 9 sessions respectively. Figure 19 shows
that left-key response rates increased with increasing values of M for both pigeons.
<Insert Figure 19 about here>
We have taken the position that the delay gradient is a fundamental property of reinforced
behavior and that other properties of the performance, such as the approximate matching of
responses per run to the respective left (M) and right (N) requirements, are indirect products of
this primary contingency as it interacts with differential reinforcement operating over extended
periods of time. We do not appeal to contiguity, which in our view has been out of favor in
accounts of operant performances maintained by schedules for half a century or more (Catania,
2012; Dews, 1962; Staddon & Simmelhag, 1971). Those who begin with the assumption that
allocation and matching is primary (Baum, 2012), with other details dropping out as a
consequence of their interaction with other processes, will presumably disagree. One problem we
have with accounts in terms of allocation based on matching is that they force all responding into
a mold in which the sum of all behavior equals one, thereby precluding the possibility that the
total behavior of an organism can vary in quantity (cf. Catania, 1983).
Catania et al., Page 21
Experiment 2: Varying Delay by Varying Number of Intervening Responses (Part II)
Whichever position is held, one obvious next step is to examine gradients obtained when
required left-key responses (M) do not vary. Experiment 2 held M constant and varied N, the
required right-key responses leading up to the reinforcer.
Method
The same pigeons and apparatus were used as in Experiment 1. The primary difference in
procedure is in the required M and N responses of the higher-order RI 60 s (tandem FR M FR N)
schedule. This time, M was held constant at 8 while N was varied. Order of conditions, sessions
and other details are summarized in Appendix 2.
Results
With these parameter values, the time from the last left peck to the next reinforced right
peck changed roughly linearly with required right pecks, as shown in Figure 20. This implies that
right-key responding (changeovers from left to right and local right-key rates during runs)
remained fairly stable across experimental conditions. Figure 21 shows left-key rates as a
function of delay, i.e., the time from the last left peck to the next reinforced right peck. Like the
functions in Figure 13, all are reasonably fitted by exponential functions, though on the whole
they are less steep than the earlier functions.
<Insert Figures 20 and 21 about here>
The combined left plus right response rates and the right-key response rate alone are shown
in Figure 22. Despite the larger M+N ratio contingencies, from 8+1 = 9 to as high as 8+16 = 24
for Pigeon 76, overall rates did not change consistently, decreasing for Pigeon 73, increasing for
Pigeons 74 and 76, and increasing only very slightly for Pigeon 78. Although these data are not
Catania et al., Page 22
completely consistent with the assumptions of the topographical tagging procedure, given the
changing contingencies across conditions neither do they contradict those assumptions.
<Insert Figure 22 about here>
Right-key response rates increased with increasing delays for left-key responding and the
correlated decreases in left-key response rates, but as in Experiment 1 these data were not well
fitted by exponential functions; the fits shown are logarithmic.
Discussion
Right-key responding fills the time between left-key pecks and the later reinforcer, but a
return to the left key before the right-key sequence has been completed resets the sequence, just
as an early response resets the delay in a single-response resetting-delay procedure (cf. Figure 3).
The topographical tagging procedure substitutes alternative behavior for the period of no
responding in the resetting delay procedure, but simply providing an alternative response cannot
eliminate the reset contingency. On the other hand, if the assumptions of topographical tagging
are appropriate, the responding that becomes the alternative behavior stands in the same relation
to later reinforcers as it would have in single-key performances. Thus, it is probably still
reasonable to regard left-key responding as just the amount of behavior that can be maintained
by a given delay between responding and a later reinforcer. In other words, if there is some
maximum quantity of behavior that can be supported by a given delay of reinforcement, then our
data should at least approximate that limit, thereby perhaps narrowing the range of functions that
can describe the relation between responding and reinforcer delays.
Catania et al., Page 23
Experiment 3: Varying Delay by Varying Intervening Time
The procedures of Experiments 1 and 2 specified numbers of responses (N) intervening
between the last left peck and the next reinforced right peck. Actual delays were determined by
the rate of the intervening right responses (cf. Figures 16 and 20). Another approach is to arrange
a minimum time between the last left peck and the next reinforced right peck. In this case, a
minimum time is specified and the number of intervening right responses may vary. Analogous
to the previous RI 60-s (tandem FR M FR N) schedules, therefore, this experiment arranged RI
60-s (tandem FR M FI T-s) schedules, or, in other words, the RI 60-s reinforcement of a higherorder succession of FR M and FI T-s components, where the FR determines the required number
of left pecks and the FI determines the minimum time after which a left response can be followed
by a later reinforced right response.
Method
Subjects and Apparatus. Three male Homing Pigeons, 11, 54, and 64, served in daily
sessions in the CMU laboratory. A fourth pigeon with problems of variability and topography
was dropped as being an unsuitable preparation for the study (cf. Methodological Note in
Catania et al., 2004). The sessions were arranged in a three-key pigeon chamber in which only
two keys were functional (cf. General Method).
Procedure. Reinforcers were always 3 s of access to white millet. After some preliminary
sessions, training included tandem schedules in which reinforcers were contingent upon FR 1 for
a left-key peck followed by FR 1 for a right-key peck, with both keys lit red. Subsequently,
contingencies were gradually modified, via tandem FR 1 (left) FI 1-s (right) schedules, tandem
FR 4 (left) FI 0.5-s (right) schedules in which the FR reset if a right-key peck occurred before the
ratio was completed, and then an RI 2.5-s (tandem FR 4 FI 0.5-s) schedule. Schedule parameters
Catania et al., Page 24
were changed over roughly the next two weeks of sessions until they had reached RI 60-s
(tandem FR 4 FI 0.25-s), at which point the experiment proper began and during which sessions
lasted until 50 reinforcers had been delivered or 50 minutes had elapsed, whichever came first.
As in Experiments 1 and 2, calculations of left-key and right-key response rates excluded
reinforced sequences and postreinforcement pauses (cf. Figure 11).
For all pigeons, FI values of 0.25, 0.50, 1.00, 2.00 and 4.00 s were arranged in ascending
order. Each value was maintained for a minimum of twenty sessions, with stability then judged
based on the most recent six sessions. Stability criteria included two components: no upward or
downward trend in left-key response rates could be evident across the six sessions; and the
difference between the mean response rate over the last three sessions and over the preceding
three sessions could not exceed 5% of the mean for all six sessions. Once both criteria were
satisfied, the pigeon was advanced to the next FI value. Sessions and data are summarized in
Table 1.
<Insert Table 1 about here>
Results
Figure 23 shows the rate of left-key responding as a function of the time between the last
left-key peck and the next reinforced right-key peck for each pigeon, with data fitted by
exponential functions. As shown in Figure 24, right-key response rates tended to increase with
increasing delay for all three pigeons, whereas combined left-key and right-key rates decreased.
Given the variability and the limited number of data points, no fits are shown. As in Experiment
2, the actual delay from the last left peck to the next reinforced right peck increased roughly
linearly with the scheduled delay contingency, which in this case was the value of the terminal FI
schedule (see Table 1).
Catania et al., Page 25
<Insert Figures 23 and 24 about here>
Discussion
In judging whether the functions in Figure 23 should be interpreted as delay-ofreinforcement gradients, it might be useful to characterize them mathematically. We have shown
exponential fits in our data presentations primarily for reasons of precedent and expedience.
Table 2 provides least-squares fits of exponential, hyperbolic and logarithmic functions to the
delay data of Experiments 1, 2 and 3. These functions and their variations do not exhaust the
possibilities, but the comparisons are nonetheless instructive.
The procedures of Experiment 3 provided consistently stronger relations between delay and
left-key responding (r2 ranging from 0.93 to 0.99) than those of Experiments 1 and 2 (r2 ranging
from 0.48 to 0.92), but none of the fits was substantially or consistently superior to any of the
others. The respective mean r2 values for the exponential, hyperbolic and logarithmic functions
across all pigeons and experiments were, respectively, 0.804, 0.799 and 0.802, and these
differences would not even have been evident if the means had been rounded to only two
decimal places. Certainly the differences are well within the range of error to be expected from
behavioral preparations.
These findings seem to lead to the unsettling conclusion that we cannot resolve the issue of
which functions best characterize delay gradients by appealing to experimental data. It may be
useful to examine which functions would be most compatible with the fits to right-key data and
with those cases in which total response rates (left plus right) approximated constancy. Not to
keep our readers in suspense, we must state here that this issue will still be unresolved by the
time we conclude this report.
<Insert Table 2 about here>
Catania et al., Page 26
Experiment 4: Varying Delay by Varying Both Number and Time
One reason that these experiments began with number rather than time as the dimension
to fill the delay between left-key responding and the later reinforced right-key peck is that
responding rather than time is often implicated in important behavioral phenomena. The human
verbal learning literature, for example, is filled with many relevant cases, as when forgetting is
determined more strongly by intervening activities such as items read or problems solved than by
elapsed time (e.g., see Catania, 2013, Chapters 21, 26 and 27; White, 2001; Wright et al., 1990).
In addition, differential reinforcement can be effectively and independently applied to either
duration or number (Catania, 1991, pp. 13-17; LeFever, 1973).
Experiment 4 attempted to tease apart the contributions of number and time to delay. Its
rationale is illustrated in Figure 25. The left side of the figure shows what might happen to a
sequence of four lefts and four rights in the topographical tagging procedure if delay effects
depended solely on time, and the right side shows what might happen if delay effects depended
solely on number. The question is whether, given appropriate contingencies, the data conform to
the kinds of effects shown on the left or to those shown on the right. The example assumes
negligible induction (response generalization) of response rates from one key to the other.
<Insert Figure 25 about here>
At start, with no added contingencies (a and b), the performances are identical with
regard to both the time and the number of responses separating the last left from the later
reinforced right response, so here time-based delays and number-based delays are necessarily
equal. In c and d, a short-IRT contingency has shaped more rapid responding during the right
run; in c, these shorter IRTs mean a shorter time-based delay, which engenders faster left
Catania et al., Page 27
responding; but in d, where delay effects depend on number rather than time, the number-based
delay is unchanged, so the faster right responding has no effect on left responding. In e and f, the
right sequence is lengthened so that the run now takes as long as it had in a and b. Again, in e
time rather than number matters, so the time-based delay is equivalent to that in a and left
responding returns to its earlier a level; but in f the increased responding creates a longer
number-based delay, so left responding slows down. The inverse cases are shown in g through j.
With a long-IRT contingency in g, the time-based delay increases with slower right responding
and engenders slower left responding; but when the required right sequence is shortened in i so
that the time-based delay is the same as in a, left responding returns to its a level. In h, however,
the long-IRT contingency has no effect on the number-based delay, so left responding is
unaffected; but when in j the sequence is shortened, left responding increases because the
number-based delay has been reduced.
The target of this experiment, then, was to arrange contingencies for right-key runs that
would allow us to tell whether delays work like the time-based ones on the left side of the figure
or the number-based ones on the right side.
Method
Four White Carneaux in the UMBC laboratory, male Pigeons 60, 75, 77 and female Pigeon
81, served in daily sessions in a two-key chamber with the left key lit amber and the right key lit
green. As in prior experiments, the procedures were higher-order schedules in which RI 60-s
contingencies set up reinforcers for tandem left-key right-key sequences. This time, the left-key
requirement of the tandem schedules (M) was held constant at eight while right-key
contingencies varied. These variations, produced by the shaping of longer or shorter right-key
runs and of higher or lower right-key response rates within runs, were introduced in order
Catania et al., Page 28
independently to vary time-based delays and number-based delays as measured from the last left
peck to the reinforced right peck.
The right-key contingencies included, separately and in combination, variations in the
required number of right-key pecks (N), differential reinforcement of high right-key rates (drh),
in which the sum of the right-key interresponse times (IRTs) within a run had to be shorter than
some criterion value to be eligible for reinforcement, and differential reinforcement of low rightkey rates (drl), in which the final IRT of the right-key run had to be longer than some criterion
value to be eligible for reinforcement. Thus, if the tandem component consisted of FR 8 for the
left key (M=8) followed by a drh 2-s contingency imposed on FR 5 for the right key (N=5), a set
up reinforcer would be delivered given eight successive pecks on the left key followed by five
successive pecks on the left key emitted in less than 2 s (i.e., four IRTs averaging less than 0.5 s).
Similarly, if the tandem component consisted of FR 8 for the left key (M=8) followed by a drl
0.4-s contingency imposed on FR 4 for the right key (N=4), a set up reinforcer would be
delivered given eight successive pecks on the left key followed by four successive pecks on the
right key, the last two of which were separated by more than 0.4 s (i.e., IRT greater than 0.4 s).
The joint continua of time-based and number-based delays were explored for each pigeon.
At various points during shaping, contingencies were maintained over several sessions for
purposes of data collection. In most instances, data were collected only when performance was
judged stable according to the criteria of zero or one uncollected setups over successive sessions
and of visual inspection of the data, but the criteria were relaxed at extreme parameter values,
where it was sometimes difficult to maintain performance. In some instances, especially with drh
requirements, it was necessary to relax contingencies by dropping back to earlier parameter
values.
Catania et al., Page 29
Given these considerations and given that these shaping procedures were conducted over
more than two years of daily sessions for Pigeons 60, 75 and 77 and almost a year of daily
sessions for Pigeon 81, it would be impractical to present the details of each pigeon’s sessions
and data. Appendix 3 shows selected measures for each pigeon from the sessions that provided
the data figures to follow. For Pigeons 60, 75, 77 and 81 over the course of shaping sessions, the
respective ranges of right-key requirements were 1 to 12, 1 to 8, 1 to 12, and 1 to 16, the ranges
of drh requirements (sum of right IRTs) were 0.12 to 1.12 s, 0.16 to 1.40 s, 0.16 to 1.00 s, and
0.48 to 2.10 s, and the ranges of drl requirements (last right IRT) were 0.14 to 0.79 s, 0.12 to
0.77 s, 0.13 to 0.79 s, and 0.19 to 0.28 s.
After these procedures, all pigeons were given additional sessions of RI 60-s tandem FR 8
FR 1, which required only a single right response and therefore a delay equal to the most recent
left to right changeover (25, 8, 15 and 11 sessions respectively for each pigeon) and of an RI 60-s
schedule arranged for left-key responding only (15, 41, 24 and 21 sessions respectively for each
pigeon). The respective pairs of left response rates for the two condition were 58.5 / 55.3, 26.0 /
26.1, 45.0 / 33.2, and 50.4 / 48.5 responses per min. Except for the third pigeon, 77, the
differences between these two-key and one-key rates are small. We will not use these rates
elsewhere but provide them as possibly useful baselines for those who may wish to have them in
examining other data from these pigeons.
Results
Figure 26 shows data in XYZ coordinates from Pigeon 60 over the course of its shaping
procedures, where X equals the number of required right-key responses (N) between the last left
peck and the reinforcer (number delay), Y equals the time from the last left peck to the next
reinforced right peck (time delay), and Z equals the left-key response rate obtained at that
Catania et al., Page 30
intersection of X and Y or, in other words, at a given number-delay time-delay combination.
Data from sessions with no differential reinforcement of local right-key rates are shown as filled
circles; those from sessions with differential reinforcement of those local rates are shown as open
triangles with apex pointing up for drh sessions and down for drl sessions. The middle
coordinates are shown from a perspective that allows an overview of the relations among the
variables. The coordinates on the left are rotated so that they are seen from the number side,
thereby emphasizing the relation between response rates and number delay, whereas those on the
right are rotated so that the same data are seen from the time side, thereby emphasizing the
relation between response rates and time delay. From these perspectives, the number-based data
(left) show relatively shallow rate changes with changes in number whereas the time-based data
(right) provide a gradient similar to those obtained in the earlier experiments.
<Insert Figure 26 about here>
Figure 27 shows the data for each pigeon in XY coordinates, with left-key response rates
plotted against number of responses to the reinforcer (number delay) in the left column and time
to the reinforcer (time delay) in the right column. Sessions obtained with drh contingencies (apex
up triangles) drl contingencies (apex down triangles) or no rate contingencies (circles) arranged
for responding during right-key runs have been separately fitted with exponential functions (no
fit can be shown for response delay with the drl contingency for Pigeon 81 because all sessions
with that condition were conducted at a single N value, N = 4).
<Insert Figure 27 about here>
These graphs may be thought of as data projections against the rear wall of the XYZ
coordinates first when facing from the number side (left column) and then when facing from the
Catania et al., Page 31
time side (right column), and therefore corresponding to the perspective views shown in the left
and right sections of the triptych in Figure 26.
Although the data from Pigeon 75 are less orderly than those from the other pigeons, as a set
all support the argument that time delays more tightly control left-key responding than do
response delays. On the whole, the fits are closer together in the time-delay graphs on the right
than in the response-delay graphs on the left, and most of the fits on the left are of shallower
slope than those on the right, with one of the latter (Pigeon 75) even showing an increasing rather
than a decreasing function.
Discussion
The relation between response delays and response rates (left column) seems weaker than
that between time delays and response rates (right column), but it can still be argued from these
data that intervening response number, like intervening time, makes some contribution to the
effects of delayed reinforcers. This argument can be made even in the light of data such as those
of Dews’ interrupted fixed intervals (Figure 1), because at any time in the interval responding is
separated from the reinforcer by the intervening FI responses as well as by the remaining FI
time.
<Insert Figure 28 about here>
Decreased responding with increased number delays, however, are not sufficient evidence
that number is an effective delay variable. Figure 28 shows why this may be so. In the nine XYZ
coordinates, each row provides three views of a set of delay functions as seen from different
perspectives. In the top row, set A, Z decreases only with X and remains constant over changes
in Y (left column); a gradient is not evident with viewing from the Y side (middle column) but
one appears with viewing from the X side (right column). In the middle row, set B, Z decreases
Catania et al., Page 32
only with Y and remains constant over changes in X (left column); this time a gradient is evident
with viewing from the Y side (middle column) but not with viewing from the X side (right
column). In the bottom row, set C, Z is a joint function of X and Y, so viewing from either the Y
side or the X side shows decreasing Z with increases in the value of either the X or the Y
parameter.
Now consider the function defined by the filled circles, which are identical in the three XYZ
sets. If data samples of Z were obtained only along the diagonal of the XY base or, in other
words, only when X and Y were equal, then the outcome would not discriminate among the A, B
and C alternatives because each would generate identical data, as illustrated by the filled circles
in the various perspective views. Figure 29 provides an approximation to such a case in an XYZ
plot of the data of Pigeon 81. Over the course of shaping, number delays and time delays tended
to be correlated; this correlation across the data from all four pigeons is illustrated in Figure 30.
<Insert Figures 29 and 30 about here>
The implication is that flat functions should not have been expected in the left graph of
Figure 27. For example, delays tend to grow with increased numbers even with added drh
contingencies, so it is difficult to move behavior along a shaping trajectory that remains parallel
either to X or to Y in the XYZ coordinates. Given such constraints, the path of the XY
coordinates will to some extent move diagonally across the XY space during shaping. Responsedelay gradients that are not flat may be totally compatible with behavior that is determined solely
by time delay. The inverse, of course, is also a possibility.
One feature of the performances of our pigeons that is not evident in the summary data of
Appendix 3 is that it often became very difficult to budge responding once shaping had moved it
some way along either the time or the number continuum. Perhaps this was because at those
Catania et al., Page 33
points we were approaching limits on the behavior that can be supported by a given delay of
reinforcement. One way in which that could happen has been demonstrated within a simulation
of operant behavior built upon some of the assumptions about delay that we have been exploring
here (cf. Catania, 2005e, Section 3.2).
If delay gradients involved well-defined limits on behavior, then variability is a relevant
dimension in judging the priorities of accounts in terms of response delay or in terms of time
delay. Table 3 shows r2 values for the functions shown fitted to the data in Figure 27. Based on
different assumptions about the distributions underlying exponential functions, more than one
way to calculate their variances accounted for is available (e.g., Cameron & Windmeijer, 1997);
those presented here were generated by curve-fitting programs in DeltaGraph7© (Red Rock
Software).
<Insert Table 3 about here>
In ten of the eleven possible comparisons available in Table 3, r2 was larger for delays
measured in terms of time than for delays measured in terms of number, sometimes by large
magnitudes (e.g., Pigeon 60, condition Lo; Pigeon 75, condition Hi; and Pigeon 81, condition O).
A difference in the same direction is also evident in comparing the data from the number-based
procedures of Experiments 1 and 2 (Figures 13 and 21) with those of the time-based procedure of
Experiment 3 (Figure 23). The weight of the evidence implies that time, not number, is the
appropriate metric for assessing delay of reinforcement.
Experiment 5: Delay and Reinforcer Duration
The research up to this point has been conducted with fixed reinforcer durations. The
literature suggests that reinforcer magnitude, and duration in particular, is not typically a
Catania et al., Page 34
powerful variable, especially when compared with rate of reinforcement (e.g., Catania, 1963;
Doughty, Galuska, Dawson, & Brierley, 2012; Grace, 1995; Jenkins & Clayton, 1949; Ludvig,
Conover, & Shizgal, 2007). Some properties of delay gradients that have been considered here
suggest why this may be the case.
Consider a thought experiment in which a fixed-interval (FI) schedule is arranged for
responding maintained by a 3-s reinforcer. Hypothetical gradients that might be in play in the
procedure are shown in Figure 31. First, the schedule operates by itself (A) and engenders a
characteristic gradient. Then a second schedule is introduced that provides another 3-s reinforcer
partway through the first interval (B). That reinforcer truncates the tail of the original gradient
(cf. Figure 9), but responding is given by the total area under the two segments, so the two 3-sec
reinforcers together generate more responding than had been maintained by the single 3-s
reinforcer alone in A. Next, the added reinforcer is moved closer to the end of the original
interval (C) and then closer still (D), each time cutting off a greater area of the original gradient
than the tail of the new gradient brings in. Eventually, the interpolated reinforcer is scheduled so
late in the original interval that the second 3-s reinforcer follows almost immediately after the
earlier one (E). At this point, the two have effectively become a single 6-s reinforcer, but the
gradient maintained by the original 3-s reinforcer has been almost totally truncated. Because
maintained responding is given by the total area under the gradient, this new 6-s reinforcer
maintains about the same quantity of behavior as had the 3-s reinforcer when it was all by itself.
<Insert Figure 31 about here>
If delay gradients operate consistently with the gradients of this thought experiment, then
changing reinforcer durations for the performances we have been studying should have little if
any effect on response rates. Experiments in which changes in the value of some parameter have
Catania et al., Page 35
no effect are ordinarily of little interest, but in this instance it is worthwhile to confirm or
disconfirm the predicted outcome.
Method
Three White Carneaux in the UMBC laboratory, Pigeons 73,, 74 and 76, served in daily
sessions in a two-key pigeon chamber. As in Experiments 2 and 3, their pecking was maintained
by the RI 60-s reinforcement of a tandem FR M (left key) FR N (right key) sequence; M was
held constant at four and N at eight. The maximum session duration was 60 min, but maximum
reinforcers per session was adjusted so as to reduce the differences in postsession feeding that
would otherwise have followed from the different reinforcer durations. Successive conditions for
each pigeon are outlined in Table 4, which also summarizes the data.
<Insert Table 4 about here>
Given that response rates sometimes change systematically within sessions (McSweeney &
Hinson, 1992; McSweeney, Roll, & Cannon, 1994; Weatherly, McSweeney, & Swindell, 1995),
response rates calculated over the entire session were supplemented with rates calculated over
only the first twenty min of each session, but in each case while still adhering to the constraints
on data collection illustrated in Figure 11. Stability criteria and other details were as described in
General Method.
Results
In Figure 32, left-key response rates are shown for the first twenty minutes of each session
and for the entire session, averaged over the last five sessions of each condition for each pigeon.
Response rates measured over the first twenty min were higher than rates measured over the full
session, indicated that response rates early in the session tended to drop off as the session
continued. Despite these differences in magnitude, however, the relations between response rates
Catania et al., Page 36
and reinforcer durations were similar across the two measures: left-key responding did not vary
consistently with reinforcer duration. Linear fits showed positive slopes for Pigeon 73 and
negative slopes for Pigeons 74 and 76,
<Insert Figure 32 about here>
Discussion
Usually a finding of no reliable differences across changes in the value of some parameter is
of minimal interest. In this case, however, the finding is consistent both with prior data and with
the implications of our assumptions about how delayed reinforcement operates. In particular, if
an interpolated reinforcer engenders its own gradient and blocks the gradients engendered by
later reinforcers from acting upon the responses that preceded it, as in Figures 9 and 21, then it
follows that the onset of a reinforcer rather than its duration may be the major source of its
potency. If this is how it works, it should be no surprise that small reinforcers can have large
effects.
Experiment 6: Delay and the Random-Interval Parameter
The delay gradients in the research so far have all been obtained with an RI 60-s schedule
maintaining the higher-order terminal sequence of responses on the left and right keys. Longer or
shorter RI schedules would not only provide lower or higher overall rates of reinforcement; they
would also allow longer or shorter tails in the delay gradient, and perhaps they would also have
other effects. We have assumed that beyond the limits of the left-key runs specified in our
contingencies, gradients will be shallow and also that, at that remove and therefore not being
subject to those contingencies, responses will consist of a mix of responses on the two keys,
probably approximating the mix observed in the relative rates of responding on the two keys.
Catania et al., Page 37
The implication is that changes in RI value should have small effects, if any, on the delayu
gradients obtained with topographical tagging procedures. Once again, as with Experiment 5, it
may be of interest to examine a parameter changes in the value of which are expected to have
little or no effect. This last experiment explored the RI parameter both by varying delays with RI
15-s, RI 30-S and RI 60-s schedules, and by holding delay roughly constant with a fixed M plus
N sequence while the RI value was varied over a range from 5 to 120 s.
Method
Four White Carneaux, male Pigeons 60, 73, 76 and female Pigeon 77, and two Silver King
males, Pigeons 47 and 59, served in daily sessions in two-key chambers in the UMBC
laboratory. Procedures were similar to those of earlier experiments. In the first, with the required
left-key requirement (M) held constant at 4, right-key requirements (N) of 2, 4 and 8 were
examined during sessions of RI t-s (tandem FR M FR N) schedules. Pigeons 73 and 77 served in
one two-key chamber, with t set to 15, 30 and 60 s; the maximum session duration was set to 30
min across all condition for these two pigeons, to reduce the impact of changes in response rate
within sessions (cf. Experiment 5, Method). Pigeons 47 and 59 served in a different two-key
chamber in sessions at a different time, with t set either to 15 s or to 60 s, and with maximum
session duration set to 30 min for RI 15-s and 75 min for RI 60-s. In all conditions for all
pigeons, maximum reinforcers per session was set to 60. Order of conditions and other details
are provided in Table 5.
<Insert Table 5 about here>
In a second procedure, conducted in still another two-key chamber with Pigeons 60, 73 and
76, the respective left-key and right-key requirements of the tandem component of the schedule
Catania et al., Page 38
were held constant at M = 8 and N = 4, and the RI t-s schedule was varied over a range of t from
5 to 120. Table 6 provides order of conditions and other details.
<Insert Table 6 about here>
Results
Figure 33 shows left-key response rates as a function of the delay from the last left peck to
the next reinforced right peck, with the value of the RI schedule as a parameter. Exponential
functions have been fitted to each data set. No systematic differences in the slopes of the
functions, in their ordering along the y-axis or in variability are evident across the four pigeons.
<Insert Figure 33 about here>
The left column of Figure 34 shows left-key and right-key response rates as a function of the
value of the random interval, with the tandem schedule held constant at M=8 and N=4. The
functions decrease with RI value for Pigeon 60, increase for Pigeon 73, and also increase, though
only slight, for Pigeon 76. The right column of the figure shows that the changes in rate were
correlated with corresponding inverse changes in the delays to reinforcement with changes in RI
value. Not only were the effects of changes in the RI parameter unsystematic across pigeons, but
those changes that did occur are probably better attributed to varying delays of reinforcement
over conditions than to any direct effects of RI value per se.
<Insert Figure 34 about here>
Discussion
Once again we have a finding of no reliable differences across changes in the value of a
parameter. In this instance the finding is reassuring, because it implies that data-based
generalizations about delay gradients engendered by RI scheduling need not be limited to a
Catania et al., Page 39
narrow range of schedule values. Whether a similar generality will hold with respect to other
parameters remains to be seen.
General Discussion
Our findings with topographical tagging have direct implications for the analysis of
performances maintained by concurrent schedules of reinforcement. Accounts in terms of the
changing reinforcement probabilities for continued responding on one key and for changing over
to the other (Shimp, 1966) need to be expanded, because each reinforcer operates on prior
responding on both keys, no matter which peck produced it. A concern about such extended
effects of reinforcers was in fact implicit in supplements to concurrent procedures such as the
changeover delay (COD), which prevented a response on one key from being followed
immediately by a reinforced response on the other key (e.g., Catania, 1976). The interactions
among concurrent responses and their reinforcers provide a challenge for computer simulations
(Catania, 2005e); those that treat reinforcement and shaping as a variety of selection seem
especially promising (McDowell, 2014).
If delay effects depend mainly on the onset of reinforcers, as implied by the results of
Experiment 5, then even small reinforcers can be potent agents of behavior change. One such
event is the onset of a discriminative stimulus, and one response that may be reinforced by a
discriminative stimulus is looking at it. In observing-response procedures (Dinsmoor, 1983,
1989; Kelleher, Riddle, & Cook, 1962), seeing the stimulus is not an appreciably delayed
consequence but the reinforcers later produced in its presence are, and the durations of their
delays may determine the potency of the stimulus as a reinforcer of looking or observing
responses (Grice, 1948).
Catania et al., Page 40
Figure 35 shows that delay gradients maintained by discriminative stimuli as reinforcers are
similar in form to those maintained by consumable reinforcers, such as those examined here. The
left key in a two-key chamber was an observing-response key; the right key was a schedule key
(Catania, 2012). Equal proportions of FI or extinction (EXT) components operated during yellow
on the right key, unless observing responses, pecks during a pre-schedule stimulus, changed the
contingencies so that, instead of yellow for both, green accompanied the FI schedule and red
accompanied EXT. In this procedure, therefore, changes in the FI value changed the delay
between the onset of green on the right key and the later delivery of the FI reinforcer.
<Insert Figure 35 about here>
Delay gradients engendered by a range of reinforcers, including stimulus onset in the
maintenance of observing behavior, have been incorporated into a model of ADHD (attentiondeficit hyperactivity disorder) by Sagvolden and colleagues (Catania, 2005a, 2005b; Sagvolden,
Johansen, Aase, & Russell, 2005). In that account, delay gradients that decrease more rapidly
than those in a general population can lead to hyperactivity when they differentially reinforce
rapid response sequences, because the gradients cannot support the earlier responses of those
sequences if responding is emitted more slowly. They can also lead to attention deficit when
those gradients reduce the likelihood of observing responses, because stimulus onset becomes a
less effective reinforcer if responding does not quickly produce reinforcers in the presence of that
stimulus. Furthermore, consistent with ADHD as studied in populations of children, the model
shows that different delay-gradient slopes can engender different proportions of the hyperactivity
and attention-deficit components of the syndrome.
A frequent component of experimental procedures that involve complex discriminations is a
correction procedure, which prevents a trial containing errors from being promptly followed by a
Catania et al., Page 41
trial containing a reinforced correct response. The recognition that reinforcers produced by
correct responses can also act on earlier incorrect responses is implicit in this procedure.
Similar contingencies operate in educational environments, so applied practitioners and
teachers in general must be alert for situations in which they might be strengthening incorrect
responses along with the corrects that they reinforce (cf. Catania, 2011a). Reinforcing every
correct response and repeating any trial with errors until a child gets it right guarantees that any
error sequence will eventually be followed by a reinforced correct response. The correct
responses will probably dominate eventually, because they are most closely followed by
reinforcers. But errors might diminish slowly and perhaps continue indefinitely at modest levels
because they are reliably followed later by reinforced correct responses.
When errors or problem behavior share in the effects of a reinforcer, we may mistakenly
conclude that the reinforcer is not doing its job very well. But if the reinforced behavior includes
response classes that we did not intend to reinforce, it may simply be doing very well a job other
than the one we wanted it to do. Behavior that we do not want to reinforce should be kept from
getting consistently close to reinforcers produced by other responses.
Delay of reinforcement undoubtedly has many more implications for and applications to
human behavior (Stromer, McComas, & Rehfeldt, 2000). Another example of course involves
the effects of intersecting steeper and shallower delay gradients, leading to analogues of the
behavior that we discuss in terms of self-control (Rachlin & Green, 1972). But any extensions to
delay discounting with humans (e.g., Green & Myerson, 2004; Johnson & Bickel, 2002) are most
likely tenuous and indirect. Somehow the delay gradients that interact with behavior in
nonhuman preparations must be converted to mathematical or other nontemporal relations when
humans deal with temporally distant events and discount remote consequences more than those
Catania et al., Page 42
more imminent. It seems unlikely that this problem can be addressed without an account of
verbal behavior, and that may be more attainable with research on how relevant histories of
mathematical behavior are created than by studying relations obtained from populations for
which the relevant history is probably variable and mostly inaccessible. But the topic is crucial,
because verbal behavior seems to allow us to bypass some of the otherwise inevitable effects of
delayed reinforcers.
Delay of reinforcement is ubiquitous, and these experiments have only scratched the surface
of issues related to it. For example, we do not know enough about whether its effect depends on
how responses that occur during the delay are related to the response that produces the
reinforcer, as in the concept of coupling (Killeen, 1994; Killeen & Sitomer, 2003). The effect
produced by the relation between the reinforcer and the response that produced it seems to have
special potency. Does that imply that a response like the one that produced the reinforcer will be
strengthened more than one that is very different? Might the creation of one operant class
insulate members of that class from the delayed effects of reinforcers later produced by members
of another class?
We also do not know enough about how delayed reinforcement might be involved in the
knitting together of temporally extended response sequences, so they may become behavioral
units in their own right (e.g., the distinction between chaining and chunking; see Catania, 2013,
pp. 127-129). We do not know enough about the effects of delayed reinforcement on variability
(Neuringer, 2002; Odum, Ward, Barnes, & Burke, 2006; Wagner & Neuringer, 2006). Does
responding become more or less variable with increasing temporal separation from the
reinforcer? We do not know enough about the role of delayed reinforcers in transitions from
schedules in which reinforcers are contingent upon responses to those of free or adventitious
Catania et al., Page 43
reinforcer deliveries (Catania, 2005d; Sizemore & Lattal, 1977; Williams & Lattal, 1999). We do
not know enough about tradeoffs between delayed reinforcement and changes in reinforcement
probability (Aucella, 1984; Lattal, 1974). And we do not know enough about the effects of
delayed reinforcers on measures other than response rate, such as accuracy (Blough, 1959),
choice (Fantino & Abarca, 1985; Gentry & Marr, 1980; Hackenberg, 1992; Mazur, 1995), or
resistance to change (Doughty & Lattal, 2003; Nevin & Grace, 2000; Podlesnik & Shahan,
2008). We could multiply such questions, but one thing we do know: More research on delay of
reinforcement is needed.
Acknowledgments
Many have contributed in many different ways to the research reported here. We especially
thank Rosie Mills, the UMBC Animal Caretaker. A number of UMBC undergraduates over
several acsdemic years assisted with a variety of tasks related to daily experimental sessions,
animal care, and the maintenance of data and records. These included, among others, Pearl
Friedman, Megan Glaze, Mariko Johnson, Stacey Long, Kevin Stine, Irina Vishnevetsky, and
Crescianciana Malinis. Experiment 5 was derived from an undergraduate Honors project by Lara
Kowalsky Kehle, and Leanne Valentine contributed both to Experiments 1, 2 and 3 and to the
research reported in Figure 35. Koji Hori of Rikkyo University in Japan also participated during
a academic sabbatical at UMBC. The statistical and data analysis skills of Eric J. French of CMU
smoothed the way for the development of Table 2. We have undoubtedly omitted some others,
but we are grateful to them along with those we were able to list here.
Our departments at both campuses also deserve thanks. The UMBC department in particular
allowed us to keep working even through times with negligible outside funding of the research.
Catania et al., Page 44
As grants for operant research became increasingly hard to come by, one set of proposal reviews
declined support on the grounds that the experiments were not feasible, and that even if they
were feasible the results would be of little interest. Delays of reinforcement sometimes seemed to
be intrinsically embedded within the entire enterprise.
We must also thank our pigeons for their service. Among them, female Pigeon 77 and male,
Pigeon 81 had long been housed in adjacent cages. With a change in experimental procedures
one of them was moved so they were no longer next to each other. Both stopped working in their
experimental chambers. They only got back to work when we moved them next to each other
again. Upon the closing of the UMBC pigeon laboratory we were able to place all our pigeons
with a local pigeon fancier, who inquired about their histories. There is more to pigeon behavior
than pecking and eating. We were pleased to learn that he housed both of these pigeons together
in a roomy pigeon coop.
Last but certainly not least, we regret that Eliot Shimoff did not survive to contribute to or
even to see this product of our collaborative research. His laboratory skills, his programming
expertise, and above all his enthusiasm were invaluable. This research could not have been done
without him.
Catania et al., Page 45
References
Aucella, A. (1984). Delay versus probability of reinforcement in discrimination learning. Jouranl of
Experimental Psychology: Animal Behavior Processes, 10, 452-460.
Baum, W. M. (2004). Molar and molecular views of choice. Behavioural Processes, 66, 349-359.
Baum, W. M. (2012). Rethinking reinforcement: Allocation, induction, and contingency. Journal of the
Experimental Analysis of Behavior, 97, 101-124.
Berg, J. P., & McDowell, J. J. (2011). Quantitative, steady-state properties of Catania's computational
model of the operant reserve. Behavioural Processes, 75, 71-83.
Blough, D. S. (1959). Delayed matching in the pigeon. Journal of the Experimental Analysis of Behavior,
2, 151-160.
Cameron, A. C., & Windmeijer, F. A. G. (1997). An R-squared measure of goodness of fit for some
common nonlinear regression models. Journal of Econometrics, 77, 329-342.
Catania, A. C. (1963). Concurrent performances: A baseline for the study of reinforcement magnitude.
Journal of the Experimental Analysis of Behavior, 6, 299-300.
Catania, A. C. (1969). Concurrent performances: Inhibition of one response by reinforcement of another.
Journal of the Experimental Analysis of Behavior, 12, 731-744.
Catania, A. C. (1970). Reinforcement schedules and psychophysical judgments: A study of some
temporal properties of behavior. In W. N. Schoenfeld (Ed.), The theory of reinforcement schedules
(pp. 1-42). New York: Appleton-Century-Crofts.
Catania, A. C. (1971). Reinforcement schedules: The role of responses preceding the one that produces
the reinforcer. Journal of the Experimental Analysis of Behavior, 15, 271-287.
Catania, A. C. (1973). Self-inhibiting effects of reinforcement. Journal of the Experimental Analysis of
Behavior, 19, 517-526.
Catania, A. C. (1976). Concurrent performances: Rate constancies without changeover delays. Journal of
the Experimental Analysis of Behavior, 25, 377-387.
Catania, A. C. (1983). Is not-responding behavior? Behavioral and Brain Sciences, 6, 321-322.
Catania et al., Page 46
Catania, A. C. (1991). Time as a variable in behavior control. In I. H. Iversen & K. A. Lattal (Eds.),
Experimental analysis of behavior. Part 2 (pp. 1-19). Amsterdam: Elsevier.
Catania, A. C. (2002). The watershed years of 1958-1962 in the Harvard Pigeon Laboratory. Journal of
the Experimental Analysis of Behavior, 77, 327-345.
Catania, A. C. (2005a). Attention-deficit/hyperactivity disorder (ADHD): Delay-of-reinforcement
gradients and other behavioral mechanisms. Behavioral and Brain Sciences, 28, 419-424.
Catania, A. C. (2005b). Attention-deficit/hyperactivity disorder (ADHD): One process or many?
Behavioral and Brain Sciences, 28, 446-450.
Catania, A. C. (2005c). Eliot Shimoff, 1943-2004. Journal of the Experimental Analysis of Behavior, 83,
197-199.
Catania, A. C. (2005d). The nonmaintenance of behavior by noncontingent reinforcement. European
Journal of Behavior Analysis, 6, 89-94.
Catania, A. C. (2005e). The operant reserve: A computer simulation in (accelerated) real time.
Behavioural Processes, 69, 257-278.
Catania, A. C. (2011a). Basic operant contingencies: Main effects and side effects. In W. W. Fisher, C. C.
Piazza & H. S. Roane (Eds.), Handbook of applied behavior analysis (pp. 34-54). New York:
Guilford.
Catania, A. C. (2011b). Model versus model or model versus data? A commentary on Berg and
McDowell’s “Quantitative, steady-state properties of Catania’s computational model of the operant
reserve.”. Behavioural Processes, 87, 84-87.
Catania, A. C. (2012). The pursuit of experimental analysis. European Journal of Behavior Analysis, 13,
269-280.
Catania, A. C. (2013). Learning (5th ed.). Cornwall-on-Hudson, NY: Sloan.
Catania, A. C., de Souza, D., & Ono, K. (2004). Free-choice preference when one alternative is rarely or
never chosen. Braxilian Journal of Behavior Analysis, 1, 69-80.
Catania et al., Page 47
Catania, A. C., & Keller, K. J. (1981). Contingency, contiguity, correlation, and the concept of causation.
In P. Harzem & M. D. Zeiler (Eds.), Predictability, correlation, and contiguity. Advances in analysis
of behaviour, Volume 2 (pp. 125-167). Chichester: Wiley.
Catania, A. C., & Reynolds, G. S. (1968). A quantitative analysis of the behavior maintained by interval
schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 11, 327-383.
Catania, A. C., Sagvolden, T., & Keller, K. J. (1988). Reinforcement schedules: retroactive and proactive
effects of reinforcers inserted into fixed-interval performances. Journal of the Experimental Analysis
of Behavior, 49, 49-73.
Chung, S.-H. (1965). Effects of delyed reinforcement in a concurrent situation. Journal of the
Experimental Analysis of Behavior, 8, 439-444.
Church, R. M., & Gibbon, J. (1982). Temporal generalization. Jouranl of Experimental Psychology:
Animal Behavior Processes, 8, 165-186.
Dews, P. B. (1960). Free-operant behavior under conditions of delayed reinforcement: I. CRF-type
schedules. Journal of the Experimental Analysis of Behavior, 3, 221-234.
Dews, P. B. (1962). The effect of multiple SΔ periods on responding on a fixed-interval schedule. Journal
of the Experimental Analysis of Behavior, 5, 369-374.
Dews, P. B. (1966). The effect of multiple SΔ periods on responding on a fixed-interval schedule: V.
Effect of periods of complete darkness and of occasional omissions of food presentation. Journal of
the Experimental Analysis of Behavior, 9, 573-578.
Dews, P. B. (1970). The theory of fixed-interval responding. In W. N. Schoenfeld (Ed.), The theory of
reinforcement schedules (pp. 43-61). New York: Appleton-Century-Crofts.
Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6,
693-728.
Dinsmoor, J. A. (1989). Selective observing: Pigeons turn discriminative stimuli on and off by pecking
separate keys. In N. W. Bond & D. A. T. Siddle (Eds.), Psychobiology: Issues and Applications (pp.
443-456). North-Holland: Elsevier.
Catania et al., Page 48
Doughty, A. H., Galuska, C. M., Dawson, A. E., & Brierley, K. P. (2012). Effects of reinforcer magnitude
on response acquisition with unsignaled delayed reinforcement. Behavioural Processes, 90, 287-289.
Doughty, A. H., & Lattal, K. A. (2003). Response persistence under variable-time schedules following
immediate and unsignalled delayed reinforcement. Quarterly Journal of Experimental Psychology
Section B, 56, 267-277.
Fantino, E., & Abarca, N. (1985). Choice, optimal foraging, and the delay-reduction hypothesis.
Behavioral and Brain Sciences, 8, 315-330.
Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York: Appleton-Century-Crofts.
Gentry, G. D., & Marr, M. J. (1980). Choice and reinforcement delay. Journal of the Experimental
Analysis of Behavior, 33, 27-37.
Grace, R. C. (1995). Independence of reinforcement delay and magnitude in concurrent chains. Journal of
the Experimental Analysis of Behavior, 63, 255-276.
Green, L., & Myerson, J. (2004). A discounting framework for choice with delayed and probabilistic
rewards. Psychological Bulletin, 130, 769-792.
Grice, G. R. (1942). An experimental study of the gradient of reinforcement in maze learning. Journal of
Experimental Psychology, 30, 475-489. doi: 10.1037/h0055908
Grice, G. R. (1948). The relation of secondary reinforcement to delayed reward in visual discrimination
learning. Journal of Experimental Psychology, 38, 1-16. doi: 10.1037/h0061016
Hackenberg, T. D. (1992). Choice in situations of time-based diminishing returns: Immediate versus
delayed consequences of action. Journal of the Experimental Analysis of Behavior, 57, 67-80.
Hamilton, E. L. (1929). The effect of delayed incentive on the hunger drive in the white rat. Genetic
Psychology Monographs.
Hineline, P. N. (2001). Beyond the molar-molecular distinction: We need multiscaled analyses. Journal of
the Experimental Analysis of Behavior, 75, 342-347.
Hull, C. L. (1943). Principles of behavior. New York: Appleton-Century-Crofts.
Catania et al., Page 49
Jenkins, W. O., & Clayton, F. L. (1949). Rate of responding and amount of reinforcement. Journal of
Comparative and Physioogicval Psychology, 42, 174-181.
Johnson, M. W., & Bickel, W. K. (2002). Within-subject comparison of real and hypothetical money
rewards in delay discounting. Journal of the Experimental Analysis of Behavior, 77, 129-146.
Katz, I., & Catania, A. C. (2005). Concurrent performances: Extinction, noncontingent reinforcement and
variably delayed reinforcement. European Journal of Behavior Analysis, 6, 95-108.
Kelleher, R. T., Riddle, W. C., & Cook, L. (1962). Observing responses in pigeons. Journal of the
Experimental Analysis of Behavior, 5, 3-13.
Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. New York: Appleton-CenturyCrofts.
Killeen, P. R. (1988). The reflex reserve. Journal of the Experimental Analysis of Behavior, 50, 319-331.
Killeen, P. R. (1994). Mathematical principles of reinforcement. Behavioral and Brain Sciences, 17, 105172.
Killeen, P. R. (2001). Writing and overwriting short-term memory. Psychonomic Bulletin and Review,
8(1), 18-43.
Killeen, P. R. (2011). Models of trace decay, eligibility for reinforcement, and delay of reinforcement
gradients, from exponential to hyperboloid. Behavioural Proceeses, 87, 57-63.
Killeen, P. R., & Sitomer, M. T. (2003). MPR. Behavioural Processes, 62, 49-64.
Laties, V. G., Weiss, B., Clark, R. L., & Reynolds, M. D. (1965). Overt "mediating" behavior during
temporally spaced responding. Journal of the Experimental Analysis of Behavior, 8, 107-116.
Lattal, K. A. (1974). Combinations of response reinforcer dependence and independence. Journal of the
Experimental Analysis of Behavior, 22, 357-362.
Lattal, K. A. (1984). Signal functions in delayed reinforcement. Journal of the Experimental Analysis of
Behavior, 30, 11-17.
Lattal, K. A. (2010). Delayed reinforcement of operant behavior. Journal of the Experimental Analysis of
Behavior, 93, 129-139.
Catania et al., Page 50
LeFever, F. F. (1973). Instrumental response chains and timing behavior. (Doctoral Dissertation), New
York University, University Microfilms, Ann Arbor, MI. (73-19,943)
Ludvig, E. A., Conover, K., & Shizgal, P. (2007). The effects of reinforcer magnitude on timing in rats.
Journal of the Experimental Analysis of Behavior, 87, 201-218.
Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E.
Mazur, J. A. Nevin & H. Rachlin (Eds.), Quantitative Analyses of Behavior, Vol. 5, The Effect of
Delay and of Intervening Events on Reinforcement Value (pp. 283-308). Hillsdale, NJ: Erlbaum.
Mazur, J. E. (1995). Conditioned reinforcement and choice with delayed and uncertion primary
reinforcers. Journal of the Experimtal Analysis of Behaivor, 63, 139-150.
McDowell, J. (2014). Predictions of an evolutrionary theory of behavior dynamics. Paper presented at the
Society for the Quantitative Analyses of Behavior, Chhicago, IL.
McSweeney, F. K., & Hinson, J. M. (1992). Patterns of responding within sessions. Journal of the
Experimental Analysis of Behavior, 58, 19-36.
McSweeney, F. K., Roll, J. M., & Cannon, C. B. (1994). The generality of within-session patterns of
responding: Rate of reinforcement and session length. Animal Learning and Behavior, 22, 252-266.
Murre, J. M. J., & Chessa, A. G. (2011). Power laws from individual differences in learning and
forgetting: Mathematical analyses. Psychonomic Bulletin and Review, 18, 592-597.
Neuringer, A. J. (1969). Delayed reinforcement versus reinforcement after a fixed interval. Journal of the
Experimental Analysis of Behavior, 12, 375-383.
Neuringer, A. J. (2002). Operant variability: Evidence, functions, and theory. Psychonomic Bulletin and
Review, 9, 672-705.
Nevin, J. A., & Grace, R. C. (2000). Behavioral momentum and the Law of Effect. Behavioral and Brain
Sciences, 23, 73-130.
Odum, A. L., Ward, R. D., Barnes, C. A., & Burke, K. A. (2006). The effects of delayed reinforcement on
variability and repetition of response sequences. Journal of the Experimental Analysis of Behavior,
86, 159-179.
Catania et al., Page 51
Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London: Oxford University Press.
Perin, C. T. (1943). A quantitative investigation of the delay-of-reinforcement gradient. Journal of
Experimental Psychology, 32, 37-51.
Podlesnik, C. A., Jimenez-Gomez, C., Ward, R. D., & Shahan, T. A. (2006). Resistance to change of
responding maintained by unsignaled delays to reinforcement: A response-bout analysis. Journal of
the Experimental Analysis of Behavior, 85, 329-347.
Podlesnik, C. A., & Shahan, T. A. (2008). Response–reinforcer relations and resistance to change.
Behavioural Processes, 77, 109-125.
Rachlin, H., & Green, L. (1972). Commitment, choice and self-control. Journal of the Experimental
Analysis of Behavior, 17, 15-22.
Reilly, M. P., & Lattal, K. A. (2004). Within-session delay-of-reinforcement gradients. Journal of the
Experimental Analysis of Behavior, 82, 21-35.
Renner, K. E. (1964). Delay of reinforcement: A historical review. Psychological Bulletin, 61, 341-361.
doi: 10.1037/h0048335
Sagvolden, T., Johansen, E. B., Aase, H., & Russell, V. A. (2005). A dynamic developmental theory of
Attention-Deficit/Hyperactivity Disorder (ADHD) predominantly hyperactive/impulsive and
combined subtypes. Behavioral and Brain Sciences, 28, 397-468.
Seward, J. P. (1942). The delay-of-reinforcement gradient in maze learning. Journal of Experimental
Psychology, 30, 464-474.
Shahan, T. A., & Lattal, K. A. (2005). Unsignaled delay of reinforcement, relative time, and resistance to
change. Journal of the Experimental Analysis of Behavior, 83, 201-219.
Shimp, C. (1966). Probabilistically reinforced choice behavior in pigeons. Journal of the Experimental
Analysis of Behavior, 9, 443-455.
Sizemore, O. J., & Lattal, K. A. (1977). Dependency, temporal contiguity, and response-independent
reinforcement. Journal of the Experimental Analysis of Behavior, 27, 119-125.
Catania et al., Page 52
Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York: AppletonCentury-Crofts.
Skinner, B. F. (1940). The nature of the operant reserve. Psychological Bulletin, 37, 423.
Staddon, J. E. R. (1965). Some properties of spaced responding in pigeons. Journal of the Experimental
Analysis of Behavior, 8, 19-27.
Staddon, J. E. R., & Simmelhag, V. L. (1971). The "superstition" experiment: A reexamination of its
implications for the principles of adaptive behavior. Psychological Review, 78, 3-43.
Steinman, W. M. (1968). Response rate and varied reinforcement: Reinforcers of similar strengths.
Psychonomic Science, 10, 35-36.
Stromer, R., McComas, J., & Rehfeldt, R. A. (2000). Designing interventions that include delayed
reinforcement: Implications of recent laboratory research. Journal of Applied Behavior Analysis, 33,
359-371.
Wagner, K., & Neuringer, A. (2006). Operant variability when reinforcement is delayed. Learning and
Behavior, 34, 111-123.
Watson, J. B. (1917). The effect of delayed feeding upon learning. Psychobiology, 1, 51-59.
Weatherly, J. N., McSweeney, F. K., & Swindell, S. (1995). On the contributions of responding and
reinforcement to within-session patterns of responding. Learning and Motivation, 421-432.
White, K. G. (2001). Forgetting functions. Animal Learning and Behavior, 29, 193-207.
Williams, A. M., & Lattal, K. A. (1999). The role of the response-reinforcer relation in delay-ofreinforcement effects. Journal of the Experimental Analysis of Behavior, 71, 187-194.
Wixted, J. T., & Gaitan, S. C. (2002). Cognitive theories as reinforcement history surrogates: The case of
likelihood ratio models of human recognition memory. Animal Learning and Behavior, 30, 289-305.
Wolfe, J. B. (1934). The effect of delayed reward upon learning in the white rat. Journal of Comparative
Psychology, 17, 1-21.
Catania et al., Page 53
Wright, A. A., Cook, R. G., Rivera, J. J., Shyan, M. R., Neiworth, J. J., & Jitsumori, M. (1990). Naming,
rehearsal, and interstimulus interval effects in memory processing. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 16, 1043-1059.
Catania et al., Page 54
Authors Note
The first author may be contacted at catania@umbc.edu and the second author at
reill1mp@cmich.edu.
<No footnotes>
Catania et al., Page 55
Table 1. Left-key (L) and right-key (R) responding in Experiment 3.
Data for three pigeons. PRP = post-reinforcement pause, rfr = reinforcer. Details in text.
Delay
Pigeon F! (s) (s)
11 0.25
0.99
0.50
1.50
1.00
2.55
2.00
2.92
4.00
5.15
L
resp/min
73.5
51.7
28.9
30.8
12.1
R
resp/min
14.5
9.1
5.5
13.7
20.3
L+R
resp/min
88.0
60.8
34.4
44.5
32.4
PRP
rfr/min (s)
Sessions
1.09
3.15
25
1.01
4.24
56
1.07
5.71
37
1.07
7.43
21
0.99
11.03
33
Delay
F! (s) (s)
54 0.25
0.59
0.50
0.87
1.00
1.14
2.00
2.16
4.00
4.34
L
resp/min
67.4
52.9
45.5
29.2
12.5
R
resp/min
15.5
17.0
34.0
61.6
35.5
L+R
resp/min
82.9
69.9
79.5
90.8
48.0
PRP
rfr/min (s)
Sessions
1.06
2.18
22
0.96
2.39
21
1.10
3.26
20
1.02
6.34
21
0.85
11.75
38
Delay
F! (s) (s)
64 0.25
0.83
0.50
0.98
1.00
1.37
2.00
2.77
4.00
4.61
L
resp/min
57.6
62.8
50.0
38.6
21.8
R
resp/min
24.8
29.2
29.7
25.9
30.9
L+R
resp/min
82.4
92.0
79.7
64.5
52.7
PRP
rfr/min (s)
0.89
3.73
1.01
3.75
0.98
4.25
1.04
6.14
1.02
8.83
Sessions
22
33
24
20
22
Catania et al., Page 56
Table 2. Least-square fits to left-key data from Experiments 1, 2 and 3.
The fitted equations were exponential, X = Be-KD, hyperbolic, X = B/(1+KD), and logarithmic,
X = AlnD + C, where X is left-key resp/min, D is delay in s, and B, K and C are constants.
Fits
Experiment Pigeons
1
73
(M+N=16)
74
76
78
2
73
(M=8)
74
76
78
3
11
(FI T-s)
54
64
Exponential
r2
B
K
138.8 0.59 0.83
125.0 0.46 0.72
91.4 0.41 0.79
68.6 0.34 0.79
112.2 0.51 0.92
54.3 0.20 0.48
48.9 0.18 0.75
46.8 0.23 0.65
115.3 0.49 0.97
85.6 0.50 0.98
74.8 0.26 0.96
Hyperbolic
r2
B
K
191.3 1.56 0.77
529.2 4.93 0.70
255.8 2.75 0.79
82.8
0.70 0.75
185.4 1.74 0.89
60.5
0.33 0.50
59.9
0.41 0.82
51.9
0.40 0.66
653.9 7.92 0.98
148.0 2.02 0.99
89.6
0.55 0.94
Logarithmic
r2
A x (-1)
C
-44.8
76.2 0.80
-46.2
82.0 0.73
-33.0
62.9 0.79
-21.8
49.5 0.75
-38.4
67.7 0.91
-14.0
46.0 0.50
-13.4
42.2 0.82
-11.7
36.9 0.66
-36.8
69.1 0.95
-26.9
50.6 0.98
-22.0
57.8 0.93
Catania et al., Page 57
Table 3. R-squared estimates for exponential functions in Figure 27.
No fit was available for Pigeon 81, condition Lo. N = number, T = time. Details in text.
Contingency
Pigeon 60
Pigeon 75
Pigeon 77
Pigeon 81
N delay data
O (none)
Hi (drh)
0.488
0.672
0.632
0.110
0.862
0.631
0.436
0.663
Lo (drl)
0.102
0.318
0.333
(none)
T delay data
O (none)
Hi (drh)
0.728
0.476
0.814
0.788
0.934
0.695
0.754
0.803
Lo (drl)
0.717
0.338
0.869
0.608
Catania et al., Page 58
Table 4. Sessions and data for Experiment 5
Reinforcer durations (s), sessions, and maximum reinforcers (rfr) per session, with left (L) and
right (R) response rates shown for the first 20 minutes of sessions and for full sessions for each
pigeon and condition.
Reinforcer
Pigeons duration (s)
3.0
73
4.5
6.0
4.5
3.0
2.0
3.0
4.5
6.0
3.0
Maximum
Sessions rfr/session
14
50
14
35
21
22
21
35
14
50
14
60
28
50
14
35
14
22
14
50
1st 20 min
L
R
resp/min resp/min
42.9
111.5
41.9
105.0
42.5
96.0
36.4
95.9
35.0
101.5
34.8
97.6
40.3
85.9
40.1
87.6
37.2
85.6
31.8
92.5
Session
L
R
resp/min
resp/min
21.7
112.5
29.4
102.6
27.4
93.0
28.8
97.9
21.7
103.3
23.6
95.9
28.6
97.1
27.0
90.6
23.2
86.0
18.8
100.0
74
2.8
4.5
3.0
2.0
3.0
4.5
6.0
3.0
6.0
4.5
3.0
14
14
21
21
14
14
14
14
14
14
14
50
35
50
60
50
35
22
50
22
35
50
30.0
21.0
23.0
22.7
31.3
23.7
22.0
25.5
22.4
27.7
19.4
47.5
50.0
54.9
56.8
45.6
59.3
56.0
55.9
61.6
69.5
61.3
23.1
14.7
17.0
14.4
25.1
13.9
14.7
16.8
13.8
16.6
16.9
49.3
51.6
55.3
56.6
47.7
59.6
56.4
54.3
61.3
66.9
58.3
76
3.0
4.5
6.0
4.5
3.0
2.0
3.0
6.0
3.0
14
35
21
14
14
14
28
14
14
50
35
24
35
50
60
50
22
50
27.4
24.8
22.5
24.3
22.2
23.4
28.1
25.7
27.6
59.8
67.3
67.5
62.9
63.2
66.0
73.4
69.7
64.0
18.4
14.8
14.3
15.8
14.7
14.2
17.1
16.3
19.9
58.3
64.3
68.0
62.3
62.3
69.2
73.0
70.4
63.6
Catania et al., Page 59
Table 5. Effects of varying N with different random-interval (RI) schedules
Required left-key responses = N; delay = time from last left (L) peck to next reinforced right
peck. Details in text.
Pigeon
73
47
N
(right)
2
4
8
4
2
2
4
8
4
2
2
4
8
4
2
RI
(s)
30
2
4
8
4
2
2
4
8
4
2
60
15
60
15
delay
(s)
2.13
2.61
4.94
2.79
2.21
2.31
2.79
4.31
2.38
1.80
1.79
2.14
3.98
2.69
1.91
L
resp/min
22.1
16.8
12.0
19.9
18.8
18.3
20.8
15.3
19.4
23.5
24.7
29.8
19.4
23.0
27.4
0.45
0.63
1.82
0.83
0.50
0.60
0.90
3.04
1.06
0.68
38.1
35.7
20.8
42.8
42.3
30.0
23.8
24.5
22.7
38.7
Pigeon
77
59
N
(right)
2
4
8
4
2
2
4
8
4
2
2
4
8
4
2
RI
(s)
30
2
4
8
4
2
2
4
8
4
2
15
60
15
60
delay
(s)
1.14
1.82
3.06
1.71
0.97
0.97
1.66
3.40
1.64
1.10
0.92
1.55
3.27
1.60
0.93
L
resp/min
22.8
26.5
18.1
21.1
25.1
25.5
19.0
19.5
26.0
26.7
27.4
15.8
11.5
15.1
19.7
Sessions
17
16
25
14
12
17
8
7
9
7
7
7
7
7
7
0.63
1.11
2.30
1.12
0.69
0.69
1.52
2.84
1.41
0.67
51.7
38.0
36.2
42.3
42.7
43.5
40.7
27.3
33.8
51.9
14
12
13
8
10
16
10
14
10
13
Catania et al., Page 60
Table 6. Effects of random interval (RI) on responding maintained by delayed reinforcement
Required left (M) = 8, required right (N) = 4; Max = maximum session duration. Details in text.
Pigeon 60
Pigeon 73
Pigeon 76
RI (s)
30
10
120
60
30
5
90
20
30
15
45
L to rfr
(s)
2.04
2.11
2.70
1.85
1.93
1.64
2.25
1.62
1.75
1.89
1.84
L
resp/min
24.9
15.7
15.3
12.8
17.1
16.2
19.5
27.2
23.7
21.2
26.7
R
Resp/min
47.1
59.6
39.0
54.5
62.7
70.1
52.1
66.9
58.6
70.0
58.4
Max
min
60
30
120
60
60
20
90
30
60
30
60
Sessions
30
19
16
19
14
14
14
14
14
14
21
30
120
60
10
30
90
5
20
30
15
45
3.26
2.82
4.11
5.52
6.62
6.69
*
9.11
9.16
6.99
7.00
17.3
21.1
11.8
4.5
5.8
8.0
*
6.8
9.4
6.9
7.3
51.1
52.6
46.2
40.9
37.4
32.4
*
30.5
30.8
47.6
40.7
60
120
60
30
60
90
20
30
60
30
60
30
22
16
19
14
14
14
14
14
14
21
30
10.8
42.0
60
3.97
30
60
16.2
35.1
60
5.08
20
10
8.9
36.9
30
4.14
15
120
12.6
44.3
120
3.04
19
30
*
*
60
*
14
90
12.5
38.9
90
3.80
14
20
7.0
48.8
30
3.52
14
5
*
*
20
*
14
30
8.1
42.3
60
3.47
14
15
9.9
43.2
30
3.43
14
45
9.1
46.5
60
3.23
21
*Data omitted because of programming/recording problems
Catania et al., Page 61
Appendixes
Appendix 1. Detailed conditions and data from Experiment 1.
Appendix 2. Detailed conditions and data from Experiment 2.
Appendix 3. Detailed conditions and data from Experiment 4.
These Appendixes are provided separately as the pages of an Excel file:
Delay appendixes 1 2 3.xlsx
Figures (Figures 1 through 35) are provided separately in a file named:
Delay Figures.docx
Catania et al., Page 62
Fig. 1. Increases in response rate during 50-s segments of a 500-s fixed interval (FI) in
which alternating segments with light or no light ended with a reinforced response at the end of
the final lighted segment. Periods when the light was off produced low response rates and
therefore interrupted FI responding, but rate when the light was on increased in much the same
way as in a standard FI. (Data averaged across four pigeons; adapted from Dews, 1962, Fig. 2.)
Fig. 2. Schematic of reinforced response sequences. In 1, the last of two responses is
reinforced; in 2, the last of ten is reinforced. Reinforcement probabilities are respectively 1/2 and
1/10, but only two responses are followed by the reinforcer in 1 as opposed to ten responses in 2.
Even allowing for effects of delay, fewer responses are reinforced in 1 than in 2 (cf. Catania,
1971)..
Fig. 3. Schematics of two delayed reinforcement procedures, In the first (top), the delay
resets if a response occurs before it ends. A response, as at x, will always be followed by the
reinforcer after t s, as long as no other response occurs during t. In the second (bottom), the delay
does not reset with responses. After one response initiates the delay, other responses may occur
before the reinforcer is delivered, so that, as at y, the time separating the most recent response
from the reinforcer may be less than t. (Adapted from Catania & Keller, 1981, Figure 4-4.)
Fig. 4. Two hypothetical delay gradients. In both, each response contributes to subsequent
responding in proportion to the height of the gradient at the time it occurs. The contribution in
the top gradient is provided by only a single response. The contribution in the bottom one is the
sum of the contributions provided by each of the several responses.
Fig. 5. Schematic of the topographical tagging procedure, used to separate the respective
contributions to subsequent responding of the peck (b) that precedes the one that produces the
reinforcer and the peck (a) that produces the reinforcer. If the peck that precedes the one that
Catania et al., Page 63
produces the reinforcer must be emitted on Key B while the one producing the reinforcer
continues to be required on Key A, later pecking on Key B can be attributed to the strengthening
of Key-B pecking by a delayed reinforcer, where the delay is given by the time between a and b
(cf. Catania, 1971).
Fig. 6. Illustration of how a delay gradient might operate on responding in a topographical
tagging procedure. A contingency has guaranteed that a sequence of pecks on one key (top) must
be followed by a sequence on a second key (bottom) before a second-key response can produce a
reinforcer. Any response on the first key will be separated (dashed vertical line) from the
reinforcer by these second-key responses, and therefore can only contribute to subsequent
responding in proportion to the height of the delay gradient at the time at which it occurs.
Fig. 7. Data from four pigeons in a topographical tagging procedure; the smooth curves
show exponential fits. Variable-interval (VI) schedules arranged reinforcers for sequences
consisting of one left-key peck followed by N right-key pecks, with N ranging from 1 to 11.
Left-key response rates are plotted against the estimated mean time between the left-key peck
and the reinforcer produced by the last peck of the right-key sequence (mean B-key IRT x N;
mean changeover times from B to A were unavailable). The plotted data are derived from
Catania (1971), Table 2.
Fig. 8. Two hypothetical gradients. Some procedures use measures that effectively
determine the height of the gradient at a given delay of reinforcement (vertical line, top),
whereas others use measures that allow estimates of the area under the gradient up to that delay
(filled area, bottom). Details in text.
Catania et al., Page 64
Fig. 9. Schematic example of a delay gradient generated by a reinforcer (rfrx), at the top,
and the truncation of that gradient by the introduction of an earlier reinforcer (rfry), at the
bottom. Details in text.
Fig. 10. Standard pigeon key equipped with a Hall Effect switch instead of a make-break
electrical contact. Details in text.
Fig. 11. Schematic of responding on left (L) and right (R) keys during the time between two
reinforcers (RFR) in a segment of a topographical-tagging session with random-interval (RI)
scheduling of reinforcers (PRP = postreinforcer pause). Details in text.
Fig. 12. Schematic illustration of consequence of the traditional method of arranging RI
schedules and the method used in the present studies. In the former, timing stops with
uncollected setups; in the latter, timing continues and setups can accumulate. The difference
between obtained and scheduled reinforcers is smaller in the latter than in the former. Details in
text.
Fig. 13. Responses per minute on the left key as a function of the time between the last left
peck and the later reinforced right peck, for four pigeons. The lines are exponential fits to the
data.
Fig. 14. Overall response rate (sum of left-key and right-key rates) as a function of the
number of pecks required on the left key (M, where M plus N equals sixteen and N is the number
of pecks required on the right key). The lines are linear fits to the data.
Fig. 15. Responses per run on the left and right keys as a function of the required pecks on
the right key (N). The lines are linear fits to the data.
Catania et al., Page 65
Fig. 16. Time from the last left peck to the next reinforced right peck (delay of
reinforcement) as a function of the required pecks on the right key (N). The lines are exponential
fits to the data.
Fig. 17. Changeovers (COs) and postreinforcement pauses (PRPs) as a function of the time
from the last left peck to the next reinforced right peck (delay of reinforcement). The lines are
linear fits to the CO data only.
Fig 18. Rate of responding on each key as a function of the time from the last left peck to
the next reinforced right peck (delay of reinforcement). The lines for the left-key rates are
exponential fits. Those for the right-key data, not well fit by exponential functions, are
logarithmic fits.
Fig. 19. Responses per minute on the left key as a function of the number of pecks required
on the left key (M), with the required pecks on the right key (N) held constant at 8, for two
pigeons. The lines are linear fits to the data.
Fig. 20. Time from the last left peck to the next reinforced right peck (delay of
reinforcement) as a function of the required pecks on the right key (N), in Experiment 2. The
lines are exponential fits to the data. Cf. Figure 16.
Fig. 21. Responses per minute on the left key as a function of the time between the last left
peck and the later reinforced right peck, for four pigeons in Experiment 2. The lines are
exponential fits to the data. Cf. Figure 13.
Fig. 22. Total rates of responding (left key plus right key) and right-key rates as a function
of the time from the last left peck to the next reinforced right peck in Experiment 2. The fits are
linear for the former and logarithmic for the latter. Details in text, and cf. Figures 14 and 18.
Catania et al., Page 66
Fig. 23. Responses per minute on the left key as a function of the time between the last left
peck and the later reinforced right peck, for three pigeons in Experiment 3. The lines are
exponential fits to the data. Cf. Figures 13 and 21.
Fig. 24. Total rates of responding (left key plus right key) and right-key rates as a function
of the time from the last left peck to the next reinforced right peck in Experiment 3. Details in
text, and cf. Figure 22.
Fig. 25. Schematic of the possible effects of rate shaping based on two different
assumptions: that the effects of delay depend solely on the time separating behavior from a later
reinforcer (time delay, left column) or that they depend solely on the number of responses
separating behavior from a later reinforcer (number delay, right column). The schematic shows
different contingencies arranged for right-key runs and illustrates the contrasting effects that
delays would produce if based on one or the other of these assumptions. Details in text.
Fig. 26. Three perspectives on an XYZ plot of data for Pigeon 60. With the overall relation
shown in the center, the left graph emphasizes the relation between number delay and left-key
responses rates and the right graph emphasizes the relation between time delay and left-key
responses rates. Details in text.
Fig. 27. Left-key response rates plotted against number delays (left column) and time delays
(right column) over various contingencies that shaped lengths of right-key runs and local rightkey response rates for four pigeons (O = none, Hi = drh, Lo = drl). Fits are logarithmic. Details
in text.
Fig. 28. Three perspectives on XYZ coordinates showing how a single function relating
response rate to number delay and time delay (filled circles within each graph) can be compatible
Catania et al., Page 67
with a function totally determined by variable X (row A), or with one totally determined by
variable Y (row B), or with one jointly determined by both X and Y (row C).
Fig. 29. Data in XYZ coordinates for Pigeon 81, showing data the variation of which with
number delays and time delays resembles the hypothetical data with filled circles in Figure 28.
Symbols and conditions are as in Figure 26.
Fig. 30. Points in the number-delay time-delay coordinate space (the base of the XYZ
graphs of Figures 26 and 29) explored with each pigeon during Experiment 4.
Fig. 31. Hypothetical delay gradients showing how the truncation of gradients as one
reinforcer moves closer in time to another (as in B through E) can lead to a gradient engendered
by two reinforcers (E) that is essentially equivalent to one engendered by a single reinforcer (A).
Details in text.
Fig. 32. Left-key response rates maintained as a function of reinforcer durations for four
pigeons. Data are shown both for the first twenty min of sessions and for full sessions to
accommodate possible effects of within-session changes in response rates. The lines provide
linear fits.
Fig. 33. Response rates as a function of time to the reinforcer with the value of the RI
schedule as a parameter, for four pigeons. The data are shown with exponential fits.
Fig. 34. Left-key and right-key response rates as a function of the value of the RI schedule
(left column), and delays from the last left peck to the next reinforced right peck (time delay) as
a function of the value of the RI schedule (right column), for three pigeons. Details in text.
Fig. 35. Observing-response rates as a function of the time to a reinforcer at the end of a
fixed-interval, with exponential fits for three pigeons. (Adapted from Catania, 2012, Figure 2).