Journal of Vision (2012) 12(3):17, 1–22
http://www.journalofvision.org/content/12/3/17
1
Mutual inhibition and capacity sharing during parallel
preparation of serial eye movements
Supriya Ray
National Brain Research Centre, Manesar, Haryana, India
Neha Bhutani
National Brain Research Centre, Manesar, Haryana, India
Aditya Murthy
Centre for Neuroscience, Indian Institute of Science,
Bangalore, Karnataka, India
Many common activities, like reading, scanning scenes, or searching for an inconspicuous item in a cluttered environment,
entail serial movements of the eyes that shift the gaze from one object to another. Previous studies have shown that the
primate brain is capable of programming sequential saccadic eye movements in parallel. Given that the onset of saccades
directed to a target are unpredictable in individual trials, what prevents a saccade during parallel programming from being
executed in the direction of the second target before execution of another saccade in the direction of the first target remains
unclear. Using a computational model, here we demonstrate that sequential saccades inhibit each other and share the
brain’s limited processing resources (capacity) so that the planning of a saccade in the direction of the first target always
finishes first. In this framework, the latency of a saccade increases linearly with the fraction of capacity allocated to the other
saccade in the sequence, and exponentially with the duration of capacity sharing. Our study establishes a link between the
dual-task paradigm and the ramp-to-threshold model of response time to identify a physiologically viable mechanism that
preserves the serial order of saccades without compromising the speed of performance.
Keywords: double step, serial order, decision making, accumulator model
Citation: Ray, S., Bhutani, N., & Murthy, A. (2012). Mutual inhibition and capacity sharing during parallel preparation of serial
eye movements. Journal of Vision, 12(3):17, 1–22, http://www.journalofvision.org/content/12/3/17, doi:10.1167/12.3.17.
Introduction
Karl Lashley (1951) in his seminal work on the
formulation of the serial order in behavior postulated a
simultaneous neural representation of all serial elements of
an action before the action began. Several models, including a class of neural networks that assume a simultaneous
representation of competitive elements of a sequence, have
been put forth to account for serial order in different types
of behavior like typing, speech, etc. (Botvinick & Plaut,
2004; Cooper & Shallice, 2006; Glasspool & Houghton,
2005; Grossberg, 1978; Jordan, 1997; Rhodes, Bullock,
Verwey, Averbeck, & Page, 2004; Sandamirskaya &
Schöner, 2010). Neurons in the prefrontal cortex and the
frontal eye field indeed exhibit concurrent representation
of forthcoming sequential manual movements (Averbeck,
Chafee, Crowe, & Georgopoulos, 2002; Barone & Joseph,
1989; Funahashi, Inoue, & Kubota, 1997; Mushiake,
Saito, Sakamoto, Itoyama, & Tanji, 2006) and saccadic
eye movements (Fujii & Graybiel, 2003; Tian, Schlag, &
Schlag-Rey, 2000), respectively. The eyes move in an
ordered sequence often during regular activities like
reading or scanning natural images (Land, 2009; Morrison,
1984; Rayner, 1998). Nevertheless, no reported attempt
has been made to model the serial order of saccades.
doi: 1 0. 11 67 / 1 2 . 3 . 1 7
Previous studies that posited simultaneous planning of
competing (Camalier et al., 2007; Findlay & Walker,
1999; Ludwig, Farrell, Ellis, & Gilchrist, 2009) or
consecutive saccades (Dominey & Arbib, 1992; Ludwig,
Mildinhall, & Gilchrist, 2007; Van Loon, Hooge, & Van
den Berg, 2002) mainly focused on their spatial orientations or latency distributions.
We have examined how the brain preserves the order of
successive saccades despite stochasticity in the rate of
concurrent planning by modeling behavior of human
subjects when they shift the gaze to the locations of two
targets in the order of appearance of the targets. Previous
studies have reported that the duration of fixation between
consecutive saccades may decrease below the average
latency of saccade made in isolation to a single target,
indicating that the preparation of the second saccade prior
to the end of first saccade is feasible (Becker & Jürgens,
1979). Further, the intersaccadic interval (ISI) decreases
down to a limit as the second target appears progressively
earlier than the onset of saccade to the first target, which
suggests that consecutive saccades directed to two visible
targets are processed in parallel. The latency of both
saccades in a sequence progressively increase as the delay
between appearances of the targets decreases from 200 to
50 ms (Ray, Schall, & Murthy, 2004). A similar pattern of
increase in the response latency occurs in the dual-task
Received July 8, 2011; published March 19, 2012
ISSN 1534-7362 * ARVO
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
paradigm as well (Kahneman, 1973; McLeod, 1977; Navon
& Gopher, 1979; Pashler, 1994), when two independent
tasks are performed in a rapid succession (Tombu &
Jolicœur, 2002). The capacity-sharing theory of dual-task
interference postulates that the smaller the fraction of
capacity allocated to a task, the longer a subject takes to
produce the corresponding response (Navon & Miller,
2002; Tombu & Jolicœur, 2003). The response to the first
task may even be generated after the response to the second
task, if the first task is extremely difficult (Leonhard,
Fernández, Ulrich, & Miller, 2011), suggesting that the
distribution of limited resources (capacity) of information
processing between tasks is critical to preserve the order
of responses.
We have designed a model to test our hypothesis that
the planning stages of successive saccades share capacity
when they overlap in time to ensure that the first saccade
is always executed in the direction of the first target and
the subsequent saccade in the direction of the second
target. In our model, two interactive signals concurrently
rise to a threshold at stochastic rates to generate
successive saccades. The strength of interaction between
processes planning saccades depends on the proportion of
capacity allocated to each saccade. Whether successive
saccades share capacity has never been tested and the
neural mechanism of sharing capacity between tasks is
unknown as well. Nonetheless, studies on monkeys that
relate the buildup activity of cortical and subcortical
neurons to saccade initiation (Dorris, Paré, & Munoz,
1997; Hanes & Schall, 1996; Ipata, Gee, Goldberg, &
Bisley, 2006; Tanaka, 2007) and parallel planning of
successive saccades (McPeek & Keller, 2002; Murthy et al.,
2007; Phillips & Segraves, 2010) partially support the
physiological feasibility of this model. Simulation of our
model suggests that the sharing of capacity and mutual
inhibition between a pair of processes planning saccades
preserve the order of saccade execution and account for
many latency phenomena observed during sequential
saccadic eye movements.
Methods
Experiment
The task, stimuli, and recording techniques used in the
experiment have been described in detail elsewhere (Ray
et al., 2004). In summary, fourteen healthy human
subjects participated in a modified double-step task
(Figure 1a). All subjects gave their informed consent in
accordance with the Institutional Human Ethics Committee
of the National Brain Research Centre, India. In 60% of
2
total trials, called no-step trials, a green target appeared
after a random fixation period between 300 and 800 ms at
one among eight possible locations, which were 45- apart
from each other at an eccentricity of 12- visual angle from
the central fixation spot. In the remaining trials, called
step trials, a second target of red color appeared at a
randomly chosen location non-adjacent to the green target
on the imaginary circular array. These targets were
identical in size, shape, and luminance and remained on
the screen until the end of the trial. The delay between the
appearances of the targets (commonly called the stimulus
onset asynchrony or SOA) was randomly chosen from 50,
100, 150, and 200 ms. Subjects were instructed to shift
gaze to the location of the green target in no-step trials
and generate successive saccades rapidly to the location of
the green and red targets in the order they appeared in step
trials. No-step and step trials were randomly interleaved.
To test the stability of our model, we examined two
additional subjects who were not explicitly instructed to
generate saccades “as quickly as possible,” unlike other
fourteen subjects. Eye movements were recorded using an
infrared eye tracker (ISCAN, MA, USA) at 200-Hz
sampling rate. TEMPO/VIDEOSYNC (Reflective Computing, MO, USA) software generated stimulus and
collected and stored the sampled eye positions in every
trial. The beginning and end of each saccade were
demarcated offline when the eye velocity crossed a 30-/s
threshold. The latency of a saccade was calculated from
the delay between the appearance of a target and the onset
of the saccade directed to the target. Trials that produced
blink-perturbed saccades were not considered for subsequent analyses.
Capacity sharing by sequential saccades
Tombu and Jolicœur (2003) previously established a
relationship between response time and SOA in the dualtask paradigm based on the assumption that the rate of
processing a response is linearly proportional to the
fraction of capacity allocated to the corresponding task.
Following their basic assumptions, we derived equations
for the latency of saccades directed to the first (RT1V) and
second (RT2V) targets (see Appendix A):
SP j 1
SP j 1
A1 þ B1
RT1 ¼
SOA þ
A2 þ
;
SP
SP
SP
0
ð1Þ
RT20 ¼ A1 þ B1 þ B2 j SOA:
ð2Þ
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
3
Figure 1. (a) Schematic of the temporal sequence of stimuli and behavior in the task. In no-step trials, subjects made saccades to a
singleton green target. In step trials, a second red target appeared after SOA. Subjects directed their gaze from the fixation spot to the first
target and, subsequently, to the second target. (b) Latency distributions of saccades directed to the first (gray) and second (black) targets for
individual subjects (thin lines) and across the population of subjects (thick lines; first: green, second: red). (c) Plot of mean intersaccadic
interval (ISI) against parallel processing time (PPT) across subjects. Horizontal broken line shows the mean saccade latency in no-step
trials. (d) Average latencies of saccades directed to the first (circle) and second (diamond) targets are contrasted at different SOAs. Error
bars in (c) and (d) show the standard error of the mean ISIs and saccade latencies, respectively.
In Equations 1 and 2, A1 (A2) and B1 (B2) refer to the
duration of the visual processing stage and planning
stage of a saccade directed to the first (second) target,
respectively, when saccades are generated in isolation
or do not share capacity (Figure 2a). The delay between
onsets of the targets is SOA. The sharing proportion
(0 G SP G 1) is the fraction of capacity allocated for
planning a saccade in the direction of the first target.
The remaining capacity (1 j SP) is assumed to be allocated
for planning another saccade in the direction of the second
target.
We modified Equations 1 and 2 such that the sharing
proportion (SP) from saccade latency can be derived
without explicit knowledge of the duration of planning
stages (B1, B2). Suppose that the duration of capacity
sharing between planning stages of saccades is D (e.g.,
D1/D2 in Figure 2b). In Equation 1, substituting SOA by
(RT1Vj A2 j D) and (A1 + B1) by RT1, we get
SP j 1
SP j 1
0
RT1 ¼
ðRT1 j A2 j DÞ þ
SP
SP
RT1
;
A2 þ
SP
SP j 1
RT1
RT10 ¼
ðRT10 j DÞ þ
;
SP
SP
RT10
1 j SP
RT1
;
¼
Dþ
SP
SP
SP
0
RT10 ¼ ð1 j SPÞ D þ RT1:
ð3Þ
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
4
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
Similarly in Equation 2, substituting SOA by RT1Vj A2
j D and (A2 + B2) by RT2, we get
RT20 ¼ RT1 þ B2 j RT10 þ A2 þ D;
5
exponentially with the duration of overlap between processing stages of consecutive saccades in contradiction to
the prediction by the linear capacity-sharing model
(Figure 2d). The data obeyed the following equations:
RT20 ¼ ðRT1 j RT10 Þ þ ðA2 þ B2Þ þ D;
n
o
RT10 ¼ erðPPTjA2Þ ð1j SPÞ ðPPT j A2Þ þ RT1; ð5Þ
RT20 ¼ jð1 j SPÞ D þ RT2 þ D;
and
RT20 ¼ SP D þ RT2:
ð4Þ
Note that according to Equations 3 and 4, the latency of a
saccade increases linearly with the fraction of capacity
allocated to the other saccade in a sequence and the
duration (D 9 0) of capacity sharing between saccades.
Interactive ramp-to-threshold model
of sequential saccades
n
o
RT20 ¼ erðPPTjA2Þ SP ðPPT j A2 þ RT2;
ð6Þ
where PPT (parallel processing time) is the time of onset of
saccade to the first target relative to the time of appearance
of the second target (e.g., PPT1/PPT2 in Figure 2b), and
r (90) is a constant. Substituting (PPT j A2) by D in the
above equations, we get
RT10 ¼ erD ð1 j SPÞ D þ RT1;
ð7Þ
RT20 ¼ erD SP D þ RT2:
ð8Þ
Model design
The empirical data show that the latency of saccades
directed to the first and second targets increased
Figure 2. (a) A model of parallel processing of sequential
saccades in which capacity is unlimited (left). The duration of
sensory, planning, and execution stages of a saccade directed to
the first (second) target is A1 (A2), B1 (B2), and C1 (C2),
respectively. The latencies of saccades directed to the first (RT1)
and second (RT2) targets remain constant when the parallel
processing time decreases from PPT1 to PPT2 as the delay
between onsets of targets increases from SOA1 to SOA2 (right).
(b) A capacity-sharing model of sequential saccades in which
planning stages share capacity when they overlap in time (left).
The duration of the planning stage of a saccade directed to the
first target increases from B1 to B1V(top, green) and from B1 to
B1W (bottom, red), and that of a saccade directed to the second
target increases from B2 to B2V(top, green) and from B2 to B2W
(bottom, red), when the delay between onsets of the targets
increases from SOA1 to SOA2. Because the duration (D) of capacity
sharing decreases as the stimulus onset asynchrony (SOA)
increases, this model predicts that the latency of saccades directed
to the first and second targets will also decrease with decreasing
parallel processing time (left). (c) A Gaussian fit of the histogram of
the reciprocal of saccade latency in correct no-step trials to
estimate the average (TSD) processing rate of saccades. (d) The
mean latency of saccades directed to the first (circle) and second
(diamond) targets in correct step trials increased with the mean
parallel processing time (PPT). The sharing proportion (SP = 0.655)
was estimated from an exponential (solid green) fit of the latency of
saccades directed to the first target. In contrast, a linear (dotted
green) relationship between saccade latency and PPT as predicted
by the classical capacity-sharing theory estimated SP = 0.349.
Exponential (solid red) and linear (dotted red) fits of the latency of
saccades directed to the second target are contrasted. Error bars
show the standard deviations of the mean saccade latencies.
Equations 7 and 8 differ from Equations 3 and 4 by an
additional exponential (erD) term on the right-hand side
of each equation.
We modeled sequential saccades to understand the
mechanism underlying exponential increase of latency
with the duration of capacity sharing. We extended the
central idea of the ramp-to-threshold models of saccadic
reaction time. This class of models assumes that a decision
signal rises to a threshold at a stochastic rate to generate a
saccade (Brown & Heathcote, 2008; Carpenter, 1981;
Laming, 1968; Luce, 1986; Nakahara, Nakamura, &
Hikosaka, 2006; Purcell et al., 2010; Smith, 1995). In
our model, two decision signals simultaneously rise at
stochastic rates to elicit saccades in respective directions
upon arriving at a threshold. From a physiological perspective, a ramp-to-threshold signal may reflect an accrual of
information over time by means of recurrent self-excitation
of a neural computing unit. Models that incorporate a
recurrent self-excitation mechanism sample and accumulate evidence in favor of alternative choices and provide a
common framework to understand motor planning and
decision making. The resulting accumulated information
reliably mimics the time-varying neural activity of nonhuman primates performing decision-making tasks (Lo &
Wang, 2006; Mazurek, Roitman, Ditterich, & Shadlen,
2003; reviewed in Gold & Shadlen, 2007).
In the absence of an interaction, sequential saccade
planning can be modeled by a pair of decision signals,
each rising from the baseline (= 0) to a threshold (= 1) to
initiate a saccade in the direction of the corresponding
target. When these signals overlap in time, their ratio may
provide a means to calculate how likely the first saccade is
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
going to be directed to the first target. Suppose that u1 and
u2 are the signals corresponding to saccades in the
direction of the first and second targets, respectively
(Figure 3, left). The rate of increase of u1 and u2 remain
fixed in a trial but vary across trials. In light of previous
6
findings that while the likelihood ratio merely provides an
instantaneous evidence in support of a hypothesis
between alternatives (Carpenter & Williams, 1995), the
log-likelihood ratio serves as a natural currency for making
a decision (Bogacz, Usher, Zhang, & McClelland, 2007;
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
Wald & Wolfowitz,
1948), we defined a Confidence Index
u1
(CI) as log u2 to quantify the level of confidence with
which one can predict the direction of the first saccade.
When u1, u2 9 0, the first saccade is more likely to be
directed to the first target if uu12 9 1 or CI 9 0. On the other
hand, uu12 G 1 or CI G 0 implies that the saccade aimed at the
second target is more likely to be executed first. The
direction of the first saccade is completely ambiguous
when CI = 0 or, stated differently, when decision signals
for saccades are identical (i.e., u1 = u2).
In addition, we defined the Degree of Concurrency (DoC)
as the duration of overlap between the planning stages of
consecutive saccades in a step trial divided by the average
duration of saccade planning in no-step trials, i.e., DoC =
D
B1. The maximum value DoC can achieve is 1 when D =
B1, and DoC e 0 implies serial processing of saccades.
n
o
Þ
(see Appendix A), we
Replacing D by B1jðSOAþA2jA1
SP
hn
o
i
B1jðSOAþA2jA1Þ
B1
. Because A1, A2,
get DoC =
SP
B1, and SP are assumed constant, the degree of concurrent
preparation of saccades increases as SOA decreases. In
contrast, the possibility of the first saccade being elicited
in the direction of the first target decreases as SOA
decreases, due to the stochastic rate of saccade planning.
1
D
This
DoC ò CI
. Replacing
implies
DoC by B1 and CI by
u1
u
D
log u2 , we get B1
¼ jn log u12 , where n (90) is a
proportionality constant. We assumed that r1 is the rate
of saccade planning to a singleton target in a no-step
trial. Because no-step and step trials were randomly
interleaved, subjects were expected to start planning a
saccade to the first target at the same rate (r1) in the step
7
trial unaware that a second target would be forthcoming.
Replacing the duration
(B1) of saccade planning in the
no-step trial by r11 , we get
n
u1
u2
r1 D
or e
¼
:
r1 D ¼ jn log
u2
u1
ð9Þ
We measured the coefficient r (= 0.0033 msj1) in
Equations 7 and 8 from the fit (Figure 2d). Because
1
r (= 300 ms) is of the order of a visually guided saccade
latency in ordinary conditions, we speculate that r is
related to the rate (r1) of planning the saccade in isolation
rD
(no-step trial). Hence erD ò er1D . nSubstituting
in
oe
Equations 7 and 8 by ðm er1 D Þ or m m (90) is a proportionality constant, we get
u2
u1
n
, where
n
u2
ð1j SPÞ D þ RT1;
RT1 ¼ m u1
ð10Þ
n
u2
SP D þ RT2:
RT2 ¼ m u1
ð11Þ
0
0
Equations 10 and 11 establish a link between the capacitysharing model and the linear ramp-to-threshold model of
sequential saccade latency. Net increases in the latency of
saccades directed to the first and second targets due to
capacity sharing for duration D are
n
u2
$RT1 ¼ RT1 j RT1 ¼ m ð1j SPÞ D;
u1
0
0
ð12Þ
Figure 3. (a) Schematic of the capacity-sharing model (above)
and an equivalent interactive ramp-to-threshold model (below) of
parallel preparation of sequential saccades directed to two targets
that appeared in succession after SOA. An, Bn, and Cn refer to
the duration of visual processing, planning, and execution stages,
respectively, of the nth saccade in the sequence; n = 1, 2. In the
absence of capacity sharing, the latency of the nth saccade RTn =
An + Bn. When planning stages of saccades share capacity, Bn
increases to Bn V, giving rise to saccade latency RTn V= An + Bn V.
In the equivalent ramp-to-threshold model of saccade, Bn = 1/rn,
where rn is the growth rate of activity of a process planning a
saccade in the direction of nth target. In our model, rn remains
fixed in a trial but varies normally across the trials. In the
attenuation stage, processes that plan saccades inhibit each
other during the overlap period (D) to maintain the order of
consecutive saccades (ISI 9 0). (b) Sample simulation of 20 step
trials in Simulink. In the model, two integrators independently
accumulate information at constant rates r1 and r2 sampled from
normal distributions. Each of the two attenuators receives output
signals u1 and u2 from the accumulators and inhibit them to X1
and X2, respectively, which trigger sequential saccades upon
arriving at a threshold activity (= 1).
and
n
u2
$RT2 ¼ RT2 j RT2 ¼ m SP D:
u1
0
0
ð13Þ
Therefore, with every unit increase in D, the respective
saccade latency increases by
n
u2
ð1j SPÞ;
$RT1 ¼ m u1
ð14Þ
n
u2
$RT2 ¼ m SP:
u1
ð15Þ
0
and
0
Given that the proportion of allocated capacity (i.e., SP)
is fixed in a trial, Equations 14 and 15 suggest that the
ratio of magnitudes of corresponding decision signals is
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
critical to ensure (SOA + RT2V) 9 RT1V, which is a must to
maintain the order of saccades in a sequence. Therefore,
attenuation of one signal should take the present state of
the other signal into account to prevent ordering error.
This may be perceived as a mutual inhibition between
consecutive saccades. We hypothesized that the magnitude of a pair of signals simultaneously planning saccades
in the direction of the first (u1) and second (u2) targets is
reduced at every time step by the amount given on the
right-hand side of Equations 14 and 15, respectively.
When the rate of accumulation of information to generate
a saccade to the second target is considerably higher than
that to the first target, exertion of an inhibitory control
may be a means to prevent the signal rising to generate the
second saccade from reaching threshold before the first
saccade is generated. The purpose of such inhibitory
control is to dampen the ascent of the signal thereby
elongating the planning stage of the saccade to the second
target. This will cause the duration (D) of capacity sharing
to increase, which in turn will increase the latency of the
saccade to the first target (RT1V). Such an increase in RT1V
will demand further slowdown of planning the saccade to
the second target, and the interaction between planning
stages will continue until the signal rising to generate a
saccade to the first target reaches the threshold. To test the
idea that both mutual inhibition and capacity sharing are
critical to preserve the order of execution of consecutive
saccades, we designed a model using Matlab Simulink
software with an embedded S-Function written in C
language, which consists of two processing stages, an
accumulation stage and an attenuation stage (Figure 3).
Model description
We made the following key assumptions to construct
the model: (1) the appearance of a target activates an
accumulator after 70 ms, which is about the average
delay of visual signals in the FEF (Buschman & Miller,
2007; Pouget, Emeric, Stuphorn, Reis, & Schall, 2005;
Schmolesky et al., 1998); (2) the duration of visual
processing of each target is the same; (3) in every trial,
activity in the accumulators at time t are denoted by
u1,t{= (r1 t)}, and u2,t{= r2 (t j SOA)}, where r1 and
r2 are independently sampled from two sets of normally
distributed numbers with means that correspond to the
mean rate of planning saccades to the first and second
targets; (4) the variance of the rate of planning saccades
in the direction of two targets are the same. (5) Each of
the two attenuators at the next stage receives signals u1,t
and u2,t from both accumulators at time t + 1, where these
input signals inhibit each other and generate output
signals X1,t+1 and X2,t+1, respectively. In the absence of
any interaction, the output from an attenuator merely
reflects the activity of its corresponding accumulator. (7)
The strength of inhibition exerted by each decision signal
on another depends on the proportion of capacity they
8
share. (8) Because the firing rate of a neuron cannot be
negative, our model prevents X1 and X2 from falling below
zero. (9) The interaction continues until either X1 or X2
reaches the threshold normalized to 1 to elicit a saccade in
the direction of the corresponding target. During simulation of the model, X1,t and X2,t were computed as
X1;0 ¼ X2;0 ¼ 0 and
9
>
>
>
>
>
>
=
X1;tþ1 ¼ u1;t
if u2;t ¼ 0 or X2;t ¼ 1
n
;
u2;t
if u1;t ; u2;t 9 0 and X1;t ; X2;t G 1>
¼ u1;t j m ð1j SPÞ >
>
>
u1;t
>
>
;
¼0
if X1;tþ1 G 0
ð16Þ
u2;t
X2;tþ1 ¼ u2;t j m SP u1;t
¼ u2;t
¼0
n
9
>
>
if u1;t ; u2;t 9 0 and X1;t ; X2;t G 1 >
=
:
if X1;t ¼ 1
>
>
>
;
if X2;tþ1 G 0
ð17Þ
Finding parameters of the model
We collated data from all correct trials across the
population of fourteen subjects to find the value of two free
parameters (m and n) of the model that are independent of
any idiosyncratic behavior of individual subjects. No-step
trials that yielded identical (T10j4 msj1) reciprocals of
saccade latencies were grouped together to estimate the
mean (2) and standard deviation (A) of rate of saccade
processing (visual processing and motor planning) from2a
xj2
ffi2A
j p
1 ffi
fit by a Gaussian function of the form f (x) = pffiffiffiffiffiffiffi
e
,
2:A2
where 2 (= 0.005) and A (= 0.00095) were the coefficients
of the fit (Figure 2c). In this analysis, we did not include
trials that produced saccades of latency shorter than 100 ms.
The goodness of fit in terms of R2 (= 0.97), which was close
to the maximum (= 1.0), suggested that the Linear
Approach to Threshold with Ergodic Rate (LATER) model
that assumes a Gaussian distribution of the rate of saccade
processing can emulate saccade latencies in no-step trials
(Reddi & Carpenter, 2000). A two-sample Kolmogorov–
Smirnov test showed that the distribution of the reciprocal
of saccade latency in 2404 correct no-step trials was
indifferent (P = 0.178) from that of equal number of
random samples from a normal distribution with mean
(TSD) of (0.005 T 0.00095) msj1.
Two free parameters of the model, m (= 17.62) and
n (= 4) were optimized by minimizing the difference
between expected saccade latencies and simulated saccade
latencies in 1000 step trials at SOA = 50 ms, using “Least
Square Method” in “Parameter Estimation” tool of
Simulink (The Mathworks) software. We derived the
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
expected latencies of saccades directed to the first and
second targets using SP (= 0.655), r (= 0.0033 msj1),
RT1 (= 208 ms), and RT2 (= 318 ms),as obtained from
1
exponential fits (Figure 2d), and D ¼ r1 j SOA in
Equations 7 and 8, respectively. Since the duration of
visual processing of both targets were assumed the same
(i.e., A1 = A2), the duration (D) of overlap between
planning stages of consecutive saccades was equal to the
difference between the duration of planning a saccade to the
first target and SOA. The rate (r1) of planning a saccade to
the first target varied from trial to trial and was calculated,
first, by subtracting 70 ms from the expected duration of
saccade processing, then calculating the reciprocal of the
difference. The reciprocal of a value sampled from a
normal distribution with mean (= 0.005 msj1) and
standard deviation (= 0.00095 msj1) provided the
expected duration of saccade processing. The same value
of r1 was used in the model as the rate of increase of u1.
The rate (r2) of increase of u2 in each simulated step trial
1
=
was sampled from a distribution of mean (= RT2jA2
j1
j1
1
ms
)
=
0.004
ms
and
standard
deviation
=
318j70
0.00095 msj1.
Simulation of the model
A total of 1000 step trials for each of the four SOAs
(50, 100, 150, and 200 ms) were simulated in a discrete
time step of 1 ms for 1 s. Values of parameters used in the
model are summarized in Table 1. Simulation of a step
trial began at t = 0 when an integrator started accumulating information. Another integrator started accumulating
information after SOA. A pair of attenuators individually
received output from both integrators and attenuated
them. When the output signals of attenuators reached the
threshold, the simulator registered the onset of saccades
in the corresponding directions. The interval between the
time when an integrator started accumulation and the
corresponding attenuated signal reached the threshold
was considered as the duration of the planning stage of a
saccade. We calculated saccade latency by adding a
visual processing delay of 70 ms to the duration of the
9
planning stage and an additional 50 ms as the duration
of saccade execution to calculate the time of saccade
end.
Results
Summary of experimental data
Figures 1b–1d summarize the saccadic behavior of
human subjects in a modified double-step task (see Ray
et al., 2004 for details). Despite idiosyncrasies in the
latency distributions of sequential saccades [mean (TSD)
skewness, first: 1.4 T 0.68; second: 0.83 T 0.87] as shown
in Figure 1b (thin lines), two trends were observed across
the population of subjects. First, the duration of fixation
between consecutive saccades or intersaccadic interval
(ISI) decreased with increasing parallel processing time
(PPT) or the duration of overlap between processing
stages of saccades [mean (TSD) linear regression slope =
j0.29 (T0.23), P G 0.05]. Second, the latency of
consecutive saccades increased as the stimulus onset
asynchrony (SOA) decreased [mean (TSD) linear regression slope, first: j0.15 T 0.21; second: j0.59 T 0.36; P G
0.05]. In the aggregated data across subjects, a decrease in
ISI down to a limit at longer PPTs (Figure 1c) and an
increase in saccade latency with decreasing SOA
(Figure 1d) indicated a capacity-limited mechanism of
saccade programming.
Capacity-sharing account of behavior
How the distribution of capacity may affect latencies of
consecutive saccades have been contrasted schematically
in two hypothetical cases: In one model, consecutive
saccades do not share capacity during parallel processing
(Figure 2a), and in the other, consecutive saccades share
capacity when their planning stages overlap in time
Parameter
Value
21 (mean 1st saccade processing rate)
A (standard deviation of 1st saccade processing rate)
r1 (rate of 1st saccade planning)
0.005 msj1
0.00095 msj1
1
r1 = 1 j70
, R was sampled from a normal distribution with mean (= 21)
22 (mean rate of 2nd saccade planning)
r2 (rate of 2nd saccade planning)
SP (sharing proportion)
m
n
Table 1. List of parameters used in the model for simulation.
R
and standard deviation (= A)
0.004 msj1
Sampled from a normal distribution with mean (= 22)
and standard deviation (= A)
0.655
17.62
4
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
(Figure 2b). Suppose that the latency of a saccade directed
to the first target is RT1 (= A1 + B1) and that to the
second target is RT2 (= A2 + B2). The symbols A1 and
A2 refer to the duration of visual processing of the first
and second targets, respectively; B1 and B2 refer to the
duration of planning saccades in the direction of the first
and second targets, respectively. When the second target
appears after SOA2 (9SOA1), processing stages of
saccades overlap for a relatively shorter period of time
(PPT2 G PPT1). Because the ongoing preparation of a
saccade does not prevent preparation of the subsequent
saccade when each saccade in a sequence uses independent pool of resources (Figure 2a, left), latencies of
consecutive saccades should remain invariant across all
parallel processing times (PPTs) as shown in Figure 2a
(right). In the capacity-limited condition, planning stages
of consecutive saccades share capacity for an interval of
time denoted by D1, when the second target appears after
SOA1 (Figure 2b, top). As a result, B1 increases to B1V,
and B2 increases to B2V. Accordingly, RT1 increases to
RT1V(= A1 + B1V) and RT2 increases to RT2V(= A2 + B2V).
When the second target appears after SOA2 (9SOA1), the
processing stages of saccades overlap for a relatively shorter
period of time (PPT2 G PPT1), and capacity is shared for a
shorter duration (D2 G D1; Figure 2b, bottom). As a result,
RT1 increases to RT1W (= A1 + B1W) and RT2 increases
to RT2W (= A2 + B2W), where RT1 G RT1W G RT1V and
RT2 G RT2W G RT2V. The capacity-limited model predicts
that the latency of saccades directed to the first and second
targets will increase with PPT (Figure 2b, right).
The experimental data show that the reciprocal of the
first saccade latency in both no-step and step trials are less
outlier-prone from a normal distribution, in comparison to
the reciprocal of the second saccade latency [Figures 1b
and 2c; kurtosis: first (no step) = 4.0, first (step) = 3.2,
second (step) = 9.5], similar to what Van Loon et al. (2002)
previously observed. The first (green) and second (red)
saccade latencies both increased with PPT (Figure 2d), as
predicted by the capacity-sharing model. To test whether
a capacity-sharing model can account for distributions of
saccade latencies in step trials, we measured PPT from the
time of onset of saccade directed to the first target relative
to the time of appearance of the second target. Step trials
in which the processing stages of saccades did not overlap
(i.e., PPT G 0) were not considered for subsequent
analyses, and the rest were divided into groups so that
trials in each group yielded identical (T25 ms) PPT.
Figure 2d shows the plot of the mean latency of saccades
directed to the first (RT1V) and second (RT2V) targets
against the mean PPT for each group of trials. The
reciprocal of the standard deviation of the corresponding
mean saccade latency weighted every data point on the
plots. To fit the data, we replaced D in Equations 3 and 4
by (PPT j A2), where we considered an average visual
delay of 70 ms in the frontal eye field as the duration of
visual processing (A2) of the second target. From the
linear fit (R2 = 0.94; green dotted line) of the data by
10
RT1V = (1 j SP) (PPT j A2) + RT1, we obtained
RT1 = 209.1 ms and SP = 0.349. To account for the
non-linearity in the plot, we also fitted RT1Vby RT1V=
{er(PPTjA2)} (1 j SP) (PPT j A2) + RT1, where
r 9 0. From the exponential fit (R2 = 0.99; green solid
line), we obtained RT1 = 208.5 ms, SP = 0.655, and r =
0.0033 msj1. Note that in correct no-step trials, the mean
(TSD) saccade latency across the population of subjects
was 205 (T40) ms. Subsequently, we fitted the plot of
RT2V versus PPT by RT2V = SP (PPT j A2) + RT2,
using SP (= 0.349) as obtained from the linear fit of RT1V,
and by RT2V= {er(PPTjA2)} SP (PPT j A2) + RT2,
using SP (= 0.655) and r (= 0.0033 msj1) as obtained
from the exponential fit of RT1V. We obtained RT2 =
398 ms and 318.4 ms from the linear (red dotted line) and
exponential (red solid line) fits, respectively. The exponential function fitted the data better than the linear
function (R2: linear = 0.52, exponential = 0.93) and
closely approximated the average (TSD) latency (328 T
95 ms) of saccades to the second target that appeared
maximum 70 ms prior to the onset of the first saccades
(i.e., 0 G PPT e 70).
Mutual inhibition model of sequential
saccades
We speculated that the non-linear increase in latencies
of consecutive saccades with the duration of capacity
sharing was due to interactions between neural processes
that planned the saccades. We examined whether a pair of
independent and non-interacting integrators, each of
which accumulating evidence at a constant rate, can
simulate sequential saccades in the right order. Successive
appearances of the targets triggered these integrators to
generate two decision signals that linearly increased from
a baseline (= 0) to initiate sequential saccades upon
arriving at a fixed threshold (= 1). In a simulated step trial,
the growth rate of each of these signals was sampled from
a normal distribution with mean (TSD) of 0.005
(T0.00095) msj1. Simulation of 2000 step trials at the
stimulus onset asynchrony (SOA) of 50 and 200 ms each
showed that saccades targeted to the second target were
executed first in 17.35% and 9% trials, respectively.
Based on the observation that, on the one hand, the
classical capacity-sharing model is inadequate to account
for the latency distributions of consecutive saccades, while,
on the other hand, an independent ramp-to-threshold model
is inadequate to account for the order of consecutive
saccades, we designed a model in which a pair of processes
that progressively raised activity to a threshold to generate
consecutive saccades shared capacity and inhibited each
other (Figure 3). In this model, a pair of integrators
generated decision signals that increased from a baseline
(= 0). Each of these integrators independently accumulated information to generate the decision signal at a rate
that varied from trial to trial but remained constant in a
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
11
trial. Signals from both integrators were fed into each of
the two attenuators where they inhibited each other with
strength that depended on the proportion of capacity they
shared and the ratio of information accumulated by
individual integrators up to that point of time. The timevarying outputs from attenuators initiated consecutive
saccades in the direction of corresponding targets, upon
arriving at a fixed threshold (= 1).
Validation of the model
Simulated and observed data are compared in Figures 4a–
4c to show that capacity sharing and mutual inhibition
between saccades are critical for (1) the preservation of the
serial order of saccades, (2) a progressive increase in the
latency of consecutive saccades with decreasing stimulus
onset asynchrony (SOA), and (3) a gradual decrease in the
intersaccadic interval (ISI) down to a limit with increasing
parallel processing time (PPT). Distributions of the
simulated and observed saccade latencies are compared
in Figure 4a. The mean (TSD) simulated latency (first:
213 T 58 ms; second: 373 T 112 ms) was not different
from the corresponding mean (TSD) observed latency
(first: 225 T 65 ms; second: 371 T 112 ms). On average
(TSD), a total of 336 (T64) correct step trials contributed
data to calculate the mean saccade latency at each SOA.
The mean (TSD) observed latency of the first and second
saccades significantly (P G 0.001) increased from 219
(T63) ms to 252 (T71) ms and from 342 (T90) ms to 445
(T121) ms, respectively, as SOA decreased from 200 to
50 ms. Similarly, the mean (TSD) simulated latency of the
first and second saccades significantly (P G 0.001)
increased from 207 (T41) ms to 232 (T82) ms and from
343 (T76) ms to 438 (T147) ms, respectively, as SOA
decreased from 200 to 50 ms (Figure 4b). Step trials
with identical (T10 ms) PPTs were grouped together to
plot the mean ISI against the mean PPT for each bin
spanning 20 ms (Figure 4c). Each data point was
weighted by the reciprocal of the standard deviation of
the corresponding mean ISI for the fit by a function of
the form f (PPT) = a ejbPPT + c. Slopes of the
exponential fits (R2: simulation = 0.92, experiment = 0.75)
reached above j0.1 at PPT of 134 ms (observed) and
157 ms (simulated) and leveled off {ISImin = limPPTYV f
(PPT) = c} at ISI of 196 ms (observed) and 190 ms
(simulated). Simulated saccades directed to the second
target started at least 67 ms and on average (TSD) 237
(T93) ms after the end of saccades directed to the first
target, indicating that the order of sequential saccades was
always maintained.
Stability of the model
In principle, a stable capacity-sharing model that
accounts for the progressive postponement of consecutive
Figure 4. (a) Distributions of simulated (dotted) and observed
(solid) latencies of saccades directed to the first (green) and
second (red) targets. (b) Average simulated (unfilled) and
observed (filled) latencies of saccades directed to the first (circle)
and second (diamond) targets are contrasted at different SOAs.
Error bars show the standard errors of the mean saccade
latencies. Significant (P G 0.001) differences between two sets
of saccade latencies at different SOAs are marked by asterisks.
(c) Exponential fit of intersaccadic interval (ISI) plotted against
parallel processing time (PPT) for observed (solid) and simulated
(broken) data, respectively.
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
responses due to the shortening of the delay between the
appearances of stimuli should also be able to fit the
behavior in a situation when the distribution of capacity is
highly biased in favor of a response. We recorded eye
movements of two additional subjects in the same task,
but they were not encouraged to generate the subsequent
saccades “as fast as possible,” unlike other subjects
examined previously. We anticipated that possibly more
capacity would be allocated to saccades directed to the
first target in the absence of an urgency of generating
subsequent saccades to the second target. The left panel of
Figures 5a and 5b each shows the plots of the mean ISI
against the mean PPT (top) in ten bins each spanning
15 ms on the abscissa and the mean latency of the first
and second saccades against SOA (bottom) for subjects
KR and KV, respectively. Previously, we showed that
the decrease of ISI with increasing PPT between 0 and
150 ms is a hallmark of parallel processing of consecutive
saccades (Ray et al., 2004). Our data indicate that while
subject KR exhibited parallel processing of sequential
saccades, in case of subject KV, preparation of saccades
to the second target was apparently postponed until the
onset of saccades to the first target [regression slope:
j0.55 (KR), 0.08 (KV); correlation coefficient: j0.24 (KR),
0.06 (KV); P G 0.001 (KR), P = 0.21 (KV)]. Saccades
directed to the second target were progressively postponed as
SOA decreased [regression slope: j0.63 (KR), j0.80 (KV);
correlation coefficient: j0.97 (KR), j0.99 (KV); P G 0.05].
In contrast, none of the subjects postponed saccades
directed to the first target when SOA decreased.
In order to test the stability of our model with the
optimized free parameters m (= 17.62) and n (= 4), we
compared the simulated data with the data recorded from
individual subjects. We measured the sharing proportion
(SP), the mean rate of processing saccades in the direction
of the first target ( 21), the mean rate of planning saccades
in the direction of the second target ( 22), and the variance
(A) in the mean rate of saccade processing for each
subject following the techniques described in the Finding
parameters of the model section. Values of these parameters are shown in Table 2. Both subjects allocated about
18% (i.e., 1jSP) of total capacity for planning saccades
to the second target in comparison to 34.5% in previous
cases when other subjects were encouraged to speed up
saccades to the second target. Figures 5a and 5b (right
panels) show the plots of the mean simulated ISI against
the mean simulated PPT (top) in ten bins, each spanning
15 ms on the abscissa, and the mean RT of the simulated
first and second saccades against SOA (bottom) for
subjects KR and KV, respectively. Our model successfully simulated the behavior of subject KR who exhibited
some degree of parallel processing of sequential saccades
and subject KV as well who apparently processed consecutive saccades in series [regression slope: j0.27 (KR),
j0.09 (KV); correlation coefficient: j0.09 (KR),
j0.03 (KV); P G 0.05 (KR), P = 0.49 (KV)]. Saccades
directed to the second target were progressively postponed
12
as SOA decreased [regression slope: j0.60 (KR),
j0.82 (KV); correlation coefficient: j0.99; P G 0.05].
In contrast, the simulated latency, like the observed
latency, of saccades directed to the first target did not
show correlation with SOA.
Predictions of the model
Figure 6a shows how the distribution of capacity
influences simulated saccade latency. In a complete
bottleneck condition (i.e., SP = 1), planning a saccade to
the second target is postponed until the onset of the
preceding saccade to the first target, which causes a sharp
rise in the mean second saccade latency at the shortest
SOA, but the mean first saccade latency remains almost
invariant across SOAs. As SP decreases (i.e., second
saccade uses larger portion of capacity), the first saccade
latency increases and the second saccade latency
decreases at the shortest SOA. Simulation of our model
also shows that the serial order of saccades is not always
preserved without mutual inhibition (m = 0), while other
parameters of the model are kept fixed (Figure 6b). Note
that the order of saccade execution is switched in less than
50% of total simulated trials even when two targets appear
simultaneously (SOA = 0 ms), because the average rate of
planning saccades to the second target was naturally
slower than that to the first target (Table 1). The
attenuated signals X1 and X2 in a simulated step trial and
that averaged across the trials are shown in Figures 6c
and 6d, respectively. The order of saccades in a sequence
may be switched if a saccade to the second target is
planned independently (m = 0) at a relatively faster speed.
Our model suggests that the order of consecutive saccades
is maintained by mutual inhibition (m 9 0) between
decision signals rising to a threshold to generate saccades,
irrespective of the speed of their rise (Figure 6c). Figure 6d
shows the output of each attenuator averaged across 1000
trials and aligned on the beginning of the simulated first
saccades, which suggests that the rate of presaccadic
increase in the activity of neurons related to saccade
planning is slower when the planning stage of a saccade
temporally overlaps with that of another saccade, in
comparison to when a saccade is planned in isolation.
A steady rise of the decision signal corresponding to the
first saccade slowly halts after the appearance of the
second target until È100 ms prior to the saccade onset
due to an inhibitory interaction.
Discussion
The primary objective of this study was to identify a
mechanism that allows simultaneous preparation of
sequential saccades as an alternative to one in which
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
13
Figure 5. (a) Mean ISI is plotted against the mean PPT for ten bins each spanning 15 ms on the abscissa (top), and the mean latency of the
first (circle) and second (diamond) saccades are plotted against SOA (bottom), measured from the data recorded from subject KR (left panel)
and simulated on the computer (right panel). (b) The same plots derived from the data recorded from subject KV (left panel) and simulated on
the computer (right panel). Error bars show the standard errors of the mean saccade latencies and the mean intersaccadic intervals.
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
Parameter
Subject
KR
KV
SP
21 (msj1)
22 (msj1)
A (msj1)
0.8155
0.8179
0.0056
0.0061
0.0030
0.0044
0.0012
0.0013
Table 2. Sharing proportion (SP), average rate of processing of
saccades directed to the first target ( 21), average rate of planning
of saccades directed to the second target ( 22), and variance (A) in
the mean rate of saccade processing for two subjects. These
parameters in addition to the subject-independent fixed parameters of the model (m = 17.62, n = 4) were used to simulate
behavior of subjects KR and KV.
14
individual saccades are planned strictly in series (Goldberg
& Bruce, 1990; Robinson, 1973) or the entire sequence is
organized as a preprogrammed package to preserve the
serial order of saccades (Ditterich, Eggert, & Straube,
1998; Lévy-Schoen & Blanc-Garin, 1974; Zingale &
Kowler, 1987). The intersaccadic fixation duration
decreases with increasing delay between onsets of the
target for the second saccade and the first saccade and may
even fall below the average saccade latency, suggesting
parallel processing of consecutive saccades (Becker &
Jürgens, 1979; McPeek & Keller, 2002; Murthy et al.,
2007; Murthy, Ray, Shorter, Schall, & Thompson, 2009).
Figure 6. (a) Mean latency of simulated saccades directed to the first (green) and second (red) targets are shown as a function of stimulus
onset asynchrony (SOA) with different proportion of allocated capacity (diamond: SP = 1, circle: SP = 0.75, square: SP = 0.50, polygon:
SP = 0.25). The effect of capacity sharing is most prominent at the shortest SOA. (b) In the absence of mutual inhibition between
saccades, the percentage of simulated step trials in which the targets were followed in the opposite order of their appearances decreased
exponentially (solid, R 2 = 0.98) as displayed with 95% prediction bound (dotted). (c) An example of the preservation of serial order of
saccades by means of mutual inhibition. In the absence of mutual inhibition (solid: m = 0), a saccade in the direction of the second target
(red) may be elicited before a saccade in the direction of the first target (green), if planned at a faster rate. Mutual inhibition between
saccades (e.g., dotted: m = 1, dashed: m = 2) preserves the order of their directions. (d) Activity in an attenuator averaged across step
trials corresponding to saccades generated in the direction of the first (green) and second (red) targets that appeared 50 ms apart and that
averaged across no-step trials (black).
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
On the contrary, an inverse relationship between the
stimulus onset asynchrony (SOA) and saccade latency
suggests a potential processing bottleneck in the brain
(Lünenburger, Lindner, & Hoffmann, 2003; Ray et al.,
2004). In this study, to reconcile these two apparently
antagonistic phenomena together, we designed a model in
which two interactive decision signals rise to a threshold
to initiate successive saccades. Our model reliably
accounts for an increase in saccade latency with decreasing SOA and a decrease in ISI down to a limit with
increasing PPT. Most importantly, simulation of the
model suggests that consecutive saccades share capacity
and inhibit each other when processed in parallel to
maintain their order without using feedback information
of the progress in saccade planning, presumably to
expedite saccade execution. Our model can accommodate
cognitive control (e.g., allocation of attention) over
sequential saccades as well by preferentially allocating
capacity to one or the other saccade in the sequence, in
other words, by modulating the value of the critical
parameter SP in Equations 16 and 17.
Although the influence of the appearance of the second
target on the first saccade latency cannot be ruled out due
to the remote distractor effect (Ludwig, Gilchrist, &
McSorley, 2005; Walker, Deubel, Schneider, & Findlay,
1997), it is unlikely that the increase in the second saccade
latency was due to the presence of the first target, because
a remote distractor has no effect on the saccade latency
when it appears È80 ms or earlier than the onset of the
saccade target (Bompas & Sumner, 2009). Furthermore,
because the frequencies of the shorter and longer SOAs
were equal, it is also unlikely that SOA played any role in
favoring the parallel mode over the serial mode of saccade
processing (Miller, Ulrich, & Rolke, 2009).
Limitations of the model
In our model, for simplicity and compatibility with
literature on the capacity-sharing model of dual-task
interference, we assumed that the visual processing,
saccade planning, and saccade execution occur in series
(Pashler, 1994; Tombu & Jolicœur, 2003). Whether
saccadic reaction time is the sum of duration of distinct
subprocesses (Donders, 1868/1969; Sternberg, 1969) or
information flows continuously resulting in an overlap
between visual processing and motor planning stages
(Eriksen & Schultz, 1979; Meyer, Osman, Irwin, &
Yantis, 1988) is controversial. However, electrophysiological studies on behaving animals show that the saccade
planning in the frontal eye field (FEF) begins after
completion of visual processing of the target (Thompson,
Hanes, Bichot, & Schall, 1996; Woodman, Kang, Thompson, & Schall, 2008) and lend support to the concept of
compartmentalization of sensory and motor processing
stages. Our data showed that the saccade latency increases
exponentially with the parallel processing time (PPT) that
15
includes both the duration of visual processing and
saccade planning; therefore, the nature of transfer of
information between these stages is not critical to describe
the phenomenon. Our assumption that the saccade planning starts strictly 70 ms after the target onset in every
trial may not be correct, but the quality of fit of the data by
the model suggests that this delay adequately serves the
purpose across many trials.
Another important point to be noted here is that
although the average latency of simulated saccades
matched the average of experimental saccade latencies,
their distributions were slightly different as shown in
Figure 4a. An Ansari–Bradley dispersion test showed that
the latency distribution of simulated and observed saccades directed to the first target were identical in two
cases, when SOA was 150 ms and 200 ms, but the latency
distribution of simulated and observed saccades directed
to the second target were different for all SOAs. The
reason for this anomaly was the assumption that uncertainties in the speed of saccade planning to both targets
were the same, which seems to be incorrect. Unfortunately, because there was no means to measure the
variability in the latency of saccades made in isolation to
the second target in the correct step trials, this shortcoming of our model was unavoidable. However, the
validity of our model does not hinge on its ability to
reproduce the latency distributions rather on the ability to
successfully simulate the relationship between the average
saccade latency and SOA and between ISI and PPT
(Figures 4b and 4c). An invariance of the first saccade
latency with SOA for subjects who were not explicitly
instructed to generate saccades rapidly to the second target
suggests a processing bottleneck, which in theory corresponds to the situation SP = 1 in Equation 7. On the
contrary, our model suggests that the subjects did not
allocate full capacity (SP , 0.82) for planning saccades in
the direction of the first target. This anomaly can be
resolved if we consider that faster processing of saccades
to the first target (21,KR = 0.0056 and 21,KV = 0.0061
compared to 21,Population = 0.005) effectively reduced the
duration of capacity sharing. Thus, the magnitude of
postponement of first saccades decreased as well, resulting
in a putative processing bottleneck.
Capacity limitations in the context
of eye movements
In the literature, the term “capacity” has been described
in many ways: as a structural resource for simultaneously
holding information of multiple objects in memory
(Fukuda, Awh, & Vogel, 2010), as a form of attention
enabling stimulus identification (Marois, Chun, & Gore,
2000), and as a strategic resource for response selection
(Pashler, 1999) and executive control (Sigman &
Dehaene, 2006). Several electrophysiological and imaging experiments have been carried out on humans and
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
non-human primates to identify neural substrates of
capacity limitations in the working memory and executive
control systems (e.g., Buschman, Siegel, Roy, & Miller,
2011; D’Esposito et al., 1995; Luck, 1998; Palva, Monto,
Kulashekhar, & Palva, 2010). Nonetheless, whether
capacity limitation represents a structural bottleneck or a
strategic inadequacy is still vague (Marois & Ivanoff,
2005), and “the precise nature of this limited-capacity stage
of processing remains unspecified” (Tombu & Jolicœur,
2005).
Results from previous attempts to identify a bottleneck
in planning eye movements are controversial. Some
studies that combined eye and hand movements toward
the same target did not notice a delay in either movement
(Mather & Fisk, 1985; Mather & Putchat, 1983; Megaw &
Armstrong, 1973), but others found interference between
tasks, one of which involved an eye movement (Malmstrom,
Reed, & Randle, 1983; Pashler, Carrier, & Hoffman, 1993;
Stuyven, Van der Goten, Vandierendonck, Claeys, &
Crevits, 2000; Remington, Wu, & Pashler, 2011).
Although our task does not recruit two independent tasks,
but in the context of parallel processing of sequential
saccades, the observed slowing of the second response
with decreasing stimulus onset asynchrony (SOA), which
is known as the “psychological refractory period” (PRP)
effect (Telford, 1931) in the dual-task paradigm, suggests
that the saccade planning is capacity limited.
Interactive ramp-to-threshold models
of response time
Our study establishes a link between the dual-task
paradigm (Welford, 1952) and the double-step paradigm
(Wheeless, Boynton, & Cohen, 1966) using concepts
derived from the capacity-sharing model and the LATER
model of response time. Previous attempts to relate a
ramp-to-threshold model of reaction time to dual-task
interference suggested that an interaction between neural
populations accumulating information is required to
account for a response time correlation during coordinated
eye–hand movements (Dean, Martı́, Tsui, Rinzel, &
Pesaran, 2011) and a central response selection bottleneck
that enforces postponement of the planning of the second
response until the onset of the first response (Sigman &
Dehaene, 2005; Zylberberg, Slezak, Roelfsema, Dehaene,
& Sigman, 2010). In the domain of non-linear accumulator
models, the concept of an inhibitory interaction between
competing decision signals has been implemented successfully to fit the pattern of the neural activity underlying the
behavior in tasks that mainly probed choice, performance,
or accuracy (e.g., Arai, Keller, & Edelman, 1994; Bogacz
et al., 2007; Boucher, Palmeri, Logan, & Schall, 2007;
Cutsuridis, Smyrnis, Evdokimidis, & Perantonis, 2007;
Gold & Shadlen, 2003; Ratcliff, Hasegawa, Hasegawa,
Smith, & Segraves, 2007; Trappenberg, Dorris, Munoz, &
16
Klein, 2001; Usher & McClelland, 2001; Wong & Wang,
2006). On the other hand, an interaction between a pair of
LATER units has been used to fit the distribution of
response time (Leach & Carpenter, 2001; Story &
Carpenter, 2009) and the duration of fixation between a
pair of evoked or spontaneous saccades (Roos, Calandrini,
& Carpenter, 2008). Our preference for the linear
accumulator(s) was solely intended to reduce the number
of parameters of the model. We do not claim that our
model is the only model that can explain the serial order
of saccades; however, we do believe that the introduction
of the idea of capacity sharing and mutual inhibition
between ramping decision signals is an exciting prospect
and the proposed framework is a plausible one.
Physiological feasibility of the model
A common belief that the retinal information about the
difference between the current position of the gaze and the
location of the target is the only source to generate an eye
movement (Robinson, 1973) fails to explain how a
saccade can be directed to a target that disappears before
the onset of the preceding saccade directed to another
target (Hallett & Lightstone, 1976), suggesting that the
retinal information of the location of the second target
combined with the extraretinal signal of present eye
position is critical for the production of rapid sequential
saccades (Bock, Goltz, Belanger, & Steinbach, 1995;
Collins, 2010; Doré-Mazars, Vergilino-Perez, Collins,
Bohacova, & Beauvillain, 2006; Honda, 1997; Joiner,
FitzGibbon, & Wurtz, 2010; Munuera, Morel, Duhamel,
& Deneve, 2009), which has been confirmed physiologically (Mays & Sparks, 1980). Nevertheless,
research is ongoing to understand how these signals are
combined (Sommer & Wurtz, 2002; Vaziri, Diedrichsen, &
Shadmehr, 2006) or the type of extraretinal signalsVeye
position or eye displacementVare used (Goossens &
Opstal, 1997). Furthermore, the ability to plan consecutive
saccades in parallel requires an obligatory control to
prevent a saccade from being directed to an intermediate
location between respective targets (Chou, Sommer, &
Schiller, 1999; Findlay & Gilchrist, 1997; Ottes, Van
Gisbergen, & Eggermont, 1984) or being curved (McPeek,
2006). More recent works show that the oculomotor system
prepares motor commands by converting goals into corresponding movement vectors (Quaia, Joiner, FitzGibbon,
Optican, & Smith, 2010) to bypass an update in the “goal
space” due to the gaze shift (Duhamel, Colby, & Goldberg,
1992), suggesting that the ramping signals in the accumulators of our model represent motor plans and not the goals
for saccades.
Movement neurons in the frontal eye field (FEF) of the
primate brain seemingly increase firing frequency linearly
before saccade execution at a rate inversely related to the
saccade latency (Hanes & Schall, 1996), but a delayed
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
presaccadic augmentation in the firing frequency of the
FEF visuomovement neurons occurs exclusively when the
saccade is inevitable (Ray, Pouget, & Schall, 2009). The
biphasic pattern of simulated activity in attenuators
(Figure 6b) resembles visuomovement activity in the
FEF that has been modeled in different other contexts by
a lateral inhibition between competing decision signals
rising to a threshold (Hamker, 2005; Stanford, Shankar,
Massoglia, Costello, & Salinas, 2009). Alternatively,
mutual inhibition may also be mediated by the basal
ganglia that disinhibit superior colliculus (SC) through the
direct and indirect pathways (Hikosaka, Takikawa, &
Kawagoe, 2000). Visuomovement neurons in the SC have
been seen to contribute to processing consecutive saccades in parallel (McPeek & Keller, 2002). The presence
of multiple pathways might be a particularly attractive
architecture to implement such a capacity limitation
through inhibitory control.
Appendix A
We assumed that A1 (A2), B1 (B2), and C1 (C2) refer
to the duration of the visual processing stage, planning
stage, and execution stage of a saccade directed to the first
(second) target, respectively, when saccades are generated
in isolation or do not share capacity (Figure 2a). The
duration of the planning stage of a saccade directed to the
first target increases from B1 to B1V, and that of a saccade
directed to the second target increases from B2 to B2V, if
consecutive saccades share capacity (Figure 2b). When
the second target appears after SOA (i.e., SOA1/SOA2 in
Figure 2b, top), planning stages of consecutive saccades
overlap (SOA + A2) after the onset of the first target and
planning of both saccades continue simultaneously until
the onset of a saccade. Suppose that the fraction of
capacity allocated for planning a saccade in the direction
of the first target, denoted by the sharing proportion, is SP
(0 G SP G 1). The remaining capacity (1 j SP) is assumed
to be allocated for planning another saccade in the
direction of the second target. A saccade in the direction
of the first target is planned with full capacity (SP = 1) for
a period of (SOA + A2 j A1) until planning of a saccade
to the second target begins. During this period, the
oculomotor system finishes {(SOA + A2 j A1) SP}
or (SOA + A2 j A1) part of planning of a saccade
directed to the first target. The remaining part of the
planning, which
is {B1 j (SOA
+ A2 j A1)}, takes time
n
o
B1jðSOAþA2jA1Þ
equal to
to finish with a fraction of
SP
capacity (0 G SP G 1). The net amount of n
time these saccades
o
.
share capacity is denoted by D = B1jðSOAþA2jA1Þ
SP
Therefore, the delay (RT1V) between the onset of the first
target and a saccade in that direction is the sum of the
17
duration of the visual processing stage and planning stage
with full and partial capacities as shown in order in the
following equation:
RT10 ¼ ½A1 þ ðSOA þ A2 j A1Þ þ D;
B1 j ðSOA þ A2 j A1Þ
;
RT10 ¼ A1 þ ðSOA þ A2 j A1Þ þ
SP
SP j 1
SP j 1
A1 þ B1
SOA þ
A2 þ
:
RT10 ¼
SP
SP
SP
ðA1Þ
During
capacity sharing,
h
i [D (1 j SP)] or
B1jðSOAþA2jA1Þ
ð1 j SPÞ part of planning of the
SP
saccade in the direction
n of the second target is accomo
ð
1
j
SP
Þ
part
plished, leaving B2 j B1jðSOAþA2jA1Þ
SP
to be planned
n with full capacity (SP = 1), which
o takes time
B1jðSOAþA2jA1Þ
equal to B2j
ð1 j SPÞ . Therefore,
SP
the delay (RT2V) between the onset of the second target and
a saccade in that direction is the sum of the duration of the
visual processing stage and planning stage with partial and
full capacities as shown in order in the following equation.
B1 j ðSOA þ A2 j A1Þ
ð1 j SPÞ ;
RT20 ¼ A2 þ D þ B2 j
SP
B1 j ðSOA þ A2 j A1Þ
RT20 ¼ A2 þ
SP
B1 j ðSOA þ A2 j A1Þ
ð1 j SPÞ :
þ B2 j
SP
ðA2Þ
After simplification, we get
RT20 ¼ A1 þ B1 þ B2 j SOA:
ðA3Þ
Acknowledgments
This work was supported by a grant from the Department of Biotechnology, Government of India. Author S.R.
was supported by a fellowship from the Council of
Scientific and Industrial Research, India.
Commercial relationships: none.
Corresponding author: Supriya Ray.
Email: supriya@ski.org.
Address: The Smith-Kettlewell Eye Research Institute,
2318 Fillmore Street, San Francisco, CA 94115, USA.
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
References
Arai, K., Keller, E. L., & Edelman, J. A. (1994). Twodimensional neural network model of the primate
saccadic system. Neural Networks, 7, 1115–1135.
Averbeck, B. B., Chafee, M. V., Crowe, D. A., &
Georgopoulos, A. P. (2002). Parallel processing of
serial movements in prefrontal cortex. Proceedings of
the National Academy of Sciences of the United
States of America, 99, 13172–13177.
Barone, P., & Joseph, J. P. (1989). Prefrontal cortex and
spatial sequencing in macaque monkey. Experimental
Brain Research, 78, 447–464.
Becker, W., & Jürgens, R. (1979). An analysis of the
saccadic system by means of double step stimuli.
Vision Research, 19, 967–983.
Bock, O., Goltz, H., Belanger, S., & Steinbach, M. (1995).
On the role of extraretinal signals for saccade generation. Experimental Brain Research, 104, 349–350.
Bogacz, R., Usher, M., Zhang, J., & McClelland, J. L.
(2007). Extending a biologically inspired model of
choice: Multi-alternatives, nonlinearity and valuebased multidimensional choice. Philosophical Transactions of the Royal Society of London B: Biological
Sciences, 362, 1655–1670.
Bompas, A., & Sumner, P. (2009). Temporal dynamics of
saccadic distraction. Journal of Vision, 9(9):17, 1–14,
http://www.journalofvision.org/content/9/9/17,
doi:10.1167/9.9.17. [PubMed] [Article]
Botvinick, M., & Plaut, D. C. (2004). Doing without
schema hierarchies: A recurrent connectionist
approach to normal and impaired routine sequential
action. Psychological Review, 111, 395–428.
Boucher, L., Palmeri, T. J., Logan, G. D., & Schall, J. D.
(2007). Inhibitory control in mind and brain: An
interactive race model of countermanding saccades.
Psychological Review, 114, 376–397.
Brown, S. D., & Heathcote, A. (2008). The simplest
complete model of choice response time: Linear
ballistic accumulation. Cognitive Psychology, 57,
153–178.
Buschman, T. J., & Miller, E. K. (2007). Top-down versus
bottom-up control of attention in the prefrontal and
posterior parietal cortices. Science, 315, 1860–1862.
Buschman, T. J., Siegel, M., Roy, J. E., & Miller, E. K.
(2011). Neural substrates of cognitive capacity limitations. Proceedings of the National Academy of
Sciences, 108, 11252–11255.
Camalier, C. R., Gotler, A., Murthy, A., Thompson, K. G.,
Logan, G. D., Palmeri, T. J., & Schall, J. D. (2007).
Dynamics of saccade target selection: Race model
analysis of double step and search step saccade
18
production in human and macaque. Vision Research,
47, 2187–2211.
Carpenter, R. H. S. (1981). Oculomotor procrastination. In
D. F. Fischer & R. A. Monty (Eds.), Eye movements:
Cognition and Visual Perception (pp. 237–246).
Hillsdale, NJ: Erlbaum.
Carpenter, R. H. S., & Williams, M. L. L. (1995). Neural
computation of log likelihood in control of saccadic
eye movements. Nature, 377, 59–62.
Chou, I., Sommer, M. A., & Schiller, P. H. (1999).
Express averaging saccades in monkeys. Vision
Research, 39, 4200–4216.
Collins, T. (2010). Extraretinal signal metrics in multiplesaccade sequences. Journal of Vision, 10(14):7, 1–14,
http://www.journalofvision.org/content/10/14/7,
doi:10.1167/10.14.7. [PubMed] [Article]
Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas
and goals in the control of sequential behavior.
Psychological Review, 113, 887–916.
Cutsuridis, V., Smyrnis, N., Evdokimidis, I., & Perantonis, S.
(2007). A neural model of decision-making by the
superior colliculus in an antisaccade task. Neural
Networks, 20, 690–704.
Dean, H. L., Martı́, D., Tsui, E., Rinzel, J., & Pesaran, B.
(2011). Reaction time correlations during eye–hand
coordination: Behavior and modeling. Journal of
Neuroscience, 31, 2399–2412.
D’Esposito, M., Detre, J. A., Alsop, D. C., Shin, R. K.,
Atlas, S., & Grossman, M. (1995). The neural basis of
the central executive system of working memory.
Nature, 378, 279–281.
Ditterich, J., Eggert, T., & Straube, A. (1998).
Fixation errors and timing in sequences of memoryguided saccades. Behavioral and Brain Research, 95,
205–217.
Dominey, P. F., & Arbib, M. A. (1992). A corticosubcortical model for generation of spatially accurate
sequential saccades. Cerebral Cortex, 2, 153–175.
Donders, F. C. (1868/1969). On the speed of mental
processes. In W. G. Koster (Ed. & Trans.), Attention
and performance II (pp. 412–431). Amsterdam, The
Netherlands: North-Holland Publishing. (Original
work: Over de snelheid van psychische processen.
Onderzoekingen gedaan in het Physiologisch Laboratorium der Utrechtsche Hoogeschool, Tweede
reeks, 1868–1869, II, 92–120)
Doré-Mazars, K., Vergilino-Perez, D., Collins, T.,
Bohacova, K., & Beauvillain, C. (2006). The use
of recurrent signals about adaptation for subsequent
saccade programming depends on object structure.
Brain Research, 1113, 153–162.
Dorris, M. C., Paré, M., & Munoz, D. P. (1997). Neuronal
activity in monkey superior colliculus related to the
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
initiation of saccadic eye movements. Journal of
Neuroscience, 17, 8566–8579.
Duhamel, Colby, C. L., & Goldberg, M. E. (1992). The
updating of the representation of visual space in parietal
cortex by intended eye movements. Science, 255, 90–92.
Eriksen, C. W., & Schultz, D. W. (1979). Information
processing in visual search: A continuous flow
conception and experimental results. Attention,
Perception, & Psychophysics, 25, 249–263.
Findlay, J. M., & Gilchrist, I. D. (1997). Spatial scale and
saccade programming. Perception, 26, 1159–1167.
Findlay, J. M., & Walker, R. (1999). A model of saccade
generation based on parallel processing and competitive inhibition. Behavioral and Brain Sciences, 22,
661–674.
Fujii, N., & Graybiel, A. M. (2003). Representation of
action sequence boundaries by macaque prefrontal
cortical neurons. Science, 301, 1246.
Fukuda, K., Awh, E., & Vogel, E. K. (2010). Discrete
capacity limits in visual working memory. Current
Opinion in Neurobiology, 20, 177–182.
Funahashi, S., Inoue, M., & Kubota, K. (1997). Delayperiod activity in the primate prefrontal cortex
encoding multiple spatial positions and their order
of presentation. Behavioral and Brain Research, 84,
203–223.
Glasspool, D. W., & Houghton, G. (2005). Serial order
and consonant–vowel structure in a graphemic output
buffer model. Brain and Language, 94, 304–330.
Gold, J. I., & Shadlen, M. N. (2003). The influence of
behavioral context on the representation of a perceptual decision in developing oculomotor commands.
Journal of Neuroscience, 23, 632–651.
Gold, J. I., & Shadlen, M. N. (2007). The neural basis of
decision making. Annual Review of Neuroscience, 30,
535–574.
Goldberg, M. E., & Bruce, C. J. (1990). Primate
frontal eye fields: III. Maintenance of a spatially
accurate saccade signal. Journal of Neurophysiology,
64, 489–508.
Goossens, H., & Opstal, A. J. (1997). Local feedback
signals are not distorted by prior eye movements:
Evidence from visually evoked double saccades.
Journal of Neurophysiology, 78, 533–538.
Grossberg, S. (1978). A theory of human memory: Selforganization and performance of sensory-motor codes,
maps, and plans. In R. Rosen & F. Snell (Eds.),
Progress in theoretical biology (vol. 5, pp. 233–374).
New York: Academic Press.
Hallett, P. E., & Lightstone, A. D. (1976). Saccadic eye
movements to flashed targets. Vision Research, 16,
107–114.
19
Hamker, F. H. (2005). The reentry hypothesis: The
putative interaction of the frontal eye field, ventrolateral prefrontal cortex, and areas V4, IT for
attention and eye movement. Cerebral Cortex, 15,
431–447.
Hanes, D. P., & Schall, J. D. (1996). Neural control of
voluntary movement initiation. Science, 274, 427–430.
Hikosaka, O., Takikawa, Y., & Kawagoe, R. (2000). Role
of the basal ganglia in the control of purposive
saccadic eye movements. Physiological Reviews, 80,
953–978.
Honda, H. (1997). Interaction of extraretinal eye position
signals in a double-step saccade task: Psychophysical estimation. Experimental Brain Research, 113,
327–336.
Ipata, A. E., Gee, A. L., Goldberg, M. E., & Bisley, J. W.
(2006). Activity in the lateral intraparietal area
predicts the goal and latency of saccades in a freeviewing visual search task. Journal of Neuroscience,
26, 3656–3661.
Joiner, W. M., FitzGibbon, E. J., & Wurtz, R. H. (2010).
Amplitudes and directions of individual saccades can
be adjusted by corollary discharge. Journal of Vision,
10(2):22, 1–12, http://www.journalofvision.org/content/
10/2/22, doi:10.1167/10.2.22. [PubMed] [Article]
Jordan, M. I. (1997). Serial order: A parallel distributed processing approach. In J. W. Donahoe &
V. P. Dorsel (Eds.), Advances in psychology (vol. 121,
pp. 471–495). Amsterdam, The Netherlands: NorthHolland.
Kahneman, D. (1973). Attention and effort. Englewood
Cliffs, NJ: Prentice Hall.
Laming, D. R. J. (1968). Information theory of choicereaction times. London: Academic Press.
Land, M. F. (2009). Vision, eye movements, and natural
behavior. Visual Neuroscience, 26, 51–62.
Lashley, K. S. (1951). The problem of serial order in
behavior. In L. A. Jeffress (Ed.), Cerebral mechanisms in behavior (pp. 112–136). New York: Wiley.
Leach, J. C. D., & Carpenter, R. H. S. (2001). Saccadic
choice with asynchronous targets: Evidence for
independent randomisation. Vision Research, 41,
3437–3445.
Leonhard, T., Fernández, S. R., Ulrich, R., & Miller, J.
(2011). Dual-task processing when task 1 is hard and
task 2 is easy: Reversed central processing order?
Journal of Experimental Psychology: Human Perception and Performance, 37, 115–136.
Lévy-Schoen, A., & Blanc-Garin, J. (1974). On oculomotor programming and perception. Brain Research, 71,
443–450.
Lo, C. C., & Wang, X. J. (2006). Cortico-basal ganglia
circuit mechanism for a decision threshold in reaction
time tasks. Nature Neuroscience, 9, 956–963.
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
Luce, R. D. (1986). Response times: Their role in
inferring elementary mental operations. New York:
Oxford University Press.
Luck, S. J. (1998). Sources of dual-task interference:
Evidence from human electrophysiology. Psychological Science, 9, 223–227.
Ludwig, C. J. H., Farrell, S., Ellis, L. A., & Gilchrist, I. D.
(2009). The mechanism underlying inhibition of
saccadic return. Cognitive Psychology, 59, 180–202.
Ludwig, C. J. H., Gilchrist, I. D., & McSorley, E. (2005).
The remote distractor effect in saccade programming:
Channel interactions and lateral inhibition. Vision
Research, 45, 1177–1190.
Ludwig, C. J. H., Mildinhall, J. W., & Gilchrist, I. D.
(2007). A population coding account for systematic
variation in saccadic dead time. Journal of Neurophysiology, 97, 795–805.
Lünenburger, L., Lindner, W., & Hoffmann, K. P. (2003).
Neural activity in the primate superior colliculus and
saccadic reaction times in double-step experiments.
Progress in Brain Research, 142, 91–107.
Malmstrom, F., Reed, L., & Randle, R. (1983). Restriction
of pursuit eye movement range during a concurrent
auditory task. Journal of Applied Psychology, 68,
565–571.
Marois, R., Chun, M. M., & Gore, J. C. (2000).
Neural correlates of the attentional blink. Neuron,
28, 299–308.
Marois, R., & Ivanoff, J. (2005). Capacity limits of
information processing in the brain. Trends in
Cognitive Sciences, 9, 296–305.
Mather, J., & Fisk, J. (1985). Orienting to targets by
looking and pointing: Parallels and interactions in
ocular and manual performance. Quarterly Journal of
Experimental Psychology A: Human Experimental
Psychology, 37, 315–338.
Mather, J., & Putchat, C. (1983). Parallel ocular and
manual tracking responses to a continuously moving
visual target. Journal of Motivation Behavior, 15,
29–38.
Mays, L. E., & Sparks, D. L. (1980). Dissociation of
visual and saccade-related responses in superior
colliculus neurons. Journal of Neurophysiology, 43,
207–232.
Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen,
M. N. (2003). A role for neural integrators in
perceptual decision making. Cerebral Cortex, 13,
1257–1269.
McLeod, P. (1977). Parallel processing and the psychological refractory period. Acta Psychologica, 41, 381–396.
McPeek, R. M. (2006). Incomplete suppression of distractor-related activity in the frontal eye field results
20
in curved saccades. Journal of Neurophysiology, 96,
2699–2711.
McPeek, R. M., & Keller, E. L. (2002). Superior
colliculus activity related to concurrent processing
of saccade goals in a visual search task. Journal of
Neurophysiology, 87, 1805–1815.
Megaw, E., & Armstrong, W. (1973). Individual and
simultaneous tracking of a step input by the horizontal saccadic eye movement and manual control
systems. Journal of Experimental Psychology, 100,
18–28.
Meyer, D. E., Osman, A. M., Irwin, D. E., & Yantis, S.
(1988). Modern mental chronometry. Biological
Psychology, 26, 3–67.
Miller, J., Ulrich, R., & Rolke, B. (2009). On the
optimality of serial and parallel processing in the
psychological refractory period paradigm: Effects of
the distribution of stimulus onset asynchronies.
Cognitive Psychology, 58, 273–310.
Morrison, R. E. (1984). Manipulation of stimulus onset
delay in reading: Evidence for parallel programming
of saccades. Journal of Experimental Psychology:
Human Perception and Performance, 10, 667–682.
Munuera, J., Morel, P., Duhamel, J. R., & Deneve, S.
(2009). Optimal sensorimotor control in eye
movement sequences. Journal of Neuroscience, 29,
3026–3035.
Murthy, A., Ray, S., Shorter, S. M., Priddy, E. G., Schall,
J. D., & Thompson, K. G. (2007). Frontal eye field
contributions to rapid corrective saccades. Journal of
Neurophysiology, 97, 1457–1469.
Murthy, A., Ray, S., Shorter, S. M., Schall, J. D., &
Thompson, K. G. (2009). Neural control of visual
search by frontal eye field: Effects of unexpected target
displacement on visual selection and saccade preparation. Journal of Neurophysiology, 101, 2485–2506.
Mushiake, H., Saito, N., Sakamoto, K., Itoyama, Y., &
Tanji, J. (2006). Activity in the lateral prefrontal
cortex reflects multiple steps of future events in action
plans. Neuron, 50, 631–641.
Nakahara, H., Nakamura, K., & Hikosaka, O. (2006).
Extended LATER model can account for trial-by-trial
variability of both pre- and post-processes. Neural
Networks, 19, 1027–1046.
Navon, D., & Gopher, D. (1979). On the economy of the
human-processing system. Psychological Review, 86,
214–255.
Navon, D., & Miller, J. (2002). Queuing or sharing? A
critical evaluation of the single-bottleneck notion.
Cognitive Psychology, 44, 193–251.
Ottes, F. P., Van Gisbergen, J. A. M., & Eggermont, J. J.
(1984). Metrics of saccade responses to visual double
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
21
stimuli: Two different modes. Vision Research, 24,
1169–1179.
Palva, J. M., Monto, S., Kulashekhar, S., & Palva, S.
(2010). Neuronal synchrony reveals working memory
networks and predicts individual memory capacity.
Proceedings of the National Academy of Sciences,
107, 7580–7585.
Reddi, B. A. J., & Carpenter, R. H. S. (2000). The
influence of urgency on decision time. Nature Neuroscience, 3, 827–830.
Remington, R. W., Wu, S.-C., & Pashler, H. (2011). What
determines saccade timing in sequences of coordinated eye and hand movements? Psychonomic Bulletin & Review, 18, 538–543.
Pashler, H. (1994). Graded capacity-sharing in dual-task
interference? Journal of Experimental Psychology,
20, 330–342.
Rhodes, B. J., Bullock, D., Verwey, W. B., Averbeck,
B. B., & Page, M. (2004). Learning and production of
movement sequences: Behavioral, neurophysiological, and modeling perspectives. Human Movement
Science, 23, 699–746.
Pashler, H., Carrier, M., & Hoffman, J. (1993). Saccadic
eye movements and dual-task interference. Quarterly
Journal of Experimental Psychology A: Human
Experimental Psychology, 46, 51–82.
Pashler, H. E. (1999). The psychology of attention.
Cambridge, MA: The MIT Press.
Phillips, A. N., & Segraves, M. A. (2010). Predictive
activity in Macaque frontal eye field neurons during
natural scene searching. Journal of Neurophysiology,
103, 1238–1252.
Pouget, P., Emeric, E. E., Stuphorn, V., Reis, K., &
Schall, J. D. (2005). Chronometry of visual responses
in frontal eye field, supplementary eye field, and
anterior cingulate cortex. Journal of Neurophysiology, 94, 2086–2096.
Purcell, B. A., Heitz, R. P., Cohen, J. Y., Schall, J. D.,
Logan, G. D., & Palmeri, T. J. (2010). Neurally
constrained modeling of perceptual decision making.
Psychological Review, 117, 1113–1143.
Quaia, C., Joiner, W. M., FitzGibbon, E. J., Optican, L. M.,
& Smith, M. A. (2010). Eye movement sequence
generation in humans: Motor or goal updating?
Journal of Vision, 10(14):28, 1–31, http://www.
journalofvision.org/content/10/14/28, doi:10.1167/
10.14.28. [PubMed] [Article]
Ratcliff, R., Hasegawa, Y. T., Hasegawa, R. P., Smith,
P. L., & Segraves, M. A. (2007). Dual diffusion model
for single-cell recording data from the superior
colliculus in a brightness-discrimination task. Journal
of Neurophysiology, 97, 1756–1774.
Ray, S., Pouget, P., & Schall, J. D. (2009). Functional
distinction between visuomovement and movement
neurons in macaque frontal eye field during saccade
countermanding. Journal of Neurophysiology, 102,
3091–3100.
Ray, S., Schall, J. D., & Murthy, A. (2004). Programming
of double-step saccade sequences: Modulation by
cognitive control. Vision Research, 44, 2707–2718.
Rayner, K. (1998). Eye movements in reading and
information processing: 20 years of research. Psychological Bulletin, 124, 372–422.
Robinson, D. A. (1973). Models of the saccadic eye
movement control system. Biological Cybernetics,
14, 71–83.
Roos, J. C. P., Calandrini, D. M., & Carpenter, R. H. S.
(2008). A single mechanism for the timing of
spontaneous and evoked saccades. Experimental
Brain Research, 187, 283–293.
Sandamirskaya, Y., & Schöner, G. (2010). An embodied
account of serial order: How instabilities drive
sequence generation. Neural Networks, 23, 1164–1179.
Schmolesky, M. T., Wang, Y., Hanes, D. P., Thompson,
K. G., Leutgeb, S., Schall, J. D., et al. (1998). Signal
timing across the Macaque visual system. Journal of
Neurophysiology, 79, 3272–3278.
Sigman, M., & Dehaene, S. (2005). Parsing a cognitive
task: A characterization of the mind’s bottleneck.
PLoS Biology, 3, e37.
Sigman, M., & Dehaene, S. (2006). Dynamics of the
central bottleneck: Dual-task and task uncertainty.
PLoS Biology, 4, e220.
Smith, P. L. (1995). Psychophysically principled models
of visual simple reaction time. Psychological Review,
102, 567–593.
Sommer, M. A., & Wurtz, R. H. (2002). A pathway in
primate brain for internal monitoring of movements.
Science, 296, 1480–1482.
Stanford, T. R., Shankar, S., Massoglia, D. P., Costello,
M. G., & Salinas, E. (2009). Perceptual decision
making in less than 30 milliseconds. Nature Neuroscience, 13, 379–385.
Sternberg, S. (1969). The discovery of processing stages:
Extensions of Donders’ method. In W. G. Koster (Ed.),
Attention and performance II (pp. 276–315). Amsterdam,
The Netherlands: North-Holland Publishing.
Story, G. W., & Carpenter, R. H. S. (2009). Dual LATERunit model predicts saccadic reaction time distributions in gap, step and appearance tasks. Experimental
Brain Research, 193, 287–296.
Journal of Vision (2012) 12(3):17, 1–22
Ray, Bhutani, & Murthy
Stuyven, E., Van der Goten, K., Vandierendonck, A.,
Claeys, K., & Crevits, L. (2000). The effect of
cognitive load on saccadic eye movements. Acta
Psychologica, 104, 69–85.
Tanaka, M. (2007). Cognitive signals in the primate motor
thalamus predict saccade timing. Journal of Neuroscience, 27, 12109–12118.
Telford, C. W. (1931). The refractory phase of voluntary
and associative responses. Journal of Experimental
Psychology, 14, 1–36.
Thompson, K. G., Hanes, D. P., Bichot, N. P., & Schall,
J. D. (1996). Perceptual and motor processing stages
identified in the activity of macaque frontal eye field
neurons during visual search. Journal of Neurophysiology, 76, 4040–4055.
Tian, J., Schlag, J., & Schlag-Rey, M. (2000). Testing
quasi-visual neurons in the monkey’s frontal eye field
with the triple-step paradigm. Experimental Brain
Research, 130, 433–440.
Tombu, M., & Jolicœur, P. (2002). All-or-none bottleneck
versus capacity sharing accounts of the psychological
refractory period phenomenon. Psychological
Research, 66, 274–286.
Tombu, M., & Jolicœur, P. (2003). A central capacity
sharing model of dual-task performance. Journal of
Experimental Psychology: Human Perception and
Performance, 29, 3–18.
Tombu, M., & Jolicœur, P. (2005). Testing the predictions
of the central capacity sharing model. Journal of
Experimental Psychology: Human Perception and
Performance, 31, 790–802.
Trappenberg, T. P., Dorris, M. C., Munoz, D. P., & Klein,
R. M. (2001). A model of saccade initiation based on
the competitive integration of exogenous and endogenous signals in the superior colliculus. Journal of
Cognitive Neuroscience, 13, 256–271.
Usher, M., & McClelland, J. L. (2001). The time course of
perceptual choice: The leaky, competing accumulator
model. Psychological Review, 108, 550–592.
22
Van Loon, E. M., Hooge, I. T. C., & Van den Berg, A. V.
(2002). The timing of sequences of saccades in visual
search. Proceedings of the Royal Society of London
B: Biological Sciences, 269, 1571–1579.
Vaziri, S., Diedrichsen, J., & Shadmehr, R. (2006). Why
does the brain predict sensory consequences of
oculomotor commands? Optimal integration of the
predicted and the actual sensory feedback. Journal of
Neuroscience, 26, 4188–4197.
Wald, A., & Wolfowitz, J. (1948). Optimum character of
the sequential probability ratio test. The Annals of
Mathematical Statistics, 19, 326–339.
Walker, R., Deubel, H., Schneider, W. X., & Findlay, J. M.
(1997). Effect of remote distractors on saccade
programming: Evidence for an extended fixation
zone. Journal of Neurophysiology, 78, 1108–1119.
Welford, A. T. (1952). The ‘psychological refractory
period’ and the timing of high-speed performanceV
A review and a theory. British Journal of Psychology,
43, 2–19.
Wheeless, L., Boynton, R., & Cohen, G. (1966). Eye
movement responses to step and pulse-step stimuli.
Journal of the Optical Society of America, 56,
956–960.
Wong, K. F., & Wang, X. J. (2006). A recurrent network
mechanism of time integration in perceptual decisions. Journal of Neuroscience, 26, 1314–1328.
Woodman, G. F., Kang, M. S., Thompson, K., & Schall,
J. D. (2008). The effect of visual search efficiency
on response preparation. Psychological Science, 19,
128–136.
Zingale, C. M., & Kowler, E. (1987). Planning sequences
of saccades. Vision Research, 27, 1327–1341.
Zylberberg, A., Slezak, D. F., Roelfsema, P. R., Dehaene,
S., & Sigman, M. (2010). The brain’s router: A cortical
network model of serial processing in the primate
brain. PLoS Computational Biology, 6, 1385–1411.