LETTER
Communicated by Liam Paninski
Variable Timescales of Repeated Spike Patterns
in Synfire Chain with Mexican-Hat Connectivity
Kosuke Hamaguchi
hammer@brain.riken.jp
RIKEN, Brain Science Institute, Wako-shi, Saitama, 351-0198, Japan
Masato Okada
okada@k.u-tokyo.ac.jp
RIKEN, Brain Science Institute, Wako-shi, Saitama, 351-0198, Japan; Department of
Complexity Science and Engineering, University of Tokyo, Kashiwa, Chiba, 277-8561,
Japan; and Intelligent Cooperation and Control, PRESTO, JST, Saitama, 351-0198,
Japan
Kazuyuki Aihara
aihara@sat.t.u-tokyo.ac.jp
Institute of Industrial Science, University of Tokyo, Meguro, Tokyo 153-8505, Japan,
and ERATO Aihara Complexity Modeling Project, JST, Shibuya-ku, Tokyo 151-0065,
Japan
Repetitions of precise spike patterns observed both in vivo and in vitro
have been reported for more than a decade. Studies on the spike volley
(a pulse packet) propagating through a homogeneous feedforward network have demonstrated its capability of generating spike patterns with
millisecond fidelity. This model is called the synfire chain and suggests a
possible mechanism for generating repeated spike patterns (RSPs). The
propagation speed of the pulse packet determines the temporal property
of RSPs. However, the relationship between propagation speed and network structure is not well understood. We studied a feedforward network
with Mexican-hat connectivity by using the leaky integrate-and-fire neuron model and analyzed the network dynamics with the Fokker-Planck
equation. We examined the effect of the spatial pattern of pulse packets on RSPs in the network with multistability. Pulse packets can take
spatially uniform or localized shapes in a multistable regime, and they
propagate with different speeds. These distinct pulse packets generate
RSPs with different timescales, but the order of spikes and the ratios
between interspike intervals are preserved. This result indicates that the
RSPs can be transformed into the same template pattern through the
expanding or contracting operation of the timescale.
Neural Computation 19, 2468–2491 (2007)
C 2007 Massachusetts Institute of Technology
Variable Timescales of Repeated Spike Patterns
2469
1 Introduction
Repeated spike patterns (RSPs) are, literally, patterns of spikes that repeatedly appear amid the apparent random activities of a neuron population.
They have been observed both in vivo and in vitro through several measurement methods, such as multielectrode array recordings and calcium
imaging (Abeles, Bergman, Margalit, & Vaadia, 1993; Prut et al., 1998; Mao,
Hamzei-Sichani, Aronov, Froemke, & Yuste, 2001; Ikegaya et al., 2004). The
generating mechanism of RSPs is, however, not clear yet. One possible
mechanism is the propagation of spike synchrony through a feedforward
network, which is often called the synfire chain. Abeles (1991) defined the
term synfire chain as a network that can support the synchronous spike
propagation mode under a certain condition. A homogeneous feedforward
network of spiking neurons is the simplest neural entity of the synfire chain.
Its capability of transmitting a synchronized spike volley has been extensively studied in several spiking neuron models (Diesmann, Gewaltig, &
Aertsen, 1999; Câteau & Fukai, 2001; Gewaltig, Diesmann, & Aertsen, 2001;
Kistler & Gerstner, 2002). A feedforward network model with synaptic delay
does not show the explicit synchrony of spikes, but the essential mechanism
of activity propagation—the synchrony of arriving spikes—is still required
(Izhikevich, 2006). The feasibility of such synchronized activity propagation
has been experimentally confirmed in an iteratively constructed biological
network in vitro (Reyes, 2003). In this letter, we refer to the synchronized
spike discharge in one layer as a spike volley and refer to its propagation
through a network as a pulse packet.
The pulse packet propagation can explain the RSP phenomenon as follows. Assume that a multielectrode array was injected into the feedforward
neural network, and several events of pulse packet propagation occurred.
When one pulse packet propagates, electrodes can detect the spike event
correlated to the activity propagation. The development of the propagating
speed has been shown to be quite stable (Gewaltig et al., 2001). Therefore,
those spikes detected over several electrodes have a specific interspike interval (ISI). Since each synchronous spike volley propagates through the
network with approximately the same speed in all trials, the same ISIs
repeatedly appear among uncorrelated spikes from spontaneous firings.
Those statistically significant ISIs are defined as RSPs. Therefore, the speed
of a pulse packet is an important property of RSPs.
In this letter, our interest resides in the relationship between the propagation speed of a pulse packet and the resultant RSPs in a multistable
network. If a network had multistability, the pulse packets in different stable fixed points would have different propagation speeds depending on
the configuration of the network structure. Given that, the temporal order
of an evoked precise spike sequence would not change, but the intervals
of spikes within the RSPs would change. Many of the studies on synfire
chains, however, have treated a homogeneous network structure, and they
2470
K. Hamaguchi, M. Okada, and K. Aihara
had only one stable pulse packet shape: a spatially uniform synchronized
activity. In this case, there is no chance to observe different RSPs. To understand the mechanism of RSPs, we will consider a biologically plausible
network structure and its resultant activities.
Electrophysiological and anatomical data indicate that the cerebral cortex is spatially organized (Mountcastle, 1997). The columnar activities are
widely observed in several cortical regions, including the primary visual
cortex (Hubel & Wiesel, 1977) and prefrontal cortex (Goldman & Nauta,
1977). Both the orientation and ocular dominance columns have been identified through an intensive study with the optical imaging method (Blasdel,
1992). The recent development of the two-photon calcium imaging technique allows us to study the population of neuron activity with cellular-level
resolution. The spontaneous activity of slices from the visual cortex shows
that the repetition of activity pattern is also spatially organized (Cossart,
Aronov, & Yuste, 2003). To understand the properties of repeated activity patterns in the spatially organized network, we chose the Mexican-hat
connectivity, which is widely accepted as one of the biologically realistic
connectivities in the cortex.
The studies on spatially localized activity date back to the 1970s. Intensive modeling studies have shown that spatially localized activity is stable
in recurrent neural circuits with nearby excitation and global inhibition
(Wilson & Cowan, 1972; Amari, 1977; Ben-Yishai, Bar-Or, & Sompolinsky,
1995; Compte, Brunel, Goldman-Rakic, & Wang, 2000), which is also called
the Mexican-hat type interaction. This interaction is widely used as a neural
substrate for general columnar activities (Wang, 2001). A numerical study
of a network with feedforward Mexican-hat connectivity has shown that
without the reverberation loop, the localized propagating activity is stable (van Rossum, Turrigiano, & Nelson, 2002). The coexistence of uniform
and localized activity propagation has been shown in analysis with binary
neurons (Hamaguchi, Okada, Yamana, & Aihara, 2005), but because of the
discrete time dynamics of a binary neuron model, the speed of the pulse
packet was not studied there.
In this letter, we report our study on the dynamics of feedforward networks with Mexican-hat connectivity composed of leaky integrate-and-fire
(LIF) neurons without refractoriness. Combined with the simulations with
the LIF neurons, we used the equivalent Fokker-Planck equation (FPE) for
the analysis. The FPE generates continuous firing-rate dynamics, which is
useful for the analysis of pulse packet propagation, especially when the
firing rate is very low. Our strategy is to embed multistability into the network, observe what type of RSPs appear, and seek the common feature
among them. We describe the activity of a network with a small number of
macroscopic variables indicating the population firing rate, localization parameter, and position of the activity. It allows us to reduce the complexity of
the spatially organized network dynamics into a low-dimensional parameter space. Using this approach, we studied the stability of spike packets
Variable Timescales of Repeated Spike Patterns
2471
within a certain parameter region and found the multistable regime. To
understand the nature of RSPs generated in multistable networks, we simulated several trials of multielectrode array recording in the multistable
synfire chain. We show that RSPs generated from different pulse packets
are similar but have different time constants because of the difference in
propagation speed of pulse packets.
2 Network Structure and Neuron Model
In this section, we first define the dynamics of the neuron model that we
use in this letter. We then connect neurons by giving the network structure,
or synaptic efficacy between neurons.
The dynamics of the membrane potential vθl of a neuron indexed with θ
in layer l is described by a differential equation,
C
vl
dvθl
= − θ + Iθl (t) + µ + Dξθ (t),
dt
R
∞
W(θ − θ )
dτ
,
Iθl (t) =
α(τ )δ t − tθl−1
,k − τ
θ
0
(2.1)
(2.2)
k
where C is the membrane capacitance and R is the membrane resistance. The
index θ = {−π, −π + 2π
, . . . , π − 2π
} is the functional distance between
N
N
two neurons. We assume that the input to the soma of neuron θ consists of
Iθ (t), a weighted sum of outputs from presynaptic neurons, and white gaussian noise with mean µ and standard deviation D. The white gaussian noise
is drawn from the independently identically distributed gaussian distribution ξθ (t), which satisfies ξθ (t) = 0 and ξθ (t) · ξθ (t ) = δθ θ δ(t − t ). The
first summation over θ in equation 2.2 is a sum of different synaptic currents from neuron θ with dimensionless weight W(θ − θ ), and the second
summation over k is a sum of different spikes arriving at time t = tθl−1
,k ,
where tθl−1
,k is the kth spike timing of neuron θ in layer l − 1. Excitatory
postsynaptic current (EPSC) or inhibitory postsynaptic current (IPSC) time
courses are described with α(t), where α(t) = βα 2 t exp(−αt)H(t) and β is
chosen such that a single excitatory postsynaptic potential (EPSP) generates
14 mV depolarization from the resting potential. Here, H(t) is the Heaviside
step function. Note that α(t) will be normalized by system size N later in
equation 2.3. The convolution of the function α(t) and spikes gives the current from presynaptic neurons. The membrane potential dynamics follows
the spike-and-reset rule: when vθl reaches the threshold Vth , a spike is fired,
and vθl is reset to the resting potential Vrest .
2472
K. Hamaguchi, M. Okada, and K. Aihara
Figure 1: Network architecture. Each layer consists of N units of leaky integrateand-fire (LIF) neurons, which are arranged in a circle. Each neuron projects
its axons to the postsynaptic layer with Mexican-hat connectivity. The actual
dynamics of one layer, which corresponds to the collective dynamics of LIF
neurons, is calculated by using the FPE.
Mexican-hat type synaptic efficacy is described with a uniform term W0
and a spatial modulation term W1 cos(θ ) (Ben-Yishai et al., 1995),
W(θ − θ ) =
W0
W1
+
cos(θ − θ ).
N
N
(2.3)
Note that this connection is only feedforward, and there is no recurrent
interaction within one layer. The whole network is a structured feedforward
network composed of identical LIF neurons, aligned in one-dimensional
ring layers (see Figure 1). Throughout this letter, the parameter values
are fixed as follows: C = 100 pF, R = 100 M, Vth = 15 mV, Vrest = 0 mV,
D = 100, µ = 0.075 pA, α = 2 ms−1 , and β = 1.7 × 10−4 . For the LIF neuron
simulations, N = 104 .
3 Fokker-Planck Equation and Macroscopic Variables
The probability distribution of the membrane potential evolving in time
according to the dynamics of equation 2.1 can be described by the following
Fokker-Planck equation (FPE) (Risken, 1996),
∂t Pθl (v, t)
= ∂v
2 v − R Iθl (t) + µ
1 D
+
∂v Pθl (v, t),
τ
2 C
(3.1)
Variable Timescales of Repeated Spike Patterns
2473
∂
where τ = RC, ∂t = ∂t∂ and ∂v = ∂v
. The probability distribution Pθl (v, t)
indicates the probability that the membrane potential of neuron θ in layer l is
vθl = v at time t. Since we regard Pθl (v, t) as the probability, the normalization
condition is
Vth
dv Pθl (v, t) = 1.
(3.2)
−∞
The resetting mechanism of the LIF neuron requires absorbing boundary
conditions at threshold potential Vth and the current source at resting potential Vrest . The boundary conditions of the partial differential equation are
Pθl (Vth , t) = 0,
2
1 D +
−
l
− ∂v Pθl (Vrest
r (θ, t) ≡
, t) + ∂v Pθl (Vrest
, t) ,
2 C
2
1 D
∂v Pθl (Vth , t).
=−
2 C
(3.3)
(3.4)
(3.5)
Here, r l (θ, t) means the instantaneous firing rate of neurons θ in layer l at
time t. The initial condition for the membrane potential distribution is the
stationary distribution Pst (v) for no external input, Iθ (t) = 0. Details of the
derivation of Pst (v) are given in appendix A.
Given initial condition Pθ (v, 0) = Pst (v) and the boundary conditions in
equations 3.3 to 3.5, the dynamics of the membrane potential distribution
is calculated by numerical integration of equation 3.1, and the firing rate is
obtained through the boundary condition (see equation 3.5).
The firing-rate dynamics of one layer depends on the firing rate of the
presynaptic layer. Therefore, we start our calculation from the l = 1 to Lth
layer in a sequential manner. The input current Iθl (t) is now a function of
the firing rates of neurons in the (l − 1)th layer:
π
∞
dθ W(θ − θ )
dτ α(τ )r l−1 (θ , t − τ ).
(3.6)
Iθl (t) =
2π
−π
0
Note that equation 3.6 for FPE is the counterpart of equation 2.2 for the
LIF model. Let us further transform equation 3.6 to introduce some useful
macroscopic variables of the network. By introducing equation 2.3 into
equation 3.6 and exchanging two integrals, we get
∞
π
dθ l l
Iθ (t) =
dτ α(τ ) W0
r (θ , t − τ )
0
−π 2π
π
dθ l r (θ , t − τ ) cos(θ ) cos(θ ) + sin(θ ) sin(θ ) , (3.7)
+W1
−π 2π
2474
K. Hamaguchi, M. Okada, and K. Aihara
∞
=
0
dτ α(τ ) W0 r0l−1 (t − τ ) + W1 rcl−1 (t − τ ) cos(θ )
+rsl−1 (t − τ ) sin(θ ) ,
(3.8)
where
r0l (t) =
π
−π
rcl (t) =
π
−π
rsl (t) =
π
−π
dθ l
r (θ, t),
2π
(3.9)
dθ l
r (θ, t) cos(θ ),
2π
(3.10)
dθ l
r (θ, t) sin(θ ).
2π
(3.11)
Here, r0l (t) is the population firing rate, and both rcl (t) and rsl (t) are the coefficients of the Fourier transformation of the spatial firing pattern, which
represent the degree of localization around θ = 0 or π/2 at time t, respectively. These macroscopic parameters have a dimension of firing rate. Since
the network has translational invariance, we can transform (rcl (t), rsl (t)) into
the following variables to separate the dependence of the position of the
bump:
2 2
rcl + rsl ,
φ l (t) = arctan rsl (t)/rcl (t) .
r1l (t) =
(3.12)
(3.13)
From the above transformations, we obtain the bump-position-invariant
index r1l (t), which indicates the degree of localization at time t, and φ l (t)
indicating the position of the bump in terms of angle. In this way, equation
3.8 is expressed as
Iθl (t) =
0
∞
dτ α(τ ) W0 r0l − 1 (t − τ ) + W1 r1l−1 (t − τ ) cos(θ − φ l−1 (t − τ )) .
(3.14)
We have simplified the activity of the network into a small number of
variables without any approximations. Pulse packet propagation, or any
other type of activity propagation, in this feedforward network can be
understood as the transformation of these macroscopic variables. The actual
procedure of our calculation is as follows. Given the time courses of the l − 1
layer macroscopic variables (r0l−1 (t ), r1l−1 (t ), φ l−1 (t )) in the range of t < t,
the firing rate of each neuron at time t is calculated from equation 3.5. By
summing up the firing rates of postsynaptic neurons using the definitions
in equations 3.9 to 3.13, we can obtain the next macroscopic variables on
Variable Timescales of Repeated Spike Patterns
2475
the postsynaptic neural layer (r0l (t), r1l (t), φ l (t)) at time t. These steps are
performed recursively until the last neural layer is reached.
We note that the FPE in equation 3.1 is essentially equivalent to the
stochastic ordinary equation in equation 2.1 in the limit of large neuron
number N. When the network consists of neurons with inhomogeneous
properties such as different mean input, one FPE is not enough to capture
the inhomogeneity. However, by using more than one FPE to interpolate
the difference, we can use the Fokker-Planck analysis, provided that the
number of FPEs is large enough to cover the spatial frequency of inputs.
Since the input takes the form of, at most, a unimodal shape, we found
that a few Fokker-Planck equations are enough to give qualitatively similar
results to the LIF simulations. Here, we divide θ space into 100 regions to
guarantee quantitatively good agreement with the LIF simulation.
For the actual numerical calculation of the FPE, we used a modified
Chang-Cooper algorithm (Chang & Cooper, 1970), which is a fully implicit
method for solving the advective-diffusion equation. To support the FokkerPlanck analysis, we performed LIF neuron network simulations using the
stochastic second-order Runge-Kutta algorithm (Honeycutt, 1992). Their
numerical complexity is compared in appendix C. All code was simulated
in Matlab.
4 Results
4.1 Membrane Potential Distribution Dynamics. In this section, we
show the dynamics of the membrane potential distribution of one neural
layer. First, we define input currents to neurons in the first layer. They are
formulated in terms of the firing rate of the presynaptic virtual layer (l = 0)
activity as follows:
r 0 (θ, t) =
r00 + r10 cos(θ )
(t − t̄ 0 )2
exp −
.
√
2(σ 0 )2
2πσ 0
(4.1)
Throughout this letter, we use r00 = 500, r10 = 350 for the localized stimulation (see Figure 2a) and r00 = 900, r10 = 0 for the uniform stimulation case.
Here, the temporal dispersion of input spikes σ 0 and timing of the stimulation t̄ 0 are set to σ 0 = 1 and t̄ 0 = 5. Note that this localized stimulation has
φ 0 (t) = 0 for all t.
When a neural layer is activated by localized stimuli with φ 0 (t) = 0,
membrane potentials around the origin are activated and reach the thresholds (see Figure 2b). Higher-probability regions of Pθ (v, t) are shown in
black. Probability densities of membrane potential distributions averaged
over θ at several timings are indicated by shaded regions in Figure 2c. Figure 2d shows the dynamics of the membrane potential distribution of FPEs
at position θ = {0, π/2, 3π/4}. The membrane potential distributions were
driven to the threshold Vth at approximately t = 5 ms and reappeared from
2476
K. Hamaguchi, M. Okada, and K. Aihara
Figure 2: Overview of the dynamics of the membrane potential distributions
of one layer in response to a localized stimulus. (a) Input rate in terms of r0 (t)
and r1 (t) generated from equation 4.1 with parameters r0 = 500 and r1 = 350. (b)
Snapshots of membrane potential distributions. The horizontal axis is space θ,
and the vertical axis is the membrane potential. The darker region has a higher
density of membrane potential Pθ (v, t). (c) Probability density of membrane
potential distribution averaged over θ . Numerical calculations of the FPEs are
shown in the shaded region, and simulations from 104 LIF neurons are shown by
solid lines to support the Fokker-Planck calculation. They are almost identical.
(d) Time courses of membrane potential of neurons at different positions θ =
0, π/2, 3π/4.
Variable Timescales of Repeated Spike Patterns
2477
the resting potential Vrest . After the stimulation, the membrane potential
distribution gradually relaxed toward its stationary distribution.
Numerical simulations of many identical LIF neurons are used to support the analysis by the FPE (Brunel & Hakim, 1999; Omurtag, Knight, &
Sirovich, 2000; Brunel, 2000). To support our analysis, we simulated 104 LIF
neurons distributed over a ring layer. Their membrane potential distributions are superposed in Figure 2c. It shows good agreements between the
FPEs (shaded regions) and the LIF simulation (solid lines) results.
4.2 Dynamics of Macroscopic Variables. In section 3, we saw that some
averaged quantities can help us to reduce the dimension of the output.
Although the macroscopic variables defined in equations 3.9 to 3.13 can
fully explain the output of one layer, they are still functions of time t.
Therefore, let us further consider a method of reducing the complexity in
the temporal direction in order to illustrate network activity.
We first show typical time courses of r01 (t) and r11 (t) in response to the
local stimulation in Figure 3a for both the numerical simulation of LIF
neurons and the FPE. Here, the firing rate of LIF neurons at each timing
was calculated from the number of spikes within 0.5 ms sliding windows.
For a quantitative evaluation of the evolutions of the spike packet
shape, the gaussian-approximation method provides a powerful method
for capturing the temporal profile of a spike volley (Diesmann et al., 1999).
Diesmann et al. characterized a spatially uniform spike volley by two parameters of the gaussian function: its area and the variance of the gaussian function. These correspond to the number of spikes and the temporal
dispersion of the spikes, respectively. We follow this approach to characterize the activity of a localized spike volley. The effect of a spike volley
on the next layer can be fully described by the time courses of population firing rate r0 (t), localization parameter r1 (t), and position φ(t). Without
loss of generality, we can neglect the position parameter φ l (t) as long as
φ l (t) is constant because of the translational invariance of the network. We
therefore characterize a spike volley by approximating r0 (t) and r1 (t) with
two gaussian functions (see Figure 3b), respectively. We first approximate
r0 (t) − ν0 with the gaussian function and derive three parameters—r0 , σ ,
and t̄—which correspond to the area, standard deviation, and peak time of
the gaussian. Here, ν0 is the spontaneous firing rate (see equation A.2). Since
the Mexican-hat type interaction adds new dimension r1 (t), we then derive
another parameter r1 from the area of the r1 (t)—approximating gaussian
function.
The practical advantage of using the FPE is that the firing rate at any
moment can be obtained as a smooth and continuous time series. This
is useful for approximating a spike volley with the gaussian, especially
when the firing rate is very low. In contrast to FPE, a simulation with LIF
neurons gives the firing rate through statistical sampling of their spiking
events. In the low-firing-rate regime, statistical sampling of spikes gives
2478
K. Hamaguchi, M. Okada, and K. Aihara
(t) [spk/s]
(a)
r0(t)
400
200
0
FP
LIF
r1(t)
0
5
10
15
t [ms]
(b)
r0(t)
r1(t)
Gaussian approximation
r1
r0
tFigure 3: (a) Response of a neural layer driven by a localized stimulation described by macroscopic variables r0 (t) and r1 (t). The solid line, r0 (t), and the
dashed line, r1 (t), were calculated from equation 3.1. LIF neuron simulations
are shown again to confirm our analysis. Population firing rate r0 (t) and degree
of localization r1 (t) are plotted as squares and triangles, respectively. (b) Four
parameters that characterize the temporal profiles of a spike volley: r0 , r1 , σ,
and t̄. They were derived from the gaussian approximation of the temporal profiles of r0 (t) and r1 (t). Three of them, r0 , σ , and t̄, are derived from the gaussian
approximating r0 (t), as the area, standard deviation, and mean time. Here, r1
was obtained from the area of the gaussian, which approximates the temporal profile of r1 (t). The parameters of the gaussian functions are obtained by
minimizing the mean squared error with r0 (t) or r1 (t) and the gaussian function.
relatively big fluctuations, which is not suitable for estimating gaussian
parameters.
4.3 Propagation of Pulse Packets. So far, we have considered the activities of one layer to prepare for the analysis in multilayer cases. To understand the stability of the pulse packet propagation, we numerically calculated the FPE for 20 layers and checked the convergence of parameters r0 , r1 ,
and σ . In the spontaneous firing state, firing rates satisfies r (θ, t) = ν0 . It also
indicates r0 (t) − ν0 = 0; therefore, r0 = 0. Since the firing profile is spatially
uniform, the spontaneous firing state is described as r0 = r1 = 0 in terms of
the estimated gaussian parameters. On the other hand, positive r0 values
after convergence indicate that a pulse packet is stable. When the activity
Variable Timescales of Repeated Spike Patterns
Uniform stim.
Local stim.
Layer Uniform stim.
Uniform phase
(W0,W1) = (1, 0.6)
1000
1
2
3
4
5
6
7
8
800
600
400
ing rate
Local stim.
Non ing phase
(W0,W1) = (0.7, 1)
2479
200
0
1
2
3
4
5
6
7
8
0
5
10
Localized phase
(W0,W1) = (0.7, 2.5)
15
0
5
10
15
Multi-stable phase
(W0,W1) = (1, 1.4)
1
2
3
4
5
6
7
8
1
2
3
4
5
6
7
8
0
5
10
15 0
5
10
15
t [ms]
Figure 4: Typical activity profiles of the feedforward networks in terms of
firing rate in the four phases: N, U, L, and M. The evolutions of firing rates
are illustrated with colors (see the right color bar). Equation 4.1 was used to
describe the input current to the first layer with parameters r00 = 500, r10 = 350
for the localized stimulus cases and r00 = 900, r10 = 0 for the uniform stimulus
cases. The other parameters are σ 0 = 1 and t̄ 0 = 2.
profile is spatially uniform, r1 (t) vanishes. Therefore, a spatially uniform
spike volley is defined as r0 > 0 and r1 = 0. If the localized pulse packet is
stable, r0 > 0 and r1 > 0. To summarize, we classified the network state into
three states: spontaneous firing (r0 = r1 = 0), uniform pulse packet mode
(r0 > 0, r1 = 0), and localized pulse packet mode (r0 > 0, r1 > 0). This classification is consistent with the analysis of the network with feedforward
Mexican-hat type connections composed of binary neurons (Hamaguchi,
Okada, Yamana et al., 2005).
In Figure 4, typical examples of network dynamics in response to the local or uniform stimulus to the first layer are exhibited in terms of firing rate.
The network has four phases depending on parameters (W0 , W1 ). When the
values of both W0 and W1 are small, no pulse packet can propagate, and the
2480
K. Hamaguchi, M. Okada, and K. Aihara
spontaneous firing state is the only stable state: nonfiring phase (N). When
the uniform excitation term W0 is sufficiently strong, a uniform pulse packet
is stable in addition to the spontaneous firing: uniform phase (U). When
the Mexican-hat amplitude W1 is strong enough, a localized pulse packet
is stable in addition to the spontaneous firing: localized phase (L). When
W0 and W1 are balanced, there exist multistable states where both the uniform and localized pulse packets are stable depending on the initial input:
multistable states (M). Note that spontaneous firing is stable in any phase.
We note that localized and uniform pulse packets are the only possible
states among the possible propagating pulse packets. This is because the
inputs are described with spatially constant term and cos(θ ) function as
described in equation 3.14, so the possible inputs to a network are spatially
unimodal or flat no matter how spatially complex the form of previous
layer activity is. Therefore, the uniform and localized bump state is the
only possible state. If we used higher-frequency cosine functions in the
weight function W(θ ), it would be possible to make more than one localized
bump propagation mode. If we went beyond the pulse packet mode by
increasing the connection strength even more, there would be other states
such as bursting-instability mode where the firing rate always increases
as the activity propagates through the layers. In this letter, we avoid such
unrealistic cases.
The evolution of a pulse packet and its convergence to an attractor can
be illustrated in flow diagrams (Diesmann et al., 1999; Câteau & Fukai,
2001). Figures 5a to 5c show the evolutions of pulse packets in the (r0 , σ, r1 )
space. One line segment with a small arrowhead indicates the evolution of
Figure 5: Flow diagrams for U, L, and M phases. (a–c) These panels depict
the flow diagram in three-dimensional space (r0 , σ, r1 ). Each line with a small
arrowhead indicates the evolution of a spike volley per layer, and one sequence
of a line represents the evolution of a pulse packet. The colors of the arrowheads
and lines represent the final stable states of the pulse packets: spontaneous firing
state (blue), uniform pulse packet mode (green), or localized pulse packet mode
(red). Insets indicate the connection profiles W(θ ), and the weight highlighted
with color is the one used in the flow diagram. The blue rectangle frame in a
to c correspond to d to f . The position of the frame was chosen to include at
least one attractor of the pulse packet mode. Big arrowheads in d to f show the
direction of the evolution of the spike packet evolution. The choice of colors is
the same as in a to c. (a) Flow diagram in uniform phase (U) with parameters
(W0 , W1 ) = (1, 0.6). The bottom panel, d, is the r1 = 0 plane. (b) Flow diagram in
localized phase (L) with parameters (W0 , W1 ) = (0.7, 2.5). The bottom panel, e, is
the σ = 0.37 plane. (c) Flow diagram in multistable phase (M) with parameters
(W0 , W1 ) = (1, 1.5). The bottom panel, f , includes two attractors: uniform and
localized pulse packet modes. The broken lines indicate the schematic boundary
of the basin of the attraction.
0
500
1000
(d)
0
500
r0
1000
0
200
400
0
W( )
0
0
2
4
0.2
0.2
0 2
0.4
0.4
0
200
400
(e)
0
r0
0.4
200
1000
500
0
(f )
400
500
r0
1000
500
r0
0.2
0 2
400
(c)
1000
0
W( )
0
0
2
4
r1
0
200
400
(b)
r1
r1
r1
r0
0
200
r1
(a)
1000
0
0
0
500
W( )
r
0
2
4
0
0.2
0 2
0
0.4
Variable Timescales of Repeated Spike Patterns
2481
2482
K. Hamaguchi, M. Okada, and K. Aihara
a spike volley from one layer to the next, and the whole sequence of the
line represents the evolution of a pulse packet. The colors of the arrows
represent the stable states of the pulse packets after convergence: spontaneous firing state (blue), uniform pulse packet mode (green), or localized
pulse packet mode (red). Insets indicate the connection profiles W(θ ), and
the weight highlighted with color is the one used in the flow diagram. The
blue rectangular frame in Figures 5a to 5c corresponds to Figures 5d to 5f
to show the details of the flow. The position of the frame was chosen to
include at least one attractor of the pulse packet mode.
Depending on the connection profile, the attracting point where several
lines converges changes. The U phase with parameters (W0 , W1 ) = (1, 0.6)
described in Figure 5a shows that the pulse packets converge on the r1 = 0
plane. It indicates that the pulse packet becomes spatially uniform even
though the initial state is localized. The green lines converge to high firing
rate r0 with low spike timing dispersion σ point (uniform pulse packet
mode). The blue arrows converge to low-firing-rate r0 with high spike timing dispersion σ (failure of pulse propagation). Figures 5d to 5f show the
flow diagram on the r1 = 0 plane. Large arrowheads show the direction of
the spike packet evolution. This is qualitatively equivalent to that of the
conventional synfire chain model (Diesmann et al., 1999).
When W1 was increased, the L phase appeared. The flow diagram of
the L phase with parameters (W0 , W1 ) = (0.7, 2.5) illustrates the existence
of the attracting point in the nonzero r1 region (see Figure 5b). Several red
lines converge to the high firing-rate r0 , high localized parameter r1 , and
with low temporal dispersion σ (localized pulse packet mode), indicating
the stable propagation of localized pulse packets. Figure 5e shows a flow
diagram on the plane σ = 0.37. This plane contains the attracting point of
the localized pulse packet.
In the M phase, depending on the initial stimulus, pulse packets converge
to either uniform or localized pulse packet mode. In Figure 5c, green and red
lines converge to uniform and localized pulse packet attractor, respectively.
Figure 5f includes both of these attractors. The broken lines in Figures 5d to
5f indicate the schematic boundary of the basin of the attraction.
The converging dynamics toward the attractor of a synchronized spike
volley are similar in the U, L, and M phases. If a spike volley is within the
basin of the attractor of the pulse packet mode, spike volleys are rapidly
shaped into a stable spike packet of high firing rate with submillisecond
dispersion. Otherwise, spike packets gradually die out (r0 → 0 and large
σ ). Near the boundary, the nonmonotonic evolution of spike packets is
commonly observed; weakly activated spike volleys, which have a submillisecond dispersion of spikes with a relatively small number of spikes,
evolve with increasing spike jitter (increasing σ ) in initial stages. If the initial number of spikes has exceeded a certain threshold, the network evolves
to a pulse packet mode, and the spike jitter is reduced again (σ decreases).
Otherwise, the pulse packet dies out, and σ continues to increase. This
Variable Timescales of Repeated Spike Patterns
2483
Figure 6: Phase diagram of the system in (W0 , W1 ) space. The parameter set
is as in Figure 4. In the localized phase (L) and uniform phase (U), low-firing
spontaneous firing state is stable, as well as in the nonfiring phase (N). When
the values of W0 or W1 are too high, they lead to the bursting phase (B).
nonmonotonic phenomenon is illustrated as curved red or green lines starting near the boundary of the spontaneous state. They are observed in all
the development processes of spike volleys in the U, L, and M phases.
Phase diagrams in Figure 6 show the stability of U phase and L phase
in (W0 , W1 ) space with the same parameter set. As we have seen, larger W0
leads to U phase, and larger W1 leads to L phase. In between them, there is a
multistable region M. Note that the nonfiring state (N) is stable within these
regions. If we increase connection strength more than a certain threshold,
a system becomes unstable, and the firing rate of the pulse packet goes to
infinity as it propagates to the deeper layers. We refer to this state as the
bursting state (B).
4.4 Propagation Speed of Pulse Packets. The timescale of an RSP is
strongly related to the propagation speed of a pulse packet. In this section, our interest resides in the propagation speed under multistability of
pulse packets. The time required for a pulse to propagate from the (l − 1)th
layer to the lth layer is defined as t̄l = t̄l − t̄l−1 . The mean arrival timing
t̄l is obtained from the r0l (t)-estimating gaussian function as illustrated in
Figure 3b. We let t̄ denote the characteristic propagation time of a stable
pulse packet, which is obtained after pulse packets converge to their stable
states. The characteristic propagation time of stable pulse packets is shown
in Figure 7 for various values of parameter W1 .
The characteristic propagation time t̄ depends on both (W0 , W1 ) and
(r0 (t), r1 (t)), but in M phase, the effect of activity pattern (r0 (t), r1 (t)) on the
speed can be directly studied by comparing each t̄. In Figure 7, circles
represent t̄ for a uniform pulse packet case, and triangles represent those
of localized pulse packets. Increasing W1 reduces the propagation time t̄
2484
K. Hamaguchi, M. Okada, and K. Aihara
w0 = 1
-
∆ t [ms]
1.2
local
uniform
1
0.8
1.4
1.6
1.8
w1
2
Figure 7: Plot of characteristic propagation time t̄ in M phase, where t̄ =
t̄l − t̄l−1 after convergence. Localized pulse packets propagate more slowly than
the uniform ones. Since the W0 parameter was fixed here, t of uniform pulse
packets was constant.
of localized pulse packets. In contrast, the uniform pulse packet does not
depend on W1 because uniform activity has vanishing r1 (t) values; thus the
W1 term can be neglected in equation 3.12.
In this letter, we used the LIF neuron models without refractoriness for
simplicity, and the pulse packet propagation phenomena are studied with
the parameter region without rate instability, where the firing rate of a pulse
packet does not grow to infinity as it propagates through the network. Under this condition, the propagation speeds of local pulse packets are slower
than those of uniform ones. We note that if we introduce long refractoriness
after firing and set the value of parameter W1 much larger than that of W0 ,
the speeds of local pulse packets can be faster than those of uniform ones.
4.5 Repeated Spike Patterns Generated by Multistable Synfire Chain.
In the previous section, we showed that a pulse packet has its own characteristic propagation speed. Here we consider RSPs generated from different
pulse packets. We simulated multielectrode recordings from randomly chosen neurons in the 20 layers of a multistable feedforward network as shown
in Figure 8a. Parameters were set to (W0 , W1 ) = (1, 1.5).
The simulated multielectrode recordings were performed as follows.
First, 10 recording sites {θ, l} were randomly determined. Then we performed five trials of the LIF neuron simulations for each of the uniform and
the localized pulse packet case. In Figure 8b, the spike events are plotted
with squares for the uniform pulse packet cases and triangles for the localized pulse packet cases. The vertical position of the squares and triangles
corresponds to its trial number. To show the overall shape of the probability
of observing a spike, one trial of the FPE calculation is performed for each
pulse packet shape. The probability of detecting a spike at each recording
Variable Timescales of Repeated Spike Patterns
2485
(a)
8
1
0
5
2
3
1
6
9
10
7
4
5
10
Layer
(b)
15
20
1
2000
Uniform Local
2
rate
spike
3
0
2000
0
2000
0
2000
0
2000
7
0
2000
8
uniform trial 1-5{
local
trial 1-5{
0
2000
9
# of Electrodes
6
5
4
0
2000
0
2000
r( ,t) [Hz]
position
π
10
0
2000
0
5
10
15
20
time [ms]
25
0
30
Figure 8: In silico experiments of multielectrode array recordings in a multistable synfire chain. (a) Placement of electrodes in the multistable feedforward
network with the same parameters as in Figure 5a. (b) Spike raster plot from 10
electrodes. The responses of the network were measured with five trials from
the experiments of uniform stimulation to the first layer and five trials from
localized ones. The spikes in uniform pulse packet mode (squares) are plotted
in the upper halves of raster plots in each electrode, and those of localized pulse
packet mode (triangles) are plotted in the lower halves. The firing rate r (θ, t) is
also shown to support the LIF simulations, whose firing rate is shown by the
right y-axes. Parameters were set to (W0 , W1 ) = (1, 1.5).
2486
K. Hamaguchi, M. Okada, and K. Aihara
site, r (θ, t)t, is obtained from the firing rate of the FPE at the electrode position θ . The solid and dashed lines represent the probability of observing
spikes for the uniform pulse packet propagating cases and localized pulse
packet propagating case, respectively.
RSP is defined as a specific combination of interspike intervals (ISIs)
ISI
{τiISI
j , τ jk }, which indicates that a spike from electrode unit i is followed by
a spike from unit j after exactly τiISI
j ms, followed by another spike of unit
ISI
ISI
ISI
k after exactly τ jk s. The {τi j , τ jk } ± τc allows events that occurred within
a time constant ±τc ms time window around the exact ISI. We can define
ISI
a longer combination of ISIs as {τiISI
j , τ jk . . .} ± τc . We apply a suffix to each
RSP to indicate which pulse packet an RSP is generated from. Hereafter,
ISI(U)
ISI(U)
we refer to {τi j
, τ jk
. . .} as the RSP generated from the uniform pulse
ISI(L)
ISI(L)
packets and {τi j , τ jk . . .} as generated from the localized pulse packets.
ISI
Note that {τiISI
j , τ jk } itself can be used to describe any ISIs. An RSP is
a special combination of ISIs that occurs more frequently than a certain
threshold determined from a reasonable assumption, such as the stationary
Poisson firing. Therefore, an ISI combination with high statistical significance is called a repeated spike pattern. RSPs are often searched for by
ISI
counting the number of events {τiISI
j , τ jk . . .} ± τc throughout the recording
data. However, since our simulations had a low spontaneous firing rate,
the RSPs were easily recognizable in spike rasters, as shown in Figure 8b.
Details of our definition of RSPs are given in appendix B.
In Figure 8b, the RSPs generated by a uniform pulse packet (squares)
were detected in all the randomly inserted electrodes, but the RSPs generated by a localized pulse packet (triangles) did not appear in all the
electrodes. The initial stimulation position was set to φ 0 = 0, so only the
electrodes around θ ∼ 0 could detect the RSP.
The spike patterns aligned to a specific spike timing of an electrode are
commonly used to show the RSPs (Abeles et al., 1993; Prut et al., 1998).
Here, we realigned the spike trains according to the first spike of the first
electrode, as plotted in Figure 9a. For comparison, we chose electrodes that
detected RSPs generated from both uniform and localized pulse packets. We
observed different RSPs depending on the stability of the pulse packet. Each
RSP itself retained millisecond-order fidelity, but two RSPs were clearly
separated depending on the propagating patterns.
The final goal of this letter is to find the relationship between those
ISI(U)
ISI(L)
different RSPs generated in the same network. If we plot (τi j
, τi j )
for several {i j} pairs, we can find a specific relationship between the two
RSPs. For fixed i (= 1) and varied j = {1, 2, 5, 6, 7, 9, 10} pairs, we plotted
ISI(U)
ISI(L)
(τ1 j , τ1 j ) in Figure 9b. It is clear that they are aligned on one line with
t̄U
, which satisfies
slope t̄L
ISI(L)
τi j
=
t̄L ISI(U)
τ
.
t̄U i j
(4.2)
Variable Timescales of Repeated Spike Patterns
(b)
20
10
10
ISI(L)
1j
[ms]
# of electrode
9
7
5
1
(a)
2487
0
5
10
ISI
1j
15
[ms]
20
0
0
10
ISI(U)
1j
20
[ms]
Figure 9: (a) Spike raster aligned to the first electrode unit spikes. It correISI(U)
ISI(L)
sponds to τ1ISIj . (b) Plot of (τ1 j , τ1 j ) (cross) representing the ratio of ISIs
lengths. The ISIs were measured between the first electrodes and the others,
j = {1, 2, 5, 6, 7, 9, 10}, as illustrated in Figure 8a. The dotted line represents
was obtained from the result in Figure 7.
equation 4.2, whose slope t̄t̄UL = 1.17
0.81
Parameters were set to (W0 , W1 ) = (1, 1.5).
The slope approximately equals the ratio of the propagation time t̄
between the uniform and localized spike packets. These results indicate
that the ratios of ISIs are constant over the different RSPs, and the ratio is
determined by the ratio of the propagation speed of each pulse packet. Since
the order of spikes does not change, different RSPs can be transformed to
each other by expansion or contraction of the timescale with a certain ratio.
Therefore, in a multistable feedforward network with a stable synfire chain
state, RSPs can have variable timescales, but RSPs are strongly connected
through the timescale expansion or contraction operation.
5 Conclusion
In spite of the importance of the propagation speed of neural activity in
discussions of repeated spike patterns (RSPs), the relationships among the
speed, the network structure, and the spatiotemporal patterns of the pulse
packet had not been well studied. We used the Fokker-Planck equation
to study the dynamics of a feedforward network with Mexican-hat connectivity. The network has a spatial structure in the connections, but by
using macroscopic variables to describe the network activity, we can simplify the output of one neural layer by three time courses of macroscopic
variables: population firing rate, localization parameter, and position of the
activity. The Fokker-Planck analysis allowed us to numerically calculate
the deterministic dynamics of the macroscopic variables and their stability.
We found that there are four phases in the W0 − W1 space: nonfiring (N),
localized phase (L), uniform phase (L), and multistable phase (M). In the M
phase, two types of pulse packets (the uniform spike packet and localized
2488
K. Hamaguchi, M. Okada, and K. Aihara
one) can propagate through the neural layers. These two pulse packets
have their own characteristic propagation speed, which indicates that the
speed of information processing depends on the spiking patterns, or the
representation of the stored information.
When we observe the pulse packets’ propagation through the multielectrode, the activity will be observed as the RSPs. The different pulse packets
generate different RSPs due to the difference of propagation speed, but the
different RSPs can be mapped onto the same template pattern through the
timescale expansion or contraction operation.
Appendix A: Stationary Membrane Potential Distribution and
Spontaneous Firing Rate
The initial condition for the membrane potential distribution is the stationary distribution for no external input, Iθ (t) = 0. The stationary distribution
under the dynamics of equation 3.1 and boundary conditions (threshold
potential Vth and reset potential Vrest ) is obtained as follows:
Pst (v) = e
−U(v)
Vth
du
v
2ν0 C 2
H(u − Vrest )e U(u) ,
D2
(A.1)
2
and ν0 is the spontaneous firing rate for Iθ (t) = 0
where U(v) = C(v−Rµ)
(DR)2
case. From the normalization condition (see equation 3.2), ν0 is obtained as
−1
(ν0 )
√
√ C (Vth −Rµ)
=τ π √
C
Here, erf(y) =
D
D (Vrest −Rµ)
√2
π
y
0
dy exp(y2 )(1 + erf(y)).
(A.2)
dx exp(−x 2 ).
Appendix B: Estimation of an RSP
To estimate τiISI
j , we calculated the probability of observing a spike at position θ from the FPE as shown in Figure 8b. Then we switched to spike
data from LIF simulations and collected spikes that drop in a small time
window τc = 1 around the peak of the probability of observing spikes r (θ, t)
calculated by FPE. This process is used to remove uncorrelated spikes from
the ISI estimation. Finally, we took the mean of the ISIs from these spike
sets as the estimated τiISI
j .
Variable Timescales of Repeated Spike Patterns
2489
Appendix C: Numerical Complexity
Here we compare the numerical complexity of calculating LIF and FPE. In
one layer, the number of LIF neurons and FPEs are N and θ M , respectively.
The membrane potential in a FPE is discretized in M bins.
LIF neurons’ dynamics in equation 2.1 has been calculated using the
second-order stochastic Runge-Kutta algorithm reported in Honeycutt
(1992):
t
D√
tξ
(F1 + F2 ) +
2
C
F1 = f (v(t))
D√
F2 = f v(t) + t F1 +
tξ
C
v
f (v) = − + I (t) + µ
C.
R
v(t + t) = v(t) +
(C.1)
(C.2)
(C.3)
(C.4)
This process requires 13N operations. To calculate I (t), α-function convolved macroscopic variables r0α (t), rcα (t) and rsα (t) are required. Here,
∞
r xα (t) = 0 dτ α(τ )r x (t − τ ). Assuming that these inputs are given, equation
2.2 can be written as
Iθ (t) = W0 r0α (t) + W1 rcα (t) cos(θ ) + rsα (t) sin(θ ) ,
(C.5)
which requires 5N + 1 operations. In total, N units of LIF neuron simulation
per one time step take approximately ≈ 23N operations.
Fokker-Planck calculation requires the inverse of a matrix when we use
the implicit method. Given the probability of observing a membrane potential v at time t as Pt (v) (an M × 1 column vector) and the transition
probability matrix F, we can calculate Pt+t (v) through the following linear
matrix equation,
FPt+t (v) = Pt (v),
(C.6)
where F is a tridiagonal matrix. We can therefore solve this equation through
LU decomposition and gaussian elimination (Strang, 1988), which requires
only 4M − 2 operations. When the input changes, it takes 13M operations
to construct F (for details; see Chang & Cooper, 1970.) In total, therefore, one
update requires 17Mθ M operations. The coefficient may vary depending on
the order of arithmetic operations and the number of terms included in the
equation. Here, we used N = 104 LIF neurons, θ M = 100, and M = 800 for
the best calculation. The time step sizes were the same: 0.01 ms. Therefore,
LIF simulations require 2.3 × 105 operations, and the Fokker-Planck method
2490
K. Hamaguchi, M. Okada, and K. Aihara
requires 1.36 × 106 operations per time step. In this case, the Fokker-Planck
method has a higher computational cost, but it can provide more stable
results for any firing-rate regime. This computational complexity depends
on the choice of spatial discretization M and number of equations θ M compared with the number of neurons N. For example, the FPE simulation for
a simple homogeneous synfire chain with θ M = 1 will be much faster than
LIF simulations.
Acknowledgments
The preliminary result in this letter was reported in Hamaguchi, Okada,
and Aihara (2005). This work is partially supported by the Japanese Society
for the Promotion of Science, Research Fellowships for Young Scientists,
Advanced and Innovational Research Program in Life Sciences, a Grant-inAid for Scientific Research, on Priority Areas No. 17022012, No. 14084212,
No. 18020007, No. 18079003, Scientific Research (C) No. 16500093 from
the Ministry of Education, Culture, Sports, Science, and Technology, the
Japanese Government.
References
Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. Cambridge:
Cambridge University Press.
Abeles, M., Bergman, H., Margalit, E., & Vaadia, E. (1993). Spatiotemporal firing
patterns in the frontal cortex of behaving monkeys. J. Neurophysiol., 70, 1629–
1638.
Amari, S. (1977). Dynamics of pattern formation in lateral-inhibition type neural
fields. Biol. Cybern., 27, 77–87.
Ben-Yishai, R., Bar-Or, R. L., & Sompolinsky, H. (1995). Theory of orientation tuning
in visual cortex. Proc. Natl. Acad. Sci. USA, 92, 3844–3948.
Blasdel, G. (1992). Orientation selectivity, preference, and continuity in monkey
striate cortex. J. Neurosci., 12(8), 3139–3161.
Brunel, N. (2000). Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comp. Neurosci., 8(3), 183–208.
Brunel, N., & Hakim, V. (1999). Fast global oscillations in networks of integrate-andfire neurons with low firing rates. Neural Comp., 11(7), 1621–1671.
Câteau, H., & Fukai, T. (2001). Fokker-Planck approach to the pulse packet propagation in synfire chain. Neural Networks, 14, 675–685.
Chang, J. S., & Cooper, G. (1970). A practical difference scheme for Fokker-Planck
equations. J. Comp. Phys., 6, 1–16.
Compte, A., Brunel, N., Goldman-Rakic, P. S., & Wang, X.-J. (2000). Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical
network model. Cerebral Cortex, 10, 910–923.
Cossart, R., Aronov, D., & Yuste, R. (2003). Attractor dynamics of network up states
in the neocortex. Nature, 423(6937), 283–288.
Diesmann, M., Gewaltig, M.-O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529–533.
Variable Timescales of Repeated Spike Patterns
2491
Gewaltig, M.-O., Diesmann, M., & Aertsen, A. (2001). Propagation of cortical synfire
activity: Survival probability in single trials and stability in the mean. Neural
Networks, 14, 657–673.
Goldman, P. S., & Nauta, W. J. H. (1977). Columnar distribution of cortico-cortical
fibers in the frontal association, limbic, and motor cortex of the developing rhesus
monkey. Brain Research, 122(3), 393–413.
Hamaguchi, K., Okada, M., & Aihara, K. (2005). Theory of localized synfire chain:
Characteristic propagation speed of stable spike patterns. In L. K. Saul, Y. Weiss,
& L. Bottou (Eds.), Advances in neural information processing systems, 17. (pp. 553–
560). Cambridge, MA: MIT Press.
Hamaguchi, K., Okada, M., Yamana, M., & Aihara, K. (2005). Correlated firing in
a feedforward network with Mexican-hat type connectivity. Neural Computation,
17, 1–26.
Honeycutt, R. L. (1992). Stochastic Runge-Kutta algorithms. I. White noise. Phys. Rev.
A, 45, 600–604.
Hubel, D., & Wiesel, T. (1977). Functional architecture of macaque monkey visual
cortex. Proc. R. Soc. Lond. B Biol. Sci., 198, 1–59.
Ikegaya, Y., Aaron, G., Cossart, R., Aronov, D., Lampl, I., Ferster, D., & Yuste, R.
(2004). Synfire chains and cortical songs: Temporal modules of cortical activity.
Science, 304, 559–564.
Izhikevich, E. M. (2006). Polychronization: Computation with spikes. Neural Computation, 18, 245–282.
Kistler, W. M., & Gerstner, W. (2002). Stable propagation of activity pulses in populations of spiking neurons. Neural Comp., 14, 987–997.
Mao, B., Hamzei-Sichani, F., Aronov, D., Froemke, R., & Yuste, R. (2001). Dynamics
of spontaneous activity in neocortical slices. Neuron, 32, 883–898.
Mountcastle, V. (1997). The columnar organization of the neocortex. Brain, 120(4),
701–722.
Omurtag, A., Knight, B., & Sirovich, L. (2000). On the simulation of large populations
of neurons. J. Comp. Neurosci., 8, 51–63.
Prut, Y., Vaadia, E., Bergman, H., Haalman, I., Slovin, H., & Abeles, M. (1998).
Spatiotemporal structure of cortical activity: Properties and behavioral relevance.
J. Neurophysiol, 79(6), 2857–2874.
Reyes, A. (2003). Synchrony-dependent propagation of firing rate in iteratively constructed networks in vitro. Nature Neuroscience, 6, 593–599.
Risken, H. (1996). The Fokker-Planck equation. Berlin: Springer-Verlag.
Strang, G. (1988). Linear algebra and its application. Belmont, CA: Thomson Learning.
van Rossum, M. C. W., Turrigiano, G. G., & Nelson, S. B. (2002). Fast propagation of
firing rates through layered networks of noisy neurons. J. Neurosci., 22, 1956–1966.
Wang, X.-J. (2001). Synaptic reverberation underlying mnemonic persistent activity.
Trends in Neurosciences, 24(8), 455–463.
Wilson, H. R., & Cowan, J. D. (1972). Excitatory and inhibitory interactions in localized population of model neurons. Biophysical Journal, 12, 1–24.
Received March 9, 2006; accepted September 21, 2006.