Biologia, Bratislava, 56/6: 591—604, 2001
Temporal coding and recognition of uncued temporal
patterns in neuronal spike trains:
Biologically plausible network of coincidence detectors
and coordinated time delays
Juraj Pavlásek* & Ján Jenča
Department of Neurophysiology, Institute of Normal and Pathological Physiology, Slovak Academy
of Sciences, Vlárska 5, SK-83334 Bratislava, Slovakia; tel., fax: ++421 2 5477 5428, e-mail:
unpfpavl@savba.sk
Pavlásek, J. & Jenèa, J., Temporal coding and recognition of uncued temporal patterns in neuronal spike trains: biologically plausible network of coincidence detectors and coordinated time delays. Biologia, Bratislava, 56:
591—604, 2001; ISSN 0006-3088 (Biologia). ISSN 1335-6399 (Biologia. Section Cellular and Molecular Biology).
Much of the work on sensory systems assumes that grouping of spikes by time
can carry significant information about a stimulus. The fundamental problem
of general perception is how can a neural network identify a specific temporal
pattern within the stream of pulsatile input activity. A computational model
of a neuronal network is described that recognizes a temporal pattern (a group
of spikes in a narrow time window) in continuous and uncued spike trains.
The devised network performs real-time recognition both in a single neuron
employing a temporal code and within the spiking activity of co-activated
neurons in responding pathways. The temporal resolution of the spike timing
and recognition of a temporal pattern is possible to accuracy within 1 ms limit.
Special attention in simulation experiments has been devoted to the aspects of
real time processing under condition of background spiking noise. Operation
of the network is based upon biologically plausible filtering mechanisms and
population neurodynamics.
Key words: neuronal network, spike train, temporal pattern, pattern recognition, encoding, decoding.
Introduction
The whole sensory experience is derived from processes that encode primary stimulus variables. The
receptor sheet encodes an adequate stimulus into
short-range processes the essence of which are analogue nonlinearities (e.g., receptor potential) and
their stimulus-dependent plastic changes. The re-
ceptor potential generates a long-range process –
spike potential (propagated, uniform, discrete, binary signal). At the next levels the pulse-coded
information is integrated in relay nuclei by complex cooperative and competitive processes. They
result in analogue nonlinearities (e.g., synaptic excitatory and inhibitory potentials) and their eventrelated plastic changes, as well as pulsatile sig-
* Corresponding author
591
naling with different number of spikes and timing
in their occurrence. The resulting information encoded in strings of spikes is transmitted via parallel sensory pathways to the brain. It ought to symbolize each sensory input (its modality, location,
intensity and temporal dimension) with sufficient
resolution and precision to be separable from other
sensory inputs.
The pathways in a sensory channel may mediate a specific (monomodal) and/or “unspecific”
(polymodal) information (Mountcastle, 1967).
The specificity is based on a specialization of peripheral receptors and on certain degree of preserved characteristics of their responses at the next
levels of the sensory channels (Burgess & Perl,
1973; Iggo, 1974; Cervero & Iggo, 1980). The
modality specific lines and their topographic mapping underlie the labeled line code. The convergent
activity in an “unspecific” sensory channel has its
origin in different kinds of receptors.
A temporal dimension in neuronal activation
(spatio-temporal coding) indicate that the characteristics of a sensory input may also be encoded in a fine temporal structure of strings of
spikes, e.g., that grouping of spikes by time can
carry significant information (Engel et al., 1992;
Singer, 1999). The above-mentioned ideas concerning temporal coding imply that the treatment of patterns that extend over time (Port
et al. 1995; Rose 1995) may play a fundamental role in general perception. How can a neural
network identify and utilize a specific temporal
pattern within the continuous stream of pulsatile
input activity? One possibility may be derived
from the fact that repetitive stimulation induces
strengthening or weakening of the synaptic transmission in polysynaptic pathways (Pavlásek &
Petrovický, 1994, Fig. 52) in a manner that depends on the interpulse interval (Guo-Quiang
Bi & Mu-Ming Poo, 1999). Thus, information
coded in the timing of individual spikes can be
converted into and stored as spatially distributed
patterns of persistent synaptic modifications in
a neural network. Another possibility is that a
network produces selective response to particular
pattern of spiking activity entering the network
(Pavlásek, 1999). In this case some ideas originating in both experimental and computational
biology advocate the assumption that possible recognizing mechanism should comprise an “assignment clock” to label with “tags” the start and
the end of a stimulus event (Ghosh & Deuser,
1995; Port et al., 1995) as having occurred in
a particular period of time (process of segmentation).
592
Notes on terminology
The term neural code is applied to rules that
translate attributes of stimulus energy into the
activity of nerve cells: it is the minimum set of
symbols capable of representing all of the biologically significant information. Information processing in the brain is by its nature a population
phenomenon (population coding). The high dimensionality of the response space in the population
coding is based on activity correlation between different neurons (correlation coding, relational coding). Involved activities are scattered in time and
distributed in space (spatio-temporal encoding).
Neuronal spike trains (pulse code) represent neural image of a complex dynamic sensory stimulus
in neurons of the sensory channels. The temporal coordination of spike sequences (involvement
of temporal dimension) in relation to the stimulus
presentation is defined as the temporal coding. The
pulse code offers two encoding schemes: a rate coding (frequency of spike occurrence – a mean rate
code) and a temporal encoding scheme (the neuron encodes information by grouping the spikes
by time – an interval code). Their definition relies on the identification of an encoding time window, defined as the duration of a neuron’s spike
train, which is assumed to correspond to a single symbol in the neural code. In the temporal
encoding scheme, the relevant information is correlated with the timing of the spikes (a temporal
pattern) within the encoding window (Theunissen & Miller, 1995). The pattern is defined as
specific and in some way, statistically correlated
sequence of interspike intervals combined in linear order. An indispensable constituent of temporal encoding and temporal codes is the existence
of decoding scheme, which utilizes them for differentiation, and recognition of signals enabling the
adequate behavioural responses. In this sense the
term decoding does not mean the stimulus reconstruction. Recognition occurs if at least one neuron
produces selective response to specific interspike
interval or to particular sequence of intervals (a
pattern recognizer).
Material and methods
The computer model JASTAP obeys the principles concerning the physiology of a biologically realistic neuron with chemical transmission of information. Details of the model have been reported
elsewhere (Janèo et al., 1994).
The basic element of the network is a model
neuron (neuroid) behaving as an integrate-and-fire
element. It is described by:
1) Instantaneous membrane potential (Mp).
Mp is a dimensionless quantity within the h−1, 1i
range.
2) Membrane potential determined as the
sum of postsynaptic potentials (Psp) limited by
the non-linear function
Mp(t) = (2/π) · arctg
X
Psp(t)
(1)
3) A threshold (Th) from the interval h0, 1i.
4) The frequency of spikes (Sp) is restricted
by the absolute refractory period. This is managed
by setting minimum (Imn) and maximum (Imx)
interspike intervals. The actual interspike interval
(Ia) is determined as
Ia = Imn + (Imx − Imn) · (2/π)·
· arctg((Mp − Th)/(1 − Mp))
(2)
The standard value for Imn was 1 ms, and
Imx ranged from 2 ms to 10 ms. The relative value
of Th was close to 0.5 for all neuroids.
Every neuroid can have 8 synaptic inputs but
a single output. The program treats the synapse
as a part of the neuroid. The output can be connected to one or several synapses in the network
of neuroids. A synapse is characterized by:
a) The input connected to it
b) The shape of a Psp prototype, which is
evoked by Sp arriving at this synapse (particular
waveform is selected from a set of prototype Psp
shapes stored in a buffer of Psp waveforms). The
Psp prototype is described by
Psp(t) = k · (1 − exp(−t/t1))2 · exp(−2 · t/t2) (3)
The waveform simulates whether the in question synapse is located on the soma or on the dendritic tree (the time-course and the attenuation of
its amplitude). In this presentation uniform Psp
time-courses were used. Experimental data (Redman & Walmsley, 1983) were simulated with parameters t1 = 0.3 and t2 = 2.7 ms (The rising
phase of an EPSP lasts 1–2 ms, and the falling
phase lasts 10–15 ms) (Fig. 1B).
c) The latency (time delay) of the synaptic
transmission and/or axonal conduction.
d) The synaptic weight (Sw) has its value
from the interval h−1, 1i. Sw simulates effectiveness of a synaptic input (a synchronously activated
set of axons of the same type or a cluster of the
terminal branches of an axon).
The computer program JASTAP has been
written in C++ language. The program can define
a network by simple command language and simulate its activity in discrete time intervals (0.5 ms
steps). Samples of simulated activity can be presented in the form of intracellular recording with
a microelectrode, or as a raster map of the spike
potentials.
Results
Temporal code generation
Approach to solving the problem
In the spatio-temporal encoding hypothesis receptor cells and neurons display a casual sequence of
spikes in relationship to a stimulus configuration.
Thus time-course of a complex dynamic stimulus evokes specific temporal pattern of the spike
trains generated in receptor cells and conveyed via
a set of primary afferent fibers and consequently
via tract cells of the specific and/or “nonspecific”
sensory pathways. In the presented network the
temporal code emerged in a “relay nucleus” from
local interactions based on biologically plausible
implementations.
Applied morphological and functional principles
The structural constituents of the first-order “relay nucleus” in a “sensory channel” (Fig. 1A) are
represented by four neuroids: three excitatory (0,
2, 3) and one inhibitory (1). Two of the firstorder neuroids (0, 2) are monosynaptically connected with primary afferent fibers (PAF) conveying fictive spike trains generated by imaginary receptor cells (R1, R2); the different timing of the
R1 and R2 activation is simulated (Fig. 1A, R1,
R2, Fig. 1C, traces R1, R2). The neuroids 0 and
2 play at the same time the role of the tract cells
giving rise to axons belonging to the specific sensory pathways (Fig. 1A, SpP). Collaterals of their
axons converge on neuroid 3 which represent a
tract cell in an “unspecific” pathway (supposing
the different specificity of R1 and R2 receptors).
The diffuse projection of an “nonspecific” pathway is mimicked by the divergence of neuroid’s 3
axon into four parallel pathways (Fig. 1A, USpP,
1, 2, 3, 4). The temporal encoding of the activity
arriving via PAFs is performed at the level of firstorder neuroids by two different kinds of filtering:
a) The capability of neuroid 0 to transmit higher
frequency is limited by the inhibitory feedback (recurrent inhibition) from neuroid 1 (Fig. 1A, loop
0-1-0), b) The subthreshold excitatory influence of
PAF on neuroid 2 makes necessary the temporal
summation of postsynaptic potentials to exceed
the threshold. The reactions of neuroid 3 results
from spatio-temporal summation (convergence) of
593
A
1
R1 PAF
SpP
0
USpP
B
SP
TH
0
10
PAF
R2
x
1
d1
2
d2
3
3
d3
t [ms]
4
SpP
2
x x x x
C
PR
PAF
R1
R2
1.1
1.2
2.1
PAF
0
1.3
CL 4
1.4
2.2
2.3
15
2.4
45
60
ms 75
30
45
60
75
30
45
60
75
60
75
60
ms 75
30
1.1
0
4
1.4
-60
-100
0
15
0
15
1
-60
-100
2.2
2
2.4
-60
-100
0
15
1.1
3
30
i1
2.2
45
i2 1.4
i3
2.4
-60
mV
-100
0
15
30
45
Fig. 1. Temporal coding. A. A simplified scheme of the first-order relay nucleus in a sensory channel consisting
of five model neurons (neuroids 0–4) and inputs from imaginary receptor cells (R1 and R2), specific for different
modalities. Connections marked by bars (circles) are excitatory (inhibitory); the crosses indicate subthreshold
excitatory influence. PAF – primary afferent fibers, SpP – specific pathways, USpP – “unspecific” pathways
(1–4). d1-d3 delays in pathways which mediate activity to a pattern recognizing neuroid (PR 4). CL 4 is a
coded line originating in PR 4. B. Time course of excitatory postsynaptic potential (EPSP) used in simulation
experiments with model network. Dash-dot-dot line – membrane potential, dashed line – threshold (TH) for
generation of the propagated spike potential (SP). An arrow indicate time at which EPSP was evoked. C.
Spatio-temporal encoding. Two upper traces simulate fictive SPs generated in R1 (1.1–1.4; 1.1. indicates the
first spike), R2 (2.1.–2.4.) and conveyed via PAFs to neuroids 0 and 2. In the lower part there is the simulation
of intracellularly recorded postsynaptic potentials (PSPs) evoked by arriving spikes in three excitatory (0, 2,
3) and one inhibitory (1) neuroid; upward (downward) deflections simulate excitatory (inhibitory) PSPs. The
numbering of the spikes is the same as in PAFs. Abscissa – simulation time in ms, ordinate – simulation of the
transmembrane potential in mV providing an approximate range of PSP amplitudes in a biologically realistic
neuron.
594
the suprathreshold excitatory inputs from neuroids 0 and 2.
Decoding of temporally encoded information
Approach to solving the problem
We have considered the decoding of a stimulus
from the response it evokes. The decoding scheme
is based upon transition of a temporal pattern to
the activity of a neural cell (a place-cell code) or
a neuronal ensemble of spatially distributed units
(spatialization of a temporal sequence). The onset and offset of a pattern – the duration of the
encoding time window – is determined by coordinated delays in an array of parallel pathways. The
decoding scheme can be based on decoding a temporal pattern in a single channel (a neuron in a
synaptic pathway) by a within “reading” or in a
co-activated set of parallel pathways – population
decoding by an across “reading”. Recognition of
each temporal pattern is causally related to a specific pattern recognizer.
Applied morphological and functional principles
The structural constituents of the devised pattern
recognizer are simple: one neuroid (Fig. 1A, PR 4,
Fig. 2A, PR 1) supplied with a synaptic input
from collateral pathways (convergence) branched
off from afferent sensory pathway(s) (Figs 1A, 2A,
3A, 4A, 5A). All collateral pathways excite PR
neuroid (spatio-temporal summation) with subthreshold intensity and with different (organized)
delays (d1-d3 in Figs 1A, 2A, 3A, 5A and d1d4 in Fig. 4A); therefore the propagated spike
in the axon of the PR neuroid(s) (Fig. 1A, CL
4, Fig. 2A, Fig. 3A, CL 1, Fig. 4A, CL1 and 2,
Fig. 5 CL1) is set up only when a temporal coincidence of spikes arriving to PR(s) occurs and
maximal spatio-temporal summation of the excitatory influences is induced (coincidence detection).
Thus the organized delays, temporal coincidence
and the mechanism of coincidence detection determine the recognized temporal pattern. The PR
neuroid plays a role of a coded line (carried information is just the presence or absence of the
signal): it means that a single spike in the PR neuroid axon might bring information about the occurrence of the whole temporal pattern (contraction of the temporal domain, shrinking the code
length).
Simulation experiments
Network parameters
Figure 1B shows the time-course of the simulated
suprathreshold (1.2 times threshold value) excitatory postsynaptic potential (EPSP) evoked in neu-
roids 0 and 3 by each spike arriving in them via
excitatory ending(s). The synaptic weights in remaining neuroids had different values: the excitatory influence of the neuroid 0 on neuroid 1 was set
to 1.6 times the threshold value (TH), the coefficient of the synaptic strength of the PAF synaptic
terminals on neuroid 2 was set to 0.9 TH, and the
synaptic weight for each of four collaterals derived
from neuroid 3 pathways (USpP, 1, 2, 3, 4) establishing synapses with PR 4 were set 0.3 times TH
(Fig. 1A). The simulated inhibitory postsynaptic
potential (IPSP) evoked from neuroid 1 in neuroid
0, had the same time-course as EPSP but reversed
polarity and lower amplitude.
Spatio-temporal encoding
The temporal structure of a spike train evoked by
a stimulus is determined by both nature and dynamics of the stimulus and characteristics of the
neural encoding process. The train of four spikes
with constant 7 ms interspike interval (143 Hz
frequency) arriving from R1 to the PAF synaptic terminal in the tenth ms of the simulation
time (Fig. 1C, R1, 1.1–1.4) caused in neuroid
0 a sequence of excitatory and inhibitory shifts
in the membrane potential lasting for about 40
ms (Fig. 1C, 0). Changes in the membrane potential resulted in generation of two propagated
spikes (1.1, 1.4). Four spikes generated in R2 were
grouped into two pairs of spikes with 5 ms intespike interval; the first pair arrived at the PAF
synaptic terminal eighteen ms and the second one
forty eight ms after the start of the simulation time
(Fig. 1C, R2, 2.1–2.4). The subthreshold intensity
of the excitatory influence of PAF on neuroid 2
caused that only the second spike in each twin of
spikes evoked EPSP which exceeded the threshold
and consequently generated the propagated spike
in neuroid 2 (Fig. 1C, 2, 2.2, 2.4). The spatiotemporal encoding in neuroid 3 (Fig. 1C, 3) resulted from the integration of the information received from both sources (neuroid 0 and 2). Thus
the sequence of four spikes (1.1, 2.2, 1.4, 2.4) with
three interspike intervals (i1 = 12 ms, i2 = 8 ms,
i3 = 22.5 ms) encoded the fictive sensory stimulus
in the time window 42.5 ms long (PATTERN 1).
Recognition of a temporal pattern in a neuron employing a temporal code
The spike sequence in a neuron represents information in two dimensions: a cellular specificity (e.g.,
afferent and efferent connections) and spike timing. We defined the pathway under consideration
(Fig. 2A, USpP 1) as a “nonspecific” (e.g., neuroid 3 in Fig. 1A); therefore the available informa-
595
A
PAT T E R N1
i3 = 22.5
i2 = 8
*
i1 = 12
USpP
1
d1
d3=22.5
d2=30.5
d1=42.5
d2
x x x x
10 ms
PR
t
B
*
1
d1
1
d2
1
d3
1
d3
i1
i2
1
CL 1
i3
*
*
*
0
60
ms 1 2 0
60
ms 1 2 0
t
C
1
20
-20
-60
mV
-100
0
Fig. 2. Recognition of a temporal pattern in a neuron employing a temporal code. A. The same spike train
as in Figure 1C, 3 (an asterisk indicates the first spike) mediated via an “unspecific” pathway (USpP, 1) and
delay pathways. The interspike intervals (i1, i2, i3, values in ms) defined a specific temporal pattern (PATTERN
1). PATTERN 1 was transmitted in parallel via four pathways with specific delays (d1-d3, values in ms) to a
pattern recognizing neuroid (PR 1). d1/d2/d3 corresponded to the time intervals between the first/second/third
and the last spike in PATTERN 1. CL 1 is a coded line originating in PR 1. Time calibration 10 ms. Other
symbols as in Figure 1. B. Temporal coincidence. The raster display of spikes illustrates temporal shifts in
PATTERN 1 arrival at PR 1 (1) via four pathways. The indicated organized delays (d1, d2 and d3) resulted in
simultaneous arrival (an arrow in time t) of the first, second, third and fourth spike (indicated with asterisks)
to PR 1. C. Coincidence detection. Simulation of postsynaptic potentials (PSPs) recorded from PR 1 (1). The
eight horizontal lines above the simulated recordings represent possible synaptic inputs and the small vertical
bars superimposed on them indicate spikes arriving at the presynaptic endings; short horizontal bars on the
right-hand side marked active inputs. The temporal coincidence of arriving spikes (an arrow at time t) resulted
in suprathreshold summation of excitatory PSPs and made the PR 1 to generate the propagated spike with a
monosynaptic delay. Other symbols and notation as in Figure 1.
tion was recognition of PATTERN 1 by “within”
reading in USpP 1. In Figure 2A the sequence
of four spikes representing PATTERN 1 was mediated to the locus where four collateral pathways branch off from USpP1. The same activity
596
(PATTERN 1) was transmitted in parallel via all
four collateral pathways to PR 1 (convergence).
The identification is performed by mechanism of
spatio-temporal summation of subthreshold EPSPs evoked at the level of PR 1 by successively ar-
A
*
d1
d2
*
*
USpP
1
d1 = 42.5
2
*
d2 = 30.5
d3
3
d3 = 22.5
4
10 ms
x x
PR
t
B
1
x
x
CL 1
*
1
d1
1
d2
1
d3
1
*
*
*
0
60
ms 1 2 0
60
ms 1 2 0
t
C
1
20
-20
-60
mV
-100
0
Fig. 3. Recognition of a temporal pattern in a co-activated population of neurons. A. The spike trains (vertical
bars) mediated in four “unspecific” pathways (USpP 1–4) to the locus where delay lines branched off from them.
Four spike potentials indicated with asterisks represent a temporal pattern causally related to a specific stimulus.
Timing of the marked spikes was set to correspond to PATTERN 1 (Fig. 2A); other spikes simulated “noise”.
PATTERN 1 was transmitted in parallel via four pathways with specific delays (d1-d3, values in ms) to a pattern
recognizing neuroid (PR 1). d1/d2/d3 corresponded to the time intervals between the first/second/third and
the last spike in PATTERN 1. CL 1 is a coded line originating in PR 1. Time calibration 10 ms. B. Temporal
coincidence. The raster display of spikes illustrates temporal shifts in PATTERN 1 arrival to PR 1 (1) via
four pathways. The outcome of the indicated organized delays (d1, d2 and d3) was simultaneous arrival (an
arrow in time t) of the four spikes indicated with asterisks at PR 1 (1). C. Coincidence detection. Simulation of
postsynaptic potentials (PSPs) recorded from PR 1 (1). The temporal coincidence of arriving spikes (an arrow
at time t) resulted in summation of excitatory PSPs exceeding the threshold and made the PR 1 to generate
the propagated spike. Other symbols and notation as in Figs. 1 and 2.
597
riving spikes that represent the pattern. Because
the synaptic weights of synaptic terminals were
low (0.3 times TH) the maximal summation of
EPSPs was necessary in order to evoke a propagated spike in PR 1. It meant that all four spikes
forming PATTERN 1 had to arrive at PR 1 at the
same time (t). In other words the first spike had
to “wait” for the last one for 42.5 ms, the second
spike for 30.5 ms and the third one for 22.5 ms
(Fig. 2A). This condition was met by an appropriate waiting in each delay pathway (Fig. 2A, d1,
d2, d3). “Within” reading based on different time
shifts of the PATTERN1 in the delay pathways
enhances the processing time; comparing with the
length of its encoding window (42.5 ms), the integration time of PR 1 was about 100 ms (Fig. 2C,
1). All indicated spikes arrived at PR 1 via different pathways at the same time (Fig. 2B, 1, 2C
1). This temporal coincidence resulting from the
organized delays was detected by PR 1 (Fig. 2C,
1). Illustrated simulations indicated that a single
neuron might be a decoder.
Recognition of a temporal pattern represented by
the activity of different neurons in a co-activated
population of pathways
According to spatio-temporal coding hypothesis
information about specific features of a stimulus
lies in precise relationships of spikes across coactivated neurons (population and relational coding). The scheme in Figure 3A simulates spiking activity in four USpPs (1, 2, 3, 4). In illustrated time interval only four spikes marked with
asterisks were considered as having causal relationship to a specific stimulus; the accessible information was recognition of the temporal pattern read “across” USpPs 1–4. In this case one
specific delay pathway supplying a particular PR
neuroid branched off from each USpP (Figs 1A,
3A). Thus the spatio-temporal coding comprises a
combinatorial dimension because the same number of spikes (four) with the same timing of their
occurrence (the same temporal pattern) can be
transmitted in USpPs 1–4 in 24 different ways.
It is evident that the rise of propagated response
of PR neuroid depends not only on the temporal
pattern of arriving activity but also on distribution of relevant spikes across the sensory pathways involved. This moment is decisive for determination of due delays in individual collateral
delay pathways. The timing of the marked spikes
was set to correspond to PATTERN 1 (Fig. 2A).
Therefore the delays involved in processes of coincidence detection (d1, d2, d3) were the same as
in Figure 2A. d1/d2/d3 corresponded to the time
598
intervals between the first/second/third and the
last spike in PATTERN 1. The spikes relevant to
PATTERN 1 were mixed with other spikes, which
simulated “noise” from internal and/or external
sources (Fig. 3A). The resulting spike sequences
were transmitted to PR neuroid (1) with delays
corresponding to the distribution of spikes pertinent to PATTERN 1 within USpPs 1 − 4. All four
spikes forming PATTERN 1 arrived at PR 1 at the
same time (Fig. 3B, time t) and their coincidence
was detected by PR 1 (Fig. 3C). Simulation experiments indicated that the presented decoding
mechanism is able to recognize a specific temporal
pattern even if it is “contaminated” with a background spiking “noise”.
In the simulation experiment illustrated in
Figure 4A two PR neuroids (1, 2) were activated
from identical group of four USpPs (1, 2, 3, 4).
The arriving stream of spiking activity contained
two temporal patterns (PATTERN 1 and PATTERN 2) (Fig. 4B). The delay pathways to PR 1
comparing with PR 2 were set different (d3 and
d4) (Fig. 4A); thus PR 1 was tuned to recognize PATTERN 1, and PR 2 was tuned to identify PATTERN 2. Both patterns consisted of four
spikes and three intervals (12 ms, 8 ms, and 22.5
ms in PATTERN 1 compared with 12 ms, 5 ms and
25.5 ms in PATTERN 2) within 42.5 ms long encoding window. The first, second, third and fourth
spike in both patterns arrived at PR neuroids via
USpP 1, 2, 3, and 4. The propagated response of
each PR neuroid was caused by a specific pattern
(Fig. 4B, 1, 2); the spiking sequence representing
PATTERN1 was for PR 2 identifying PATTERN
2 just a “meaningless” subthreshold input activity (and vice-versa). The propagated responses of
the PR neuroids were generated in a “real” time
– with a monosynaptic delay after the last spike
of the particular temporal pattern arrived. As is
evident, both temporal patterns were recognized
without cueing their starts and ends by special
“tags”.
Possible precision of the coincidence detection
mechanism
When considering the theory of temporal coding
(the pattern theory) the crucial moment is the
precision with which we must measure spike occurrence in order to extract most of the information from a neuronal response. This precision
determines the temporal resolution of the neural code. The concept of pattern recognizers presupposes the existence of neurons, which tend to
respond to very precise temporal patterns. The
higher temporal sensitivity of such recognizers the
A
USpP
1
d1=42.5
d1=42.5
2
d2=30.5
d2=30.5
x x
x x x x
PR
B
PR
1
*
i2
2
x
4
x
CL 2
CL 1
PATTERN 1
i1
3
d4=25.5
d3=22.5
PATTERN 2
i3
i1 i4
*
i5
0
60
120
180
ms 240
0
60
120
180
240
0
60
120
180
ms 240
1
20
-20
-60
-100
2
20
-20
-60
mV
-100
Fig. 4. The specificity of pattern recognizing neuroids. A. The sketch illustrates two sets of delay lines branching off from four “unspecific” pathways (USpP, 1–4). Each set of delay pathways converged on own pattern
recognizing neuroid (PR 1, PR 2). The delays in the pathways converging on PR 1 (d1, d2, d3) compared with
delays in the pathways establishing synaptic contacts with PR 2 (d1, d2, d4) were different. d1/d2/d3 (values in
ms) corresponded to time intervals between the first/second/third and the last spike in PATTERN 1; d1/d2/d4
corresponded to the time intervals between the first/second/third and the last spike in PATTERN 2 (see in B).
B. Raster display in upper trace simulate spiking activity with two discernible four-spikes groups (the asterisks
indicate the first spikes); the interspike intervals within each of them defined two temporal patterns: PATTERN
1 and PATTERN 2. The lower two traces simulate intracellularly recorded postsynaptic potentials from PR 1
(1) and PR 2 (2). Other symbols and notation as in Figures 1, 2.
better chance of differentiating patterns or of identifying a pattern in the background “noise”. As
spatio-temporal summation of postsynaptic potentials is one of the processes underlying the coincidence detection, their time-courses (Fig. 1B)
obviously co-determine the accuracy of the coincidence detection mechanism. Figure 5A shows a
micronetwork consisting of one PR neuroid (1)
and another neuroid (0) acting as an inhibitory interneuron. PR 1 was set to recognize PATTERN
1 (Fig. 4B), but parameters of the micronetwork
were changed. The synaptic weights of delay pathways establishing synapses with PR neuroid were
set unequally: at 0.6 times threshold value (TH)
599
A
B
a
1
USpP
d1=42.5
2
d2=30.5
TH
3
d3=22.5
0
4
10
t [ms]
b
x
PR
x
1
x
x
TH
CL 1
0
10
t [ms]
0
C
1
20
c
d
e
-20
-60
mV
-100
30
0
D
60
d
ms 120
90
c
e
t[ms]
-2.0
-1.5
-1.0
-0.5
0
0.5
1.0
1.5
spike
0
0
1
1
1
0
0
0
Fig. 5. Possible precision of the coincidence detection. A. A pattern-recognizing micronetwork consisting of two
neuroids (0, 1). The organized delays (d1, d2, d3, values in ms) in delay lines branched off from unspecific
pathways (USpP, 1–4) made PR 1 specific for identification of PATTERN 1 (see Fig. 4B). The subthreshold
excitatory influences on PR 1 were set unequal: USpP 2–4 (0.2 TH, crosses); USpP 1 (0.6 TH, encircled cross).
The excitatory influence of the USpP 1 was modified by a feed-forward inhibition from neuroid 0. B. a. Time
courses of simulated postsynaptic potentials evoked in PR 1: excitatory (0.6 TH – upper trace; 0.2 TH –
middle trace) and inhibitory (lowermost trace). b. The time course of the excitatory postsynaptic potential
(EPSP) corresponding to 0.6 TH was modified by the feed-forward inhibition; this EPSP was used in simulation
experiments documented in C. C. The influence of minute delay changes on coincidence detection. c. The ideal
temporal coincidence of spikes (arrow) arriving at PR 1 via all four delay lines; PR 1 responded with propagated
spike. d. Shortening of d3 by 1.5 ms (from 22.5 to 21 ms, arrow) eliminated the propagated spike in the PR 1. e.
Prolongation of d3 by 0.5 ms (from 22.5 to 23 ms, arrow) prevented the PR 1 to generate propagated response.
D. Table giving information about possible precision of the coincidence detection. c, d, and e correspond to
situations illustrated in C. Number 1 in the lower row indicates the occurrence of propagated spike in PR 1.
Other symbols and notation as in Figures 1, 2.
for delay pathway from USpP 1 (indicated by an
encircled cross) and at 0.2 times TH for each of the
remaining three. The time-courses of simulated
EPSPs evoked in PR 1 are illustrated in Figure 5B,
a. PR 1 was also slightly inhibited by feed-forward
inhibition induced by discharge activity of neuroid
0 (Fig. 5A). This inhibition (Fig. 5B, a) dimin-
600
ished the falling phase of the EPSP caused by the
most effective excitatory synapse and shortened
the whole EPSP duration to about 5 ms (Fig. 5B,
b). The results of simulation experiments in Figure 5C document the effects of tiny changes of
the delay (d3) with which spike transmitted via
USpP 3 (the third spike in PATTERN 1) arrived
at PR 1. In the case of the ideal coincidence (the
delays indicated in Fig. 5A) the spiking activity
of PR 1 confirmed its ability to recognize PATTERN 1 (Fig. 5C, 1, c). The shortening of d3 by
1.5 ms (from 22.5 to 21 ms) or its prolongation by
0.5 ms (to 23 ms) prevented the PR neuroid from
generating spikes (Fig. 5C, 1, d, e). Our results
confirmed that the temporal resolution of the spike
timing and recognition of a temporal pattern is
possible to accuracy within 1 ms limits (Fig. 5D).
Discussion
Temporal encoding and decoding
The generally accepted concept of temporal coding is supported with experimental observations
and theoretical studies (Bialek et al., 1991; McClurkin et al., 1991; Thorpe & Gautrals,
1997). The temporal code results from the interplay between stimulus and encoding dynamics
in each relay nucleus of the sensory channel. As
shown in model experiments an analogue information evoked by a stimulus can be encoded by
action potential timing (Hopfield, 1995). Local
inhibition modifying activity in a simple model
network can generate temporal codes (Buonomano & Merzenich, 1999). The temporal structuring of neuronal firing in millisecond range may
be achieved by sequential activity propagation
in a non-ring neuronal assembly supervised by
a tonic excitatory activity in a set of inputs
(Pavlásek, 1997).
An indispensable constituent of temporal encoding and temporal codes is the existence of
mechanisms, which utilize them. The efforts in
neural modelling aimed at processing and recognition of time intervals and temporal patterns have
led to the assumption that there is a transformation from the temporal domain to a population
code (a place-cell code, a spatial code) (Buonomano & Merzenich, 1995; Covey et al., eds.,
1995, Pavlásek et al., 1996).
Physiological plausibility of the model
Our model is a sort of mechanistic model, which
addresses the questions of how nervous system
operates on the basis of known general anatomical and physiological principles. It is reasonable
to suppose that common computational primitives are involved in low level sensory processing which compute special features very quickly.
“Real-time” processing of the temporal structure
of spike trains accomplished at an early stage in
the system (Casseday & Covey, 1995) may be
translated at next stages into a different code that
is resistant to degradation across synapses (population code, coded lines) (Konishi, 1990). This
“bottom-up” processing could be done by separate modules performing selective filter operations (Rose, 1995). Distinct streams of processing project through several stages of the brainstem in diverging and converging ways. It is well
known that variable signal delays (synaptic transmission, dendritic and axonal conduction time)
along neuronal pathways are omnipresent in the
brain (Nowak & Bullier, 1997).
The mechanisms implemented in our model
network are simple, widespread, and biologically
plausible: the anatomical constituents are represented by parallel pathways, convergence, and divergence; the functional mechanisms comprise excitatory influences (supra- and subthreshold), inhibitory influences (recurrent, feed-forward), delay
pathways, temporal coincidence and coincidence
detection. Our results confirmed that a spiking
pattern can be recognized by a single neuron sensitive to coincidence. The decoding procedure uses
organized delays, temporal coincidence and coincidence detection. Time delays are organized in such
a way that the encoded features, although occurring sequentially at different times, produce signals which arrive simultaneously at a PR neuroid.
The shorter is the integration time of PR neuroid
(the EPSP duration) the higher is its ability to
discriminate different patterns. The summation of
EPSPs evoked by spikes arriving from different afferents is then more sensitive to the relative times
of their arrival (König et al., 1996). Thus, information about specific patterns is encoded in the
activity of distinct groups of neurons. The temporal and spatial dimension influence each other;
timing of signals from the periphery and the processes of maturation of the target tissue combine
to trigger anatomical architecture that determines
activity timing (Waite et al., 1998).
The computation is not only scattered in
time but also distributed across the network
(Longuet-Higgins, 1969). In such a situation
(population encoding, spatio-temporal encoding)
any decoding scheme must be able to extract information from spike trains in a population of coactivated neurons. The described results indicated
that morphological and functional principles we
used may be an effective decoding scheme. Nevertheless the extraction of information contained
in the relationships between the firing patterns
of different neurons represents a persistent challenge.
The numbers of cells receiving multimodal
(convergent) inputs increases in the upward di-
601
rection (Brooks, 1969). A frequent feature is
that some of them react more strongly to the
same temporal sequence (Barlow, 1969) or complex stimuli (response specificity); such ”tuned”
cells (Rose, 1995) could play a role of ”sequence
detectors” (Granger et al., 1995) or complex
pattern recognizers. Experimental data confirmed
the presence of such neurons in a primate visual
system pathway (Richmond et. al., 1987; McClurkin et al., 1991). Increasingly fewer impulses
are transmitted, but in more numerous fibers. It
may be supposed that part of the purpose of the
brainstem circuitry is to create a system of coded
lines, delay lines and to establish mechanisms for
coincidence detection (Casseday & Covey, 1995;
Hopfield, 1995; Pavlásek et al., 1996).
Advantages and potential limitations of the proposed model.
In the devised model the pattern itself represents
an “access code” (no cueing is necessary) and the
recognition mechanism eliminates the necessity of
special search processes. Responding neuroids are
structural representation of a pattern; there is no
problem how to “decode” the signal from a population of activated neuroids. The network (PR
neuroid) is able to identify each pattern element in
a “real time”. Devised circuitry eliminates the necessity of extensive specific descriptions of the patterns, and/or complex knowledge systems, which
often require a huge number of iterative steps or
long training periods to reach recognition.
Decomposition of complex information into
elementary constituents (temporal patterns) as
well as wiring architecture that underlies formation of feature neuroids (PR) leads to the problem of their “togetherness” in sensory perception of a stimulus (a binding problem). There are
two theories trying to elucidate this puzzle. The
first doctrine is based on combination of coding
cells. Serial processing of a stimulus in hierarchically organized neural networks with parallel
and divergent projections may result in integration at higher levels by convergence on common
neuronal pools (an anatomical connectivity determined by genetic information, activity-induced
functional connectivity). Such morphological and
functional concepts create an “intelligent” neuron
(a grand-mother cell, an object specific neuron,
a cardinal cell, a pontifical cell, a gnostic cell, a
key neuron) or a group of “intelligent” neurons
(a master area) with potency to discriminate and
identify very specific patterns (Barlow, 1972).
When parallel processing of a stimulus in divergent dynamic systems is the case, its neural image
602
may be represented by a spatially distributed activity; in this situation the second mechanism of
binding performed by synchronization of oscillatory responses of the relevant neurons (relational
code) is suggested (Crick & Koch, 1990; Singer,
1993; Von Der Malsburg, 1995). The coherent
oscillatory discharges of spatially distributed neuronal groups may be the result of the convergence
of stimulus-dependent activity in modality-specific
afferent pathways with oscillatory activity generated in unspecific sensory systems (Pavlásek,
1998).
The length of the encoding window (42.5 ms),
number of spikes (four) forming patterns in four
co-activated parallel lines and the conceivable discriminability of decoding produced even in the
narrow time window a large amount of “basic”
patterns. A delay scatter in coded lines (delay
coding) creates ground for “higher-order temporal encoding”. A combinatory explosion based on
the relational codes among higher-order recognizers (recognizers of recognizers, Edelman, 1987),
may be the foundation on which a stimulus perception relies. In the proposed network “pre-wired”
delay lines determined the encoding time window. One can anticipate the constraints influencing the length of delays arising from the evolutionary forces; nevertheless, the median latencies to flashed visual stimuli recorded in “slow
brain” cortical areas of awake monkeys ranged
from 100 ms to 150 ms (Nowak & Bullier,
1997).
The generation and processing of temporal patterns in the brain remains one of the major challenges in both experimental and computational
neuroscience
A variety of different statistical measures have
been proposed for extracting a temporal pattern
out of a continuous spike stream; but mathematics is not natural to elementary neural circuits. It
is highly probable that in addition to delay lines
many other mechanisms participate in the temporal processing (e.g., short-term memory, plastic alterations of the functional connectivity and
modifications of the dynamics of the activity flow).
There is an entire range of biophysical processes,
which could be linked to such “computations”
(Segev, 1998; Koch, 1999); besides them one can
reasonably anticipate further emergent “tricks”
underlying information processing in the nervous
system. The next biologically realistic models will
continue in the exploration of brain functions at
the interface between theory and experiment at
both the cellular and network levels.
Acknowledgements
This work was supported, in part, by Slovak Grant
Agency VEGA (grant No. 2/1011/21). Thanks are due
to Mr. Nicholas P. Lee who has assisted by correcting
English where necessary.
References
Barlow, H. B. 1969. Trigger features, adaptation and
economy of impulses, pp. 209–226. In: Leibovic,
K. N. (ed.) Information Processing in the Nervous
System. Springer-Verlag, Berlin, Heidelberg, New
York.
Barlow, H. B. 1972. Single units and sensation:
a neuron doctrine for perceptual psychology. Perception 1: 371–394.
Bialek, W., Rieke, F., Ruyter van Sieveninck,
R. R. & Warland, D. 1991. Reading a neural
code. Science 252: 1854–1857.
1969. Information processing in the
motosensory cortex, pp. 231–243. In: Leibovic,
K. N. (ed.) Information Processing in the Nervous
System. Springer-Verlag, Berlin, Heidelberg, New
York.
Buonomano, D. V. & Merzenich, M. 1995. Temporal information transformed into a spatial code by
a neural network with realistic properties. Science
267: 1028–1030.
Buonomano, D. V. & Merzenich, M. 1999. A neural network of temporal code generation and position invariant pattern recognition. Neural Comput.
11: 103–116.
Burges, P. R. & Perl, E. R. 1973. Cutaneous
mechanoreceptors and nociceptors, pp. 29–78. In:
Iggo, A. (ed.) Handbook of Sensory Physiology.
Somatosensory System. Springer Verlag, Vol. II,
Berlin, Heidelberg, New York.
Casseday, J. H. & Covey, E. 1995. Mechanisms for
analysis of auditory temporal patterns in the brainstem of echolocating bats, pp. 25–51. In: Covey,
E. et al. (eds) Neural Representation of Temporal
Patterns. Plenum Press, New York, London.
Cervero, F. & Iggo, A. 1980. The substancia gelatinosa of the spinal cord. A critical review. Brain
103: 717–772.
Covey, E., Hawkins, H. L. & Port, R. F. (eds).
1995. Neural Representation of Temporal Patterns.
Plenum Press, New York, London.
Crick, F. & Koch, Ch. 1990. Some reflections on visual awareness, pp. 953–962. In: The Brain. Cold
Spring Harbor Symposia on Quantitative Biology,
Vol. LV, Cold Spring Harbor Laboratory Press,
New York.
Edelman, G. M. 1987. Neural Darwinism. The Theory of Neuronal Group Selection. Basic Books, New
York.
Brooks, V. B.
Engel, A. K., König, P., Kreiter, A. K., Scillen,
T. B. & Singer, W. 1992. Temporal encoding in
the visual cortex: New vistas on integration in the
nervous system. Trends Neurosci. 155: 218–226.
Ghosh, J. & Deuser, L.
1995. Classification of spatiotemporal patterns with applications to recognition of sonar sequences, pp. 227–250. In: Covey,
E. et al. (eds) Neural Representation of Temporal
Patterns. Plenum Press, New York, London.
Granger, R., Taketani, M. & Lynch, G. 1995.
Special purpose temporal processing in hippocampal fields CA1 and CA3 patterns with applications
to recognition of sonar sequences, pp. 183–195. In:
Covey, E. et al. (eds) Neural Representation of
Temporal Patterns. Plenum Press, New York, London.
Guo-Qiang, B. I. & Mu-Ming Poo. 1999. Distributed synaptic modification in neural networks
induced by patterned stimulation. Nature 401:
792–796.
Hopfield, J. J. 1995. Pattern recognition computation using action potential timing for stimulus representation. Nature 376: 33–36.
Iggo, A. 1974. Cutaneous receptors, pp. 347–404. In:
Hubbard, J. I. (ed.) The Peripheral Nervous System. Plenum Press, New York, London.
Janèo, J., Stavrovský, I. & Pavlásek, J. 1994.
Modeling of neuronal functions: A neuronlike element with the graded response. Comput. Artif.
Intellig. 13: 603–620.
Koch, Ch. 1999. Biophysics of Computation. Oxford
Univ. Press, New York, Oxford.
Konishi, M. 1990. Similar algorithms in different sensory systems and animals, pp. 575–597. In: The
Brain. Cold Spring Harbor Symposia on Quantitative Biology, Vol. LV, Cold Spring Harbor Laboratory Press, New York.
König , P., Engel, A. K. & Singer, W. 1996. Integrator or coincidence detector? The role of the cortical neuron revisited. Trends Neurosci. 19: 130–
137
Longuet-Higgins H. C. 1969. The non-local storage and associative retrieval of spatio-temporal
patterns, pp. 37–46. . In: Leibovic, K. N. (ed.)
Information Processing in the Nervous System.
Springer-Verlag, Berlin, Heidelberg, New York.
McClurkin, J. W., Optican, L. M., Richmond,
B. J. & Cawne, T. J. 1991. Concurrent process-
ing and complexity of temporally encoded neuronal
messages in visual perception. Science 253: 675–
677.
Mountcastle, V. B. 1997. The problem of sensing
and the neural coding of sensory events, pp. 393–
408. In: Qarton, G. C. et al. (eds). The Neurosciences, The Rockefeller University Press, New
York.
Nowak, L. G. & Bullier, J. 1967. The timing of
information transfer in the visual system, pp. 205–
238. In: ROCKLAND, K. S. et al. (eds) Cerebral
Cortex, Vol. 12. Plenum Press, New York.
Pavlásek, J. 1997. Timing of neural commands:
a model study with neuronal networks. Biol. Cybern. 77: 359–365.
603
Pavlásek, J.
1998. The binding problem in population neurodynamics: A network model for stimulusspecific coherent oscillations. Gen. Physiol. Biophys. 17: 323–340.
Pavlásek, J. 1999. Temporal patterns recognized by a
network of coordinated time delays and coincidence
detectors. Gen. Physiol. Biophys. 18: 249–255.
Pavlásek, J. & Petrovický, P. 1994. The Reticular
Formation and the Reticulo-Spinal System. Veda,
Publishing House of the Slovak Academy of Sciences, Bratislava.
Pavlásek, J., Poledna, J. & Jagla, F. 1996. Time
intervals comparing neural network. Neural Networks 9: 1131–1140.
Port, R. F., Anderson, S. E. & McAuley, J. D.
1995. Toward simulated audition in open environments, pp. 77–106. In: Covey, E. et al. (eds) Neural Representation of Temporal Patterns. Plenum
Press, New York, London.
Redman, S. & Walmsley, B. 1983. The time course
of synaptic potentials evoked in cat spinal motoneurones at identified groups Ia synapses. J.
Physiol. (Lond.) 343: 117–133.
Richmond, B. J., Optican, L. M., Podel, M. &
Spitzer, H. 1987. Temporal encoding of two-
dimensional patterns by single units in the primate
inferior temporal cortex I. Response characteristics. J. Neurophysiol. 57: 132–146.
Rose, G. J. 1995. Representation of temporal patterns
of signal amplitude in the anuran auditory system
and electrosensory system, pp. 1–24. In: Covey,
E. et al. (eds) Neural Representation of Temporal
Patterns. Plenum Press, New York, London.
Segev, I. 1998. Sound grounds for computing dendrites. Nature 393: 207–208.
Singer, W. 1993. Synchronization of cortical activity
and its putative role in information processing and
learning. Ann. Rev. Physiol. 55: 349–374.
Singer, W. 1999. Time as coding space? Curr. Opin.
Neurobiol. 9: 189–194.
Theunissen, F. & Miller, J. P. 1995. Temporal encoding in nervous system. A rigorous definition. J.
Comput. Neurosci. 2: 149–162.
Thorpe, S. J. & Gautrals, J. 1997. Rapid visual
processing using spike asynchrony, pp. 901–907. In:
Mozer, M., C. et al. (eds) Advances in neural information processing systems, 9. MIT Press, Cambridge, MA.
Von Der Malsburg, C. 1995. Binding in models of
perception and brain function. Curr. Opin. Neurobiol. 5: 520–526.
Waite, M.E., Marotte, L. R., Leamey, C.A. &
Mark, R.F. 1998. Development of whisker-related
patterns in marsupials: factors controlling timing.
Trends Neurosci. 21: 265–269.
Received March 29, 2001
Accepted September 3, 2001
604