A Gaussian Approach to Noisy Non-isolated Neural Nets
with Chemical Markers
AXILLEAS N.ANASTASIADIS, ATHANASIA KOTINI
Lab. of Medical Physics, Medical School, Democritus University of Thrace, 68100
Alexandroupolis, Greece
Abstract :Previous studies with probabilistic isolated neural nets with chemical markers and firing
threshold fluctuations, in which the neural connections are set up by means of chemical markers,
revealed the existence of multiple memory domains. This behaviour appears both in Poisson and
Gaussian approaches for the neural connectivity. Here, we extend these studies to non-isolated neural
nets with Gaussian interneuronal connectivity by considering the intrinsic noise of the systems,
caused by the spontaneous release of synaptic transmitter substance. A simple mathematical model is
developed, the dynamics of which may be compared with the corresponding Poisson model and the
simulation results.
Keywords: Neural modelling, noisy neural nets, chemical markers, Poisson distribution of
connectivities, Gaussian distribution of connectivities.
In this work, the basic assumptions of the
1. Introduction
model are the same as in previous work [6,7]
In previous studies [1-3], the dynamical
and the mathematical formalism is similar to
behaviour of isolated and non-isolated neural
that of isolated networks [4]. Neural nets are
nets with chemical markers and high
assumed to be constructed of discrete sets of
connectivity, and the relationship between
randomly interconnected neurons with similar
structure, as expressed in patterns of
structure and function, but the neural
interneuronal synaptic connectivity and
connections are set up by means of chemical
"spontaneous" activity were investigated. In
markers carried by the individual cells,
further investigation [4], the dynamical
according to the theory of neural specificity
behaviour of isolated neural nets with high
[8-10]. Thus, the whole network is divided to
connectivity was examined by considering the
neural subpopulations, each of them
intrinsic noise of the systems, caused by the
characterized by each own marker. The
spontaneous release of synaptic transmitter
neurons are bistable elements and operate
substance. This study was focused on the
synchronously at discrete times. The present
concept of the Gaussian pattern of
model focuses on the concept of the Gaussian
interneuronal connectivity and the intrinsic
pattern of interneuronal connections and the
noise of the systems for simple neural nets and
intrinsic noise of the system for non-isolated
nets with chemical markers. In the present
neural nets with chemical markers.
study, we extend this investigation to nonIn our model, a neural net with N
isolated neural nets with chemical markers and
markers is assumed to be constructed of A
Gaussian interneuronal connectivity. The
formal neurons. A fraction h (0  h  1) of
netlet under consideration is assumed to be
them are inhibitory neurons while the rest are
attached to a cable of afferent fibres receiving
excitatory. Each neuron receives, on the
through it sustained inputs from another netlet
average,   excitatory postsynaptic potentials
with the same structure. In constructing
models of such neuron assemblies,
(EPSPs) and
  inhibitory postsynaptic
connectivity among individual elements may
potentials (IPSPs). The size of the PSP
be specified to follow a given probability law,
produced by an excitatory (inhibitory) unit is
maintaining all other parameters constant [5].
K  K  . If m1 , m 2 , ..., m N are the fractions
of neurons in the network corresponding to
2. The neural net model and assumptions
each
subpopulation,
then
m1  m 2  ...  m N  1 . The netlet is
 
attached to a cable of afferent fibres, receiving
through it sustained inputs from another netlet
of A neurons with the same structure. A
fraction
h o 0  h o  1
 
of
them
are
    we denote the
inhibitory. With
average number of neurons in each subsystem
with which an external excitatory (inhibitory)
neuron makes its synaptic connections in the
netlet, while K o K o are the corresponding
strengths of the synaptic coupling coefficients.
The neurons are characterized by the
absolute refractory period, the firing threshold
 and the synaptic delay  . It is assumed
here that the refractory period is greater than
the synaptic delay, but less than twice the
synaptic delay. A parameter r for the
refractory period may be used, taking, in
general, any integer value. For our purposes,
r was given the value r  1 when
refractoriness is assumed and r  0 otherwise.
The neural activity is restricted to discrete
times, i.e. if a number of neurons fire
simultaneously at time t , then all neural
activity resulting from this initial activity will
be restricted to times t  , t  2, ...
 
3. Mathematical Formalism
Consider a netlet of A neurons and N
markers, which is attached to a cable of
afferent fibres, receiving through it sustained
inputs from another netlet of A  neurons
having the same structure. The dynamic
variable of interest is the level of activity a n ,
i.e. the fractional number of neurons in the
netlet that are active at time t  n . We
denote with  the fraction of external active
fibres, i.e. those carrying action potential at a
particular instant. It has been shown in the
past that such a netlet may exhibit sustained
steady-state activity even if it is isolated. This
activity is sometimes referred to as
endogenous or spontaneous activity. Here, the
term “spontaneous activity” is used to describe
the firing of neurons, occurring independently
of the activity of other neurons, i.e. without
triggering by the activity of other neurons. It is
clear now, that in an isolated netlet exhibiting
spontaneous activity there are generally two
components in its steady-state activity: the
spontaneous activity or “noise”, and the
activity of neurons triggered by the preceding
activity of the netlet.
The model for the origin of such
spontaneous activity is mathematically simple.
The PSPs, which in the usual non-noisy neural
models were assumed to be zero in the
absence of presynaptic activity, are now
allowed to undergo spontaneous random
fluctuations. This is somewhat similar to the
random end plate potentials, originating from
the spontaneous release of synaptic transmitter
substance in motorneurons [11]. PSPs
generated by presynaptic activity will be
added linearly to these fluctuations, the total
PSP determining again whether or not a
neuron will fire. The random PSPs are
functionally equivalent to fluctuations in the
threshold of the neuron. Furthermore, it is
assumed that these fluctuations of the neural
firing thresholds may be positive or negative,
and they have a Gaussian distribution. The
spontaneous activity of the netlet is then
described by a simple parameter, the standard
deviation  (  j for the jth subpopulation) of
the Gaussian distribution.
We assume here that, if the average
number of active inputs per neuron becomes
sufficiently large, the number of PSPs per
neuron will follow a Gaussian distribution
according to de Moivre’s approximation. Due
to the well-known theorem which states that
the sum of any number of independent
Gaussian distributions is also Gaussian, we
can proceed with the following simplification
[7]: let the total PSP input to a neuron of the
jth marker at time t  (n  1)  be given by
e j,n1  l jK   i jK   ljK0  ijK0
(1)
where l j , i j are the numbers of EPSPs and
IPSPs, respectively, emanating from the netlet
itself and lj , ij are the numbers of external
EPSPs and IPSPs, respectively, emanating
from the axons, received by the neuron at that
instant.
If all the quantities l j , i j , lj , ij are
sufficiently large, their distributions may be
approximated by normal distributions about
their average values l j  a n  j 1  h j m j ,


1
a j 
2
i j  a n  h j m j , lj  A o / A  1  h o m j

j
and
ij  A o / A o h o m j .

o
Thus,
the
distribution of e j,n 1 will be also a Gaussian
distribution with average value:


e j,n 1  a n m j  1  h j K   h j K 

j


j


A0
m j   1  h  K     h  K 
A

e

x2
2
dx
j /  j
(4)
Obviously, the neural activity of the
whole net is given by the sum of the
spontaneous activities a  j ( j  1, 2, ..., N) of
all the constituting subsystems:
N
a   a j

j1
(5)
(2)
For the calculation of the activity at
subsequent times t  0 in the above nonand variance:
isolated netlet, we have to define the

 
  
K    h K  
 2j,n 1  a n m j  j 1  h j  K    j h j K 

A
 m j   1  h 
A
2
 2




2
 2

(3)
since the probabilities for l j , i j , lj , ij are
independent of each other.
In an isolated neural net with N markers,
which is completely quiescent at one instant,
the activity one synaptic delay later, called
a , will be entirely due to the spontaneous
firing of the neurons of all subsystems [4].
The resulting random PSPs will be
functionally equivalent to fluctuations in the
thresholds of the neurons of each subsystem. It
is further assumed that these fluctuations may
be both positive and negative and therefore
they may have Gaussian distributed firing
thresholds with standard deviations  j
( j  1, 2, ..., N) . Thus the spontaneous activity
a  j for the jth marker will be the fraction of
neurons whose thresholds at t  0 are less
than zero and it is given by
j
and
j
, namely, the average and instantaneous
values of the firing thresholds in the
subsystem characterized by the jth marker.
The activity can be found by adding the
probabilities for all possible threshold values
and PSPs that will produce firing. Therefore,
the probability that a neuron with jth marker
will receive a certain number of EPSPs or
IPSPs, that shift the membrane potential closer
to or further away from the instantaneous
threshold [1], will be given by
Pj a n , m j , ,  j  
1

e
2 x j,n 1

x2
2
dx
(6)
where
x j,n 1 
 j  e j,n 1
 j,n 1
(7)
A given combination of EPSPs and IPSPs will
produce firing of neurons whose instantaneous
thresholds are equal to or less than the sum of
these PSPs.
Thus, if we define T j ( j ) as the
probability that the instantaneous threshold of
a neuron in the jth subsystem is equal to or
less than  j , we get:

T j ( j ) 

x2
2
1
 e dx
2 (  j   j ) /  j
T j ( l j K   i j K   ljK   ijK  )
(8)
Consequently,
the
firing
probability
Pj a n , ,  j,n 1 ,  j , i.e. the probability that a

Pj a n , ,  j,n 1 ,  j   Pj (a n , m j , ,  j )

neuron of the jth marker will receive PSPs
exceeding its threshold at time t  (n  1) 
will be given by the following equation:
Pj a n , ,  j,n 1 ,  j   Pj (a n , m j , ,  j )
(10)
The expectation value of the activity for a
noisy netlet with N markers which is
attached to a cable of afferent fibres, receiving
through it sustained inputs from another netlet
with the same structure will be given by the
equation:
 a n 1  (1  a n ) m j Pj a n , ,  j,n 1 ,  j 
N
j1
(11)
lj
lj
ij
The factor (1  a n ) in this equation is
neglected if no refractoriness is assumed.
The equations, like (10) and (11), which
has derived in this work for the noisy nonisolated neural nets, are reduced to those
derived for the noisy isolated nets in previous
work [4], by taking   0 .
ij
 T j ( j )
l0 i 0 l0 i0
or
Pj a n , ,  j,n 1 ,  j   Pj (a n , m j , ,  j )
lj
ij
lj
ij
 T
l 0 i 0 l0 i0
j
4. Conclusions
(l j K   i j K   lj K   ij K  )
(9)
Since the quantities
l j , i j , lj , ij are
sufficiently large, the multiple sum of the
probability T j ( j ) for the jth marker will be
equal to the probability of the average value of
l j , i j , lj , ij for the jth marker. Thus, equation
(6) take the following form:
Summarizing, in this work the appropriate
mathematical model, describing the activity of
noisy non-isolated neural nets with chemical
markers
and
Gaussian
interneuronal
connectivity, has been constructed. The model
leads to an analytical expression for the
expectation value of the system activity. On
the basis of our model, the phase diagrams and
the hysteresis curves of such neural systems
can be drawn successfully, extending the
accumulated work that has been already
performed on neural nets having an intrinsic
noise [4]. In future work we intend to explore
in detail the differences between the Gaussian
model presented here and the Poisson model.
Moreover, the picture describing the
intrinsically noisy nets will be completed by
appropriate computer simulations.
References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
Fournou, E., Argyrakis, P. and
Anninos, P.A. Neural Nets with
Markers and Gaussian - distributed
Connectivities. Connection Science, 5,
1993, pp. 77
Fournou, E., Argyrakis, P., Kargas, B.
and Anninos, P.A. A Gaussian
Approach to Neural Nets with
Multiple
Memory
Domains.
Connection Science 7, 1995, pp. 331
Fournou, E., Argyrakis, P., Kargas, B.
and Anninos, P.A. Hybrid Neural Nets
with
Poisson
and
Gaussian
Connectivities. Journal of Statistical
Physics 89,1997, pp. 847
Kotini, A and Anninos, P. A.
Dynamics of Noisy Neural Nets with
Chemical Markers and Gaussiandistributed Connentivities. Connection
Science 9, 1997, pp.381
Anninos, P.A. and Elul, R. Effect of
structure on function in model nerve
nets. Biophysics Journal, 14, 1974,
pp. 8
Anninos, P.A. and Kokkinidis, M. A
neural net model for multiple memory
domains. Journal of Theoretical
Biology, 109, 1984,pp. 95
Anninos, P.A., Beek, B., Csermely,
T.J., Harth, E.M. and Pertile, G.
Dynamics of neural structures.
Journal of Theoretical Biology, 26,
1970, pp. 121
Prestige, M.C. and Willshaw, D.J. On
a role for competition in the formation
of patterned neural connections.
Proceedings of the Royal Society,
London, B, 190, 1975, pp. 77
Sperry, R.W. Visumotor co-ordination
in the newt (Triturus viridecens) after
regeneration of the optic nerve.
Journal of Comparative Neurology,
79, 1943, pp. 33
Sperry, R.W. Chemoaffinity in the
orderly growth of the nerve fibre
patterns and connections. Proceedings
of the National Academy of Sciences,
USA, 50, 1963, pp. 703
Katz, B. Nerve Muscle and Synapse.
McGraw-Hill, New York, 1966
Anninos, P.A., Anogianakis, G.,
Lehnertz, G., Pantev, C.H. and Hoke,
M. Biomagnetic measurements using
SQUID. International Journal
Neuroscience 37, 1987, pp.149
of