Novel emergent dynamics in biologically inspired
globally coupled stochastic networks
Supervisor:
Prof. Nigel Stocks (Engineering))
Project to run: March-June or June-Sept.
Background
Biological sensory systems are remarkable; they posses a sensitivity, dynamic range and
adaptability that is unsurpassed by man-made signal processing systems. Paradoxically, they
achieve this superior level of functionality despite the influence of large amounts of internal
(neuronal) noise. Although the origins of the noise are numerous1, they all combine to
produce a noise source that is irreducible and intrinsic to the system. Indeed, it is now well
established2 that sensory neurons only have output signal-to-noise-ratios (SNRs) of order
0dB. When compared to the output SNR of even a modest "hi-fi" system (typically 70dB) one
infers that the internal noise levels found in biological sensory systems are more the 1 million
times larger in biological sensory systems.
This astonishing fact raises questions of significant scientific weight regarding the
role of noise in biological sensory systems3, it also has, potentially, profound implications for
the way engineers view internal noise. Engineers are commonly faced with situations (e.g.
sonar, radar, satellite systems etc) where weak signals (relative to the internal noise of the
system) have to be detected and coded. This is precisely the situation that occurs in biological
sensory systems. Therefore, it does not seem unreasonable to assume that, through millions of
years of evolution, Nature has found novel methods of signal encoding that take account of
the internal noise level. Consequently, there is the real expectation that the study of biological
sensory neural networks may suggest improved engineering approaches to signal coding in
noisy environments. This idea has received additional impetus in recent years with the
discovery that noise can actually be beneficial for the efficient transmission of neural
information3-4. It now seems likely that the way engineers traditionally think about noise may
well need to be re-examining4.
The precise mechanisms by which neural systems code and transmit sensory information are
still largely unknown. What is clear, however, is that they combat the large levels of intrinsic
noise by processing information using networks. It is now believed that the neural coding of
information emerges from the collective response (dynamics) of coupled neural populations.
Whilst the study of such networks has received a lot of attention, the effect of noise on the
dynamics of these networks remains largely unexplored. This gives an opportunity for new
discoveries to made in this area. For example, preliminary studies indicate that the presence
of noise can introduce qualitatively new dynamics by re-organising the phase space. An
alternative way of viewing this is to state that noise can lead to different emergent behaviour
– with the magnitude of the noise controlling the structure of the resulting dynamics. This is
in contradiction the popular held view that noise simply ‘smears out’ or averages the
dynamics across the phase space. Consequently, this non-trivial effect of noise could well be
central to understanding the ability of complex stochastic networks to code and process
information.
The aim of this project is to characterise the dynamics of some simple biologically inspired
neural networks that are subject to global inhibitory or excitatory coupling and noise. Recent
results indicate that these types of network are capable of enhanced signal processing5, but, at
present, the dynamics are relatively poorly understood.
Project Objectives
Figure 1. Each neuron (represented by large circle) produces a 0 or 1 (spike) in response to an analogue
input (in general the input can be weighted by a factor
 i ). These responses are then summed and
globally fed-back (dashed line) to each neuron with a coupling strength K. This coupling can be chosen
to be inhibitory (negative) or excitatory (positive).
Initial studies will focus on binary (McCulloch Pitts) neurons with recurrent (feedback)
connection that will be either excitatory or inhibitory (see Fig.1). By analogy with time
dependent rate coding theory, the response of the network is taken to be the summed response
of the individual units. The dynamics are controlled by the strength of the coupling K, the
level of the internal noise and the duration of the time history feedback to the neurons. It is
this latter variable that controls the diversity of the dynamics. If only feedback from a single
time sample is fed back, then only one fixed point and period 2 dynamics are possible. As
more samples are fed-back, the number of dynamical states increases. Initially, the main
objective will be to characterise the types of attractors observed and determine their basin of
attraction. The ability of the system to code information is directly related to the number of
attracting states. How these structures depend on the level of internal noise will then be
investigated. It is anticipated that the noise will play a nontrivial role – with non-zero noise
leading to qualitatively new dynamics and attracting states. The robustness of these networks
in the coding of signal information will also be assessed and, where possible, theoretical
results will be obtained. However, there is plenty of scope for diversifying the project based
on individual skills and interests.
Skills required
The project will primarily be based on Monte Carlo simulation of the networks. However, the
networks (initially binary and discrete time) will be chosen to be sufficiently simple that some
theoretical analysis will be possible. An interest in probability theory, stochastic processes,
information theory, nonlinear dynamics and statistical physics (e.g. theory of Ising models)
would be useful.
Relevance to End Users
This project forms part of a larger ongoing project to investigate the role of noise and
fluctuation in nonlinear signal coding systems. We currently, have two funded programmes,
one that is utilising noise to improve neural coding in cochlear implants (prosthetic devices
that restore hearing to the deaf) and another on the design of low SNR sensor arrays based on
biomimetic techniques. This later project is being done in collaboration with Prof. JiangFeng
Feng in Computer Science. In the long term this work should underpin the development of
future nano-based technologies where (due to reduced power and size) noise cannot be treated
using conventional engineering approaches. We envisage the noise being incorporated
synergistically into the design to improve performance. This project could develop into a full
PhD programme for the right individual and has not been offered to the Systems Biology
DTC.
References
1
Somjen G., Neurophysiology - the essentials, (Willaims & Wilkins, Baltimore 1983).
F. Reike, D. Warland, R.R. Steveninck and W. Bialek, “Spikes: Exploring the neural code” (MIT press
1997)
3
S. F. Traynelis and F. Jaramillo, "Getting the most out of noise in the central nervous system", Trends
in Neuroscience 21,pp137-145 (1998)
4
D. G. Luchinsky, R. Mannella, P. V. E. McClintock and N. G. Stocks, “Stochastic resonance in
electrical circuits I. Conventional stochastic resonance”, IEEE Trans. on Circuits & Systems, 46
pp1205-1214 (1999); L. Gammaitoni, P. Hanggi, P. Jung and F. Marchesoni “Stochastic Resonance“,
Rev. Mod. Phys., 70, p223 (1998). F. Moss and K. Wiesenfeld “The benefits of back-ground noise”
5
D. J Mar et al “Noise shaping in populations of coupled neurons”PNAS 96, 10450-10455 (1999).
2