University of Oslo
Department of Informatics
Controlled
Introduction of
Noise to an
Integrate & Fire
Neuron
Knut Frode Pettersson
knutfp@ifi.uio.no
Thesis Candidatus
Scientiarum
9th August 2004
This thesis looks at the introduction of controlled dynamic noise on an Integrate & Fire Neuron in Neuromorphic Electronics. The idea behind this is
to decorrelate the outputs of a population of neurons, both to improve the
throughput in Addresse-Event Representation and for Spatial Noise Shaping of population spike codes.
Several methods for noise generation are presented. The approach used
to generate the noise is by amplification of thermal noise, which is then led
together with the input current to charge the membrane of the neuron.
Preface
The idea of Neuromorphic Electronics is inspired by the structure and
function of neurons and neural networks found in the human and animals nervous systems. There exist several approaches of how to implement
neurons in analog VLSI, among others, the I&F-Neuron. The communication
of spikes from the neurons rely on an Addresse-Event Representation for
connectivity between the respective neurons. Dependent on the exact AER
scheme applied, and the coding of the spike information, benefits can be
gained from some form of decorrelation of the outputs of the neurons.
The approach for deccorelation, and Spatial Noise Shaping of neuron
population codes, presented in this thesis, is by applying a small well regulated noise influence on each individual neuron. Several approaches to
generating and introducing this noise is presented. Further study of the
option of introducing an input noise current, generated from amplified
thermal noise found in resistors, are conducted. Neurons with this noise
implementation are designed, and the simulations and calculations to predict the effect of the noise is conducted, and finally attempts to measure
this on a processed chip are done.
This thesis is divided into ten chapters concerning the following subjects:
Chapter 1 Contains an introduction to the field, and important aspects, of
Neuromorphic Electronics.
Chapter 2 Presents the motivations for enquiring into the problems this
text tries to solve.
Chapter 3 Is concerned with different models and implementations of the
Integrate & Fire Neuron.
Chapter 4 Summarises some of the most important noise models, and their
influence on the design of a Noisy Integrate & Fire Neuron.
Chapter 5 Looks at the noise requirements for deccorelation, enhanced
sampling, spatial noise shaping and other applications of the noisy neuron.
Chapter 6 Uses the models presented in chapter 4 to look at different principles for the generation and introduction of noise on the neuron.
Chapter 7 Is concerned with the design of the test circuit. Considering
necessary components, and giving an theoretical estimate of their required
proportions.
Chapter 8 Presents the calculations and simulation results on the complete
circuit design, thus trying to predict the noise influence which would be
feasible to expect from the processed circuit.
VI
Preface
Chapter 9 Looks at the implementation of the design found in chapter
7. The layout of the circuit and finally the measurements done on the
processed circuit. Revealing the most important discoveries.
Chapter 10 Proposes improvements which can be done in the design, looks
into what the further development of the design may be concerned with,
and the inclusion of other additions to the Integrate & Fire Neuron. And
finally some concluding remarks about the discoveries done.
Acknowledgements
This thesis had not been possible to write had it not been for the help,
support and knowledge of several people. Those who introduced me to
the field, and represent the main source of my knowledge in it. Philipp D.
Häfliger, my tutor, and T.S. ‘Bassen’ Lande. My fellow students and other
participants of the fortnightly Neuromorphic Engineering Colloquium. The
important contributors to the field, which have been a pleasure to read,
and who are duly acknowledged in the bibliography. Are Linnerud among
others, for non curriculum contributions. And, of course, Maria Costa, for
immeasurable patience and support.
Oslo, 9th August 2004
Knut Frode Pettersson
Contents
Preface
V
1 Neuromorphic Electronics
1.1 The Nervous System . . . . . . .
1.2 Representation of Neurons . .
1.3 Rate or Temporal Codings . . .
1.4 Addresse-Event Representation
1.5 Decorrelation . . . . . . . . . . .
1.6 Spatial Noise Shaping . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2 Problem
1
1
2
4
4
6
6
7
3 Implementations of Neurons
9
3.1 The Basic Integrate & Fire Neuron . . . . . . . . . . . . . . . . . .
9
3.2 Expanding the Integrate & Fire Neuron . . . . . . . . . . . . . . . 10
3.3 Relevant Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4 Noise Models
4.1 Thermal Noise .
4.2 Shot Noise . . . .
4.3 Flicker Noise . .
4.4 Mismatch Noise
4.5 Final Remarks . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
13
13
14
14
15
15
5 Statistical Noise Requirements
17
5.1 Decorrelation in AER . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.2 Spatial Noise Shaping . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.3 Applied Requirement . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6 Introduction of Noise
19
6.1 Point of Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.2 How to Generate and Introduce the Noise . . . . . . . . . . . . . 21
7 Circuit Design
25
7.1 The Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
7.2 The Noise Generator . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8 Simulations
35
8.1 The Complete Neuron . . . . . . . . . . . . . . . . . . . . . . . . . . 35
8.2 System Transfer Characteristics . . . . . . . . . . . . . . . . . . . 37
8.3 Noise Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
VIII
Contents
9 Implementation
9.1 Layout . . . . . . . . . . . . . . .
9.2 Parasitic Analysis . . . . . . . .
9.3 Measurement Setup and Tools
9.4 Collected Data . . . . . . . . . .
9.5 Discoveries . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
47
47
49
50
52
58
10 Conclusions and Outlook
10.1 Improvements . . . . . .
10.2 Additional Components
10.3 Alternative Approaches .
10.4 Concluding Remarks . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
59
59
60
60
61
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
A Layout
65
B Pin allocations
67
C NIF-SAQ
69
D Applications and Tools
73
List of Parameters
74
List of Figures
76
List of Tables
78
Bibliography
81
1 Neuromorphic Electronics
We start by confining ourselves to an incredibly impoverished
world, and out of that, we try to build something that makes
sense.
— Carver Mead (1990)
The common digital computer of today are based on simplified digital representation of information developed by humans, initially to easier understand the workings of electronics. Quite differently, the idea of Neuromorphic Electronics is inspired by the structure and function of neurons
and neural networks developed in human and animal nervous systems and
brains throughout millennia of evolution. Implementing these structures
through analog VLSI will let us tap into ‘all the beautiful physics that is
built into those transistors’ (Mead, 1990). There are two main motivations
for this development. To build better electronic systems and, through this,
to achieve a deeper understanding of the nervous system.
In Neuromorphic systems, algorithms typically rely on collective computation in parallel networks, whereas digital computers most often use
serial computation by one or few central processors. The memory are
spread out, spatially close to the processing units that wield it. Even learning and adaption is localised within the individual computation elements.
This combined with the fact that the transistors mostly are operated in sub
threshold (weak inversion), leads to high-density, low-power implementations of certain functions, that are computationally intensive in other paradigms.
Neuromorphic Electronics, according to Mead (1990), presents us with
the tools to build wafer-sized chips, with small power dissipation, adaptability, powerful processing potential and the ability to learn. But there are
still problems to solve before we can hope to wield this technology to its
full extent.
1.1
The Nervous System
The nervous systems main building blocks are neurons and the connection
between these. A single neuron, like the one illustrated in figure 1.1 on
the following page, has its soma as the central computational entity. It
receives its input through the dendrites, which also has the function of
pre-processing the information before it reaches the soma. From the soma,
through the axon hillock, the neuron sends the processed signals through
a network of axons which connects to the dendrites of other neurons via
synapses. The human brain consists of approximately 1011 such neurons
2
Neuromorphic Electronics
Myelin sheath
Axon
Soma
Axon hillock
Node of Ranvier
Dendrites
Synapse
Figure 1.1: A conceptual model of a neuron, displaying all its major components
each with about 104 connections to other neurons. The functionality of a
nervous system lies both in the neurons and in the network of connections.
Therefore, it comes as no surprise, the major problem when constructing a
neuromorphic system, beside the representation of the neurons, is that of
constructing the network.
1.2
Representation of Neurons
Several techniques have been developed for representation of neurons on
different levels of abstraction. Three popular models are; Perceptrons, Integrate & Fire Neuron and Compartmental Neuronal Models. All three have
advantages and limitations.
Perceptrons and Spike Response Model (SRM) are mathematical and ‘phenomenological model[s] of neuronal dynamics’ (Gerstner and Kistler, 2002).
Their description of the functions of neurons are foremost applicable
to theoretical calculations and simulation on the characteristics of neurons. But aVLSI1 implementations have been designed too. The works of
McCulloch and Pitts (1943) are given much regard when considering perceptrons and SRM. Enough so that McCulloch-Pitts-neurons are often used
as a name for most variations of perceptrons. McCulloch and Pitts discoveries are also applicable to other neural models.
‘In coming to their conclusion, they stated five physical assumptions: The activity of the neuron is an “all-or-none” process; a certain fixed number of synapses must be excited within
1 Analog
Very Large Scale Integration
1.2 Representation of Neurons
3
the period of latent addition in order to excite a neuron at any
time, and this number is independent of previous activity and
position on the neuron; the only significant delay within the
nervous system is synaptic delay; the activity of any inhibitory synapses absolutely prevents excitation of the neuron at any
time; the structure of the net does not change with time.’
Eberhart and Dobbins (1990)
Compartmental Neuronal Models or Silicon Neurons give a close representation of the bio-chemical processes in the nervous system.
The most appreciated of these models,
I
the Hodgkin-Huxley model, illustrated in figure 1.2, is based on the research of Sir Alan
C
R
K
Na
Hodgkin and Sir Andrew Huxley on the squid
giant axon (Hodgkin and Huxley, 1952; Gerstner and Kistler, 2002). To describe the action potential ‘[...] they developed a detailed
mathematical model of the voltage-dependent
and time-dependent properties of the Na+ Figure 1.2: Schematic diaand K + conductances’ (Nelson, 2004), and gram of the Hodgkin-Huxley
thus developed the HH-model. Even though model
they can give good representation of the actual workings of the neuron, the
aVLSI implementations that have been done so far is complicated to use for
implementation of large scale neural networks.
Integrate & Fire Neuron is, according to Gerstner (1998), the most widespread version of Threshold Fire Models, which are a special case of SRM.
The I&F-Neuron model was first presenI
V
action potential
ted by Lapicque (1907). His basic model consisted of a capacitance and a resistance in
parallel. ‘Lapicque postulated that when the
C
threshold
R
membrane capacitor was charged to certain
threshold potential, an action potential would
10 ms
V
reset
be generated and the capacitor would discharge, resetting the membrane potential’
Figure 1.3: Lapicques original
(Abbott, 1999)
Others have taken Lapicques model fur- Integrate & Fire Neuron
ther and developed working electronically implementable models with great
success. Of these the Integrate & Fire Neuron developed by Carver Mead
(1989) is the most popular. Since this is the neuron model best suited for,
and most used in, aVLSI implementations, among others by the CAVIARproject2 , this thesis will focus on using this approach. Further presentation
of, and considerations around, Meads Integrate & Fire Neuron can found in
chapter 3 on page 9, and are the model used throughout the rest of this
text.
rest
2 European
Commission funded project to develop a multi chip vision system based on
Address-Event Representation communication of spike events, where UiO is one of the
contributors.
4
1.3
Neuromorphic Electronics
Rate or Temporal Codings
There are several theories and models of where the information in the
pulses generated by the neurons lie, or should be placed in the case of an
electronic implementation. According to Gerstner (1998), the different approaches can be divided into two major groups. These two are rate codes,
which are the older of the approaches, and pulse codes.
Average discharge rate, spike count, looks at the average activity of a
single neuron during a given time-interval. This concept requires a time
window, to collect enough neurons to find the fire frequency, which cannot
accommodate for the fast millisecond reactions which we find in parts of
the nervous system. But it is applicable to the slow build up of muscle activity. Other rate codes look at population average and average over several
iterations. Of whom the last quite certainly are not representative for the
actual workings of the nervous system, but can be useful for simulations.
Population average, on the other hand, is a very useful coding scheme from
several points of view. It gives fast response by the change of the population activity, and is independent of imprecision or temporal deviations of
single neurons.
Three of the major coding strategies based on spike timing are; Time-toFirst-Spike, Phase and Correlations and Synchrony. Time-to-First-Spike is a
coding strategy where only the first spike, and its precise timing, emitted
from a neuron is regarded. It has been shown that during the first 20-50ms
most of the information of the new stimuli is conveyed. This strategy is
only an idealisation, but can still be used, especially together with other
strategies. An alternative method of the Time-to-First-Spike is by referring
the pulses to some background oscillation. The change from one phase to
another would signal change in input. Synchrony or Correlations between
several neurons is also an aspect which might bring more information to
the pulse code. If several neurons fire simultaneously, or with some form
of synchrony it might signify a special event, which is not necessarily conveyed in the firing rate of the individual neurons. And certain firing patterns might signify certain stimuli.
Which model, or what combination of several models, are used have
influences on how to propagate and read the information contained, as
well as having an important influence on the grade of temporal distortion
each spike can endure before information is lost or altered.
1.4
Addresse-Event Representation
A severe problem encountered in the application of the neural model on an
aVLSI chip is that of connectivity. The brain is an object with expansion in
all three spatial dimensions, which leaves much room for the thousands of
connections per neuron. The chip, on the other hand, is restrained to only
two dimensions, though several layers may be present, in which to rout
all these connections. Together with limitations by the number of pins
for inter-chip communications an aVLSI implementation requires a different approach for network connection. The Addresse-Event Representation
1.4 Addresse-Event Representation
5
(AER) is one such approach, which does not require individual direct connections between every neuron and its peers.
The basic principle of AER is that the action potentials from N neurons are sent to a
Address
Digital
Address
Encoder
Bus
Decoder
common bus, consisting of log2 N wires, en1
1
coded with an individual digital address that
2 123 31 2 21 31 2
2
uniquely identifies each sending neuron. At
Time
3
3
the receiving end of the bus this address is
decoded, and the signal propagated forwarded to the neurons it is intended for.
This is possible as a consequence of the Figure 1.4: A basic schematic
advantage of the speed difference between ‘a of AER
neuron (hundreds of hertz) and a digital bus
(tens of megahertz), enabling us to replace thousands of dedicated point-topoint connections with a handful of high-speed metal wires and thousands
of switches (transistors)’ (Boahen, 2000). Furthermore, the receiving neurons are coded into the decoder, instead of hardwired, making it possible
for computer-controlled post-process reconnection of the network. The
limitation of this scheme lies in the access to the bus.
There are several different approaches to the handling of bus capacity
and collision control including; free-for-all, discarding or unfettered (Mortara et al., 1995), full-arbitration (Mahowald, 1992; Lazzaro et al., 1993;
Boahen, 1999) and ageing-versus-loss trade off (Lande, Marienborg et al.,
1996).
Discarding or unfettered AER does not present any limitations on the access to the bus. As the name implies, any collision is solved by discarding
the action potentials in question. The discarding is done at the receiving
end by recognition of event addresses generated by a collision. This protocol is quite easy to implement and does not put any delays on the spike
timing. On the other hand pulses will be lost, and it is not as efficient at
operating at full capacity.
Full-Arbitration schemes is based on a hand-shaking protocol at the sender.
Whenever a neuron wishes to send a pulse, the arbiter receives a request.
When the bus is free the next neuron in queue is granted access. In his
work Boahen (2000) concludes that arbitration is a more efficient approach
than the unfettered, since the unfettered only ‘offers higher transmission
rates than the arbitered(sic.) channel [...] if it is more than five times faster,
since [...] the arbitered(sic.) channel operates fine at 95% capacity’ (Boahen,
2000). On the other hand, arbitration requires a more complex circuitry,
a delay is often encountered for the spikes and most arbitration schemes
can easily give an unfair access to the bus.
Ageing-Versus-Loss Trade Off, the third approach, have been put forth by
Lande, Marienborg et al. (1996). It considers the advantages and problems
of the other approaches and tries to make a fusion from the best of both.
The main principle is that the requesting neurons enter a first-in-first-out
(FIFO) queue, for fair arbitration. If the delay since the request becomes
too long, the pulse is discarded. Unfortunately the implementations, so
far, have revealed itself to be slow and requiring complex circuitry.
6
1.5
Neuromorphic Electronics
Decorrelation
An important characteristic of the Addresse-Event Representation, is its
information throughput. The throughput is ‘defined as the usable fraction
of the channel capacity’ (Boahen, 2000), an represents the bandwidth for
the amount of spike carried information the system can handle. One way
to increase the throughput is by reducing the need for collision handling.
Collisions occur whenever several neurons try to access the bus simultaneously. No matter whether the AER discards or delays spikes trying to
reach the bus simultaneously, the information carried by the spike will be
altered or destroyed. Consequently, it would be beneficial for all representations if the rate of collisions were reduced.
Depending on the homogeneity of the neuron population, some sort
of random deviation of the spike times may reduce the correlation of the
neurons, and thus the collision frequency, improving the throughput. If
two neurons would spike simultaneously, under given circumstances, a discarding scheme would cause one of the spikes, and its information, to be
lost. If both had a slight random deviation they would not collide, and no
information would be lost. With arbitration, the same circumstance would
only cause one of the spikes to be delayed the time it takes the other spike
to propagate through the system. With a random deviation the delay would,
ideally, be divided equally between the two spikes. Instead of one arriving a
t
time t late, they would both arrive 2 late or early. Furthermore the random
variation would reduce the potential unfairness of the arbitration scheme.
This deviation can possibly be achieved by introducing some, appropriately scalable, random dynamic noise. But there are limits to the level of
deviation which can be successfully introduced without information getting
lost. And for some applications such noise could be counterproductive by
destroying more information than it promotes. This approach will be of no
use if the collisions are totally random, and there are no actual correlation
between the neurons.
1.6
Spatial Noise Shaping
When using a population rate coding, the information capacity can be increase by shaping the correlation between the neurons. Mar et al. (1999)
has proposed a scheme for this. ‘Inspiration for [their] method comes from
the concept of noise shaping, used in certain electronic analog-to-digital
converters.’ (Mar et al., 1999) A population of neurons receiving the same
input does not, if they are perfectly identical, improve the amount of information retrieved. But if they are totally asynchronous the would increase the sampling of the information, and thus increase the information
retrieved, by a factor equal to their number.
Some asynchrony between the neurons will always exist in an aVLSI implementation. But a well designed neuron will, and should, have very little
uncontrolled, unintended mismatch, or other, noise. To decrease the synchrony of the neurons, and increase the sampling rate, some sort of temporal deviating noise, cross inhibition, or a combination thereof, can be
introduced.
2 Problem
The subject of this report is to look at possibilities for optimising the
design of Integrate and Fire Neurons. First and foremost through the effects of controlled noise introduction.
As the previous chapter illustrates, there are several possible advantages to gain from the introduction of noise on neurons. This thesis will
look at the different approaches of introducing dynamic noise to decorrelate the output of a set of neurons.
There are several aspects to be resolved to enable introduction of dynamic noise in an beneficial fashion to the neuron. Decisions of what kind
of noise to introduce, the source of this noise and at what point to introduce it to the neuron, must be decided. The possibility of adjusting the
noise, its magnitude and influence, after the circuit has been processed,
would be preferable. These considerations will of course be dependent on
the precise implementation of the Integrate and Fire Neuron, what coding
scheme and which, if any, AER is used.
We will look at some of these considerations, and what influence the
will have on a noise introducing design. The most important common requirements will be considered. While the design presented throughout this
thesis will try to be as generic as possible, enabling it to be adjusted and
applied for most variations of the Integrate & Fire Neuron.
3 Implementations of Neurons
In this chapter we will look into the different possibilities of implementing
an aVLSI neuron with appropriate characteristics to introduce noise. As
mentioned earlier in section 1.2 on page 2 there are several ways to implement a neuron. The Integrate & Fire Neuron was a natural choice for a noisy
design. Not only because it is the prevalent design, but there are several
advantages for noise implementation, as we will see.
3.1
The Basic Integrate & Fire Neuron
The most common and popular aVLSI implementation of the I&F-Neuron was presented
by Mead (1989). The basic design consist of
just an amplifier and two capacitances. This
simple yet elegant design, as shown in figure 3.1, implements the primary functions
of the axon hillock of a neuron.
C2
Vin
A
Vout
C1
The capacitances integrates the input signal until it has accumulated enough charge Figure 3.1: Carver Meads basic Integrate & Fire Neuron
to trigger the amplifier. At this point the C2
capacitance gives a positive feedback of the
output ensuring that the increased input will force the output to quite instantaneously reach VDD , creating a pulse. But, this circuit is a ‘one-spike’
design, with no reset capabilities.
The self-resetting neuron, is presented by Mead in the same work (Mead,
1989) in which he introduces the basic I&F-Neuron. This is actually what
most commonly is regarded as Meads Integrate & Fire Neuron. This expansion is displayed in figure 3.2 on the next page.
Upon triggering, the reset transistor, Q2, is opened and pulls the Soma
voltage, Vmem , back toward ground. Vb , the pulse-length-voltage, delays
this drain, and as such controls the duration (i.e. length) of the output
pulse. Once the voltage again falls below the threshold of the amplifier
the output falls, and the membrane voltage is pulled down through the
feedback capacitance C2 .
But simplicity is not always the main focus of an implementation, and
several additions and modifications have been introduced to accommodate
shortcomings, or introduce new capabilities, in Meads original design.
10
Implementations of Neurons
Q5
Q3
V1
Iin
Vout
C2
C1
Q4
Q6
Ipl
Q2
Q1
Vb
Figure 3.2: Carver Meads self-resetting neuron
3.2
Expanding the Integrate & Fire Neuron
Dependent on the application and purpose of the I&F-Neuron implementation, there exists several additional or expanded capabilities in for the
neuron. This section will give a short presentation of some of the more
popular alterations. Some example implementations are also shown, but
there are several different ways to achieve the same functions, with their
respective sets of virtues and flaws.
Leakage Certain ‘passive characteristics of the membrane’ (Schultz and
Jabri, 1995) which can be necessary for the correct simulation and workings of a neuron exist. These can easily be approximated by the use of
a leakage of the membrane potential. This realised through the use of a
single nMOS transistor, Q16, controlled by Vlk , the leakage controlling bias
voltage. One of these passive characteristics the leakage introduces, is that
the membrane is slowly reset to its resting potential if no further input
excites the membrane over time.
Threshold Regulation The threshold voltage of the membrane has so far
been set by the threshold of the first inverter, Q3 and Q4, in the amplifier.
But it can be desirable to be able to regulate the threshold, by a bias voltage,
to achieve different neuron characteristics. This is done in van Schaik et al.
(1996) to be able to simulate different kinds of neurons using the same
physical layout. One approach to achieve this is a differential amplifier
which compares the membrane potential with a threshold potential V thr .
When Vmem exceeds Vthr the output of the amplifier is set high, and the
spike generation is initiated.
3.3 Relevant Equations
11
C2
Vpa
Vrfs
Vfas
Q8
Iin
Q9
Q3
Vmem
Q11
Q5
+
Vap
Vthr
–
Q4
C1
Q6
Q10
Vpl
Q1
Q12
Vrfd
Q7
Vfa
Vlk
Q14
Q16
Vrf
Vfam
Q15
Q2
Vfad
Q13
Crf
Cfa
Figure 3.3: An extended Integrate & Fire Neuron
Refractory Control Most actual neurons have a refractory period following
the generation of an action potential. During this period no new inputs are
allowed to affect the membrane potential. One method to implement the refractory control is with three nMOS transistor, Q7, Q9 and Q10, controlled
by the Vrfd and Vrfs biases, which work in conjunction with the capacitance
Crf . This implementation is adapted from Indiveri (2000).
Adaptive Fire Frequency In some neurons an ‘adaption in frequency following the first few spike’ (Treves, 1993) occurs. This adaption can be
implemented through an additional capacitance, Cfa , and five transistors,
named Q11 through Q15, in figure 3.3. It ‘is meant as a representation of
the summed dynamics of many individual channels, [...], valid in the limit in
which the activation of the conductance is much faster than its inactivation’
(Treves, 1993).
Noise The last addition to the basic I&F-Neuron, that we will discuss, is that
of controlled dynamic noise introduction. This, of course, is the subject for
the remainder of this text.
Which of these additions to implement on the noisy test circuit will be
discussed in chapter 7 on page 25.
3.3
Relevant Equations
These section will have a short summary of some of the important equations when doing calculations on the Integrate & Fire Neuron. They, or
derivations of them, will be used throughout the text.
The increase of Vmem upon reaching the threshold of the amplification
12
Implementations of Neurons
is dependent on the relation between the two capacitances C1 and C2 . Expressed through the equation (Mead, 1989, page 199)
∆Vmem = VDD
C2
C1 + C 2
(3.1)
With equal capacitance on C1 and C2 , this yields ∆Vmem = VDD /2. If the
inverters constructing the amplifier is well matched, or a threshold regulation is set to it, the threshold lies at VDD /2, This gives a Vmem voltage which
reaches from 0V, threshold at VDD /2 and a rising edge that reaches VDD ,
getting a total amplitude equal to VDD . The fall in voltage when coming
down again to the threshold is similar, but not equal, to ∆Vmem
The duration of the output pulse is defined as thigh , and expressed by
the equation (ibid., page 200)
thigh =
C VDD
Ipl − Iin
(3.2)
where C is the average (C1 + C2 )/2. Accordingly the equation for the time
it takes to charge Vmem to generate a pulse is given by (l.c.)
tlow =
C VDD
Iin
(3.3)
Both these equations are specialisations of Ohm’s law for capacitances,
applied to I&F-Neuron which have equal capacitances and a threshold of
VDD /2. There are circumstances where more general equations are necessary. Remembering that Ohm’s law for capacitances states that
A=C
dV
dt
By replacing for the corresponding entities from the I&F-Neuron, we get
Iin = (C1 + C2 )
Vthr
tlow
(3.4)
Solving for tlow this yields
tlow =
(C1 + C2 )Vthr
Iin
(3.5)
If C1 = C2 and Vthr = VDD /2, we see that equations 3.5 and 3.3 are identical.
4 Noise Models
Noise, in the broadest sense, can be defined as any unwanted
disturbance that obscures or interferes with a desired signal.
— Motchenbacher and Connelly (1993)
Similar to Motchenbacher and Connellys definition of noise, most definitions stress the fact that noise are unwanted, undesirable or in other ways
unwelcome. According to such definitions alone, a beneficial intended introduction of noise would be self contradictory. Other definitions focus on
the randomness of noise. Such that noise is ‘the quality of lacking any predictable order or plan.’ (WordNet, 2004) Such definitions are more in line
with the purpose to which noise is applied in this thesis.
By defining noise as ‘distortions or additions which interfere with the
transfer of information’ (OED, 2004), the focus is moved to what the noise
does to the signal. For this thesis, the most suitable definition of noise
would probably be something in the direction of: ‘Any part of a signal that
is not the true or original signal but is introduced by the communication
mechanism.’ (FOLDOC, 1993)
There are several types of noise which is relevant in the design of an
analog VLSI circuit. The most prevalent are thermal, shot and flicker noise.
The list of other noise sources is long, and include such as power supply,
crosstalk, burst (‘popcorn’), avalanche, generation-recombination, partition
and charge-sharing noise. Being lesser sources, unevenly distributed, or in
other ways unsuited for exploitation for generating a decorrelating noise,
no further regards will be given to these sources.
Furthermore we have the special case of mismatch noise. This and the
three most prevalent will be further analysed to see which influence they
have on the neural model, as well as finding which are potential candidates
for the decorrelating dynamic noise.
4.1
Thermal Noise
Thermal noise, also known as Johnson or Nyquist noise after the important
contributers, is the most commonly encountered noise in electronic circuits. It ‘is caused by the random thermally excited vibration of the charge
carriers in a conductor.’ (Motchenbacher and Connelly, 1993) Accordingly,
temperature is one of the factors which yields the noise level together with
resistance and the noise bandwidth. It has a Gaussian distribution with a
standard deviation expressed by the Root Mean Square (RMS) thermal noise
14
Noise Models
voltage Et . Et is given by the equation
q
Et = 4kT R∆f
(4.1)
Where k is Boltzmann’s constant1 , T is the temperature in Kelvin, R the
resistance and ∆f is the noise bandwidth of the system. (Motchenbacher
and Connelly, 1993, page 10)
From the definition we can see that it has a ‘white’ noise spectrum, equal
in power over all frequency intervals. Due to this, thermal noise is an appropriate noise source where an equal deviation is required for all input
currents to the neuron.
4.2
Shot Noise
The current through a pn-junction in a semiconductor device is the sum
of the charges of all the carriers crossing the junction. But this current
differs in short intervals due to the exact numbers of carriers crossing at
that precise moment. This variation is called shot noise. The variation in
the current caused by shot noise can be expressed as
q
(4.2)
Ish = 2qIDC ∆f
‘Where q is the electronic charge (1.602 × 10−19 Coulombs), IDC is the direct
current in amperes, and ∆f is the noise bandwidth in hertz.’ (Motchenbacher and Connelly, 1993, page 28)
For a MOSFET transistor this noise onlyp exist in through gate-source
leakage current, and is expressed as Ing = 2qIDC . To use the shot noise
present in a MOSFET transistor traversed by the signal current, and produce
a standard deviation of, say, 10%, the signal current must be in the order
of zepto Amperes. This will hence be a too small noise source.
Shot noise is more pronounced in bipolar transistors emitter-base junction. Leading the input current through such a junction, disregarding other
noise sources and influences, will only make it necessary with a current
in the order of nano Amperes, presuming the system has a effective noise
bandwidth in the order of giga Hertz.
One of the disadvantages with exploiting shot noise is that it is relative to the square-root of the signal current. A frequent firing neuron will
get a very large temporal deviation, caused by the shot noise introduction,
compared to the mean firing interval and will start to be dominated by the
noise signal. While a neuron firing more rarely will have small noise interaction. Since we are looking for a noise source which yields an equal
standard deviation over the mean firing interval for all firing intervals, shot
noise applied directly to the signal current seems not like a good option.
4.3
Flicker Noise
Low-frequency, pink or 1/f noise is caused by properties in the surface of
the material at hand. It is characterised by its inverse proportionality to
1 Boltzmann’s
constant = 1.38 × 10−23 J/K
4.4 Mismatch Noise
15
the frequency, ie. increasing as the lower limit of the frequency bandwidth
falls.
Looking for a noise source with equal intensity for all frequencies, it is
quit obvious that flicker noise is not a good alternative. Further studies
may prove this wrong, but flicker noise does not seem a good source for
the dynamic noise to be introduced to the neuron. On the other hand, ‘the
fluctuations of a membrane potential in a biological system have been reported to have flicker noise.’ (Motchenbacher and Connelly, 1993) It might
therefore be considered as an addition to the neuron design to achieve a
more correct representation of real neurons. But this is not the subject of
this thesis, and any further regards to flicker noise will be at diminishing
it.
4.4
Mismatch Noise
Mismatch noise is not, like the other mentioned, a natural occurring dynamic noise in the material, but is the static mismatch of components generated due to the processing of the design. Since one of the applications
intended for the introduction of noise in the neuron is to decorrelate the
output signals, in other words, to make the neurons slightly unequal to
each other, mismatch noise could seem applicable. But this noise source
is difficult to control. It is hard to predict accurately, unevenly distributed
over the geometry of the design, not possible to adjust after processing and
has an effect characteristic which are not applicable to the use for which the
required noise is intended. This is noise in the design, not in the functions
of the design.
4.5
Final Remarks
These considerations leaves us with thermal noise seemingly the best suited
candidate to achieve the dynamical noise desired. There are of course difficulties to be encountered, as we shall see, with the use of thermal noise
as the noise sources. And further studies may show that the difficulties of
other noise sources are less severe than these. But for now thermal noise
seems the more accessible approach.
The other noise sources will therefore fall under the common definition
of noise as unwanted, and should be minimised in the design. Special attention will be given to the possibilities of reducing the statical mismatch
noise.
5 Statistical Noise Requirements
When introducing controlled dynamic noise for decorrelation, the magnitude of noise required is fully dependent on the application for which
it is intended. I will in this chapter try to give an evaluation of the approximate noise required for different applications. But any precise number
will not be evident until the final implementation, for which the noise is
intended, has been chosen, and probably not until the circuit in question is
processed and measured.
Since this thesis is only concerned with the theory and design of the
noise generation, and not with implementing it for any given application,
the following considerations for noise level in different applications are
only meant as an illustration and reference.
5.1
Decorrelation in AER
As was stated in section 1.5 on page 6, the reason to implement noise
for neurons in an Addresse-Event Representation network is to reduce the
probability of collision. This to increase the information throughput by
letting more spikes come through unmodified. Whether this is achieved by
reducing the discarding rate in an unfettered, the delay in an arbitrated or
the ageing/loss in an ageing-versus-loss trade-off design is irrelevant.
For all designs the increase in information throughput must be weighted
against the possibility of altering, by the deviation of the precise spike timing, the information carried. Where the balance between this two considerations lie, is dependent on what coding scheme is applied in the system.
When the weight of the information alteration overwhelms the throughput,
the benefits of any further increase disappears.
For an unfettered design the colliding spikes will be discarded. But when
the discard ratio becomes small enough, or collision is totally random and
independent of any correlation between the different neurons, any further
increase of the noise will only reduce the information carrying ability of the
spikes. For an arbitrated design the collisions will result in a delay of the
spikes, and the system must therefore be prepared for temporal deviation
of the spikes and accordingly use an appropriate coding scheme. Thus the
level at which the deviation induced by the noise overwhelms the benefits
is higher for arbitrated AER. As with most other differences between these
two schemes, ageing-versus-loss trade-off must consider the consequences
applicable for both.
The appropriate scale of the deviation induced by the noise is dependent on how many neurons should be decorrelated, and the capacity of the
AER representation in question. The number of neurons to decorrelate is
18
Statistical Noise Requirements
not necessarily the same as the number connected to the same AER-bus,
but dependent on their probability of collision. If, for example, there are
ten neurons employing the same bus, but through some sort of inhibition
no more than five of them is allowed to spike during a given time interval. Then it would only be necessary to induce a deviation big enough for
decorrelating five neurons.
5.2
Spatial Noise Shaping
In spatial noise shaping the noise is introduced to generate an oversampling
of the population rate generated. This makes calculations for the required
noise easier accessible. The ideal noise influence should be such that the
spikes of the entire population is spread out with equal intervals to achieve
the best sampling. In other words, the decorrelation should be as large
as possible, as opposed to decorrelation in AER, where the decorrelation
should be as small as possible beyond that which is required to avoid a
collision.
There are limits to the size of the appropriate deviation. A maximal
decorrelation is not the same as unlimited deviation. If the spike of one
of two neurons, that would spike simultaneous with no decorrelation, does
not occur until after the next spike of the other neuron, an alteration of the
sampled information would occur. Therefore the deviation should be such
that probability spreads the spikes with equal intervals, but no more than
half the mean spike interval. In the case of a varying input signal and a
steady noise level, the deviation should be limited by the shortest expected
interval.
As mentioned in section 1.6 on page 6, some sort of cross inhibition
could be applied to further promote the decorrelation of the neurons.
5.3
Applied Requirement
After these considerations, we must remember that the design in this thesis
tries to generate a noise which is adjustable, by a bias, even after the circuit
have been processed. It is therefore only necessary with an approximation
of the magnitude of noise required. The ideal noise generator should be
adjustable to the extent that the application of noise will not have an influence on the parameters and size of the noise generator, but only on the
setting of the bias.
More precise calculations of the required noise is left for any applied
implementation of the noise generator. This thesis shall test whether the
noise generating design is functional. The noise requirement will therefore,
arbitrarily, be set for the generator to produce 0 to 10% temporal deviation.
Large enough to measure as an observable, intended, alteration of the equivalent noise-free circuit.
6 Introduction of Noise
The purpose of the noise is to give the timing of the output spike a temporal
deviation. The exact size of this deviation should preferably be adjustable
even after the physical layout of the noise generation has been decided.
As was established in chapter 4, the noise model we will try to exploit is
thermal noise. But there are several consideration as to where and how to
generate and apply this noise to the neuron.
6.1
Point of Introduction
There are several candidates of variables in the neuron of which to apply the noise to. The most obvious alternatives are: the threshold of the
neuron, the refractory period and the input current. We will here look at a
short review of the most important virtues and flaws of these candidates.
6.1.1
Threshold Noise
If a threshold regulating transconductance amplifier (cf. section 3.2 and figure 3.3 on page 11) is used, a thermal noise can be applied to the threshold
reference voltage. This will yield a varying threshold level, which in turn
will give a temporal deviation of the output spike.
The advantages of this approach is that,
V
V
if a threshold regulation is used, there are
V
few extra components necessary to impleV
ment this noise solution. For the noise to be
V
loud enough to influence a frequent spiking
neuron, either a huge resistance or an amplification is necessary to generate the required
noise voltage. The huge resistance solution,
not only requires much from the available Figure 6.1: Basic sketch of a
area, but will also dissipate a lot of unwanted threshold noise generator
cross-talk noise. Therefore a solution with an amplification of a thermal
noise voltage is the best solution.
bt
mem
+
amp
–
bn
thr
+
–
6.1.2
Refractory Noise
Some Integrate & Fire Neuron implementations have an explicit refractory
control which inhibits the neuron from generating another spike directly
after a spike has been emitted. An example of such refractory control were
given in section 3.2 and figure 3.3 on page 11. By applying a noise signal to
20
Introduction of Noise
one or both of the voltages Vrfs or Vrfd , the biases controlling the refractory
period, will yield a variation to the time after a spike which the neuron
is unaffected by any input. This time variation will translate directly to a
variation of the spike interval of the neuron.
However, if a leakage is used on the neuron the membrane potential
will, when little input is given, over time be pulled down again to its resting
potential. If this happens the influence of the noisy refractory period will
disappear. Furthermore, the deviation will be independent of the spike interval. So a noise level that is appropriate for a neuron spiking at a mean interval of 100ms will possibly be overwhelming and destroy the information
contained in spiking with a 20ms mean fire-rate. Since most applications
of the dynamic noise requires the deviation to be relative to the fire-rate
for all inputs to the neuron, refractory noise seems, at the moment, to be
a less ideal choice. This does not conclude that it should be disregarded
altogether, but I will, however, heed it no more attention in this thesis.
6.1.3
Current Noise
A third alternative is to introduce the noise to the neuron together with the
input signal. That is, generating a noise current, which is applied to the
input current at the input of the neuron. These two currents together will
become the effective input current of the neuron.
The noise current should be generated so that it deviates with equal
magnitude above and below 0A. Given such an offset-free noise current,
the effective input current will on average use the same amount of time
to charge the membrane to the neurons threshold, as would a noise free
input current. But the voltage will not increase at a steady rate, but with
random variations. These variations will, when the membrane approaches
the threshold, give a random deviation of the time of spiking.
One approach to achieve this is by generating two noise voltages around
the same mean voltage, and feeding them to a transconductance amplifier.
Not only will this transform the noise voltages into a current but it will at
the same time amplify the noise which lowers the requirements to the size
of the noise generating resistances.
Since the size of a amplifier in a CMOS process will be relatively small
compared to the increase necessary in a resistor to deliver the same noise
level, the cost of the extra components are feasible.
As with the threshold noise, the probability of a spike will increase as
the membrane approaches the threshold. This will yield a deviation that is
proportional to the fire-rate of the neuron. This can be assumed since the
period the membrane is close enough for the noise to excite a spike will
become longer when the membrane increases slower.
Both the threshold and current noise displays several of the characteristics
required of the noise source, and both seem eligible as the source candidate
to use. During the course of this project, the current noise was chosen as
the source to implement in the test circuit. Some further regards about the
threshold noise can be found in section 10.3 on page 60.
6.2 How to Generate and Introduce the Noise
6.2
21
How to Generate and Introduce the Noise
As outlined in the previous section, a comparative amplification between
two voltages is a good approach. The difference in the voltages will be
given by the thermal noise generated from a resistor.
At some point the resistor must be conVbias
nected to a bias or reference voltage to enVnoise
sure it is not left floating. There are two
+
Iout
possibilities of where to connect this refer–
ence. Either at one of the ends of the resistor
or somewhere in the middle of the resistor.
The last alternative will, from a schematics
point-of-view, equal two resistors in series.
Figure 6.2 shows the first alternative, while Figure 6.2: Noise generator
with the bias directly to +
figure 6.3 shows the second.
We will see that the thermal voltage difference between the two inputs
of the amplifier, denominate Eti , are equal for both designs. If we consider
that the first design has a resistance of 2R, and the second design has
both resistances at R. This will for the first design, in accordance with
equation 4.1 on page 14, yield
q
Eti = 8kT R∆f
For the second design, the thermal voltages for both resistors will be equal,
and directly given be the standard equation
q
Et = 4kT R∆f
To find Eti , we must sum the thermal voltages of both resistors. But since
they are uncorrelated RMS voltages, they must be RMS-summed1. That is
q
√
Eti = Et2 + Et2 = 2Et
(6.1)
From this we see that the two designs will yield the same standard deviation
at the inputs of the amplifier.
The distinction between the two designs
Vbias
is not the standard deviation of the inputs,
but at what voltages this deviation is found.
+
Iout
Vnoise
The arithmetic mean will be Vnoise for both
–
designs. But in the first design one input will
always be at Vnoise , while the other will deviate Eti from Vnoise . When the difference is
at its largest, within the standard deviation,
the arithmetic mean of the difference will be Figure 6.3: Noise generator
found at Vnoise ± Eti /2. This same average with two resistances
for the second design will be at Vnoise , when the difference is at its largest.
Since a transconductance amplifier will display the greatest transconductance for voltage differences close to VDD /2, the second design will yield a
slightly larger noise current, and will therefore be the design of choice.
There are two important aspects to consider in this design. First we
have the size of the resistances, which we want as large as possible, for
1 The
RMS-sum is found
by taking the square root of the sum of the square of the two
q
V12 + V22
voltages, or: V =
22
Introduction of Noise
maximal noise generation, while still considering the required layout size
and crosstalk noise dissipated from the resistances. Secondly, the specifications of the amplifier are crucial. We want an amplifier with a large
transconductance span adjustable by the bias, small offset, current independent of the load, and as always a compact design.
We will first take a theoretical enquiry on how to find the noise voltage
required to generated the required temporal deviation: Followed by how to
find resistances generating this noise voltage. The considerations for the
final design will be found in section 7.2 on page 30.
6.2.1
Calculation of Required Noise Voltage
As discussed in chapter 1 and 5, the required noise is dependent on the
application for which it is implemented. We will look at equations for calculating the noise voltage which will yield the required noise, which for this
implementation were set to 10% temporal deviation. But we will for the moment only look at the general equations, the final calculation will be done
together with other parameter settings in chapter 7 on page 25.
To find the thermal voltage which will yield the required temporal standard deviation of the neuron, we must first have a concept of what magnitudes of the input current to expect. Then we must look at what noise
currents will, in conjunction with those input currents, yield a satisfactory
temporal deviation of the spikes. Having acquired the noise currents, we
can look at which characteristics the amplifier has, and how those must be
set to generate such a noise current. Among those characteristics we will
find the noise voltage, and thus know how large that must be.
An expression for the input current was given on page 12 by equation
3.4. From that we see that the input current is dependent on the size of
the capacitances, the threshold voltage, and the spike interval required.
Repeated for convenience:
Iin = (C1 + C2 )
Vthr
tlow
An estimate for the ratio of the temporal standard deviation (tσ ) to the
mean spike interval (tµ ), which we will call tr , is found by dividing the noise
current by the input current.
tr =
In
Iin
(6.2)
As we will discover later, there are factors, which influence the deviation
ratio, that are not accounted for in this equation. But for a temporary
estimate, we get that the noise current should be approximately 10% of the
input current, or In = 0.1Iin .
Having an expression for the required input current we must look at
what factors influence the generate of the current. A transconductance
amplifier generates a current dependent on the difference between the two
inputs, multiplied by the transconductance (Gm ) of the amplifier. The noise
current generated by the amplifier will thus be
I n = E t i Gm
(6.3)
6.2 How to Generate and Introduce the Noise
23
Where Eti is the standard deviation of the inputs, and equals the RMS
sum of the two noise voltages. By solving equation 6.3 for Et , we get that
Et = √
In
2Gm
(6.4)
Neither this equation is considering all factors, but are suitable for giving a rough estimate of Et .
6.2.2
Resistance Size
Having found the noise voltage we must find an expression for the resistances necessary to generate such a voltage. A quick look at equation 4.1
reveals that we need to find the noise bandwidth. According to Motchenbacher and Connelly (1993) the noise bandwidth is given by the equation
Z∞
1
|Av (f )|2 df
(6.5)
∆f = 2
Avo 0
‘where A2vo is the peak magnitude of the voltage gain and |Av (f )|2 is the
square of the magnitude of the voltage gain over frequency’ (Motchenbacher and Connelly, 1993, page 12).
To calculate this equation for a complex circuit is quite demanding. But
by the use of the tools presented by Cadence a good approximation can be
found through simulation. The results of such simulations will be presented in chapter 7 on page 25.
When the noise bandwidth have been found we are able to find the required resistance from equation 4.1 solved for R.
R=
Et2
4kT ∆f
(6.6)
Having defined Et as the difference of V+ and V− , equation 6.6 gives us
the total resistance between the two inputs of the amplifier, and the two
resistances must therefore be one half of that each to yield the same E t .
6.2.3
Introduction
The noise bandwidth is not only dependent on the amplifier, but also on
any loads encountered later in the system which might influence the noise
current. Since the noise current shall be added to the input current, and we
do not want the noise to affect the neuron while spiking, the current must
be introduced at some point before the refractory transistor Q7, which
will be presented in figure 7.1 on page 26. The current which reaches the
source of Q7 will thence be the sum of Iin and In . The addition of currents
is straight forward, just connecting the nets in question. But this will also
have the effect that not only will the noise current be added, or subtracted
in the case of a negative current, but any loads, parasitics or offsets found
at the output of the amplifier will be applied to the input current as well.
Therefore cation must be used in the design of the amplifier to ensure that
it does not significantly affect the input current beyond the desired noise
deviation.
7 Circuit Design
We have established that the most appropriate model is to implement the
noise test neuron as an Integrate & Fire Neuron. The main focus is the
noise introduced on Iin , and its effect on the membrane potential and thus
the temporal deviation of the action potentials (ie. spikes). Most of the additional components presented in chapter 3 on page 9 are unnecessary for
this test circuit. These components are relevant in perspective of a later application of this noisy neuron, and will be further discussed in section 10.2
on page 60. To easier isolate the effects of the noise introduction from
other potential noise sources a simple and basic neuron is best suited.
In the final realisation of the test circuit shortcomings were encountered,
resulting in problems with the measurements on the circuit. These problems, which will be presented in chapter 9 on page 47, are not a consequence of the theory and principles the design are founded on. On the
other hand, they illustrates important considerations and potential fallacies regarding the design. Considerations for the further development of
the design and corrections that should be implemented for a better measurable circuit, can be found in chapter 10 on page 59.
7.1
The Neuron
The only addition to Meads self-resetting neuron (figure 3.2 on page 10) at
first required, not including the noise generation, was the pseudo refractory
control transistor Q7. Transistor Q8 is used as a current source, and will, in
a full implementation of the noisy neuron, be replaced by the input source
(eg. the synapses of other neurons or sensory inputs) of the neuron. Both
of these additions can be found in figure 7.1 on the next page.
7.1.1
Current and Capacitances
The spike interval, the most important characteristic of the Integrate & Fire
Neuron, is given by how long the input current uses to charge the capacitances to reach the threshold, as equations 3.1 through 3.5 on page 12
showed. We will try to pick a threshold voltage of approximately V DD /2 for
the neuron. It would therefore be appropriate for ∆Vmem to be the same
to give Vmem an amplitude from Vgnd to VDD . Equation 3.1 shows us that
this is achieved by using capacitances of equal size. Furthermore the size
of the capacitances affects the required input current to give the neuron a
certain spike interval. We would prefer to have this interval in the region
of 50ms at an intermediate input current. Attention must be given that
this intermediate input current is of such a scale that it is large enough
26
Circuit Design
C2
Iin
Viin
Q8
Q3
Vin
Vmem
Imem
Q5
Vmid
Vspike
Q7
C1
Ipl
Q4
Q6
Vpl
Q1
Q2
Figure 7.1: The I&F-Neuron prior to the introduction of noise. The transistor Q7 is
a pseudo-refractory control and Q8 is an adjustable current source. Transistors Q1
& Q2 are both 0.7µm by 0.35µm, Q3 & Q5 1.5/0.35µm, Q4 & Q6 0.7/0.35µm, Q7
0.7/0.7µm and Q8 is set to 10/10µm. Capacitances C1 & C2 are both set to be 35fF.
to give good measurement conditions and not too influenced by parasitics
or external signals, while not making it too large for proper influence by
the noise signal. The area the capacitances will require in the layout of the
circuit is another factor to consider.
To find the proper size for the capacitances we need to look at the current source. The current in a test circuit will be generated by the transistor
Q8 as a function of the bias voltage Viin . Since this component will not be
part of a fully implemented version of the neuron the required layout area
is not an important concern. To avoid mismatch between the currents of
the neurons a rather big transistor have been chosen for the current source.
Not only do we want an equal current for all neurons, but also constant as
the drain voltage increases. Therefore a long transistor is preferable, to
reduce the channel-length modulation.
Q8 was designed with the dimensions of 10 by 10µm. An approximation
of the source-drain current (ID ) of the transistor, which is the input current
(Iin ) of the neuron, can be found by looking at the expressions presented
by Vittoz (1994). It is expressed by ID = IF − IR (equation 1 on page 4 ibid.).
IF and IR can be found by equation 2 (l.c.)
VG −VT0 −nVS(D) 2
2nUT
IF(R) = IS ln 1 + e
(7.1)
The source of the transistor is at VDD , the drain will be somewhere between
Vgnd and VDD /2, say VDD /4. The bulk, which the voltages are referred to, is
connected to the source. This gives us VS = 0 and VD = −2.475. We know
that UT = 26mV at room temperature. Considering that IS = 2nβUT2 (l.c.)
we can find the current through Q8 for different gate voltages according to
7.1 The Neuron
27
the process parameters from austriamicrosystems (AMS, 2003)1 . Assuming
a slope factor n of 1, this gives
IS = 78.4nA
(7.2)
If we let the current source transistor operate in sub-threshold with a gate
voltage of 2.9V we get ID = 1.64pA.
We can now look for the required size of the capacitances according to
equation 3.5. Solved with respect to C, which represent each of the two
identical capacitances C1 and C2 , we get
C=
tlow Iin
2Vthr
(7.3)
Desiring a spike interval at about 50ms we would require capacitances at
about 24.8fF for a gate voltage of 2.9V. The size of the capacitances were
slightly increased to approximately 35fF after the layout revealed spare
room for this, which allows us to operate with slightly larger currents.
Restraining the current source transistor to operate at sub-threshold
will make the measurements of the circuit more difficult, since adjustments
to the gate voltage will have exponential influence on the current. To make
it easier to generate the required current and adjust it, a leakage could be
introduced. With a leakage the current source could be operated above
threshold, giving a quadratic adjustment of the current, and excess current
would be drained by the leakage.
The input current provided by Q8 will further be restrained by the
pseudo-refractory control transistor Q7. Regulated by the action potential,
the gate of this transistor will either be set to VDD or Vgnd . As a switch, the
transistor can be quite small, and still feed the current source to the membrane. A square transistor of 0.7 time 0.7µm should be sufficient. Where
the increased length, above the minimal 0.35µm, is to reduce the channellength modulation. This has also the effect of reducing the transconductance and bandwidth of the noise amplifier slightly, for a bias close to V gnd .
7.1.2
Spike Amplifier
The amplifier, which are a crucial part of the neuron, will in this design be
realised by the use of two complementary inverters. Simple, but sufficient
to start the pulse generation. With no explicit threshold regulation (see
section 3.2 on page 10), the threshold is set by the matching of the gain
factor (β) of the inverters transistors. As mentioned earlier, a threshold of
VDD /2 is preferable.
For the process parameters, given by AMS (2003), the difference in widthto-length ratio, between the nMOS and pMOS transistors, should be in the
vicinity of 3 times bigger for the pMOS transistor to compensate for the
lower gain factor. There are other factors which influence the threshold
voltage of the inverters, such as the difference between the threshold of
the two transistor types. Therefore simulation were conducted to find that
1 This
document is stated as Company Confidential and I’m not at liberty to reveal these
process parameters to a third party. Therefore only the final results of the calculations
are shown
28
Circuit Design
the width-to-length ratio had to be approximately 3.2 to achieve a V DD /2
threshold.
Simulations, on the complete circuit, showed a tendency for a decrease
in the noise regulated input current as the membrane potential approaches
VDD /2. This is caused by a leakage through the noise amplifier. Unfortunately this leakage is dependent on the membrane voltage. As a consequence small input currents can be prevented from generating a spike,
since the current is decreased as it approaches VDD /2. There are several
solutions to this problem, of whom most are concerned with the removal
of this leakage. This is a function of the noise amplifier, and will therefore
be discussed in section 7.2 on page 30.
An ad-hoc solution were implemented in
3.3
V
V
the design by lowering the threshold to decrease the leakage influence. The intention
was to refrain from feeding the leakage by
lessening the membrane voltage required to
spike. This does reduce the leakage on the 1.65
membrane, but on the other hand requires a
smaller input current to spike with the same
interval. The decrease in the input current is
proportional to the decrease in the leakage.
0
0
1.65
3.3
The modification has thus no effect. NonV
gate
etheless, it winded up staying in the design
throughout this project.
Figure 7.2: The transfer funcThe decrease was achieved by adjusting tion of the inverter pair, with
βp . The width of pMOS transistors Q3 and 1.5µm wide pMOS transistors
Q5 were reduced to approximately 2.15 times
that of the nMOS transistors, or 1.5µm. Estimates concluded that this
would give a threshold of approximately 1.46V. Simulations on the inverter
alone, shown in figure 7.2, shows a threshold which is slightly higher at
1.52V. This is an acceptable error in the test circuit, but should be readjusted to give a threshold of VDD /2 in any future implementations.
The gain of the inverters are of slighter importance. Since the form
of the rising edge of the spike foremost is set by the feedback through
capacitance C2 .
out
mid
7.1.3
Other Components of the Neuron
The last components of the neuron, which will be applied in this implementation of the I&F-Neuron, are the pulse-length regulation and the resetcontrol. The pulse-length regulation, transistor Q1 in figure 7.1 on page 26,
is controlled by the bias voltage Vpl . The only requirements of this transistor is that it can source currents enough to drain the membrane potential
from VDD to Vthr in 1ms and less. According to equation 3.2 on page 12,
solved for Ipl , the transistor have to be able to source a current greater
than 82.5pA with drain above VDD /2, and source at Vgnd . Even a minimal
transistor is able to do this in sub-threshold, so a 0.7µm wide, 0.35µm
long transistor is more than sufficient. Increased dimensions might be appropriate to get better matching between different neurons. A mismatch
7.1 The Neuron
29
could adjust the current, and accordingly the pulse length, by a few percent. Since, under normal operation conditions, the pulse length will only
constitute a small percentage of the spike interval, a slight mismatch can
be acceptable, but should normally be avoided. The same considerations
can be applied to the reset-control transistor, Q2, which will have the same
dimensions.
7.1.4
Neuron Simulations
Even though chapter 8 on page 35 is dedicated to the simulations of the
neuron, some initial simulations of the neuron core, before connecting the
noise generator, is presented here.
1µ
50
Vmem = 0
V
= 0.75
mem
V
= 1.5
Vmem = 0
V
= 0.75
mem
V
= 1.5
mem
mem
40
30
log(Iin) (A)
Iin (µA)
1n
1p
20
10
1f
0
0.825
1.65
Viin
2.475
3.3
(a) Iin while sweeping Viin for three different membrane voltages
2.5
2.6
2.7
2.8
V
2.9
(V)
3
3.1
3.2
3.3
iin
(b) Same as (a), but for Viin close to and
below the threshold of Q8
Figure 7.3: DC-characteristics of the neurons’ input current prior to introduction of
noise and parasitic components
Figure 7.3 shows the DC-characteristics of the final neuron design. If we
compare the simulated current Iin , displayed in figure 7.3(a), to the previous
calculations, we find that simulations yield a higher input current. For a
gate voltage of 2.9V the calculations predicted a current of 1.64pA, whereas
the simulation revealed a higher current of 28.15pA. This current falls to
1.69pA when the DC-simulation reached Viin of 3.0V. This inaccuracy is
negligible as the required currents are possible to generate according to
both predictions.
The input current is also dependent on the drain voltage of the input
transistor, the membrane voltage (Vmem ) of the neuron. But as the plot
in 7.3(a) shows, this dependency diminishes as Viin passes approximately
VDD /2. For Viin in sub-threshold this membrane dependency is removed
altogether giving a constant current while the membrane charges, as figure
7.3(b) shows. Figure 7.3(b) is supplied to show the magnitude of the input
current for Viin ’s operation values. The current, as displayed, can be easily
adjusted from approximately 1µA and down when Viin is at 2.5V or less.
30
Circuit Design
3.3
Viin
Viin
Viin
Viin
=
=
=
=
3.05
3.0
2.95
2.9
Vmem (V)
2.475
1.65
0.825
0
5
10
15
Time (ms)
20
25
25
50
100
150
Time (ms)
200
250
Figure 7.4: Transient analysis of the neuron without noise or parasitic influences.
The simulation was done with Vpl at 0.4V and Viin at 2.9, 2.95, 3 and 3.05V. To best
illustrate the results, the plot has been divided onto two x-axes.
Simulations were done on the neuron prior to the introduction of noise.
This to find some central and useful characteristics, such as the input current (Iin ), the inter spike period (tlow ) and the pulse length (thigh ).
These characteristics are needed, among
Viin
Iin
tlow
thigh
other things, to do calculations on the re- 3.05V 0.24pA 53.3ms 5.56µs
1.12pA 36.9ms
5.56µs
quirements of the noise generator. These 3.0V
2.95V
5.11pA 15.5ms
5.55µs
4.28ms 5.55µs
simulations were done for three values of 2.9V 23.1pA
1.47V
∆Vmem
1.51V
Viin , and the results, which are displayed in Vthr
figure 7.4, reveals the most important characteristics. The values for these key charac- Table 7.1: Important characteristics can be found in table 7.1. The dif- teristics from noise and parasitic free simulation
ference in Iin for Viin at 2.9V, between the
transient neuron simulation and the DC simulation to find the input current, is caused both by a slight reduction by pseudo refractory control Q7, a
leakage through the pulse-length regulating transistor Q1, and inaccuracies
between Cadence’s DC and transient analysis.
7.2
The Noise Generator
The basic principle of the noise generator can be seen in figure 6.3 on
page 21. How to design big resistances with minimal cross-talk noise and
area, and the design of a good amplifier are the two main problems encountered when realising the design. Furthermore, an absolute requirement is that the noise generator does not influence the neuron in other
ways than the intended noise introduction. This noise should be independent of the membrane potential and relative to the input current, I in , of the
neuron.
7.2.1
Orientation
A transconductance amplifier, consisting of a differential pair and a current mirror, is a good solution for generating a current from the voltage-
7.2 The Noise Generator
31
Ib
Vbias
Q9
V+
V–
Q10
Q11
In
Q12
Q13
Figure 7.5: Transconductance amplifier with pMOS differential pair. Transistors
Q9, Q12 and Q13 is set to 0.7/7, Q10 to 3.9/7, and Q11 to 4/7µm.
difference generated by the noise resistances. The two characteristics to
seek is a large gain and small offset independent of the load voltage. A large
gain to increase the noise as much as possible, requiring smaller resistances. And, since the output is connected to the membrane of the neuron,
efforts to diminish the difference in current depending on the membrane
potential are important. The larger gain factor of the nMOS transistors
speaks for a nMOS differential pair, with the current mirror consisting of
pMOS transistors toward VDD . But since the bias voltage, in such a design,
regulates the current toward ground, it is harder to reduce the large positive current generated toward a small membrane potential. An amplifier
with opposite orientation will loose much of the gain, but the ill-regulated
current would be negative and occur when the membrane is close to V DD .
At that point the refractory control will isolate the noise generator from
the membrane, and the current will have no effect on the spiking neuron.
The circuit with this orientation, pMOS differential pair and bias toward
VDD and nMOS current mirror toward ground, can be found in figure 7.5.
Figure 7.6 on the next page illustrates some of the aspects of p- versus
n-oriented transconductance amplifier. Figure 7.6(a) clearly illustrates the
ill-regulated current from the n-oriented amplifier when the membrane potential, the voltage at the output of the amplifier (Vload ), is close to Vgnd .
The same effect can be found on the p-oriented amplifier when the load
approaches VDD . This will have no effect since the pseudo-refractory control transistor Q7 will have disconnected the noise generator from the
membrane at this point. The difference between the positive current of
the n, and the negative of the p, is given by the difference in βn and βp .
Both plots in 7.6 show the difference in slope as the load increases. 7.6(b)
shows clearly how the n-oriented amplifiers current falls rapidly, even after
32
Circuit Design
0.8
n−oriented
p−oriented
1.5
1.66V −n
1.65V −n
1.64V −n
1.66V −p
1.65V −p
1.64V −p
0.6
0.4
1
G (µA)
0.5
m
Gm (µA)
0.2
0
−0.2
−0.4
0
−0.6
−0.5
0
1.65
Vload
3.3
(a) DC-analysis by sweeping Vload . All
transistors at process default 10µm wide
by 0.35µm long. Both V+ and V− set
at VDD /2, and the bias’ gate voltage at
|VS − 0.9V|.
−0.8
0
1.65
V
3.3
load
(b) Same as (a), but transistors regulated
according to dimensions displayed in figure 7.5 on the page before, and varying
V+ .
Figure 7.6: Transconductance of amplifiers with different orientations
passing VDD /4, as the load increases. The p-oriented amplifier keeps a more
equal current for all loads, but the trade-off in gain are immense.
In lieu of the low gain, a p-oriented amplifier is preferable if the design
shall be based on a single first-order amplifier. Alternative solutions to levelling the current, while keeping good gain, will be discussed in section 10.1
on page 59.
7.2.2
Component Parameters
Having decide upon the design of the amplifier, matching of the components to further promote the required characteristics must be considered.
Trade off between transconductance, bandwidth, offset and linearity are
the major problems. As the constraints on available space in the layout
increases, good transconductance and bandwidth becomes even more important. This to decrease the size which the resistance require in the layout.
For a test circuit, with little restrictions on area, the other considerations
can be sacrificed to ensure offset characteristics and good linearity. All
considerations should be for small difference in V+ and V− .
The increased length of the transistors are to reduce channel-length
modulation. This reduces the potential transconductance, but makes the
current more independent of the load from the membrane by reducing the
early effect. The extra current generated with a load close to ground can
be further reduced by reducing the size of βQ1 compared to βQ4 and βQ5 .
Again this has negative effects on the transconductance. The adjustments
so far yields an amplifier with good linearity. The offset is difficult to remove, but can be countered by adjusting the difference between the width
of Q2 relative to Q3. This adjusts the average current, and as such can
be used to compensate for the offset. According to these considerations
7.2 The Noise Generator
10
Vload = 0V
Vload = 0.75V
Vload = 1.65V
50
Vload = 0V
Vload = 0.75V
Vload = 1.65V
8
40
m
2
30
6
m
∆f (MHz)
3
G (µA/V)
60
V
= 0V
bias
V
= 1.6V
bias
Vbias = 2.0V
Vbias = 2.3V
Vbias = 3.3V
g (µA)
4
33
4
20
1
2
10
1
1k
1M
Freqency (Hz)
(a) Transconductance as
a function of frequency
for five different Vbias
with Vload at Vgnd
1G
0
0.825
V
1.65
(V)
2.475
3.3
0
0.825
V
bias
(b) Noise bandwidth as
a function of Vbias for
three values of Vload
1.65
(V)
2.475
3.3
bias
(c) Transconductance as
a function of Vbias for
three values of Vload
Figure 7.7: Transfer characteristics of the noise generator
and simulations on the amplifier parameters were set as shown in 7.5 on
page 31.
To reduce the complexity of simulations
Vbias
∆f
Gm
1 Hz
2.599fA
on the noise generator, a slightly inaccur- 3.3V
19.17kHz
488.5pA
ate setup was used. Setting V− at a steady 2.7V
2.4V
19.66MHz
352.3nA
28.54MHz
2.477µA
voltage of 1.65V, and applying the full vari- 2.1V
39.73MHz
8.745µA
ation to V+ , is not correct compared to the 0.0V
effect of the actual noise. This setup is equal
Table 7.2: Important amplifier
too that proposed in figure 6.2 on page 21,
transfer characteristics. Averwere Vnoise is connected directly too V− , and aged over V
load from Vgnd to
through 2R to V+ . Though this setup is inac- VDD /2.
curate, it will still give a good approximation
to the transfer function of the noise generator.
The transfer characteristics found from these simulations, and calculations on the results, are found in figure 7.7 and table 7.2. To find the noise
bandwidth a Cadence Waveform Calculator command was used:
(1/ymax(mag( if ))**2)*integ(mag( if )**2)
Where if is the output current of the amplifier. This is equivalent to equation 6.5 on page 23.
7.2.3
Resistors
After finding the transconductance and noise bandwidths of the amplifier, for different inputs and loads, the resistances to generate the required
noise current can be found. By looking at the input currents displayed in
figure 7.3 and table 7.1, and the characteristics found in figure 7.7 and table
7.2, the equation 6.4 and 6.6 can be used to find an appropriate resistances.
For the first calculation of the required resistance we will set Viin at 3V,
which yield a intermediate spike interval, and Vbias at 2.7V. This results
in a required Et of 162µV to get a 10% tr . Which in turn would require the
resistances to be in the magnitude of 85MΩ. The same calculations for V bias
at 2.4V reveals an Et of 225nV, and resistances of 160mΩ. From this we see
34
Circuit Design
that the adjustability of the amplifier is more than sufficient, if no other
factors are regarded. After repeated calculations, and test simulations for
the different values, and slight adjustment for easier implementation in the
layout, the resistances were set at approximately 29.5kΩ. Which should
yield the 10% tr at a 2.6 volt Vbias .
8 Simulations
This chapter will be concerned with finding, more precisely, what influence
the noise generator has on the neuron. First we will take a short glance at
the complete noisy neuron, and the effect of a connected but ‘turned off’
noise generator. Thereafter the relevant system characteristics are found,
which are necessary to finally predict and find the noise expected in a produced and processed circuit.
8.1
The Complete Neuron
C2
Ib
Vbias
Viin
Q9
Q8
Iin
Vnoise
Q3
Imem
Vin
Vmem
C1
Q4
Q11
Vspike
Vmid
Q7
Ipl
Q10
Q5
Q6
In
Vpl
Q1
Q12
Q13
Q2
Figure 8.1: The complete Noisy Integrate & Fire Neuron. The noise generator, on
the left, is easily recognisable from figure 7.5 on page 31. While the neuron core is
as shown in figure 7.1 on page 26.
For reference the complete schematics of the Noisy Integrate & Fire Neuron
are displayed here, in figure 8.1. The output current of the noise generator was, as mentioned earlier, connected with the input current, generated
from Q8, at the source of the pseudo refractory control transistor Q7. In
addition to a Vap output pin, reading the action potential, a Vmem output
were included, for readout of the membrane potential. All the final component dimensions are collected in table 8.1 on the next page.
36
Simulations
Name
Width
Length
Name
Width
Length
Q1
Q2
Q3
Q4
Q5
Q6
Q7
0.7µm
0.7µm
1.5µm
0.7µm
1.5µm
0.7µm
0.7µm
0.35µm
0.35µm
0.35µm
0.35µm
0.35µm
0.35µm
0.7µm
Q8
Q9
Q10
Q11
Q12
Q13
10.0µm
0.7µm
3.9µm
4.0µm
0.7µm
0.7µm
10.0µm
7.0µm
7.0µm
7.0µm
7.0µm
7.0µm
Name
Name
Res.
Cap.
C1
C2
N/A
N/A
35.1fF
35.1fF
R1
R2
Res.
29.5kΩ
29.5kΩ
Cap.
14.3fF
14.3fF
Table 8.1: Component parameters of the complete neuron
The first simulation conducted were to find some of the key characteristics of the neuron. Noiseless, ie. Vbias at 3.3V, simulations for four values of Viin gave the input current Iin , the spike interval tlow and the pulse
length thigh of the neuron, which can be found in table 8.2. A plot, showing
the waveforms of the membrane potentials from these simulations can be
found in figure 8.2.
3.3
Viin
Viin
Viin
Viin
=
=
=
=
3.05
3.0
2.95
2.9
Vmem (V)
2.475
1.65
0.825
0
10
20
30
Time (ms)
40
50
50
100
150
Time (ms)
200
250
Figure 8.2: Transient analysis of the complete neuron. Vbias set at 3.3V, Vpl at 0.4V
and Viin at 2.9, 2.95, 3 and 3.05V. To best illustrate the results, the plot has been
divided onto two x-axes. The setup of this simulation will be explained in section 8.3.
The influence of other components, when Viin
Iin
tµ
tσ
3.05V 0.24pA
N/A
N/A
comparing with the results in table 7.1 on
3.0V
1.12pA
N/A
N/A
page 30, are evident. These values are useful
2.95V 5.11pA 21.1ms
0.13µs
2.9V 23.1pA
4.61ms 0.11µs
when the noise simulations are conducted.
They help find what level of noise are necessary to achieve the required deviation of the Table 8.2: Important characspikes. The two figures tµ and tσ , found in teristics of the neuron
table 8.2, represent the average spike interval and the standard deviation
of the spike interval. The spike interval is the sum of the inter-spike period
(tlow ) and the pulse length (thigh ). In other words the time from the end of
one spike to the end of the next. More information on these figures can be
found in section 8.3 on page 38.
8.2 System Transfer Characteristics
8.2
37
System Transfer Characteristics
Having established the characteristics of the neuron, we move on to look
at the transfer characteristics of the amplifier and the soma. These are
important to calculate the expected noise influence, and to generate the
noise files which will be fed to the simulator.
AC-analysis’ were done on the noise generator, and their results imported to Matlab. Here calculations were done to find the derived characteristics of the generator. Some of the results are displayed in figure 8.3.
6
9
Et max
Et avg
Et min
5
50
Gr max
Gr avg
Gr min
8
40
7
3
2
In (nA)
6
Gr (µA)
Et (mV)
4
In max
In avg
In min
5
4
30
20
3
2
10
1
1
0
0.825
V
1.65
(V)
2.475
bias
(a) Et as a function of
Vbias . Derived from ∆f ,
as found in figure 7.7(b)
3.3
0
0.825
V
1.65
(V)
2.475
bias
(b) Gr over the noise
bandwidth as a function
of Vbias
3.3
0
0.825
V
1.65
(V)
2.475
3.3
bias
(c) Expected noise current, In . Calculated from
Eti and Gr
Figure 8.3: Derived noise amplifier characteristics. The simulations were done for
several Vload ’s, and then the maximum, minimum and average were found for each
Vbias over Vload .
Some slight inaccuracies can be observed Vbias Eta (V) Gra (A)
Ina (A)
0.70µ
2.60f
2.57z
in the plots. These are a consequence of 3.3V
96.5µ
0.38n
51.0f
inaccuracies in the Matlab script I designed 2.7V
2.4V
3.09m 19.1µ
0.83n
3.73m
1.77µ
9.28n
to calculating ∆f . All values displayed in 2.1V
4.40m
7.12µ
43.9n
tables and used in further simulations were 0.0V
recalculated manually and checked against
Table 8.3: Important derived
Cadence simulations and waveform calculanoise amplifier characteristics
tions to ensure their accuracy.
averaged over Vload ’s from
The voltage difference found at the in- Vgnd to 1.65V
puts of the amplifier, Eti , generated by the
thermal noise of the resistances, is equally divided over all frequencies of
the noise bandwidth. It can therefore not be multiplied by the transconductance of the amplifier for any single given frequency to find the power
spectrum of the noisy output current In . Every frequency component of
Eti must be multiplied by their respective Gm . This yields Inf , which is the
input current for a single frequency, according to the equation
I n f = E t f Gm f
Where Etf is the RMS thermal voltage for a noise bandwidth of 1Hz. And
Gmf are the transconductance for a given frequency. To get the total In , we
must sum Inf for all frequencies. Since Etf is a RMS voltage, they must be
38
Simulations
RMS-summed. Hence
v
v
u ∆f
u ∆f
uX
uX
u
u
2
2
(Etf Gmf ) = Etf t
Gm
In = t
f
f =1
(8.1)
f =1
Etf can be set on the outside of the sum, since the thermal voltage are equal
for all frequencies. Considering that
Et
E tf = q i
∆f
and that the RMS of Gm
r
P∆f
2
f =1 Gmf
q
Gr =
∆f
we can substitute in equation 8.1 and get
q
Et
In = q i Gr ∆f = Eti Gr
∆f
(8.2)
Gr is the RMS transconductance over all frequencies
of ∆f . Remembering
√
that Eti equals the two Et ’s RMS-summed, or 2Et , some sample calculations of In can be found in table 8.3 on the page before.
8.3
Noise Analysis
To ensure correct simulation results, the simulation process went through
several steps. First transient noise analysis was conducted on the amplifier. The results of this were compared with the noise anticipated from
calculations. Then the In found from these analysis were used to generate
noise current files, used for simulations on the remaining neuron. Again
comparison were done toward the predicted results.
Before engaging in the actual simulations, attention must be given to
the generation of the noise input.
8.3.1
Noise File
The Cadence schematic components Vpwlf and Ipwlf were used as noise
sources. These are sources which accepts a file containing voltages, or
currents, together with a set of matching time-stamps. The time-stamps is
just a column of equally spaced times from zero to the required stop time,
with intervals dependent on the required sampling rate. The noise voltages,
or currents, were generated by the use of the Matlabs function randn. This
function generates a set of normally distributed random numbers with a
standard deviation of one. By multiplying this set by the required deviation,
a complete noise file is at hand.
8.3 Noise Analysis
39
Interpolation
The noise generated from Matlab does only consist of discreet points. When
fed to Cadence, ‘the value of the source at intermediate values of time is determined through linear interpolation of the input values.’ (CDSDoc, 2003)
When a set of points with a normal distribution is interpolated, the standard deviation (σ) decreases. To compensate for this the average decrease
by the interpolation had to be found, and then increase the σ of the noise
files accordingly.
The designation σn will be used for a standard deviation which after being interpolated n times yields σ. For most simulations, ten interpolations
are sufficient for accurate simulations in Cadence. To find σ10 , a set of 1000
random numbers with σ of 1 was generated. The set was interpolated at
ten points per interval, and divided by the original set. A script was set up
to repeat this 1000 times and calculate the average standard deviation of
these. This average moved toward 0.8177, with a standard deviation of the
average in the magnitude of 104 . σ10 was set to 0.8177−1 .
To test the accuracy of this, a new set of random numbers with σ10 = 1
was generated and interpolated 10 times. Thousand repetitions of this
yielded an average σ of 1 with a three decimal precision. Other σn , where
needed, were found using the same method.
Oversampling
To ensure correct simulations of the noise, tests for the effects of oversamplings were conducted. A plain oversampling of the frequency of the
noise signal, with no adjustments to Et had no or little effect. Introducing
a noise signal for a larger noise bandwidth did, on the other hand, have
unexpected effects. By increasing ∆f and Et by the square root of the same
factor, the deviation of In increased for all Vload . A study of the spectral
analysis of In and Eti , revealed that the cause was the way the noise files
are generated. The increase in noise were found on the frequency components below ∆f , where no increase should be present as a consequence of
sampling above ∆f .
To ensure correct simulation of the noise, an oversampling of the noise
was used. To avoid correlations between sampling and signal, the noise was
sampled slightly higher than the Nyquist sampling rate of 2 times fmax . A
sampling rate of 2.5 was used.
Correlation
The time-stamps of the noise files were generated by making a matrix in
Matlab from 0, to the required stop time, with intervals of (2.5∆f )−1 . Since
the noise-file for both neurons are generated by this same method, their
timing is completely correlated. This correlation only affect the highest
frequency components, but non the less, tests were conducted to ensure
that this correlation did not affect the generated noise.
Three identical pairs of two noise files were generated. For one pair the
time stamps were delayed a random percentage on both file. The second
pair had one set delayed 50% of the interval. The last pair were untouched.
40
Simulations
Simulations on the amplifier showed a difference of In less than 2‰. No
explicit difference could be found on the frequency spectres. The concerns
for influence of the correlation of the noise files were discarded.
8.3.2
Amplifier Noise Analysis
*
Before engaging in noise simulations on the
Vbias
entire neuron, simulations were conducted
on the noise amplifier alone. This to ensure Vnoise
+
E1
Et
that the predictions of the characteristics of
–
+
the noise current were done correctly, and
–
E2
also making it possible to shorten the runtime of the full simulation, by being able to
replace the entire noise generator by a single
noisy current source, Ipwlf. Figure 8.4 de- Figure 8.4: Schematics of the
picts the setup for the analysis on the amp- noise analysis setup
lifier. The two noise voltage sources, E1 and E2 , were fed with two different
noise files, generated by Matlab as described in 8.3.1 on page 38.
i
*
VR1, VR2 (nA)
1.65
1.64
3
2
1
0
n
In the plot three vertical markers have
been placed to show examples of a small current, a zero current and a large current (in
that order), and illustrating how they are generated by the corresponding difference of the
two noise voltages in the upper half of the
figure.
1.66
I (nA)
Figure 8.5 shows a sample of a simulation
on the amplifier. This one was done for Vbias
at 2.4V and Vload at 1.5V, which according to
calculations should yield a noise current, In ,
with a standard deviation of approximately
832pA. The simulation revealed In to have a
σ of 980pA.
−1
−2
−3
0
0.5
1
Time (µs)
1.5
2
Figure 8.5: Simulation results
for Vbias = 2.4 and Vload = 1.65V
Other simulation results, and their difference from calculations, can be found in table 8.4. The most interesting
noise is when Vmem is close to Vthr , and the possibility of a spike is imminent. All simulations, the results of which are displayed in the table, were
therefore done for Vload of 1.5V.
What can be observed is that the predic- Vbias
In -sim
In -cal
Diff.
2.79zA
tion slightly underestimated the noise cur- 3.0V 493 zA
57.2fA
54.7fA
4.5%
rent. But a difference in the magnitude of ap- 2.7V
2.4V
980 pA
832 pA
17 %
10.2nA
9.49nA
7.7%
proximately 10%, is an acceptable error. Part 2.1V
of this difference can probably be ascribed
to the noise files difference from an actual Table 8.4: Simulation results
noise signal. Furthermore, the fact that the of the noise amplifier
noise bandwidth was only calculated through an approximation, should be
kept in mind.
8.3 Noise Analysis
8.3.3
41
Neuron Noise Analysis
Figure 8.6 depicts the setup for the noise
analysis on the complete neuron. The noise
In
Viin
current source is replacing the noise gener*
Vmem
ator, and fed with a noise-file in accordance
Vap
with the current standard deviation, offset
and frequency bandwidth found from the calVpl
culations and simulations of the amplifier.
The first simulation done were for Vbias of
3.3V, to see how the neuron works while the Figure 8.6: Schematics of the
noise generator are connected but turned off. noise analysis setup for the
This simulation was shown at the start of complete neuron
this chapter in, figure 8.2 on page 36. It reveals that the neuron has problems with the spike generation at low input currents. Before conducting
further simulations, the offset current which causes this must be studied.
Offset
Since the noise amplifier isn’t an ideal amplifier, there exist an offset current. This offset current has an influence on the spiking frequency of the
neuron.
Simulations were done to find this offset
2
V
=0
current. The results, depicted in figure 8.7,
Average
V
= 1.5
show that this current is most pronounced
1.5
for low membrane voltages. For high Vbias
voltages, this offset will turn into a negative
1
leakage from the membrane when the membrane potential is high enough. This leakage
0.5
is what prohibits the neuron from spiking at
a slow rate, when the noise is turned off, as
0
two of the curves in figure 8.2 on page 36
−5
illustrate.
0
0.825
1.65
2.475
3.3
V
(V)
bias
The spike inhibition is also enhanced by
the before, on page 30, mentioned leakage
Figure 8.7: Offset simulation
through the pulse length transistor Q1.
Since Io is large for low membrane poten- results
tials, it will have an significant effect on the average spike interval. It will
not, however, have an equally large influence on the temporal noise deviation tσ . Since, the significant effect of the noise, an early or delayed spike,
will usually only occur when the membrane is close to the threshold, and
Io is at its minimum. But tr will be reduced by the offset because of the
increase of tµ , without a similar increase of tσ .
Calculations for spike times and noise influences would be very complex if the varying Io current were to be calculated correctly. To ease the
calculations on the noise influence, the average offset current, Ioa , will be
used instead, and regarded as constant. Accordingly, the average offset
current, Ioa , can be added to the input current, Iin , to find the effective input current, denominated Iine , which will be used in the calculations. This
load
o
I (pA)
Io (µA)
load
42
Simulations
will yield slightly smaller tσ and tµ , but, since they are both decreased, tr
should stay approximately similar.
Noise Prediction
Before looking at simulation results a prediction of the expected noise output is appropriate. The temporal deviation of the spike timing, tσ , is a
direct consequence of the variations of the membrane charging current. In
other words, it can be calculated by looking at the relative size of the noise
current compared to the input current. And as stated in the previous paragraph, the input current to use is the effective input current, Iine , including
the offset.
In most circumstances the interesting factor is the ratio of the deviation
to the mean fire rate, and not only the absolute deviation. This ratio is t r ,
which equals tσ /tµ . In calculations tr is actually easier to find precisely
than tσ . To find tσ , we must first calculate the time it takes to charge the
membrane with In added, and then subtract the same figure without In .
Another approach is to calculated tr directly, and then multiply tµ by tr .
Dividing the noise current by the effective input current, as equation 6.2
on page 22 suggest, seems adequate to find tr . But the high frequency
components of the noise will not influence tσ to full extent, since they will
be neutralised over time. Therefore the contributions of noise frequencies
above the frequency of the neurons spikes will be removed. First dividing
by the square root of the noise bandwidth, to find the single frequency
contribution, and then multiplying the square root of the spike frequency
approximates this.
s
tµ
In
tr =
×
(8.3)
Iine
∆f
Since the transconductance decreases for higher frequency noise components, this equation will be a low estimate of the actual influence.
As figure 8.8 show, there is little differ3
V = 3.05V
V = 3.0V
ence in tr for the varying Viin . This is because
V = 2.95V
V = 2.9V
the offset current is overwhelming compared
to the actual input current for most Vbias ’s.
2
A small difference can be discerned for Vbias
from 2.5 to 2.8V, where the noise component
of the noise current have emerged slightly
1
before the offset component. This is caused
by the early effect of the transistors in the
amplifier, which inhibits the offset current
0
0.825
1.65
2.475
3.3
V
(V)
bias
slightly. Thus allowing tr to be relative to
the actual input.
The calculations reveal tr to be largest for Figure 8.8: The expected noise
a Vbias of 2.2V. Where the deviation reaches influence according to calculaabout 2.64%. As discussed in chapter 5, this tions
could be sufficient for some applications. But it would be preferable with
the possibility of increasing the noise even further. By reducing the offset
current, the noise influence would be more pronounced since, as figure 8.7
iin
iin
iin
σ
t (%)
iin
8.3 Noise Analysis
43
showed, the actual input current is neglible compared to Io for a Vbias of
2.2V.
Not only do the offset reduce the possible noise influence, but it also
removes any information that would be carried by the input signal. As
the design is now, the spiking and noise is totally independent of the input current, except for a small area between Vbias 2.5 and 2.8V. Therefore,
an applicable noise generator most have the offset reduced, or ideally removed. Measures to accomplish, or at least approach, this can be found in
section 10.1 on page 59.
Analysis Results
Even though the characteristics of the noise amplifier is below par, tests
must be done to ensure that the predicted noise from calculations is in
the vicinity of what simulations would lead us to expect. This is necessary
for the ability to use the above stated methods of calculations to reach a
satisfactory design.
The results from the analysis with Vbias set at 3.3V was displayed already
in figure 8.2 on page 36, and some of the key characteristics in table 8.2.
The simulations displayed here are a representative collection of all the
simulations conducted. The result’s divergence from the calculated estimate are in line with those found for other simulations, that are not displayed.
3.3
V
iin
Viin
Viin
Viin
=
=
=
=
3.05
3.0
2.95
2.9
Vmem (V)
2.475
1.65
0.825
0
10
20
30
Time (ms)
40
50
50
100
150
Time (ms)
200
250
Figure 8.9: Noise analysis with Viin from 2.9 to 3.05V and Vbias at 2.7V.
For Vbias of 2.7V, the analysis were con- Viin tµ (s) tσ (s)
tr
t rc
3.05V 41.0m 71.2µ 1.7h 1.3h
ducted for four values of Viin , as figure 8.9
3.0V 30.1m 35.6µ 1.2h 0.9h
shows. This plot, as is also the case with the
2.95V 14.3m 10.5µ 0.7h 0.5h
2.9V
4.18m 1.27µ 0.3h 0.2h
remaining noise simulation plots, are just a
sample of the entire simulation. A large number of spikes had to be gathered to ensure a Table 8.5: Simulation results
correct tr result. The key figures, tµ , tσ and of the noise amplifier
tr , are compared to the calculated tr in table 8.5, where we see that the
estimate, as expected, is slightly smaller than the simulation results.
The simulations for Vbias of 2.4V, and less, were only done for Viin at
2.9V. At this point the offset current is more influent than the input cur-
44
Simulations
3.3
Vmem (V)
2.475
1.65
0.825
0
100
200
300
400
500
Time (µs)
Figure 8.10: Noise analysis with Viin at 2.9V and Vbias at 2.4V
rent, as figure 8.7 predicts, and the results for different Viin are indiscernible. The 2.4 volt Vbias simulation, depicted in figure 8.10, was quite true to
the calculations. With tµ at 61.1µs, and tσ at 832ns, the simulation revealed
a tr of 1.36%. Which is quite close to the tr of 1.31%, from the calculations.
3.3
Vmem (V)
2.475
1.65
0.825
0
25
50
Time (µs)
75
100
Figure 8.11: Noise analysis with Viin at 2.9V and Vbias at 2.1V.
As stated earlier, the prediction for tr , is a low estimate according to the
factors considered. But one factor which haven’t been included in the calculations, is the small fallacy in Integrate & Fire Neuron designs including a
pseudo refractory control. When the input is disconnected during a spike,
a small charge is gathered at the refractory transistor. Usually too small to
have any influence. But when the input current, be it the intended or the
offset, becomes sufficiently large, this charge have a visible influence on the
neuron. A well designed neuron under normal operation conditions should
not be influenced by this. But with an offset, or an extremely long pulse
duration relative to tlow , this can introduce problems. Since the calculations
lack this factor, the estimates for the deviation will be more inaccurate as
the frequency of the neuron increases above ordinary operation.
With 2.1V, at the current source, the pre-charge effect has become influent. The predicted tr of 2.63% was too high compared to the 1.86% tr found
from the simulation. The mean spike interval of 9.32µs is constrained by
8.3 Noise Analysis
45
the 5.55µs pulse length, and only leaves room for a 174ns tσ on the 3.77µs
inter spike period.
3.3
Vmem (V)
2.475
1.65
0.825
0
10
20
30
40
50
Time (µs)
60
70
80
90
100
Figure 8.12: Noise analysis with Viin at 2.9V and Vbias at 1.8V. The dashed graph in
the background is the axion potential output.
The final noise analysis, presented, is for a Vbias of 1.8V. As can be
observed from the plot in figure 8.12, the inter spike period, tlow , is almost
imperceptible. The Vap output has been set as a back-drop to promote this
effect.
The pre-charge effect can be easily observed in this plot by how the
membrane is prevented from reaching toward ground. For a 1.8 volt V bias ,
the membrane doesn’t go below 0.7V, while in figure 8.9 it can be observed
almost at Vgnd . The offset and pre-charge has reduced tr to 9.26h, as opposed to the 2.12% predicted. But by looking at the 62.1ns tσ and comparing that to the inter spike period, not spike interval, found by subtracting
the pulse length from the tµ of 6.71µs, we actually find a standard deviation ratio of 5.35%. This is not however a usefull figure in this design. But
it foretells a large tr were the design to be improved.
9 Implementation
In this chapter we will look at the implementation of the test circuit. The
layout process will be presented, including considerations and solutions
for an optimal representation of the test design. Some final simulations
revealing the effects of parasitic influences found in the layout will be done
before moving on to the measurements.
9.1
Layout
The layout of the neuron core, excluding the resistances and current source
are displayed in figure 9.1. The complete design can be found in appendix A
on page 65.
Figure 9.1: The layout of the core of the Noisy Integrate & Fire Neuron
For capacitances a cpoly structure was used. They consist of two layers of poly on top of each other, found in the top and bottom right of
figure 9.1. To achieve a compact layout, slight differences in the shape
of the two capacitances were used where required. This difference might
cause slight mismatch when processed, but since the overall dimensions
are quite equal, and the main consequence of a mismatch will only be a
slight adjustment to ∆Vmem , this difference should be acceptable.
48
Implementation
The resistances were, as the previous chapters revealed, chosen to be
29.5kΩ per resistor. They were implemented using high-resistive poly in
a rpolyhc structure, as shown in figure 9.2. Even though they were given
the exact same dimensions in layout and schematics a difference in the
resistance value were encountered. In the layout, cadence calculated the
resistance to be 28.4kΩ, 1.1kΩ less than in the schematics. Calculations,
according to AMS (2003), show that both these are within the bounds of
expected resistance-per-square of the process. Since the Cadence ‘Layout
versus Schematics’-test also accepted the two resistances as being equal,
no more considerations have been given to this difference, since it will be
smaller than the process variations, and only influence the noise by the
square root of the difference.
To ensure good matching of the resistances, dummy structures were set up around
them. These dummy structures consist of
high-resistive poly with the same width as
the actual resistances. The dummy is securely connected to ground, and thus giving
guarding properties against potential crosstalk noise dissipated from the noise resistances, as well as their matching capabilities.
Together with chip required components
(such as lead frames, bonding pads, etc.) the
only additions were followers at the outputs
of the neurons. The followers were designed Figure 9.2: The layout of the
by P. Häfliger, and can be found in figure A.1 noise generating resistances
on page 65. Their only function is to buffer
the output signals for proper reading off-chip.
9.1.1
Variations
To better understand the effects of the paraNoise
Soma
Variation
Res.
Cap.
meters in the design, and compensate for
Main
27.6kΩ
35.0fF
potential miscalculations or errors, several
Big Resistor
65.8kΩ
35.0fF
Small Resistor
17.5kΩ
35.0fF
variations of the design were implemented
Big Capacitance
17.5kΩ
77.6fF
in the layout. Most important was the incluNoiseless
N/A
35.0fF
Isolated
27.6kΩ
35.0fF
sion of a pair of ‘noise free’ neurons. They
are identical to the base neuron with the exTable 9.1: Neuron variations
clusion of the noise generator. This to ensure that the neuron functions properly, potentially isolating errors of the
noise generation, and, most importantly, as a reference for the noise floor
of the circuit. No matter how good the design, some noise components will
always be present. By including the ‘noise free’ it is possible to observe
which noise components are present prior to the introduction of the noise
generator, as well as better observing unintended noise additions from the
noise generator.
Other variations were also included by adjusting the noise resistances
and the capacitances of the neuron. A list of the different neurons included
9.2 Parasitic Analysis
49
can be found in table 9.1 on the facing page. All neurons, except the isolated, were included in pairs to enable the observation of decorrelation and
discovering potentially damaged neurons. Where altered, less attention was
given to the matching of resistances and capacitances in the layout. Neither
the enlarged or decreased resistances was given dummy-structure protection. And the layout of the larger capacitances took advantage of spare
room in the design. They hence became differently proportioned, and will
thus have a larger amount of mismatch. This lowering of requirements
were allowed since these neuron variations, at most, only shall be used as
references.
The isolated neuron were introduced to test the effects of mismatchnoise as a consequence of material stress, as well as influences of cross-talk
noise. It was placed approximately 600µm apart from the other neurons.
‘Silicon is a piezoresistive in that it exhibits changes in resistivity under
stress. Variations in stress across the die will produce corresponding variations in resistor matching.’ (Weste and Eshraghian, 1993)
9.2
Parasitic Analysis
The figure 9.3 shows the difference in response between the ‘noiseless’
schematics and the analog extracted view with parasitic components. Viin
was set at 2.95V while, for this first comparison, the noise generator was
turned off with a Vbias of 3.3V.
Analog−extracted
Schematic
3.3
Vmem (V)
2.475
1.65
0.825
0
10
20
30
40
50
Time (ms)
60
70
80
90
100
Figure 9.3: Transient analysis of the neuron. Comparing the results of simulation
with and without parasitic components. Viin set at 2.95V, Vbias at 3.3V, Vpl at 0.4V
and Vfolb at 0.9V.
A difference in the spike intervals, tlow , can be observed. They differ
with approximately 5%. 21.15ms for the schematic analysis, compared to
50
Implementation
22.33ms for the extracted. This is may be ascribed to the extra, parasitic,
capacitances found on the membrane of the layout, and was fully expected.
The offset error, which will be presented later, has no influence, since the
Vbias transistor is turned off. There is also a slight difference between the
pulse lengths, thigh . The schematic simulation had a pulse width of 5.57µs,
while the parasitics caused this to increase by approximately 3%, to 5.73µs.
An expected and negligible difference.
The difference in threshold was found to be only a slight reduction from
1.52V to 1.48V for the extracted analysis. A more pronounced difference
is that of ∆Vmem . For the parasitic-free analysis ∆Vmem is 1.49V, yielding
a maximum voltage of 3.02V on the membrane. The expected parasitics
reduces ∆Vmem to 1.07V. Together with the reduction of the threshold
this lowers the maximum voltage of the membrane to 2.54V, but has little
influence on the voltage of the action potential. The reduction of ∆V mem
is present because the parasitic capacitance is toward ground, and thus
contributes more to the functions of C1 , and not to the feedback of C2 .
The 21.15ms it takes the parasitic free neuron to charge from 190mV to
thr
the threshold of 1.52V should, according to equation 3.4 (Iin = (C1 +C2 ) Vtlow
),
require an input current of 4.41pA. The Iin measured during simulation is
at 5.11pA. From this the leakage currents through the pulse length regulator, the noise generator and the follower should be subtracted. An estimate for this, from the simulations, reduces the effective membrane charging
current to an average of 4.54pA. By considering that the calculations did
not include gate capacitances, imprecision in the subtraction of leakages,
which are not constant as the membrane charges, and inaccuracies in the
averaging, this estimate can be considered good enough. Using the same
approach for calculating the effective input current for the neuron including parasitics yield a 4.52pA current.
The total capacitance, intended plus parasitic, of the membrane is approximately 115fF. This increase, of 45fF over the intended, would usually
be problematic, but for a neuron, this will only increase the currents for
which it can operate. On the other hand it will probably inhibit the noise
influence slightly.
From these observations it can be concluded that, even though the parasitic capacitances are large, they do not have devastating effects on the
functionality of the neuron, and thus can be considered as acceptable imperfections.
9.3
Measurement Setup and Tools
The measurements done on the circuit is mostly concerned with the two
main neurons1 . The noiseless neurons have been used for reference, while
considerations regarding measurements on the remaining neurons can be
found in section 9.4.6 on page 57.
Specifications of the tools used during measurement will only be referred if central for the discoveries. The complete list of tools used can be
found in appendix D on page 73.
1 <C8>
and <C10> in the pin-list B.2 on page 67.
9.3 Measurement Setup and Tools
51
PCB
32
NI-DAQ
CB-68LP
14
CHIP
Connector Block
Voltage
Regulator
9V – DC
Power Supply
2
3
6
TDS 3052
Oscilloscope
Keithley 617
Electrometer
Keithley 617
Electrometer
Potentiometers
Figure 9.4: The measurement setup
The processed circuit was mounted on a PCB-board using a 100 pin
PGA socket. The approximately 9V supplied by the DC power supply was
reduced by the on-board voltage regulator to a steady operating voltage
of 3.3V. As shown in figure 9.4, potentiometers on the board were used
to regulate the biases of the circuit. The output of the potentiometers for
the input pins were further connected to electrometers for readout of the
potentials. A list of all the inputs and their respective default values can be
found in the table B.1 in the appendices.
All the relevant output pins were connected to the NI-DAQ PCI-card, in
the computer used for measuring, through a CB-68LP Connector Block. As
well as the data collected by the NIF-SAQ program (see section 9.3.2), an
oscilloscope were used to read the output of the neurons.
9.3.1
National Instruments Data Acquisition
The data acquisition PCI-card, used to collect more extended spike time
information for calculations on the rate-codes, were a National Instruments
high-speed digital I/O card with 32 digital data lines. This card, along with
corresponding software, supplies tools necessary for constructing a spike
acquisition program. The card is able to sample the data at an interval of
100ns. As a digital I/O card it poses limitations on the read-in of the soma
voltages. Voltages from ground up to 0.8V are read as low input. While the
signal is considered high when the voltage is above 2V. The intermediate
analog voltages are not read. Frequent spiking, or short pulse lengths, can
have problems crossing these limits.
9.3.2
Noisy Integrate & Fire Neuron Spike Acquisition
NIF-SAQ is the C-based program I developed for use in conjunction with NIDAQ to acquire the measurement data. It was loosely based on the ‘Poisson’
program developed by P. Häfliger (unpublished). I will only give a short
sketch of the program. For the full code refer to appendix C on page 69.
During the call to the program the form of the measurement can be
set through command-line options. This includes which neurons to read,
52
Implementation
whether the end-condition should be time, a minimum of spikes for all or
a maximum for any neuron, and the current measurement conditions.
After configuration, the NI-DAQ card reads the digital value of the designated outputs of the neuron, and stores this into a temporary buffer.
The sampling interval is the minimum 100ns. When an entire buffer is
filled it is read by the program, which compares each consecutive bit-set
for changes. When a positive change is encountered a time-stamp is stored
for that neuron. This continues until the end-condition is satisfied.
Once all the required data has been collected, the NIF-SAQ program
writes all the spike times, as well as several derived data such as the interval and frequency, to a Matlab M-file for further computations. In Matlab
several scripts were developed to import this data, calculate further derived
characteristics, comparing and plotting the results.
9.4
Collected Data
Measurements were done for values of Viin from 2.6 to 3.1V, and Vbias from
2 to 2.9V with increments of 50mV. Additional measurements with Vbias at
VDD were done for several Viin above 2.5V. A second chip was used for a
complete set of measurements from Vbias at 0 to 3.3V, 50mV intervals, with
Viin at 2.9V. Altering the chip was done to check for potential mismatches
between the individual chips.
9.4.1
Layout Error
Through the measurements a large mismatch in the results, compared to
what was expected from the simulations, were discovered. This difference
was foremost concerned with the offset current. After considering several
explanations, and a thorough search for the source of this difference, it was
discovered that the circuit had not been implemented as intended.
The followers, design by Häfliger, contains two very wide transistors,
in each follower. They were implemented through a ‘nonconventional (sic.)
MOS transistor layout’ (Hastings, 2000, page 416). ‘The waffle transistor [...]
uses a mesh of horizontal and vertical poly strips to divide the source/drain
implant into an array of squares.’ (ibid.) This is a good approach for designing compact transistors with a large width-to-length ratio. Unfortunately
this design is not recognised correctly by Cadence.
When conducting ‘layout versus schematics’ comparisons in Cadence,
the waffle transistors causes a parameter mismatch error. Expecting these
errors messages, I did not however notice, as I should have, that there were
more error messages than waffle transistors. The other errors was caused
by a factor ten mismatch on the length of the amplifier bias transistors
(Q9 in figure 7.5 on page 31), which should have been 0.7/7µm but was
0.7/0.7µm. The consequence of this mismatch was devastating on the offset current of the implemented circuit.
9.4 Collected Data
9.4.2
53
New Amplifier Characteristics
From the measurement data it can be read that the mean spike interval decreases exponentially with a decreasing Vbias for a given Viin . This is caused
by the offset current from the noise amplifier. The presence of this offset was expected, but the magnitude was much larger than the simulations
predicted. This is in line with the mismatch found on the bias transistor of
the amplifier.
5
Isolated Neuron
Main Neuron 1
Main Neuron 2
4
tµ (ms)
Figure 9.5, showing the measurement results for three of the neurons at a Viin of 2.9V,
clearly illustrates the effect the offset has on
the mean spike interval tµ . As was discussed
in section 8.3 on page 38, this offset greatly
diminishes the influence of the noise. A increased offset current has therefore a devastating effect on the findings through measurements.
3
2
1
To recalculate what would be expected
2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8
of noise presence in the measurements, we
Vbias (V)
must consider the influences of the increased
offset current. First we will take a look at
Figure 9.5: The decrease in tµ
the magnitude change of the offset, the noise as a consequence of decreasing
bandwidth of the altered amplifier, and the Vbias for Viin at 2.9V
calculated noise current. The procedure to
reach these, and later derived results, are the same as used in chapter 8 on
page 35, no intermediate results will therefore be presented.
The first sub-figure of figure 9.6 illustrates the change in the actual offset current. The difference is an increase of a factor 9 in average. For Vmem
close to the threshold, the increase is slightly less, only approximately a
factor 5. Even before any further calculations are done, the immense influence of such a change is obvious.
50
80
Implemented
Intended
Implemented
Intended
1.5
40
60
50
30
In (nA)
o
I (µA)
∆f (MHz)
1
Implemented
Intended
70
40
20
30
0.5
20
10
10
0
0
0.825
1.65
Vbias (V)
2.475
(a) The offset with erroneous transistor length,
compared to the intended offset
3.3
0
0.825
1.65
V
(V)
2.475
bias
(b) Same comparison as
for 9.6(a), but for the
noise bandwidth
3.3
0
0.825
V
1.65
(V)
2.475
3.3
bias
(c) Ina calculated from
the Eti and Gra found
through the new ∆f
Figure 9.6: Plots illustrating the change in transfer and noise characteristics as a
consequence of the mismatch
54
Implementation
On the positive side, the noise bandwidth has also increased, slightly.
But remembering, from chapter 8, that the highest frequency components
do not influence the membrane. We can conclude that, unfortunately, this
positive change can probably be disregarded.
Finally, we come to the new noise current, found in figure 9.6(c), which
shows a great increase for all values of Vbias . Alas, again, this optimistic
observation can be discounted, since the deviation of the membrane will
be countered by the immense increase in the offset current. But from figure 8.8 on page 42, depicting tr in the intended design, we could observe
that there is a slight window, for the Vbias , where the noise influence precedes the offset. If this window can be found in the measurements, some
of the introduced noise might be discerned.
Noise Prediction
Applying the methods for calculations of the
noise, discovered in section 8.3.3 on page 41,
which we found to be satisfyingly accurate,
we reach the results depicted in figure 9.7.
Even though the noise current, In , increased
for this ill-matched implementation, the offset was increased further, and thus marginalising the noise influence.
The peak noise, found for a Vbias of 2.3V,
is no more than 3.1h. Almost indistinguishable if the noise floor was at ground. But as
the next section will reveal, the noise floor,
from which to distinguish this noise influence, is all but at ground making the intended noise difficult to observe.
9.4.4
3
Implemented
Intended
2
tσ (%)
9.4.3
1
0
0.825
1.65
2.475
Vbias (V)
3.3
Figure 9.7: Comparing the
noise influence of the processed
chip, with the intended neuron
Other Noise Sources
There will always be several noise components present in a processed aVLSI
chip. Ideally these should be as small as possible. Both on-chip noise
sources, noise sensitive components and pick-up of off-chip noise should
be well guarded, and the design and layout should accommodate this to
the furthest extent. Unfortunately some noise will always remain, as was
the case also in this chip.
Noise Floor
It was early discovered through the measurements that there existed a
50Hz noise component which was quite overwhelming for the circuit. This
can quite certainly be ascribed to pick-up from the main power supply and
grid. This, along with several other, smaller and unidentified noise sources
constitutes the noise floor off the circuit under measurement. This noise
floor varies with external conditions during the time of measurement, and
9.4 Collected Data
55
thus no exact universal value can be found for the level of the noise floor.
But readouts from the power spectrum of the circuit and comparisons with
the noise free neurons gave an estimate for this noise floor to yield an approximately 5.5h tr . Slight variations from this, due to variations in the
input currents, was found, so this noise floor should only be considered an
estimated for reference.
Sampling Errors
In the data collected by NIF-SAQ, an unexpected increase in the mean spike
interval of several neurons can be found for Viin approaching 2.6V and Vbias
approaching 2V. Also an increase in the standard deviation can be found
for small Viin independent of Vbias . The last tendency is most pronounced
for the ‘noiseless’ neurons.
Neither of these observations are actually occurring in the neuron, but is
caused by sampling errors of the NIF-SAQ program. The program is limited
by the maximum sampling rate of the NI-DAQ card. There are two errors
which might result of this.
(V)
Unregistered Spikes An increase in mean spike interval can be found when
the inter-spike period, the time between the spikes as opposed to the spike
interval which is from the start of one to the start of the next spike, becomes too small to register properly. At this point sampling errors occur
because not necessarily all low outputs are registered and two or more consecutive spikes can be registered as one. This will also lead to a erroneous
increase in the standard deviation since it is haphazard which spikes are
registered.
Since Vpl is set to 0.4V, the pulse length
2.5
should be approximately 5.8µs according to
simulations. Measurements done on the ac2
tual neurons reveal the pulse length to vary
1.5
from 4.3 to 5.3µs. Therefore a spike interval
(tµ ) of 5.4µs, leaving an inter spike period
1
(tlow ) of more than 100ns, would seem sufficient to avoid sampling errors. But most of
0.5
the outputs of the chip are not of the action
potentials, but the membrane potentials of
0
0 5 10 15 20 25 30 35 40 45 50
the neurons. The NI-DAQ card, designed for
Time (µs)
digital measurements, has an upper measurement limit for low input of 0.8V. It is thus Figure 9.8: Oscilloscope plot
only the inter-spike period of the membrane of a main neuron with Viin at
below 0.8V, which is illustrated in the oscil- 2.9V, and Vbias at 2.3V. Input
loscope plot in figure 9.8, that are possible to low and high voltage for the
measure accurately. This period only consti- NI-DAQ marked.
tutes 2-4% of the full inter-spike periode, disappearing altogether for interspike periods of approximately 1.7µs and less. From this we can conclude
that, a guarantee for no sampling errors, caused by short inter-spike periods, can only be given, without further enquiry, for spike intervals of more
than 10.5µs. Subtracting the largest pulse length of 5.3µs from 10.5µs
yields an inter-spike period of 5.2µs. Of this 2%, the least periode below
56
Implementation
0.8V, is 104ns which is sufficient for sampling. Shorter spike intervals
can be measured correctly, but should be checked to ensure that correct
sampling is possible.
The same problem could occur for reading the spikes, if Vpl were decreased. The lower limit for high inputs are 2V on the NI-DAQ card. If the
pulse length would become too short, enough to let the period above 2V
fall below 100ns, not necessarily all spikes would be read, and the mean
interval would increase.
Imprecise Spike Timing The precision of the spike timing is limited by the
sampling rate of the NI-DAQ-card. The highest precision possible is given
by the 100ns sample rate. When precision required for correct calculations
goes beyond 100ns the results from the NI-DAQ-card will be insufficient.
Under normal circumstances, and with a correctly functioning neuron, timings below micro seconds are irrelevant. But with an increasingly large
offset current, measurements below Vbias at approximately 2.4V becomes
inaccurate on this test circuit. If the sampling rate becomes too crisp to get
a very precise mean spike interval, this will become even more apparent on
the standard deviation.
This error is easy to estimate. By dividing the sample rate by the mean
spike interval we get the average imprecision caused by the sampling. In
other words, for a 1µs spike interval we can expect a 10% error of the exact
timing.
9.4.5
Measured Noise
2.2
Neuron 1
Neuron 2
2
1.8
V
mem
(V)
1.6
1.4
1.2
1
0.8
0.6
0.4
0
5
10
15
20
Time (ms)
25
30
35
40
Figure 9.9: An oscilloscope plot with both Vbias and Viin set to 2.9V. The resolution
prevents the actual spikes from being displayed properly.
As mentioned in the introduction of this chapter, measurements were done
for a variety of Viin and Vbias . Because of the lack of any conclusive noise observations due to the erroneous offset characteristics, no thorough presentation of all the measurement data should be necessary. But a short presentation of measurements for one neuron for Viin at 2.9V, and Vbias from 2.1
9.4 Collected Data
57
to 2.9V will be given. This to show the effects of the unintended noise
components and the offset current.
1m
tµ log(s)
Figure 9.10 depicts the mean spike interval, tµ , for this neuron. It has been given
a logarithmic y-axis to display the full variation of tr for different biases. As found
in the previous section, a danger of unregistered spikes will occur when tµ falls below, approximately, 10µs. This can be seen
in the plot at Vbias slightly above 2.3V. From
this we can expect a severe increase in tr for
lower Vbias ’s.
.1m
10µ
2.1
2.3
V
2.5
(V)
2.7
2.9
The actual, intended, noise will be even
more disturbed by the noise floor and the
Figure 9.10: tµ of a single
sampling error from imprecise spike timings. neuron for V at 2.9V
iin
The influence of these is plotted, together
with the measured tr , in figure 9.11. The second curve depicts the noise
floor, and the level of sampling error rising above this. The sampling error
were, as suggested earlier, found by dividing the sample rate by tµ . Furthermore it can be observed that tr as predicted increases drastically when
Vbias reaches 2.3V.
bias
9.4.6
5
tr of Neuron 1
Noise floor and
sampling error
4
3
r
t (%)
The only, so far, unexplained noise can
be found around Vbias of 2.7V. At the peak
this constitutes a 0.8h tr above the noise
floor. Similar peaks were found in some of
the other measurement sets as well. This is
a small noise component, and the observation data of it is quite inconclusive. It can
be ascribed to imprecisions in the measurements and calculations, or as external noise
influences. Furthermore the fact that it occurs at, and around, a Vbias of 2.7V where,
according to prior calculations and simulations, little or no measurable noise influence
should be present, speaks against it. But,
then again, it might actually be some fragment of the intended noise introduce in the
design.
2
1
2.1
2.3
2.5
Vbias (V)
2.7
2.9
Figure 9.11: tr of a single
neuron for Viin at 2.9V, together with the noise floor and
sampling error
Variations
No new information could be derived from the results from measurements
on the neuron variations implemented on the chip. The increased resistors,
a factor 2.4 larger than the original, will only yield a 50% increase on the
noise. The virtues of a 4.5h tr is not much different from that of a 3h tr .
58
9.5
Implementation
Discoveries
The offset current from the noise generator overwhelms the actual noise
and input currents when Vbias passes the threshold voltage of the bias transistor. This prevents the neuron from spiking at a satisfactory rate once
the noise generator is activated. The increased rate becomes too fast for
proper measurement and noise influence, and is the main cause of the lack
of any conclusive noise data. Which possible measurements can be used
to correct this will be discussed in the concluding chapter 10 on the next
page.
10 Conclusions and Outlook
After conducting an examination of the possibility of introducing noise on
the Integrate & Fire Neuron design, to deccorelate the outputs of a population of neurons, some concluding considerations have been found.
10.1
Improvements
As have been shown in chapter 8 and 9 there are several shortcomings in
the test circuit. To accommodate this several improvements which can be
introduced in the design will be presented.
10.1.1
Improved Amplifier
To achieve more noise influence, the transconductance of the amplifier
could be increased. But a better alternative is to reduce the offset. By
reducing the offset current, both negative and positive, the Vbias values for
which the neuron functions properly and the noise influence is at its full
potential, will increase. This is not just an improvement, but a requirement
if the information carried in the neurons spike-timing should be preserved.
Neither the bandwidth nor the transcon200
V = 3.05V
V = 3.0V
ductance is as important as first perceived.
V = 2.95V
V = 2.9V
The only requirement of the noise bandwidth
150
is that it is equal to or larger than the spike
frequency of the neuron. The best solution is
100
that the bandwidth is equal to or larger than
the highest frequency of the neuron to use
50
the full potential of the transconductance.
The transconductance is actually more than
sufficient in the design used in this thesis,
2
2.2
2.6
2.8
Vbias (V)
as figure 10.1 shows. The plot illustrates the
calculated tr with Io = 0. This, of course,
is only a theoretical illustration. A zero off- Figure 10.1: A more ideal
set is practically impossible to achieve, and noise amplification
a standard deviation of the spike times above 100% is self-contradictory 1 .
There are several approaches to reducing the offset. Some of whom are:
iin
iin
iin
tσ (%)
iin
• Optimising the transconductance amplifiers parameter matching.
• Introducing a filter or other offset regulating components between the
amplifier and neuron.
• Adding a second amplifier stage.
• Changing to an altogether different amplifier design.
1 For
an early spike, that would require the next spike to occur before the current spike,
and is thus impossible per definition.
60
10.1.2
Conclusions and Outlook
Removing Other Noise Components
Unintended noise on the chip is always a problem. The noise found through
sampling errors is not a noise found in the neuron, but in the measurement.
For the measurement setup used on this chip, the sampling errors would
to a large extent disappear if the offset was reduced. The 50Hz noise is
an altogether more severe problem. The possibility that it is introduced
through the input and output of the circuit makes it less relevant for a
fully integrated implementation. But if it is picked up on the chip itself,
possibly by the large resistors or capacitances, some form of regulation
would be appropriate.
To achieve good operations, countermeasures to the 50Hz AC-frequency
influence must be deployed. Since this 50Hz signal is in the frequency band
at which the neuron ideally should operate, an introduction of a band-stop
(notch) filter is inadvisable since it would inhibit the actual signal. This,
unless the component which picks up the 50Hz noise can be located, and
it is such that a notch-filter can be deployed to prevent it from propagating
the noise to other components. But designing a good compact filter, which
only removes the 50Hz component is not an easy task.
10.2
Additional Components
If a leakage control is introduced on the neuron, it is important to ensure
that it does not inhibit the noise influence by feeding it directly to ground.
But allows the noise to influence the input currents charge of the membrane, and then leak the membrane potential to ground.
Any threshold regulation introduced must be sensitive enough to register the possibly high frequency threshold crossing the noise can introduce. The bandwidth of the threshold regulator must be such, that when
the membrane crosses the threshold, it must not be allowed to cross it
again in the opposite direction directly subsequently, even if the membrane
is above for only a few nano seconds.
There would, probably, arise no problems for the noise influence by
introducing either an adaptive fire frequency or a refractory control. That
is if this components are introduced correctly. A refractory control which
allows a charge to load at the input would, on the other hand, reduce the
possible noise influence.
10.3
Alternative Approaches
It would be unwise to suggest that a threshold noise is easier to implement.
It does remove one of the major obstacles encountered by the current noise
generator, the offset current. Since it is not connected to the membrane
in any way, it can not introduce any offset or leakage to the membrane
potential or current charging it. But, since a thorough examination has not
been conducted, other problems which might arise with a noisy threshold
has not yet been discovered. This said, it is a potentially good alternative
to the current noise.
10.4 Concluding Remarks
10.4
61
Concluding Remarks
The amplifier implemented in this design is less than satisfactory, and
should be improved or replaced in any further work with any similar design.
An unfortunate error in the layout prevented any conclusive measurement
results to be established.
On the other hand, it has been established, in this thesis, that the
principle of introducing decorrelating noise, by generating a noise current
through amplification of the thermal noise found in resistances, should in
theory be feasible. The basic method of achieving this along with possible
approaches to calculating the expected influences of an implementation
have been presented, and seems accurate according to simulations.
Appendixes
A Layout
Figure A.1: Layout of the complete neuron. Including resistances, current source
and followers.
B Pin allocations
Name
Pin
Adjust
Default
Kind
Viin
Vbias
Vpl
Vis
Vnoise
Vfolb
A<7>
A<8>
A<11>
C<14>
C<15>
C<16>
A<7>
A<8>
A<11>
A<0>
A<3>
A<13>
Variable
Variable
0.4V
3.3V
1.65V
0.9V
pMOS, Input Current Gate
pMOS, Noise Amplifier Bias
nMOS, Pulse Length
source, Input Current Source
pMOS, Noise Source
pMOS, Follower Bias
Table B.1: Pin Assignment — Input
Name
Pin
Vspike
Vsoma
1a
1b
2a
2b
3
4
5
6
7
8
9
10
C<12>
C<13>
C<9>
C<8>
C<11>
C<10>
C<7>
C<6>
C<5>
C<4>
C<3>
C<2>
C<1>
C<0>
Noise
Resistance
27.6kΩ
27.6kΩ
27.6kΩ
27.6kΩ
27.6kΩ
27.6kΩ
65.8kΩ
65.8kΩ
17.5kΩ
17.5kΩ
17.5kΩ
17.5kΩ
N/A
N/A
Membrane
Capacitance
35.1fF
35.1fF
35.1fF
35.1fF
35.1fF
35.1fF
35.1fF
35.1fF
35.1fF
35.1fF
77.6fF
77.6fF
35.1fF
35.1fF
Kind
Spike, Isolated Neuron
Soma, Isolated Neuron
Spike, Main Neuron
Soma, Main Neuron
Spike, Main Neuron
Soma, Main Neuron
Soma, Big Resistor
Soma, Big Resistor
Soma, Small Resistor
Soma, Small Resistor
Soma, Small Res, Big Cap
Soma, Small Res, Big Cap
Soma, Noise Free Neuron
Soma, Noise Free Neuron
Table B.2: Pin Assignment — Output
C NIF-SAQ
#include
#include
#include
#include
#include
#define
#define
#define
#define
#define
#define
#define
#define
#define
#define
#define
#define
#define
<string.h>
<stdio.h>
<nidaq.h>
<nidaqex.h>
<time.h>
DEVICE (1)
// Defined by MAX (an NI application)
BUFFER_SIZE (131072)
// Buffer size
GROUP (2)
//
TIME_BASE (-3)
// 20MHz, 50ns
REQ_INTERVAL (2)
// In units of 50ns (defined by TIME_BASE)
CONFIG (1)
// Pattern generation with request-edge latching
REQ_SOURCE (0)
// Internal. From onboard counters.
GROUP_SIZE (2)
// 2 bytes
PORT (2)
// Ports C and D. (DIOC 0-7 and DIOD 0-7)
DIRECTION (0)
// Input
DB_MODE (1)
// Enable double buffering
OLD_DATA_STOP (1)
// No overwrit of buffer allowed when overrun
SAMPLING_INTERVAL (50e-9*REQ_INTERVAL)
FILE *outputFile = stdout;
// Data write to...
long **timeStamps;
// Pointers to pointers
int waiting;
// Clear before wait
int buffsMax, eventsMax, eventsMin;
// Measurment limits
int stopOn, inputs = 1;
// Stop on events Min or Max. Default 1 imput
int events[16];
// Events read per input
int bit[17] = {1,2,4,8,16,32,64,128,256,512,1024,2048,4096,8192,16384,32768};
/********************************************************************************/
// ERROR - Printing error messages and exiting program after clean up.
/********************************************************************************/
void error(int e){
switch (e){
case 1: printf("Malloc failed.\n"); break;
case 2: printf("DIG_Block_In failed.\n"); break;
case 3: printf("DIG_DB_Transfer failed.\n"); break;
default: printf("Unspecified error!\n");
}
free(*timeStamps);
fclose(outputFile);
exit(e);
}
/********************************************************************************/
// ILLEGAL - Handling illegal execution parameters.
/********************************************************************************/
void illegal(int arg){
printf("Illegal or missing parameter for argument -%c.\n\n", arg);
printf("Usage: nif_saq [-e <int>][-m <int>][-n <int>][-t <int>][-i <int>]\n");
printf("
[-f <filename>][-v <float>[<float>...<float>]][-h]\n\n");
exit(0);
}
/********************************************************************************/
// HELP - Displaying the -h help message.
/********************************************************************************/
void help(){
printf("\nNIF_SAQ - Noisy Integrate & Fire Neuron Spike Aquisition\n\n");
printf("Usage: nif_saq [-e <int>][-m <int>][-n <int>][-t <int>][-i <int>]\n");
70
NIF-SAQ
printf("
printf("
printf("
printf("
printf("
printf("
printf("
printf("
printf("
printf("
exit(0);
-e
-m
-n
-t
-i
-f
-v
-h
[-f <filename>][-v <float>[<float>...<float>]][-h]\n\n");
Stop on min (0) or max (1).\n");
Min number of spikes to read from a single input.\n");
Max number of spikes to read from a single input.\n");
Max time in seconds before ending measurement.\n");
Number of input to be read. Starting at DIOC 0.\n");
Output file, Matlab formated. Default stdout.\n");
Voltages at Viin, Vbias, Vpl, Vis, Vnoise and Vfolb.\n");
To be recorded in file for refference.\n");
Displays this help message.\n");
}
/********************************************************************************/
// READ ARGUMETS - Reading input arguments and assigning them to variables.
/********************************************************************************/
void readArgs(int argc, char *argv[], float volt[]){
char str[128];
// For reading file name
int i, j, time, arg;
// Incrementals. Time in sec. Argument name.
for (i = 1; i < argc; i++){
// For all input options and arguments...
arg = argv[i][1];
// Removing the ’-’
if(arg == ’h’) help();
if(argc <= ++i) illegal(arg); // If no argument to option, complain
switch(arg){
case ’e’:
// Stop on min or max?
if (sscanf(argv[i], "%i", &stopOn) != 1) illegal(arg);
break;
case ’n’:
// Max events to read
if (sscanf(argv[i], "%i", &eventsMax) != 1) illegal(arg);
break;
case ’m’:
// Min events to read
if (sscanf(argv[i], "%i", &eventsMin) != 1) illegal(arg);
break;
case ’t’:
// Max time in seconds
if (sscanf(argv[i], "%i", &time) != 1) illegal(arg);
buffsMax = (int)(time / (SAMPLING_INTERVAL * BUFFER_SIZE/2));
break;
case ’i’:
// Number of inputs
if (sscanf(argv[i], "%i", &inputs) != 1) illegal(arg);
break;
case ’f’:
// File to write data
if (sscanf(argv[i], "%128s", str) != 1) illegal(arg);
outputFile = fopen(str, "w");
break;
case ’v’:
// Voltages for refference
if (sscanf(argv[i], "%f", &volt[0]) != 1) illegal(arg);
for (j = 1; j <= 5; j++){
if(argc <= ++i) break;
if (sscanf(argv[i], "%f", &volt[j]) != 1) i--;
}
break;
default:
// None of the above
printf("Unrecognised argument -%c\n", arg);
exit(0);
}
}
}
/********************************************************************************/
// READ SPIKES - Reading the inputbuffers from the NI-DAQ card.
/********************************************************************************/
int readSpikes(i16 *halfBuffer, int buffsRead, long **timeStamps, int eventsRead){
int i, j;
for (i = 0; i < (BUFFER_SIZE/2); i++){
// For each row in input buffer...
if (halfBuffer[i] != halfBuffer[i-1]){// If there have been a change...
for(j = 0; j < inputs; j++){
// For each input...
if(events[j] < eventsMax){
// If input haven’t had max events..
if(waiting & halfBuffer[i] & bit[j]){
// Waiting,not in vain
timeStamps[j][events[j]] = i + buffsRead * BUFFER_SIZE/2;
if(++events[j] > eventsRead) eventsRead = events[j];
waiting = waiting ^ bit[j];
// Not waiting anymore
}else if(!((waiting & bit[j]) | (halfBuffer[i] & bit[j]))){
71
waiting = waiting | bit[j]; // Not waiting, set wait
}
}
}
}
}
return(eventsRead);
}
/********************************************************************************/
// CONFIGURE DAQ
/********************************************************************************/
void configure_daq(i16 *buffer){
// Configures the group for port assignment, direction, and size.
DIG_Grp_Config(DEVICE, GROUP, GROUP_SIZE, PORT, DIRECTION);
// Sets the pattern generation mode.
DIG_Block_PG_Config(DEVICE,GROUP,CONFIG,REQ_SOURCE,TIME_BASE,REQ_INTERVAL, 0);
// Sets double-buffered digital transfer operations and options.
DIG_DB_Config(DEVICE, GROUP, DB_MODE, OLD_DATA_STOP, 0);
// Initiates transfer of data to memory or return error.
if (0 != (DIG_Block_In(DEVICE, GROUP, buffer, BUFFER_SIZE))){
error(2);
}
}
/********************************************************************************/
// MAIN
/********************************************************************************/
int main(int argc, char *argv[]){
HANDLE thisProcess;
u16 buffer[BUFFER_SIZE], halfBuffer[BUFFER_SIZE/2];
// For reading buffers
int buffsRead = 0, eventsRead = 0, eventsStop = 0;
int eventsReadMax = 0, eventsReadMin = 0;
// Most & least events read
int i, j, r = 0;
// Incrementals
float volt[6] = {5,5,5,(float)3.3,(float)1.65,(float)0.9}; // Default voltage
/***** Setup *****/
thisProcess = GetCurrentProcess();
// Give Program Highest Priority
SetPriorityClass(thisProcess, REALTIME_PRIORITY_CLASS);
readArgs(argc, argv, volt);
// Read Arguments
// Allocate memory
if (NULL == (timeStamps = (long **)malloc(inputs * sizeof(long *)))) error(1);
if (NULL == (timeStamps[0] = (long *)malloc(inputs*eventsMax*sizeof(long)))){
error(1);
}
for(i = 1; i <= inputs; i++) timeStamps[i] = timeStamps[i-1] + eventsMax;
configure_daq(buffer);
// Configuring NI-DAQ Card
/***** Reading Spikes *****/
printf("Reading data...\n");
if (stopOn == 0) eventsStop = eventsMin;
// Setting appropriate stop event
else eventsStop = eventsMax;
while ((eventsRead<eventsStop)&&(buffsRead<buffsMax)) { // Not events or time
if (0 == (DIG_DB_Transfer(DEVICE,GROUP,halfBuffer,BUFFER_SIZE/2))){ // Get
eventsReadMax =readSpikes(halfBuffer,buffsRead,timeStamps,eventsRead);
eventsReadMin = eventsReadMax;
// Find least read events
for(i = 0; i < inputs; i++){
if(events[i] < eventsReadMin) eventsReadMin = events[i];
}
if (stopOn == 0) eventsRead = eventsReadMin; // Stop on min or max?
else eventsRead = eventsReadMax;
if(eventsRead >= ((eventsStop/14)*r)) printf("*", r++);// Progress bar
buffsRead++;
}else{
error(3);
}
}
/***** Writing results to file *****/
printf("\nWriting data...\n");
72
NIF-SAQ
fprintf(outputFile, "freq = [");
for (j = 0; j < inputs; j++)
if(events[j] != 0) fprintf(outputFile, "%f ",
((float)events[j] / ((float)timeStamps[j][events[j] - 1]\
* SAMPLING_INTERVAL)));
else fprintf(outputFile, "0 ");
fprintf(outputFile, "];\n");
if(outputFile != stdout){
fprintf(outputFile, "events = [");
for(i = 0; i < inputs; i++) fprintf(outputFile, "%i ", events[i]);
fprintf(outputFile, "];\n\n");
fprintf(outputFile, "inputs = %i;\n", inputs);
fprintf(outputFile, "sampling = %e;\n", SAMPLING_INTERVAL);
fprintf(outputFile, "eventsMin = %i;\n", eventsReadMin);
fprintf(outputFile, "voltages = [");
for(i = 0; i < 6; i++) fprintf(outputFile, "%5.4f ", volt[i]);
fprintf(outputFile, "];\n\n");
for (j = 0; j < inputs; j++){
if (events[j] != 0){
fprintf(outputFile, "s%2.2i = [", j);
fprintf(outputFile, "%8u*%e", timeStamps[j][0],SAMPLING_INTERVAL);
for (i = 1; i < events[j]; i++){
fprintf(outputFile, "; %8u*%e", \
timeStamps[j][i], SAMPLING_INTERVAL);
}
fprintf(outputFile, "];\n");
fprintf(outputFile, "i%2.2i = s%2.2i(2:size(s%2.2i,1),:)", j,j,j);
fprintf(outputFile, "-s%2.2i(1:(size(s%2.2i,1)-1),:);\n", j, j);
fprintf(outputFile, "o%2.2i = std(i%2.2i);\n", j, j);
fprintf(outputFile, "m%2.2i = mean(i%2.2i);\n", j, j);
}
}
if(eventsReadMin != 0){
fprintf(outputFile, "spikes = [...\n");
for (i = 0; i < eventsReadMin; i++){
for (j = 0; j < inputs; j++){
fprintf(outputFile, "%8u*%e ", \
timeStamps[j][i], SAMPLING_INTERVAL);
}
fprintf(outputFile, "\n");
}
fprintf(outputFile, "];\n");
fprintf(outputFile, "intervals = spikes(2:size(spikes,1),:)");
fprintf(Outputfile, "-spikes(1:(size(spikes,1)-1),:);\n");
fprintf(outputFile, "sigma = std(intervals);\n");
fprintf(outputFile, "mu = mean(intervals);\n");
}
}
printf("\n\n
%5i inputs\n", inputs);
printf("%5i/%5i eventsMax\n", eventsReadMax, eventsMax);
printf("%5i/%5i eventsMin\n", eventsReadMin, eventsMin);
printf("%5i/%5i cycles\n", buffsRead, buffsMax);
/***** Cleaning up *****/
free(*timeStamps);
fclose(outputFile);
yreturn(0);
}
D Applications and Tools
Applications
Cadence Design and simulation of the circuit.
Matlab Calculations and graphs in the report.
NI-DAQ Interface to the data acquisition card.
NIF-SAQ Self-written C-program for reading the NI-DAQ results.
Microsoft Visual Studio 6.0 Writing the NIF-SAQ program.
Emacs Writing report and Matlab functions.
LATEX Typesetting and page layout of the report.
Adobe Illustrator Illustrations and circuit schemas in the report.
Measurement tools
Keithley 617 & 6512 Programmable electrometer.
Hewlett Packard E3611A DC power supply.
Tektronix TDS3052 Two channel color digital phosphor oscilloscope.
DAQ PCI-6534 National Instruments high-speed digital I/O device.
CB-68LP NI-DAQ connector block.
Multimeter
Fonts
This report and the figures found herein are typeset with:
• Lucida Bright, Lucida Sans and Lucida Sans Typewriter from Linotype
by Charles Bigelow and Kris Holmes.
• Helvetica from Linotype by Max Miedinger
• Glyphs from unknown and unnamed founts may be present
List of Parameters
Av
Avo
Voltage gain . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Peak voltage gain . . . . . . . . . . . . . . . . . . . . . . . . .
23
23
βn
βp
BW
Mobility of nMOS transistors . . . . . . . . . . . . . . . . .
Mobility of pMOS transistors . . . . . . . . . . . . . . . . .
Bandwidth . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
28
75
∆f
Noise bandwidth . . . . . . . . . . . . . . . . . . . . . . . . . 33, 14
∆Vmem Increase of Vmem upon reaching Vthr . . . . . . . . . . . . .
12
Et
E ta
E tf
E ti
RMS thermal noise voltage .
Average Et over Vload . . . .
Et for a single frequency . .
Total thermal noise input of
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
q
amplifier Et2 + Et2
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. 37, 14
.
37
.
37
. 40, 21
Vgnd
Gm
Gm a
Gm f
Gr
Gr a
Ground . . . . . . . . . . . .
Transconductance . . . . .
Average Gm over Vload . . .
Gm for a single frequency
Average Gm over ∆f . . . .
Average Gr over Vload . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
25
. 32, 22
.
75
.
37
.
38
.
37
Ib
ID
IDC
IF
Iin
Iine
In
I na
I ne
I nf
Ing
Io
I oa
Ipl
IR
IS
Ish
Bias current of transconductance amplifier
Drain current of a transistor . . . . . . . . . .
Direct current in amperes . . . . . . . . . . .
Forward current of transistor . . . . . . . . .
Input current of the neuron . . . . . . . . . .
Effective input current Iin + Io . . . . . . . . .
Noise current generated by the amplifier . .
Noise current generated by the amplifier . .
Effective input current Iin + Io . . . . . . . . .
Noise current generated by the amplifier . .
Gate noise current generator . . . . . . . . .
Offset current of the noise generator . . . .
Average offset current . . . . . . . . . . . . .
Pulse length regulating current . . . . . . . .
Reverse current of a transistor . . . . . . . .
Source current of a transistor . . . . . . . . .
Shot noise current . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
31, 75
26
14
26
10, 12
41
31, 22
37
75
37
14
53, 41
41
10, 12
26
26
14
σ
Standard Deviation . . . . . . . . . . . . . . . . . . . . . . .
39
76
Applications and Tools
thigh
tlow
tµ
tr
t rc
tσ
Duration of the spike of a neuron . . . . . . . . . .
Inter spike periode of a neuron . . . . . . . . . . . .
Average spike interval . . . . . . . . . . . . . . . . .
Ratio of tσ /tµ . . . . . . . . . . . . . . . . . . . . . . .
Ratio of tσ /tµ from calculations . . . . . . . . . . .
Temporal standard deviation of the spike interval
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
12
.
12
. 53, 22
. 57, 22
.
43
.
22
Vap
Vb
Vbias
VDD
Vds
VD
Vfolb
Vgs
VG
Viin
Vis
Vlk
Vload
Vmem
V−
Vnoise
Vout
V+
Vpl
Vrfd
Vrfs
VS
Vsoma
Vthr
The readout of the action potential. . . . . . . .
Bias regulating pulse length of neuron . . . . .
Bias voltage of the transconductance amplifier
Global source . . . . . . . . . . . . . . . . . . . . .
Drain-Source voltage of a transistor . . . . . . .
Drain voltage of a transistor . . . . . . . . . . . .
Bias voltage of the output followers . . . . . . .
Gate-source voltage . . . . . . . . . . . . . . . . .
Gate voltage . . . . . . . . . . . . . . . . . . . . . .
Bias regulating input current of neuron . . . . .
Input current source of neuron . . . . . . . . . .
Bias regulating leakage of neuron . . . . . . . .
Load voltage . . . . . . . . . . . . . . . . . . . . . .
The membrane potential . . . . . . . . . . . . . .
Negative input of amplifier . . . . . . . . . . . .
Noise reference voltage . . . . . . . . . . . . . . .
Output voltage . . . . . . . . . . . . . . . . . . . .
Positive input of amplifier . . . . . . . . . . . . .
Bias regulating pulse length of neuron . . . . .
Refractory drain voltage . . . . . . . . . . . . . .
Refractory source voltage . . . . . . . . . . . . .
Source voltage of transitor . . . . . . . . . . . . .
The readout of the membrane potential . . . .
The threshold voltage of the neuron . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
11, 35
10, 9
21, 33
9
76
26
67
76
26
26, 26
67
11, 10
32, 31
11, 9
31, 23
21, 21
9, 76
31, 23
10, 28
11, 11
11, 11
26
76
11, 10
List of Figures
1.1
1.2
1.3
1.4
Conceptual model of a neuron. Adapted from Mead (1989) and Katz (1966)
Hodgkin-Huxley model. Adapted from Gerstner and Kistler (2002) . . . . .
Lapicques original I&F-Neuron. Adapted from Abbott (1999) . . . . . .
Addresse-Event Representation. Adapted from Mahowald (1992) . . . .
2
3
3
5
3.1 Carver Meads basic Integrate & Fire Neuron . . . . . . . . . . . .
9
3.2 Carver Meads self-resetting neuron . . . . . . . . . . . . . . . . . 10
3.3 An extended Integrate & Fire Neuron . . . . . . . . . . . . . . . . 11
6.1 Basic sketch of a threshold noise generator . . . . . . . . . . . . 19
6.2 Noise generator with the bias directly to + . . . . . . . . . . . . . 21
6.3 Noise generator with two resistances . . . . . . . . . . . . . . . . 21
7.1
7.2
7.3
7.4
7.5
7.6
7.7
Schematics of the Integrate & Fire Neuron . . . . . . . . . . . .
Inverters transfer function . . . . . . . . . . . . . . . . . . . . . .
Input DC-characteristics of noise-free neuron . . . . . . . . . .
Transient analysis of the neuron prior to noise or parasitics .
The transconductance amplifier . . . . . . . . . . . . . . . . . .
Transconductance for amplifier orientations . . . . . . . . . .
Transfer characteristics of the noise generator . . . . . . . . .
8.1 The complete Noisy Integrate & Fire Neuron . . . . . . .
8.2 Transient analysis of the complete neuron . . . . . . . .
8.3 Derived noise amplifier characteristics . . . . . . . . . .
8.4 Noise analysis setup for the amplifier . . . . . . . . . . .
8.5 Noise analysis of amplifier . . . . . . . . . . . . . . . . . .
8.6 Noise analysis setup for the complete neuron . . . . . .
8.7 Offset simulation results . . . . . . . . . . . . . . . . . . .
8.8 The expected noise influence according to calculations
8.9 Neuron noise analysis Vbias at 2.7V . . . . . . . . . . . .
8.10 Neuron noise analysis Vbias at 2.4V . . . . . . . . . . . .
8.11 Neuron noise analysis Vbias at 2.1V . . . . . . . . . . . .
8.12 Neuron noise analysis Vbias at 1.8V . . . . . . . . . . . .
9.1
9.2
9.3
9.4
9.5
9.6
9.7
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
26
28
29
30
31
32
33
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
35
36
37
40
40
41
41
42
43
44
44
45
The layout of the core of the Noisy Integrate & Fire Neuron
The layout of the noise generating resistances . . . . . . . .
Transient analysis comparing with parasitics . . . . . . . . .
The measurement setup . . . . . . . . . . . . . . . . . . . . . .
Offsets influence on tµ . . . . . . . . . . . . . . . . . . . . . . .
Mismatch transfer functions . . . . . . . . . . . . . . . . . . .
Inteded noise versus actual . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
47
48
49
51
53
53
54
78
List of Figures
9.8 Oscilloscope plot of a main neuron . . . . . . . . . . . . .
9.9 Another oscilloscope plot of a main neuron . . . . . . . .
9.10 Measured mean spike interval . . . . . . . . . . . . . . . . .
9.11 Measured deviation ratio, noise floor and sampling error
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
55
56
57
57
10.1 A more ideal noise amplification . . . . . . . . . . . . . . . . . . . 59
A.1 Layout of the complete neuron . . . . . . . . . . . . . . . . . . . . 65
List of Tables
7.1 Noise-free neuron characteristics . . . . . . . . . . . . . . . . . . . 30
7.2 Amplifier transfer characteristics . . . . . . . . . . . . . . . . . . 33
8.1
8.2
8.3
8.4
8.5
Component parameters of the complete neuron
Neuron characteristics . . . . . . . . . . . . . . . .
Derived noise amplifier characteristics . . . . . .
Simulation results of the noise amplifier . . . . .
Simulation results of the noise amplifier . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
36
36
37
40
43
9.1 Neuron variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
B.1 Pin Assignment — Input . . . . . . . . . . . . . . . . . . . . . . . . 67
B.2 Pin Assignment — Output . . . . . . . . . . . . . . . . . . . . . . . 67
Bibliography
Abbott, L.F. (1999), ‘Lapicque’s introduction of the integrate-and-fire model
neuron (1907)’, Brain Research Bulletin 50(5/6), 303–304.
people.
brandeis.edu/ ~abbott/ papers/ Lapique.pdf issn: 0361-9230/99/.
AMS (2003), 0.35 µm CMOS C35 Process Parameters, austriamicrosystems
AG, A-8141 Schloss Premstätten, Austria. Document Number: ENG -182,
Revision #: 2.0, Company Confidential.
Boahen, Kwabena (1999), A throughput-on-demand address-event transmitter for neuromorphic chips, in ‘Proceedings. 20th Anniversary Conference on Advanced Research in VLSI, 1999’, pp. 72–86.
Boahen, Kwabena A. (2000), ‘Point-to-point connectivity between neuromorphic chips using address events’, 47(5), 416–434.
CDSDoc (2003), Cadence SPICE Reference Manual, 5 edn, Cadence Design
Systems, Inc.
Eberhart, R.C. and R.W. Dobbins (1990), ‘Early neural network development
history: the age of camelot’, IEEE Engineering in Medicine and Biology
Magazine 9(3), 15–18.
FOLDOC (1993), The Free On-line Dictionary of Computing, Denis Howe.
http:// www.foldoc.org/ (accesed July 2004).
Gerstner, Wulfram (1998), Spiking neurons, in W.Maass and C. M.Bishop,
eds, ‘Pulsed Neural Networks’, The MIT Press, chapter 1, pp. 3–54. isbn:
0-262-13350-4.
Gerstner, Wulfram and Werner M. Kistler (2002), Spiking Neuron Models –
Single Neurons, Populations, Plasticity, Cambridge University Press. http:
// diwww.epfl.ch/ ~gerstner/ BUCH.html isbn: 0-521-81384-0.
Hastings, Alan (2000), The Art of Analog Layout, Prentice Hall. isbn: 0-13087061-7.
Hodgkin, A. L. and A. F. Huxley (1952), ‘A quantitative description of ion
currents and its applications to conduction and excitation in nerve membranes’, J. Physiol. (Lond.) 117, 500–544. Cited in Gerstner and Kistler
(2002).
Indiveri (2000), ‘Modeling selective attention using a neuromorphic analog
VLSI’, Neural Computation 12, 2857–2880.
Katz, B. (1966), Nerve, Muscle and Synapse, McGraw-Hill.
82
Bibliography
Lapicque, L (1907), ‘Recherches quantitatives sur l excitation electrique des
nerfs traitee comme une polarization’, Physiol Pathol Gen 9, 620–635.
Cited in Abbott (1999).
Lazzaro, John, John Wawrzynek, M. Mahowald, Massimo Sivilotti and Dave
Gillespie (1993), ‘Silicon auditory processors as computer peripherals’,
IEEE Transactions on Neural Networks 4(3), 523–528.
Mahowald, Misha (1992), VLSI analogs of neuronal visual processing: A
synthesis of form and function, Phd thesis, Department of Computation
and Neural Systems, California Institute of Technology, Pasadena, CA.
Mar, D.J., C.C. Chow, W. Gerstner, R.W. Adams and J.J. Collins (1999), ‘Noise
shaping in populations of coupled model neurons’, 96(18), 10450–10455.
Marienborg, Jan-Tore, Tor Sverre Lande, Mats Høvin and Ånen Abusland (1996), ‘Neuromorphic analog communication’. citeseer.ist.psu.edu/
marienborg96neuromorphic.html .
McCulloch, W. S. and W. Pitts (1943), ‘A logical calculus of ideas immanent
in nervous activity’, Bull. Math. Biophys. 5, 115–133. Cited in Gerstner
and Kistler (2002), Eberhart and Dobbins (1990) and Zhang and Zhang
(1999), among others.
Mead, Carver (1989), Analog VLSI and Neural Systems, Addison Wesley,
chapter 4: Neurons, 12: Axons.
Mead, Carver (1990), ‘Neuromorphic electronic systems’, 78(10), 1629–
1636.
Mortara, Alessandro, Eric A. Vittoz and Phillipe Venier (1995), ‘A communication scheme for analog VLSI perceptive systems’, IEEE Journal of SolidState Circuits 30(6), 660–669.
Motchenbacher, C.D. and J.A. Connelly (1993), Low-Noise Electronic System
Design, John Wiley & Sons, INC. isbn: 0-471-57742-1.
Nelson, M.E. (2004), Databasing the Brain: From Data to Knowledge, Wiley,
New York, chapter Electrophysiological Models In.
OED (2004), Oxford English Dictionary Online, third edn, Oxford University Press. Simpson, John, ed. http:// dictionary.oed.com/ (accesed July
2004).
Schultz, S.R. and M.A. Jabri (1995), ‘Analogue VLSI ‘integrate-and-fire’
neuron with frequency adaptation’, Electronics Letters 31(16), 1357–1358.
issn: 0013-5194.
Treves, Alessandro (1993), ‘Mean-field analysis of neuronal spike dynamics’, Network: Computation in Neural Systems 4(3), 259–284.
van Schaik, André, Eric Fragnière and Eric Vittoz (1996), An analogue electronic model of ventral cochlear nucleus neurons, in ‘Proceedings of
Fifth International Conference on Microelectronics for Neural Networks’,
pp. 52–59.
Bibliography
83
Vittoz, Eric A. (1994), ‘Analog VLSI signal processing: Why, where and
how?’, Analog Integrated Circuits and Signal Processing pp. 27–44.
Weste, Neil H. E. and Kamran Eshraghian (1993), Principles of CMOS VLSI
design: a systems perspective, 2nd edn, Addison-Wesley. isbn: 0-20153376-6.
WordNet (2004), WordNet, a lexical database of the English language, Cognitive Science Laboratory at Princeton University. Principal Investigator:
George A. Miller, http:// wordnet.princeton.edu/ (accessed July 2004).
Zhang, Ling and Bo Zhang (1999), ‘A geometrical representation of
mcculloch-pitts neural model and its applications’, IEEE Transactions on
Neural Networks 10(4), 925–929.