Invited Paper
Optical hybrid analog-digital signal processing based on spike
processing in neurons
Mable P. Fok1, Yue Tian1, David Rosenbluth2, Yanhua Deng1, and Paul R. Prucnal1
1
Princeton University, Princeton, New Jersey 08544, USA
2
Lockheed Martin Advanced Technologies Laboratory, Cherry Hill, New Jersey 08002, USA
ABSTRACT
Spike processing is one kind of hybrid analog-digital signal processing, which has the efficiency of analog processing
and the robustness to noise of digital processing. When instantiated with optics, a hybrid analog-digital processing
primitive has the potential to be scalable, computationally powerful, and have high operation bandwidth. These devices
open up a range of processing applications for which electronic processing is too slow. Our approach is based on a
hybrid analog/digital computational primitive that elegantly implements the functionality of an integrate-and-fire neuron
using a Ge-doped non-linear optical fiber and off-the-shelf semiconductor devices. In this paper, we introduce our
photonic neuron architecture and demonstrate the feasibility of implementing simple photonic neuromorphic circuits,
including the auditory localization algorithm of the barn owl, which is useful for LIDAR localization, and the crayfish
tail-flip escape response.
Keywords: Spike processing, optical signal processing, nonlinear optics, photonics, neuromorphic processing
1. INTRODUCTION
The success of neuromorphic engineering in emulating the biophysics of neurons has lead to important practical designs
for low power adaptive analog computing and signal processing systems. With the use of analog VLSI technology,
devices that closely replicate the capabilities of the retina and the cochlea have been implemented. Although these frontend sensor devices are small and have low-power consumption, their speed is far too slow for many real-time signal
processing applications.
Fiber optics has been known to have fast processing speed and has a large bandwidth. With the marriage of spike
processing and the characteristics of optics, we have designed and implemented a hybrid analog/digital signal-processing
device that elegantly implements the functionality of an integrate-and-fire neuron using a novel type of non-linear
optical fiber and an off-the-shelf semiconductor device. Combining the advantages of both spike processing and optics,
the resultant spike-processing device not only has the potential to be scalable and computationally powerful, but also has
large bandwidth required for high-speed processing applicationa where the use of electronic processing is too slow.
The spiking neuron comprises a small set of basic operations (delay, weighting, spatial summation, temporal integration,
and thresholding), and is capable of performing a variety of computations, depending on how its parameters (e.g., delays,
weights, integration time constant, threshold) are configured. In our photonic neuromorphic device [1], we use a novel
type of Ge-doped highly nonlinear optical fiber for thresholding, an off-the shelf semiconductor device for temporal
integration, and several simple optical components for delay, weighting, and spatial summation. Utilizing this opticsbased hybrid analog-digital processing primitive, complex and high bandwidth processing algorithms can be
implemented. In this paper, we introduce the integrate-and-fire neuron architecture used in our photonics-based
neuromorphic circuits, followed by the implementation of the spike processing device we used for different photonics
neuromorphic circuits.
2. SPIKE PROCESSING IN A LEAKY-INTEGRATE-AND-FIRE (LIF) NEURON
Our approach based on the standard leaky-integrate-and-fire (LIF) model of a neuron that operates as follows [2]: The
neuron has N inputs that are a continuous time series, consisting either of spikes or continuous analog values
representing voltages. After each input is independently weighted and delayed, they are spatially summed (summed
Optics and Photonics for Information Processing V, edited by Khan M. Iftekharuddin, Abdul Ahad Sami Awwal,
Proc. of SPIE Vol. 8134, 813402 · © 2011 SPIE · CCC code: 0277-786X/11/$18 · doi: 10.1117/12.895347
Proc. of SPIE Vol. 8134 813402-1
Downloaded From: http://spiedigitallibrary.org/ on 07/05/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx
point-wise). The resulting single time-series is then temporally integrated using an exponentially decaying impulse
response function. If the integrated signal exceeds a threshold, then the neuron outputs a spike. After the spike, there is a
short refractory period during which no other spikes can be issued. The output of the neuron consists of a continuous
time-series of spikes.
The above spiking behavior is formalized below:
Inputs to neuron:
I j (t )
, where
j = 1..N
I (t ) = Σ ω j I j (t − δ j )
n
j =1
, where ω j is the weight and δ j is the delay
Neuron outputs:
O(t )
If ν (t ) ≥ ν threshold
then
[spike equations: ν (t ) := Vr and
O(t ) = 1 ]
else
[between spikes:
ν (t ) = Vr e
−
t −t0
τm
s
1 t −t0 −
+ ∫ e τ m I (t − s )ds
c 0
, where t0 is the last time the neuron spiked
O(t ) = 0 ]:
After issuing a spike, there is a short period of time, the refractory period, during which no other spikes can be issued.
The output of the neuron consists of a continuous time series comprised of spikes. As opposed to just the integrate-andfire neuron model, the LIF neuron model allows for the application of temporal weighting function to the inputs,
providing an additional dimension to characterize the input.
3. OPTICAL IMPLEMENTATION OF A LIF SPLIKING NEURON
In our optics-based spike processing model, the LIF neuron is mimicked optically using two key elements: a
semiconductor optical amplifier (SOA) and a Ge-doped nonlinear fiber based thresholder. The functional architecture of
the integrate-and-fire device consists of three major processing blocks as shown in Figure 1 (ii) summing, (iv) temporal
integration, and (v) thresholding. The input signal is first weighted and delayed as in Figure 1(i), and then spatially
summed (Figure 1(ii)). The sampling pulse train shown in (iii) is used to provide spikes to the integrator, while the input
signals are integrated in the SOA/EAM (Figure 1(iv)). The green curve corresponds to the dynamic change in the
SOA/EAM governed by the spatially-summed input signals. The integrated output is thresholded by a HDF-based loop
mirror (Figure 1(v)) and the final output is shown in Figure 1(vi). The equations governing the SOA carrier density and
the equations governing leaky integration in a LIF neuron has an exact correspondence [3], which justifies the use of an
SOA as the embodiment of the leaky integrator in this computational primitive.
Proc. of SPIE Vol. 8134 813402-2
Downloaded From: http://spiedigitallibrary.org/ on 07/05/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx
Figure 1. Illustration of a photonic neuronn. Input 1 – Inpuut N: inputs; W1 – WN: variable weight; T1 – TN: variable time
OA: semiconducttor optical ampliifier; EAM: elecctro-absorption modulator;
m
HDF: highly Ge-doped nonlinear fibeer.
delay; SO
Note: Usiing an SOA resu
ults in an inverteed output, while an EAM results in an non-invertted output.
The SOA haas an exponenttial recovery behavior
b
that is similar to thhe integration characteristic
c
t
that
a neuron requires.
r
When an opptical pulse is launched into the SOA, thee SOA carrier density decreeases. In the presence
p
of a pumping
p
current, the carrier density
y recovers expponentially oveer time. When a second optiical pulse is laaunched into the
t SOA
before the caarrier density completely recoovers, it further decreases thee carrier densitty, resulting in a temporal inttegration
of the effectss of both inputt pulses. Figuree 1 illustrates the
t conversion between channges in carrier density
d
to outpput pulse
amplitude thhrough gain saampling The interval betweeen sampling pulses correspponds to the refractory period. We
experimentallly measured th
he relative chaange in carrier density in ressponse to diffeerent optical puulses launched into the
SOA.
(a)
(b)
Figure 2. The measured SOA
S
response too excitation by multiple
m
pulses. (a)
( Experimentallly measurementt of the pulses
b) Relative changge in SOA carrieer density as measured using a optical
o
sampling pulses. The SOA
A
launched into the SOA (b
recovery time (integration
n time constant) is 180 ps.
Figure 2(a) shows
s
a series of
o optical pulsees with differeent temporal sppacing and intennsity that are launched into thhe SOA,
while Figuree 2(b) shows th
he relative chaange in carrier density, represented by the intensity of thhe sampling puulses. An
nonlinear opptical loop mirrror is used for thresholdingg in which the fiber loop coonsists of a shoort piece of Ge-doped
G
nonlinear fibber and a tunab
ble isolator as a directional atttenuator [4]. The
T optical threesholder has a cubic transfer function
Proc. of SPIE Vol. 8134 813402-3
Downloaded From: http://spiedigitallibrary.org/ on 07/05/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx
which suppreesses low-pow
wer inputs whille saturating att high powers. In other wordds, the optical thresholder
t
am
mplitudediscriminatess the input sign
nal and the neuuron fires only when the inpuut power exceeeds a certain thhreshold. In thee context
of the spiking neuron, the thresholder
t
is used
u
to removee the undesiredd weak spikes while equalizinng the strong spikes
s
to
provide a low
w noise controll for a second-sstage neuron.
4. LIGHT
TWAVE NE
EUROMOR
RPHIC CIR
RCUITS
We demonsttrated several small-scale
s
lighhtwave neurom
morphic circuits to mimic im
mportant neuroonal behavior based
b
on
the optics-baased neuromo
orphic circuit described aboove. Here, wee would like to present tw
wo of those lightwave
neuromorphiic circuits: (i) the
t auditory loccalization algoorithm of the baarn owl, usefull for LIDAR loocalization, andd (ii) the
crayfish tail-flip escape resp
ponse, useful for
f responding to patterns.
4.1 Auditorry localization algorithm of the barn owl
Figure 3(a) shows
s
a simplee diagram of auditory
a
localizzation. Due to the difference in position off object 1 and object
o
2,
there is a tim
me difference beetween the signnals arriving thhe owl’s left seensor and rightt sensor, denoteed as ΔT2 = (t1a
1 -t1b) for
object 1 and ΔT2 = (t2a-t2b) for object 2. Therefore,
T
the neuron can bee configured too respond to a certain object location
w
and delayed
d
signals are strong enoough and
merely by addjusting the weight and delay of the neuron inputs. If the weighted
arrive withinn the integrattion window, the neuron sppikes; otherwiise no spike results. Figure 3(b)i illustrrates the
correspondinng SOA-based
d integrator response when the two weigghted and delaayed signals are
a too far appart. The
stimulated siignal cannot paass through thee thresholder and
a therefore no
n spike is obtaained. When thhe two inputs are
a close
enough, the carrier
c
density reaches the thhreshold and leads to a spike, as shown in Figure
F
3(b)ii. This
T algorithm is useful
for LIDAR loocalization.
(a)
(b)i
(b)ii
Figure 3. (a) Schematic illlustration of thee auditory localizzation algorithm
m of the barn owll. (b)i SOA carrier density - wheen
( spike). (b)ii SOA carrier dennsity - when two signals are close (spike).
two signaals are far apart (no
Figure 4 is an
a experiment showing the temporal sensittivity of the sppike processingg [5]. Figure 4(a)
4 correspondds to the
input consistts of a number of pulses (signnals) with sam
me intensity butt with differennt time intervall, i.e. (i) case I:
I signals
that are close together (du
ue to the limiteed bandwidth of the photodeetector, the meeasured pulsess seems to be a strong
pulse) (ii) caase II: signals that are furtheer apart. After temporal integgration at the SOA and thresholding at thee optical
Proc. of SPIE Vol. 8134 813402-4
Downloaded From: http://spiedigitallibrary.org/ on 07/05/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx
thresholder, a spiking output is obtained as shown in Figure
F
5(b). Duue to the gain depletion
d
propeerty of SOA, the
t spike
output is inverted. As we can
c see in Figuure 5(b), no spiike is observedd when the input signals are close
c
together (case I),
w
input signnals are furtherr apart (case II).
while spikes are observed when
(b))
(a)
Casse I
Case II
Case III
C
Case
I
Figure 4. (a) Input to the photonics neuroon (i) case I: inpuut signals are cloose together (meeasured temporall resolution limitted
by the baandwidth of the photodetector)
p
(iii) case II: input signals are furthher apart (b) Photonic neuron outtput (inverted) (ii)
case I: noo spike when signals are close (iii) case II: spike when
w
signals aree further apart.
4.2 Tail-flip
p escape respo
onse of the craayfish
Crayfish escaape from a predator by meanns of a rapid taiil-flip responsee. The correspoonding neuron circuit is confiigured to
respond to apppropriately ab
brupt stimuli but
b not respondd to stimuli froom normal water flow. Sincee this is a life--or-death
decision to thhe crayfish, the response hass to be executeed quickly and accurately. Ouur device, whicch mimics the crayfish
circuit using photonic tech
hnology, is suffficiently fast to
t be applied to
t defense appplications in which
w
critical decisions
d
m
quickly while
w
minimizinng the probabiility of false allarm. A potenttial military appplication of lightwave
need to be made
neuromorphiic signal processing based onn escape responnse could be for
f pilot ejectioon from aircrafft under seriouus attack.
By means off compact opticcal devices, thee latency can bee as low as 5000 ps.
Figure 5. Experimental reesults of a photoonic feature recoggnizer based on the crayfish tail--flip escape respponse (a) Opticall
s
– case II:
inputs to the first integrattor; (b) Output sppikes – case I: reecognizing patteern “abc” and “abb-”; (b) Output spike
recogniziing pattern “abc”” only; (c) Outpuut – case III: nonne of the input iss recognized.
The recognizzer consists of two photonic neurons
n
[6]. Thhe first integraator is configurred to respond to a set of signnals with
specific featuures, while thee second integrrator further seelects a subset of the signal from
fr
a set deterrmined by a weighting
w
and delay coonfiguration. Itt responds onlyy when the inpput stimuli andd the spike from the first inteegrator arrive within a
very short tim
me interval. Fiigure 5 shows the experimenntal results of a lightwave neuuromorphic feature recognizzer based
on the tail-fliip escape respo
onse of the craayfish, the devvice is configurred to respond to different seets of input pattterns by
Proc. of SPIE Vol. 8134 813402-5
Downloaded From: http://spiedigitallibrary.org/ on 07/05/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx
simply varying the weights and delays of the neuron inputs. Figure 5(a) corresponds to the input patterns, while the
recognizer can be configured to response to pattern “abc” and “ab-” (case I: Figure 5(b)) or only to pattern “abc” (case
II: Figure 5(c). However, when the inputs does not match with the recognizer, e.g. spikes from the first neuron arrive too
late, i.e. exceed the integration time, the second integrator will not spike (case III: Figure 5(d)).
5. SUMMARY
The optical hybrid analog/digital processing device we introduced is a result of the cross-fertilization between spike
processing in the fields of computational neuroscience and high speed processing in nonlinear optical device physics.
Utilizing the advantages of both fields, a high-speed, broadband, and scalable processing device can be achieved. We
believe this technology will have broad applications to high bandwidth signal processing, ultra-fast control loops, ultralow latency response functions, and ultra-fast adaptive processing.
REFERENCES
[1] M. P. Fok, D. Rosenbluth, K. Kravtsov, and P. R. Prucnal, “Lightwave Neuromorphic Signal Processing,” IEEE
Signal Processing Magazine, vol. 27, iss. 6, pp. 158 – 160, November 2010.
[2] W. Maass and C. M. Bishop, eds., Pulsed Neural Networks (The MIT Press, 1999).
[3] D. Rosenbluth, K. Kravtsov, M. P. Fok, and P. R. Prucnal, “A High Performance Photonic Pulse Processing
Device,” Optics Express, vol. 17, iss. 25, pp. 22767–22772, December 2009.
[4] K. Kravtsov, P. R. Prucnal, and M. M. Bubnov, “Simple nonlinear interferometer-based all-optical thresholder and
its applications for optical CDMA,” Opt. Express 15, 13114 (2007).
[5] K. Kravtsov, M. P. Fok, D. Rosenbluth, and P. R. Prucnal, “Ultrafast All-Optical Implementation of a Leaky
Integrate-and-Fire Neuron,” Optics Express, vol. 19, iss. 3, pp. 2133 – 2147, January 2011.
[6] M. P. Fok, H. Deming, M. Nahmias, N. Rafidi, D. Rosenbluth, A. Tait, Y. Tian, and P. R. Prucnal, “Signal Feature
Recognition based on Lightwave Neuromorphic Signal Processing,” Optics Letter, vol. 36, iss. 1, pp. 19 – 21,
January 2011.
Proc. of SPIE Vol. 8134 813402-6
Downloaded From: http://spiedigitallibrary.org/ on 07/05/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx