IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
241
A 2-D Velocity- and Direction-Selective Sensor with BJT-Based
Silicon Retina and Temporal Zero-Crossing Detector
Hsin-Chin Jiang and Chung-Yu Wu
Abstract—In this paper, a 2-D velocity- and direction-selective
visual motion sensor with a bipolar junction transistor (BJT)based silicon retina and temporal zero-crossing detector is proposed and implemented. In the proposed sensor, a token-based
delay-and-correlate computational algorithm is adopted to detect the selected speed and direction of moving object images.
Moreover, binary pulsed signals are used as correlative signals to
increase the velocity and direction selectivities. Each basic detection cell in the sensor has a compact architecture, which consists
of one BJT-based silicon retina cell, one current-input edge extractor, two delay paths, and four correlators. Using the proposed
architecture, an experimental 32 2 32 visual motion sensor chip
with a cell size of 100 2 100 m2 has been designed and fabricated
by using 0.6-m CMOS technology. The correct operations of the
fabricated sensor chip have been verified through measurements.
The measured ranges of selectively detected velocity and direction
in the fabricated sensor chip are 56 mm/s–5 m/s and 0–360 ,
respectively. The complete sensor system consumes 20 mW at 5
V.
Index Terms— Bipolar junction transistor (BJT)-based silicon
retina, CMOS motion-selective sensor, current-input edge extractor, direction-selective sensing, temporal zero-crossings, velocityselective sensing.
I. INTRODUCTION
V
ELOCITY and direction are two important properties
of motion. So far, many visual-motion sensors have
been proposed [1]–[14] that incorporate photoreceptors and
processing circuits on a single chip to estimate velocity
and direction of motion. The adopted motion-sensing algorithms can be divided into two categories: intensity-based
algorithms [1], [2] and token-based algorithms [3]–[14]. An
intensity-based algorithm, which is based upon the mapping
of mathematical techniques, usually requires high-precision
temporal and spatial derivatives. Therefore, it is not suitable
for an analog hardware implementation. In contrast, a tokenbased algorithm, which is inspired by biological models,
has a higher numerical stability and is suitable for analog
hardware implementations. Comprehensive comparisons of
these two algorithms and their implementations can be found
in [15]–[18].
In this paper, a bipolar junction transistor (BJT)-based
silicon-retina sensory system for velocity- and directionselective sensing is proposed. In this system, a token-based
delay-and-correlate motion computation algorithm inspired by
Manuscript received December 22, 1997; revised July 7, 1998. This work
was supported by the National Science Council, R.O.C., under Contract
NSC86-2221-E-009-082.
The authors are with the Institute of Electronics and Department of
Electronics Engineering, National Chiao-Tung University, Hsinchu, Taiwan
300 R.O.C.
Publisher Item Identifier S 0018-9200(99)00974-9.
Fig. 1. The adopted token-based delay-and-correlate motion-computation
algorithm.
biological retinal processing is adopted, which uses edges as
image tokens and correlates them with binary pulses. One of
the two significant features of the proposed sensory system
is that the circuitry of the current-input edge extractor is
simple and robust, which makes very-large-scale integration
(VLSI) implementation feasible. The other is that the binary
pulse correlation [9]–[11] is used to increase the accuracy of
velocity- and direction-selective sensing. Using the proposed
32 motion measurement
architecture, an experimental 32
chip has been fabricated by using a 0.6- m CMOS technology.
The operations are also verified through measurements.
II. MOTION-COMPUTATION ALGORITHM
The token-based delay-and-correlate motion computation
algorithm, which is similar to the Reichardt algorithm [19],
is adopted in the sensor design. The conceptual structure of
the algorithm is shown in Fig. 1. The elements perform the
functions of photoreceptors and horizontal cells in the retinas.
The photoreceptors transduce light into electrical signals,
whereas the horizontal cells perform the spatial smoothing on
perform
signals from photoreceptors. The edge extractors
the functions of the bipolar cells in the retinas, which process
the signals from both the photoreceptor and horizontal cell
to generate a signal that contains spatial edge information of
the incident image. Meanwhile, the edge extractors identify
both turn-ON and turn-OFF edges of the moving image from
the signals generated by bipolar cells. They generate edge
pulses to perform a function similar to the action potential
responses observed in transient amacrine cells of the retinas.
0018–9200/99$10.00  1999 IEEE
242
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
(a)
(b)
Fig. 2. (a) The structure of a BJT-based silicon retina with tunable image-smoothing capability, which is proposed in [20] and [21]. (b) The measured
responses of the emitter current difference of a single cell in the BJT-based silicon retina [21] with a moving light bar incident upon the chip.
The elements
are the correlators that correlate the edge
with the delayed edge pulse generated from
pulse from
The correlator
the previous cell using the delay elements
fires a pulse only when the object moves in the preferred
direction and the difference between the traveling time of
the object edge crossing the two cells and the selected delay
is smaller than the width of the edge
time of the elements
and
is used to mimic the
pulse. The combination of
direction-sensitive responses of ganglion cells in the retinas.
III. HARDWARE IMPLEMENTATION
A. The BJT-Based Silicon Retina
Fig. 2(a) shows the structure of a BJT-based silicon retina
[20], [21], which is used to realize the function of the elements
in Fig. 1. In this BJT-based silicon retina, each cell has
two BJT’s, called smoothing BJT and isolated BJT, which are
implemented using the parasitic BJT structure in a standard
CMOS process. The base region of one smoothing BJT is
connected with the base regions of the smoothing BJT’s
in its four neighbor cells via nMOS field-effect transistors
(nMOSFET’s) to form a BJT smoothing network, which is
used to implement the equivalent function of the horizontal
cells in the retina. The isolated BJT is used to realize the
photoreceptor in the retina. The outputs are the emitter currents
of the BJT’s.
The response of the bipolar cell in the retina is realized by
subtracting the emitter current of the BJT in the smoothing network from that of the isolated BJT in the same cell. Fig. 2(b)
shows the measured responses [21] of a single cell in the BJTbased silicon retina with a moving light bar projected upon it.
When the turn-ON edge and the turn-OFF edge of the moving
and
respectively,
light bar pass over the readout cell at
the temporal averaging functions of the BJT smoothing network lead to the temporal zero-crossings in the emitter current
difference of the phototransistors. The appearance of temporal
zero-crossings can be used to identify whether an edge of a
moving object passes over the readout cell.
B. The Edge Extractor
To simultaneously realize the response of the bipolar cell
and detect the temporal zero-crossings of the response as
the edges of the object image passing over the BJT-based
silicon retina cell, an edge extractor is proposed, as shown
in Fig. 3(a). In Fig. 3(a), the transistors
and
are used to virtually bias the emitters of
and smoothing BJT
in the BJT-based
isolated BJT
and readout emitter
silicon retina cell at the fixed voltage
and
respectively. This readout scheme,
currents
which has a common part as shown on the left, is proposed in
[22] as the readout circuit for the infrared detector.
To detect the zero-crossing point without ambiguity, even
under noise and disturbance, a current-input Schmitt trigger
and
is used. The input
[23]
is applied to the drain of the
, whereas the other
current
is applied to the drain of
via a cascoded
input current
–
If the current through
and
current mirror
is
the edge signal at the output of the Schmitt trigger has a
becomes greater
sudden change from VDD to GND when
This corresponds to the temporal zero-crossing
than
in Fig. 2(b). In this case, the
point of the turn-ON edge at
, and
GND level of the edge signal turns off the NMOS
is cut off from the drain of
Only if
is decreased to a
will the edge signal at the output of the
value smaller than
edge extractor change from GND to VDD. This corresponds
to the temporal zero-crossing point of the turn-OFF edge at
A temporal transition at the Edge-Signal output is further
converted into an edge pulse by using the pulse-conversion
circuit, also shown in Fig. 3(a). It consists of two inverters
used to shape the signal at the Edge-Signal node, a simple
transient detection circuit used to perform the temporal differentiation, and an inverter used to produce a narrow pulse.
The capacitor in the circuit is implemented using the PMOS
transistor with its source and drain tied together as one plate
and the gate as the other plate of the capacitor. The discharge
in the circuit is implemented by an NMOS transistor with an
adjustable bias voltage
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
243
(a)
(b)
Fig. 3. (a) The edge extractor, which detects the temporal zero-crossings from the edge signals given by the BJT-based silicon retina and generates the edge
pulse. (b) The circuit architecture, which is used to realize the element
in Fig. 1 for the generation of delayed edge pulse signals.
D
At the falling temporal transition of the input to the transient
detector, the negative voltage due to the capacitance coupling
makes the drain junction of the nMOSFET forward biased,
which inhibits the generation of a large negative voltage pulse.
Hence, another set of transient detectors and inverters is used,
with an extra inverter at the input to invert the falling temporal
transition and generate a positive voltage pulse. Then the
outputs of two pulse-conversion circuits are connected to a
NAND gate to form a single output. The circuit of Fig. 3(a)
in Fig. 1.
realizes the function of the element
C. The Delay Element
Fig. 3(b) shows the circuit architecture used to realize the
element in Fig. 1 for the generation of delayed edge pulse.
There are four signal paths in this circuit architecture. The first
two paths are used to generate delayed binary narrow pulses
at the temporal transitions of signals from the edge extractor
along the
direction, whereas the last two paths are used
direction. In each signal path,
to generate those along the
there are two transient detectors with the adjustable voltages
and
The first one with
is used
to generate specified delay time whereas the second one with
to generate narrow pulses.
D. The Correlator
Since the signals to be correlated are binary pulses, the
correlators can easily, compactly, and robustly be implemented
by simple NAND gates, as shown in Fig. 3(c). Motion is
selectively detected by correlating the edge pulse at a pixel
with the delayed edge pulse from a neighboring pixel. Only
the preferred moving direction and velocity can enable the
correlator to fire a pulse out. Four such correlators are used in
each pixel to correlate the edge pulse with the delayed edge
and
pulse from which four neighbors in four directions
244
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
(c)
Fig. 3. (Continued.) (c) The circuit architecture of the correlators.
to extract two-dimensional (2-D) velocity and direction.
In this work, the outputs of all the correlators in the pixels
along each direction are combined into one output terminal
via a simple wired OR gate. Therefore, when the object moves
with preferred velocity and direction, there are serial pulses to
appear at the output terminal.
The detection of a particular velocity and direction can be
realized by controlling the delay times of the delay paths along
and
the -axis and -axis with the tuning voltages
For example, if the selected direction of object motion is and
the selected delay times
and
along
the velocity is
the -axis and -axis, respectively, can be written as
(1)
(2)
where is the space between two adjacent BJT-based silicon
the
retina cells. If
and
in Fig. 3(c) have pulse outputs.
outputs
and
If
have pulse outputs.
IV. MEASUREMENT RESULTS
An experimental chip was designed and fabricated in a
0.6- m N-well CMOS technology, which consists of a 32
32 array of the proposed motion-detection cells. Fig. 4(a)
shows the layout diagram of the basic cell, whereas Fig. 4(b)
shows the photography of the whole chip. Since both imageacquisition elements and computation elements are integrated
into one pixel, the wiring will not increase as the size of array
increases. Thus, if the die size can be freely increased, the size
of array can be freely increased as well.
Fig. 5 shows the measured waveforms in the various computational stages of one motion-detection cell in the fabricated
2-D sensor array. The top traces show the measured emitter
currents of isolated BJT and smoothing BJT in the fabricated
BJT-based silicon retina cell, where the currents are measured
by linearly converting them into voltages via the external
circuits. The third trace shows the response at the Edge-Signal
node in the current-input edge extractor shown in Fig. 3(a).
As predicted, the response has sharp transitions when
is greater than
or
is smaller than
The
fourth trace shows the edge-pulse response of the current-input
edge extractor shown in Fig. 3(a). The last two traces show
and
delayed edge pulses from the delay element
the
shown in Fig. 3(b). In Fig. 5, the delay time of the first
delayed edge pulse and that of the second
delayed edge
pulse are respectively generated via the first two delay paths
shown in Fig. 3(b). The component mismatches between these
two delay paths induce the mismatch between these two delay
Fig. 6 shows the measured
times even under the same
output waveforms at the four output terminals of the fabricated
direction with the
chip when a bright spot moves in the
and
preferred speed 1 m/s, where only the outputs
have serial pulses as expected.
The interpixel variance of the delay times is one of the
important factors that determine the selectivity of the sensor
chip. Generally, the interpixel variance of the pulse width and
the delay time caused by process variations has a Gaussian
distribution with mean value and standard deviation. Through
and
the mean value of
the adjustment of
the delay time can be set to the desired delay time, whereas
the mean value of the pulse width is equal to the standard
deviation of the delay time. Then one can obtain around 61%
of the overall output pulses with some pulses missing. In this
case, the velocity and the direction of the moving objects still
can be detected with good selectivity. However, the minimum
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
245
(a)
(b)
Fig. 4. (a) The layout diagram of the basic detection cell and (b) the photography of the 32
which is fabricated by using 0.6-m N-well CMOS technology.
Fig. 5. The oscilloscope traces of the various computational stages of one
motion-detection cell in the 2-D array. The emitter currents of the BJT’s are
read out by using off-array op-amp circuits with negative-feedback resistors
to linearly convert them into voltages. The bias voltages are VG
3
V, VB = 1:2 V, VP = 2 V, VF = 0 V, Vpulse = 0:9 V, and
VdX = VdY = 0:7 V.
=
pulse width has a lower limit equal to the standard deviation of
the delay time. Fig. 7 shows the measured maximum variance
of the delay time of one pixel among eight fabricated chips
where the variance percentage is around 16%. The variance
is dependent on the mean value of the delay time. Using
half of the delay-time variances as the pulse widths to detect
the preferred speeds, respectively, the measured selectivity
2 32 2-D velocity- and direction-selective sensing chip,
Fig. 6. The measured output waveforms at the four output terminals of the
fabricated sensor chip when a bright spot moves in the 45 direction with the
preferred speed. The bias voltages are VG = 3 V, VB = 1:2 V, VP = 2 V,
VF = 0:3 V, pulse width = 10s, and delay time = 71s.
tolerance is around 8% for all preferred speeds with some
output pulses missing.
To verify the direction-selective function, a bright spot
moving in different directions with the same speed is projected
on the chip, and the interpulse delay time of the serial pulses at
the four output terminals is measured. The required delay times
along the -axis and -axis for the preferred direction are
set according to (1) and (2), which are equal to the interpulse
delay times of the same direction. Fig. 8 shows the measured
interpulse delay times where the negative delay times represent
246
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
TABLE I
SUMMARY OF THE CHARACTERISTICS FOR THE
FABRICATED MOTION-SELECTIVE DETECTION CHIP
Fig. 7. The measured maximum variance of the delay time of one pixel
among eight fabricated chips.
computational algorithm is adopted, and the motion-selective
detection is achieved by correlating two binary edge pulses.
Both image-acquisition elements and computation elements
are integrated into one cell, and complicated intercell wiring
is avoided. Moreover, the robust edge detection in the basic
detection cell of the sensor is achieved using the compact
structure of the BJT-based silicon retina with the current-input
edge extractor, which identifies the temporal zero-crossing
points. The operations of the proposed motion sensor have
been verified by the measurements on a 32 32 CMOS experimental chip. It has been shown that the proposed hardware
architecture of the visual motion sensor provides an efficient
solution for implementing a dense and robust 2-D visual
chip with little power consumption and real-time processing
capability.
Fig. 8. The measurement of the direction-selective function of the fabricated
sensor chip where the interpulse delay time at the four terminals is measured
and plotted for different directional angles. The bias voltages in the measure3 V, VB = 1:2 V, VP = 2 V, VF = 0:3 V, and pulse
ment are VG
width = 25s.
=
those along the
or
direction. As may be seen from
Fig. 8, the motion directions with any angles from 0 to 360
at a fixed speed of 0.5 m/s can be detected by specifying the
suitable delay times from 100 to 300 s in the four directions
and
If the specified delay
via the tuning voltages
time is infinity, the tuning voltage is set to zero to turn off
the nMOSFET.
Table I shows the summary on the characteristics of the fabricated motion-selective detection chip. From the measurement
results, the correct operations of the proposed visual motiondetection system, which is realized in CMOS technology, have
been successfully verified.
V. CONCLUSION
A 2-D velocity- and direction-selective visual motion sensor
with a BJT-based silicon-retina and temporal zero-crossing
detector has been proposed, analyzed, and experimentally
verified. In this sensor, a token-based delay-and-correlated
REFERENCES
[1] J. Tanner and C. Mead, “An integrated optical motion sensor,” in VLSI
Signal Processing II, S.-Y. Kung, R. Owen, and J. Nash, Eds. New
York: IEEE Press, 1986, pp. 59–76.
[2] J. Hutchinson, C. Koch, J. Luo, and C. Mead, “Computing motion using
analog and binary resistive networks,” IEEE Trans. Comput., vol. 21,
pp. 52–64, 1988.
[3] A. G. Andreou and K. Strohbehn, “Analog VLSI implementation of the
Hassenstein-Reichardt-Poggio models for vision computation,” in Proc.
1990 IEEE Symp. Syst., Man, Cybern., 1990, pp. 707–710.
[4] A. G. Andreou, K. Strohbehn, and R. E. Jenkins, “Silicon retina for
motion computation,” in Proc. 1991 IEEE Int. Symp. Circuits and
Systems, Singapore, June 1991, pp. 1373–1376.
[5] T. Horiuchi, J. Lazzaro, A. Moore, and C. Koch, “A delay line based
motion detection chip,” in Advances in Neural Information Processing
Systems 3, R. Lippman, J. Moody, and D. Touretzky, Eds. San Mateo,
CA: Morgan Kaufman, 1991, pp. 406–412.
[6] R. G. Benson and T. Delbruck, “Direction selective silicon retina that
uses null inhibition,” in Advances in Neural Information Processing
Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San
Mateo, CA: Morgan Kaufman, 1992, pp. 756–763.
[7] T. Delbruck, “Silicon retina with correlation-based, velocity-tuned pixels,” IEEE Trans. Neural Networks, vol. 4, pp. 529–541, May 1993.
[8] J. Kramer, “Compact integrated motion sensor with three-pixel interaction,” IEEE Trans. Pattern Anal. Machine Intell., vol. 18, pp. 455–460,
Apr. 1996.
[9] J. Kramer, R. Sarpeshkar, and C. Koch, “Pulse-based analog VLSI
velocity sensors,” IEEE Trans. Circuits Syst. II, vol. 44, pp. 86–101,
Feb. 1997.
[10] R. Sarpeshkar, W. Bair, and C. Koch, “Visual motion computation
in analog VLSI using pulses,” in Advances in Neural Information
IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 34, NO. 2, FEBRUARY 1999
247
Processing Systems 5, S. J. Hanson and C. L. Giles, Eds. San Mateo,
CA: Morgan Kaufman, 1993, pp. 781–788.
R. Etienne-Cummings, J. Van der Spiegel, and P. Mueller, “A focal
plane visual motion measurement sensor,” IEEE Trans. Circuits Syst. I,
vol. 44, pp. 55–66, Jan. 1997.
C. F. Chiu and C. Y. Wu, “Design of CMOS elementary-motion-flowselective image detector using the BJT-based silicon retina,” in Proc.
1997 IEEE. Int. Symp. Circuits and Systems, Hong Kong, June 1997,
pp. 717–720.
G. Indiveri, J. Kramer, and C. Hoch, “Analog VLSI architecture for
computing heading direction,” in Proc. Int. Vehicles Symp., 1995, pp.
24–29.
R. C. Meitzler, A. G. Andreou, K. Strohbehn, and R. E. Jenkins,
“A sampled-data motion chip,” in Proc. Midwest Symp. Circuits and
Systems, 1995, vol. 1, pp. 288–291.
T. Poggio, W. Yang, and V. Torre, “Optical flow: Computational
properties and networks, biological and analog,” in The Computing
Neuron, R. Durbin, C. Mial, and G. Mitchison, Eds. Wokingham, U.K.:
Addison-Wesley, 1989, pp. 355–370.
E. C. Hildreth and C. Koch, “The analysis of visual motion: From
computational theory to neuronal mechanisms,” Ann. Rev. Neurosci.,
vol. 10, pp. 477–533, 1987.
[17] R. Sarpeshkar, J. Kramer, G. Indiveri, and C. Koch, “Analog VLSI
architectures for motion processing: From fundamental limits to system
applications,” Proc. IEEE, vol. 84, pp. 969–987, July 1996.
[18] T. Horiuchi, W. Bair, B. Bishofberger, A. Moore, C. Koch, and J.
Lazzaro, “Computing motion using analog VLSI vision chips: An
experimental comparison among different approaches,” Int. J. Comput.
Vision, vol. 8, pp. 203–216, 1992.
[19] B. Hassenstein and W. Reichardt, “Systemtheoretische analyze der zeit-,
reihenfolgen- und vorzeichenauswertung bei der bewegungsperzeption
des russelkafers chlorophanus,” Z. Naturforsch, vol. 11b, pp. 513–524,
1956.
[20] C. Y. Wu and H. C. Jiang, “A high-density 2-dimensional BJT-based
silicon retina with tunable image smoothing capability,” in Proc. Int.
Conf. Electronics, Circuits, and Systems (ICECS’97), Dec. 1997, vol. 3,
pp. 1025–1029.
, “An improved 2-dimensional BJT-based silicon retina with
[21]
tunable image smoothing capability,” IEEE Trans. VLSI, to be published.
[22] C. C. Hsieh, C. Y. Wu, and T. P. Sun, “A new cryogenic CMOS readout
structure for infrared focal plane array,” IEEE J. Solid-State Circuits,
vol. 32, pp. 1192–1199, Aug. 1997.
[23] Z. Wang and W. Guggenbuhl, “Novel CMOS current Schmitt trigger,”
Electron. Lett., vol. 24, no. 24, pp. 1514–1516, Nov. 1988.
[11]
[12]
[13]
[14]
[15]
[16]