IEEE Santa Clara ComSoc/CAS
Weekend Workshop –
Event-based analog sensing
Theodore Yu
theodore.yu@ti.com
Texas Instruments – Kilby Labs, Silicon Valley Labs
September 29, 2012
TI Information – Selective Disclosure
1
Living in an analog world
• The world is analog
– Many different levels to sense
• Sight, sound, touch, taste, smell
– Analog interfaces are uniquely suited for each environment
• Increasingly, we turn to machines to help interpret the world
for us
– Interface through sensors and actuators with computation being
performed in digital machines
• e.g. microprocessors, cellphones, CPUs, etc.
– Digital computation is robust, easily configurable, and widespread
TI Information – Selective Disclosure
2
Analog-digital interface
-Mostly digital
A
D
• Analog world is directly
sampled into the digital domain
– e.g. all-digital
implementations
-Mostly analog
A
D
• Analog world is processed and
interpreted in analog
– e.g. traditional analog
implementations
• The placement of the boundary between analog and digital is flexible
– But transitions are expensive
• All-digital approach: send raw sensor data to digital domain
– Places the burden upon the analog-digital interconnect and digital processing power consumption
• All-analog approach: all-analog signal processing
– Often highly task specific which increases development time and reduces generalization to other
applications
TI Information – Selective Disclosure
3
Analog-digital interface – smart sensors
-Mostly digital
A
D
• Analog world is directly
sampled into the digital domain
– e.g. all-digital
implementations
-Mostly analog
A
D
• Analog world is processed and
interpreted in analog
– e.g. traditional analog
implementations
• The placement of the boundary between analog and digital is flexible
– But transitions are expensive
– Smart sensors and actuators
• Learning and interpretation of analog information
• Adaptation in analog sensor and actuator operation
TI Information – Selective Disclosure
4
Analog-digital interface
• Since the transition from analog domain to
digital domain is expensive, only transmit what
is necessary.
– Maximize information content of each digital bit
– Minimize transfer of redundant information
meaning?
0 1 0 0 1 0
• Analog sensor interface
– Objective
• Operate analog circuits in high efficiency regime for
low-power performance
• Integrated local analog signal processing circuitry
results in sparse data being transferred to the
digital domain
– Extract features of interest from sensors in the
analog domain
– Transmit as digital events to the digital domain
TI Information – Selective Disclosure
Analog to
digital
encoding
5
Event-based sensing approach
• Each digital event encodes a
feature of interest from the sensor
– Event encoding
• Feature selection
– Select what is and is not a feature from
sensor data
– Decide what feature information to
transmit for each event (i.e. spatial
position, temporal position, etc.)
Describes features
of object as timebased digital events
0 1 0 0 1 0
Analog to
digital
encoding
– Event decoding
• Digital processor must now interpret and
understand what each event means
TI Information – Selective Disclosure
6
Dynamic vision sensor (DVS)
• Frame-free image (scene) processing
– Only transmits individual pixel information when
has a change in relative log intensity
• Characteristics
–
–
–
–
Low bandwidth
Low power consumption
Low computational requirements
High sensor dynamic range
• Technical specifications
– 128x128 resolution, 120dB dynamic range,
23mW power consumption, 2.1% contrast
threshold mismatch, 15us latency
• http://www.youtube.com/embed/5NNoq1Gq4sc
TI Information – Selective Disclosure
Lichtsteiner, et. al. (ISSCC 2006, JSSC 2008)
A silicon retina that reproduces signals in
the optic nerve
• Frame-free image (scene)
processing
– Only transmits individual pixel
information when has a change in
relative log intensity
• Event decoding scheme
– ON activity corresponds to bright
pixels and OFF activity
corresponds to dark pixels
• Technical specifications
– <100mW power consumption,
3.5mm x 3.3 mm
TI Information – Selective Disclosure
Zaghloul, et. al. (J. Neural Eng. 2006)
8
Convolution chips for image processing
• Event-based image processing
– Frame-free event-based image
processing of asynchronous events
– On-the-fly processing of events results in
2-D filtered version of the input flow
• Characteristics
– Arbitrary kernel size and shape
• Technical specifications
– 32x32 pixel 2-D convolution event
processor, 155ns event latency between
output and input, 20Meps input rate, 45
Meps output rate, 350nm CMOS,
4.3x5.4mm2, 200mW at maximum kernel
size and maximum input event rate
Linares-Barranco, et. al. (TCAS 2011)
TI Information – Selective Disclosure
Silicon cochlea architecture
Input sound
-2nd order
LPF bank
Seek to emulate
cochlea performance
and functionality by
emulating cochlea
biological architecture
in silicon
-Transform into
analog signal
-Transform into
“digital” neural
event signal
Chan, et. al. (TCAS I 2007)
Digital events
-Each “event” is a data packet describing event source (LPF) and event time
TI Information – Selective Disclosure
Reconstructed silicon cochlea data
Input sound
PC reconstructs the output digital event information by
sorting by channel (LPF) number and then aligning
according to time stamp information.
Digital events
channel number
Silicon
cochlea
PC
time
TI Information – Selective Disclosure
Example data with pure tones (for one
channel)
•Count the time difference
between events
(interspike interval, ISI)
for each channel
750 Hz pure tone
channel number
Simple real-time data
processing procedure
channel number
300 Hz pure tone
time
time
bin count
•A peak in the ISI
histogram indicates a
resonant frequency
response
bin count
•Arrange the ISIs into a
histogram
ISI
TI Information – Selective Disclosure
ISI
Sound Discrimination Example
“coo” sound
Wav file
FFT
ISI histogram
TI Information – Selective Disclosure
“hiss” sound
3-D integrated silicon neuromorphic
processor
Park, et. al. (ISCAS 2012)
Receiver
Hierarchical address-event routing (HiAER)
Sender
Top metal
TI Information – Selective Disclosure
TSV
IFAT (Analog CMOS)
HiAER (Digital CMOS)
DRAM
HiAER
IFAT
Top metal
I/O pad
0.13 m CMOS
5 mm
0.13 m CMOS
5 mm
• 65,000, two-compartment neurons
– Conductance-based integrate and fire array
transceiver (IFAT)
• 65 million, 32-bit “virtual” synapses
– Conductance-based dynamical synapses
– Dynamic table-look in embedded memory
(2Gb DRAM)
• Locally dense, globally sparse synaptic
interconnectivity
– Hierarchical address-event routing (HiAER)
– Dynamically reconfigurable
– Asynchronous spike event I/O interface
5 mm
5 mm
14
Event-driven framework
Provide background on motivation
• Coincidence detection performs efficient spikebased computation
– coincidence detection
• two or more arriving events result in a stronger
response than a single arriving event
Event-based approach relies upon temporal
encoding to communicate signals.
The time of the event is the key parameter, not
the voltage value. Event-encoding is robust
against additive noise.
– applications
• event-driven sensing
–
sensors are only “on” when something important
happens
• event-driven computation
–
information is sparsely represented with events
Yu, et. al. (EMBC 2012)
TI Information – Selective Disclosure
Theodore Yu
UCSD Integrated Systems Neuroengineering Lab
Temporal code and synchrony
• At a local scale, neurons perform
coincidence detection within
temporal integration window.
• At a network scale, the temporal
delay information in events models
the spatial distribution between
neurons.
– Each scene of interest can be
encoded as a unique combination of
features
5ms delay
Input pattern
Coincidence?
Yes or no?
10ms delay
4ms delay
TI Information – Selective Disclosure
16
Temporal code and synchrony example
Event at t = 7ms
5ms delay
Event at t = 3ms
Coincidence?
No!
10ms delay
Event at t = 8ms
4ms delay
Event at t = 7ms
5ms delay
Event at t = 2ms
Event at t = 8ms
TI Information – Selective Disclosure
Coincidence?
Yes!
10ms delay
4ms delay
17
Summary
• Analog event-based sensing
– Since the transition from analog domain to digital domain
is expensive, only transmit what is necessary.
• Maximize information content of each digital event through
encoding of features in analog domain
• Minimize transfer of redundant information for sparse digital signal
processing
– Applications
• Visual and acoustic sensors for event-encoding of features
• Event-based processor performs event-decoding of features
utilizing coincidence detection in neural synchrony
TI Information – Selective Disclosure
18
Thank you
TI Information – Selective Disclosure
19