Intelligent and NoiseRobust Interfaces for
MEMS Acoustic Sensors:
Smart Microphone
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
UNIVERSITY OF MARYLAND
Electrical and Computer Engineering & Psychology Departments
Baras, Horiuchi, Krishnaprasad, Moss, Shamma
THE JOHNS HOPKINS UNIVERSITY
Electrical and Computer Engineering Department
Andreou, Cauwenberghs, Etienne-Cummings
UNIVERSITY OF SIDNEY
Electrical Engineering Department
van Schaik
SIGNAL SYSTEMS CORPORATION
Riddle, Murray
COLLABORATIONS
Institute for Neuroinformatics, ETH
Army Research Laboratory
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
PROJECT GOALS AND MISSION
Overall Mission
Formulate, design, and implement signal processing systems and
technology that can adapt, control, and utilize the noisy MEMS sensor
signals
Specific Approach
Focus Area I
Characterize and integrate
M EMS sensors with
aVLSI circuits that detect
and receive the signals.
Cent er for Auditory
and Acoust ic Research
Focus Area II
Noise control in M EMS sensor arrays
through design and fabrication of
versatile analog VLSI M EMS
interface and associated feature
extraction and analysis stages
Focus Area III
Embedding and demonstrating the
functional capability of the integrated
M EMS/VLSI sensor and signal
processing arrays in a moving robotic
vehicle.
Inst it ut e f or Syst ems Research
Universit y of Maryland
Specific Objectives
Objective 1
Formulate strategies for
interfacing with acoustic MEMS
Objective 2
Develop and implement wind-noise and
platform-noise reduction algorithms in VLSI
Objective 3
Implement VLSI co chlear frequency analysis
2000
log f
1000
log u
500
log f
log f
u
log f
250
125
100
log f
Objective 4
Design and fabricate feature extraction
algorithms
200
300
400
500
Time
(ms)
600
700
800
900
1000
NSL TOOLBOX
Cortical Decomposition of Sound
QuickTime™ and a
Photo - JPEG decompressor
are needed to see this picture.
Objective 5
Feature synthes is and recognition
Objective 6
Technology transf er and demonstrations
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Acoustic and Ultrasonic Transducers
(AGA, REC)
Prototype and Evaluate Various Types of MEMS Microphones/Speakers
Custom Microphones/Speakers
Cent er for Auditory
and Acoust ic Research
Commercial Transducers
Inst it ut e f or Syst ems Research
Universit y of Maryland
Integrated MEMS Acoustic and Ultrasonic Arrays
(AGA, REC)
Prototype and Evaluate Various Types of MEMS Microphone/Speaker Arrays
2D Piezo Arrays
Ceramics and Polymers
Cent er for Auditory
and Acoust ic Research
2D MEMS Arrays
Capacitive Micro-Membranes
Inst it ut e f or Syst ems Research
Universit y of Maryland
Vision:
• A small, low power microphone interface for acoustic
sensors that reduces turbulence and vibration induced
noise on military platforms such as battlefield robotics
Primary
Microphone
Port
Polyurethane
foam
windscreen
1/2”
Secondary
Sensor Port
for Wind
Sensing
Mounting plate
Connector
Microphones
Preamps
aVLSI noise
reduction
circuitry
1”
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
MEMS/VLSI Integration and Prototyping
(REC,AGA)
Develop Integrated Processing Electronics for Transducers
Integrated Transducers and Electronics
Cent er for Auditory
and Acoust ic Research
Transduction Electronics
Inst it ut e f or Syst ems Research
Universit y of Maryland
• Approach:
– Utilize multi-channel adaptive filtering modules based on
aVLSI biomimetic technology
• Analog filter banks with Independent Component Analysis (ICA) and
Least Mean Squares (LMS) adaptation
– Incorporate low noise preamps; acoustic and vibration sensors
– Develop specification and prototypes
– Demonstrate in acoustic duct and installed on unmanned land
vehicle
Acoustic Sensors
Wind Noise Sensor
Multi-Resolution
Low noise acoustic
signals
Adaptive Filter
Noise and Vibration
Sensors
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Cochlear Frequency Analysis
•
We will design a new silicon cochlea with the following features:
–
–
–
Increased robustness due to 2D design
Integrated Inner Hair Cell Model
Reproducible settings of the parameters
Magnitude Response Phase Response
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Stochastic Resonance
Exploit stochastic resonance (noise-induced
enhancement of spectral power amplification SPA) in
conjunction with auditory-inspired (e.g. cochlear) sensor
signal processing architectures
External
World
Band-pass
filter
+
Threshold
detector
Controlled
noise generator
circuit
MEMS
heater
Cent er for Auditory
and Acoust ic Research
K
Inst it ut e f or Syst ems Research
Universit y of Maryland
Adaptive Filtering and Blind Source Separation
(GC, AGA)
Static and Dynamic ICA (Independent Component Analysis)
Adaptive Noise and Wind Cancellation without Need for Isolated Reference
Dynamic ICA Array Processor
Cent er for Auditory
and Acoust ic Research
Adaptive Cell
Inst it ut e f or Syst ems Research
Universit y of Maryland
Filtering
Auditory-Based
Sound Separation
X1
X2
X1
X2
A
S1
S2
De-noising
1
H1
v
v
1
1
H2
v
v
1
2
w
w
ICA1
2
2
w
w
2
ICA2
2
1
1
1
2
1
2
Grouping
2
And
X1
X2
.
.
Competitive
.
.
Learning
.
.
.
X1
X2
v
v
HN
.
N
1
N
w
w
ICAN
2
w
w
1
2
w
sö  y
 wx
N
1
N
2
Early Auditory Models
lo g f
u
lo g f
S1
l
gf
u
lo g f
ICA
lo g f
u
lo g f
S2
Cent er for Auditory
and Acoust ic Research
l
gf
u
lo g f
Cochlea
Hair cell
Lateral Inhibition
Inst it ut e f or Syst ems Research
Universit y of Maryland
Cochlear Feature Extraction
(AGA, GC)
Neuromorphic implementation with asynchronous “spiking” outputs
BM in
BM out
auditory nerve
hair cells
15 channels
cochlea
AM
Energy
TZC
(single
channel)
TZC
FM
BM out
Time
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Early Auditory Processing Stages
Analysis
Transduction
Reduction
Cochlea r f ilt er s
Hair ce lls
Lat er al i nhib i t io n
2000
log f
1000
log u
500
log f
log f
u
log f
250
125
100
log f
eardrum
/ r
i
cochlea
t
basilar membrane
filt ers
a
w
a
hair cell st a ges lat era l inhibit ory
net work
200
300
400
500
Time
(ms)
600
700
800
900
1000
Audit or y Spec t rogr am
y /
4000
4000
250
250
Time ( ms )
Cent er for Auditory
and Acoust ic Research
500
Time ( ms )
500
Inst it ut e f or Syst ems Research
Universit y of Maryland
Multiresolution Preprocessor:
Auditory Filtering
Multiresolution cortical filter outputs
Fast Rate
Fine Scale
Slow Rate
Fine Scale
Upward Moving
Fast Rate
Coarse Scale
Slow Rate
Coarse Scale
Slow Rate
Fine Scale
Fast Rate
Fine Scale
Downward Moving
Slow Rate
Coarse Scale
Fast Rate
Coarse Scale
• The second filter models the multiscale processing of the signal that
happens in the auditory cortex
• A Ripple Analysis Model, using a ripple filter bank, acts on the output of
the inner ear to give multiscale spectra of the sound timbre (Wavelet
Transform)
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Auditory Processing of
Vehicle Signals: Cortex
70
60
scale
50
40
30
20
10
0
20
40
60
80
100
120
Auditory frequency
Example of multi-resolution representation from
cortical module
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Wavelet TSVQ Applied to
Acoustic Vehicle Classification
• Objective: a prototype vehicle
acoustic signal classification
system with low classification
error and short search time
• Biologically motivated feature
extraction models: cochlear filter
banks and A1-cortical wavelet
transform
• Vector Quantization (VQ) based
classification algorithm.
Including learning VQ (LVQ)
and tree structured VQ (TSVQ)
Acoustic
Recording
Preprocessing
Peripheral auditory
processing model
Cortical processing model
Feature
extraction
system
VQ based Classification
Algorithm
Classification Result
Algorithm Flowchart
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Acoustic Transient Time-Frequency Analysis
(GC, w/ APL)
Models “Ripple” Dynamics of Cells Recorded in Auditory Cortex (Shamma)
audio in
Segmenter
“Shelf” (class 10)
“Tub” (class 11)
Continuous
Wavelet Filterbank
32 (freq)
32 (freq) X 64 (time)
ATP
Time-Frequency
Template Correlator
16 (templ.)
...
16 (12 used)
Digital
Postprocessor
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
BINAURAL LOCALIZATION
NM/NL
(ITD processing)
Cochlear
filters
ICc
NA
ICx
VLVp
(ILD processing)
ABL
Cochlear
filters
NM/NL
(ITD processing)
ICc
NA
ICx
VLVp
(ILD processing)
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Stereausis: A Biologically Plausible Binaural Network.
A binaural sound localization system will
be developed using 2 silicon cochleas
and an aVLSI implementation for ILD
and ITD detection.
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Coc hlear
filt ers
A
hair cell
st ages
lat eral
Temporal
inhibit ion Sharpening
Coincidence
C ij = z(t ; i).z(t;
x=1
log f
Mat rix
j)
Z(t;
i)
log f
x=128
log f
B
r(t; x)
.125
y(t; x)
Z(t;
j)
z(t ; x)
.125
.125
.2
5
.5
.25
1
1
1
2
2
2
.25
.5
200
.5
200
C
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
BATS
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Steerable Range Gauging and Echolocation
(REC)
Develop Ranging Signal Processing Algorithms
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Massively Parallel Kernel Learning “Machines”
(GC)
Kernel “machines” subsume LVQ,
RBF and SVM classifiers
Locally adaptive, distributed memory
Scalable and modular
Factor 100-10,000 more efficient
than CPU or DSP
Parallel vector quantizer
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Integrate-and-Fire Address-Event VLSI Neural Networks
(GC & AGA)
T itle:
c hip2.eps
Creator:
Xcircuit v2.0
Preview:
T his EPS pic ture was not s aved
with a preview i nc luded i n i t.
Comment:
T his EPS pic ture wi ll pri nt to a
Pos tScri pt printer, but not to
other types of printers.
Scalable, multi-chip architecture for
“neural” computations
Address-event routing circuit
provides for arbitrary interconnection
topologies
Analog-valued synaptic weights are
implemented by probabilistically
transmitting address-events
Integrate-and-Fire Array Address-Event Transceiver Chip
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Address-Event Asynchronous
Communication and Computation
(AGA, REC, GC)
The multi-chip modular, scalable approach to system integration
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
New design: An AER cochlea chip
• Currently in fabrication
• 128 output channels
• Both for sonar and audio
• New silicon process
(0.35um minimum feature size versus 2.0um)
•AER makes inter-chip communication possible.
•AER allows manipulation of output such that projective
fields are readily implemented.
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
1-D Address-Event Transceiver with Diode-Capacitors
Integrators (AGA)
Address Event (AE) transceiver circuit
is a modular element for future multidimensional communication between
neuromorphic chips.
Input AE data is processed by the
sigmoid function of the nonlinear
diode-capacitor integrators .
The data is retransmitted using the
AE protocol with an arbitrated
queueing communication system.
1-D Address-Event Transceiver Chip
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
• Impact:
– Enable effective acoustic surveillance for Future Combat Systems
– Increase by 20 dB the turbulence induced noise rejection of acoustic
sensors relative to passive treatments of the same size, using active noise
control
– Increase by 20 dB the platform noise rejection of acoustic sensors over
existing methods
Demo III Experimental
Unmanned Vehicle
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland
The Robots
Microphones
Speakers
Wireless camera
Sonar sensors
Touch sensors
Cent er for Auditory
and Acoust ic Research
Inst it ut e f or Syst ems Research
Universit y of Maryland