Acous&c
Terrain
Classifica&on
for
Outdoor
Mobile
Vehicles
Jacqueline
Libby
Hatem
Alismail
Debadeepta
Dey
Terrain
grass
branches
leaves
PlaBorm
Data
Collec&on
Data
Labeling
Approach
Branches
Leaves
Grass
labeled
acous&c
data
Noise
filtering
• Frequency
trunca&on
• Hamming
window
Feature
Extrac&on
• Gianna
features
• STFT
Classifica&on
• Kernelized
SVM
• kNN
Feature
Extrac&on
• 
200
ms
frames
• 
each
frame

data
point
(feature
vector)
• 
~50
seconds
of
smallest
data
set
(branches)
• 
50%
overlap
• 
~500
data
points
for
each
class
Courtesy:
Vehicle
Sound
Signature
Classifica&on
by
Frequency
Vector
Principal
Component
Analysis
Wu
et
al,
1998
Short
Time
Fourier
Transform
*
=
FFT
Frame
Hamming
window
‐ To
compute
the
STFT,
repeat
this
process
for
many
overlapping
frames
and
store
their
FFT’s
‐ 
STFT
is
used
to
generate
features
for
classifica&on
KNN
Classifica&on
•  For
two
classes,
A
and
B
and
a
sound
signal
X
•  Compute
the
f=STFT(X)
•  Find
the
KNN
of
f
in
A
and
B
•  Output:
–  A,
if
the
sum
of
distances
to
A
<=
B
–  B,
otherwise
KNN
Results
•  ~500
features
per
class
•  Results
averaged
over
10
trials
–  At
each
trial,
randomly
select
95
features
and
compute
classifica&on
accuracy
Leaves
vs.
Grass
Branches
vs.
Grass
Average
accuracy
(%)
99.78
vs.
98.84
92.80
vs.
88.80
Leaves
vs.
Branches
97.47
vs.
83.78
Reducing
percentage
of
overlap
between
frames
increases
accuracy
significantly.
This
could
be
due
to
more
dis&nc&ve
STFT
features,
or
the
smaller
number
of
frames
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
•  Energy
of
the
frame
–  Square
of
the
norm
of
frequency
amplitudes
present
in
the
frame
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
•  measure
of
abrupt
changes
in
energy
•  Further
subdivide
each
frame
into
K
(=10)
subframes
•  Compute
energy
of
each
sub‐frame
•  Entropy
:
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
The
number
of
&mes
per
second
the
signal
crosses
the
zero
axis
in
the
frame
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
Frequency
bin
below
which
‘c%’
(=80%)
of
the
magnitude
of
the
DFT
coefficients
of
the
signal
is
concentrated
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
weighted
mean
of
frequency
components
Gianna
Features
•  Short
Time
Energy
•  Energy
Entropy
•  Zero
Crossing
Rate
•  Spectral
Rolloff
•  Spectral
Centroid
•  Spectral
Flux
•  measure
of
how
quickly
the
power
spectrum
of
a
signal
is
changing
•  Devia&on
of
the
power
spectrum
for
one
frame
to
the
next
SVM
Results
•  SVM
–  Kernel:
Radial
Basis
Func&on,
Sigma
=
10.0
•  ~400
features
per
class
•  50%
training
data,
50%
test
data
Leaves
vs.
Grass
Branches
vs.
Grass
Average
accuracy
(%)
99.74
vs.
97.24
95.22
vs.
95.63
Leaves
vs.
Branches
98.25
vs.
99.75