The State of Music
at LabROSA
Dan Ellis
Laboratory for Recognition and Organization of Speech and Audio
Dept. Electrical Eng., Columbia Univ., NY USA
dpwe@ee.columbia.edu
http://labrosa.ee.columbia.edu/
1. The State of LabROSA
2. Music Projects
3. The Big Picture
Music at LabROSA - Dan Ellis
2013-01-25 - 1 /10
1. The State of LabROSA
• LabROSA Music Publications per year
10
8
6
4
2
0
2003 2004 2005 2006 2007 2008 2009 2010 2011 2012
Music at LabROSA - Dan Ellis
2013-01-25 - 2 /10
But!
Brian
McFee
Matt
McVicar
Dawen
Liang
Zhuo
Chen
Colin
Raffel
Music at LabROSA - Dan Ellis
2013-01-25 - 3 /10
Current Projects at LabROSA
Zhuo Chen
• Parametric source-note models
1
0
b = 0.05
p = A6#
p = C2
1
0
b = 0.05
p = A6#
p = C2
p = A6#
p = C2
1
0
b = 0.95
time tl/d
Music at LabROSA - Dan Ellis
1
2
3
4
a = 0.25
25
1
2
3
4
a = 0.01
01
1
2
3
a = 0.01
1
2
gamma = 500
3
4
1
gamma = 5
2
3
4
1
gamma = 5
2
3
4
time tl/d
time tl/d
2013-01-25 - 4 /10
Large-Scale Cover Recognition
Bertin-Mahieux
• 2D Fourier Transform
Magnitude (2DFTM)
fixed-size feature
to capture “essence”
of chromagram:
• First results on finding covers in 1M songs
Average rank
meanAP
random
500,000
0.000
jumpcodes 2
308,369
0.002
137,117
0.020
2DFTM (50 PC)
Music at LabROSA - Dan Ellis
2013-01-25 - 5 /10
Million Song Dataset
• Many Facets
Bertin-Mahieux
McFee
Echo Nest audio features
+ metadata
Echo Nest “taste profile”
user-song-listen count
Second Hand Song covers
musiXmatch lyric BoW
last.fm tags
• MSD Challenge
Kaggle.com competition, 153 teams
winning entry used simple Collaborative Filtering
Music at LabROSA - Dan Ellis
2013-01-25 - 6 /10
Jazz Discography Project
• How can MIR help organize jazz collections?
our tools are quite genre-specific
e.g. beat tracker is fine for pop, useless for Jazz
40
30
20
10
80
60
40
20
0
84
Music at LabROSA - Dan Ellis
86
88
90
92
94
96
98
2013-01-25 - 7 /10
Local Tagging
• MFCC-statistics classifiers on 5 sec windows
trained from MajorMiner data
freq / Hz
01 Soul Eyes
2416
1356
761
427
240
135
_90s
club
trance
end
drum_bass
singing
horns
punk
samples
silence
quiet
noise
solo
strings
indie
house
alternative
r_b
funk
soft
ambient
british
distortion
drum_machine
country
keyboard
saxophone
fast
instrumental
electronica
80s
voice
beat
slow
rap
hip_hop
jazz
piano
techno
dance
female
bass
vocal
pop
electronic
rock
synth
male
guitar
drum
Music at LabROSA - Dan Ellis
50
100
150
200
250
300
1.5
1
0.5
0
−0.5
−1
−1.5
−2
40
80
120
160
200
240
280
time / s
320
2013-01-25 - 8 /10
Onset Correlation
Brian McFee
• “Ahead of” or “behind” the beat?
Tony Williams
Music at LabROSA - Dan Ellis
Elvin Jones
2013-01-25 - 9 /10
The Big Picture
• After The Echo Nest solved music
recommendation,
what’s left for MIR?
decomposing audio into events
nuance
musical structure & motifs
good vs. bad
Music at LabROSA - Dan Ellis
2013-01-25 - 10/10
Music Research Hackathon?
• Like a hack day, except
no cash prizes, corporate sponsors, flash factor
aimed at academic researchers & similar
deeper problems more than pretty demos
• @Columbia, spring/summer 2013
• See Colin’s breakout
Music at LabROSA - Dan Ellis
2013-01-25 - 11/10