Report about polyphonic music
transcription
Enabling Access to Sound Archives through Integration, Enrichment and Retrieval
What is Automatic Music Transcription
Play/Synthesis
Coordinating meeting, Nice, November
Project #033902
Transcription
Key technologies of polyphonic music transcription
 Music onset detection
 Polyphonic music transcription
Coordinating meeting, Nice, November
Project #033902
A particular time-frequency analysis tool:
Resonator Time-frequency Image (RTFI)
 Computation-efficient
 implemented by the first-order complex resonator filter bank
 development of multi-resolution fast implementation
 A uniform Framework of TF analysis for music signal
 unlike Cohen’s class and Affine class, RTFI is not limited to
either constant-band or constant-Q
 by simply setting several parameters, the RTFI can implement
different TF analysis such as constant-band, constant-Q and earlike TF analysis
 a frequency-dependent time-frequency analysis
Coordinating meeting, Nice, November
Project #033902
Music Onset Detection
 What is music onset detection
 detection of the instant when a new event begins in acoustical
signal
 hard Onset (fast transition with big energy change)
 soft Onset (slow transition with small energy change)
 How human detect onset
 energy change
 pitch change
 timbre change
Coordinating meeting, Nice, November
Project #033902
Onset detection method in EASAIER
Original Audio Signal
RTFI Average Energy
Spectrum
Time -Frequency
Processing
Adjusted
Energy Spectrum
Pitch Energy Spectrum
Normal Pitch
Energy Specturm
Smoothed Pitch Energy
Spectrum
Difference Pitch
Energy Spectrum
Energy Change
Detection
Aloghrithm
Pitch Change
Detection
Algorithm
Detected
Onsets
Detection
Algorithm
 Time-Frequency Processing:
 incorporating psychoacoustics knowledge about loudness perception
 making energy-change and pitch-change as clear as possible
 Detection Algorithms: detecting onsets by both energy and pitch change clues
Coordinating meeting, Nice, November
Project #033902
Problems in Polyphonic Pitch Estimation
 Harmonic components of different music notes may
overlap
 In-harmonic: some music instrument have inharmonic timbre
Coordinating meeting, Nice, November
Project #033902
Polyphonic Pitch Estimation Method in EASAIER
5 Steps:
1)
2)
3)
4)
5)
Coordinating meeting, Nice, November
Project #033902
Performing RTFI analysis
Extracting harmonic
components
Making preliminary estimation
of possible pitches
Cancelling the extra pitches by
checking harmonic components
( simple timbre model)
Checking pitch candidates by
spectral smoothing principle
Compared with other state-of-art methods
MIREX 2007 Evaluation
 Music onset detection
 According to the overall performance, our method wins this
contest
 Polyphonic pitch estimation (Multiple-F0 estimation task)
 our method performed third best in the submitted 16 methods.
The performance differences between our method and the first
and second best method are minor.
 but most computationally efficient, about 10 time faster than the
first best method, and 100 time faster than the second best
method
Coordinating meeting, Nice, November
Project #033902
Future plan
 To develop the method for note offset detection
 To estimate the note duration time
 To improve and evaluate the automation music
transcription system
 To apply the transcription system assisting the other
functionalities such as content-based music retrieval
Coordinating meeting, Nice, November
Project #033902