Melodic Style Detection
in Hindustani Music
Amruta Vidwans
Prateek Verma
Preeti Rao
Department of Electrical Engineering
Indian Institute of Technology
Bombay
OUTLINE
p
Introduction
n
p
p
p
p
2/27
Objective
Style Classification in Indian Classical Vocal Music
n
Literature Survey
n
Previous Work
n
Feature Description
n
Database and Listening Tests
n
Results and Extention to Turkish Music
Style Classification In Flute Alap recordings
n
Database and Annotation
n
Signal Characteristics of Discriminatory Ornaments
n
Feature Design
Conclusion and Future Work
References
Department of Electrical Engineering , IIT Bombay
40
2 ofof25
INTRODUCTION
Objective
p
Melodic features to distinguish
n
n
Hindustani, Carnatic and Turkish music
Instrument playing styles
p
Basis: Melody line alone suffices for listeners to reliably distinguish musical
styles
p
Applications: Extract important metadata automatically
3/27
Department of Electrical Engineering , IIT Bombay
40
3 ofof25
Style Classification in Indian Classical Vocal Music
4/27
Department of Electrical Engineering , IIT Bombay
4 of 25
INTRODUCTION
Literature Survey – Similar work done
p
Timbral features
n
p
Features based on timbre such as MFCC, delta-MFCC, and spectral features used
by Parul et. al (ISMIR13) to distinguish Indian music genres using Adaboost,
GMM etc.
Timbral + Rhythmic features
n
n
Kini et. al. (NCC10) used these features to do genre classification of North Indian
Classical Music into Quawali, Bhajan, Bollywood etc.
Liu et. al. (ICASSP09) used timbral, wavelet coefficients and rhythmic features to
classify different styles viz. Arabic, Chinese, Japanese, Indian, Western classical
p
p
Timbral +Melodic features
n
5/27
Emphasized that the diversity of Indian Classical Music is difficult to model
Salamon et al (ICASSP12) gave a large number of pitch based features in
addition to timbral features which improved the performance.
Department of Electrical Engineering , IIT Bombay
40
5 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Pre-processing
p
Melody extraction
n
n
n
Difficulty: Presence of multiple instruments along with voice
Accuracy of the present state of the art automatic pitch trackers ~80%
Semi automatic approach for pitch detection is used to achieve best possible
performance
Hindustani raga Jaijaiwanti by artiste Rashid Khan
original
resynthesized
Carnatic raga Dwijavanti by artiste R Vedavalli
6/27
Department of Electrical Engineering , IIT Bombay
40
6 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Feature Description
p
Localized contour shape-features
n
n
n
n
n
7/27
Previous study used Steady Note (SN) and Gamak Measure (GM)
A steady note: pitch segment of ‘N’ ms with standard deviation less than ‘J’
cents
Data driven parameters gave highest accuracy (N=400ms J=20cents)
SN: Normalized total duration of steady regions
GM: ratio of number of non steady regions(1sec) having oscillations in
3-7.5Hz to the total number of regions
Department of Electrical Engineering , IIT Bombay
40
7 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Feature Description
p
Primary Shape contour (PS)
n
n
Contour typology proposed by C. Adams [6] for categorizing melodic shapes in 15
types
Segments taken from silence to silence, assignment done using relation between
Initial, Final, Highest and Lowest pitch values.
(a)
12
(b)
Contour types 12 and 13 in alap section in raga Hindolam by Carnatic artistes
(a) T. N. Sheshagopalan
(b)M. S. Subalaxmi
13
8/27
Department of Electrical Engineering , IIT Bombay
40
8 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Feature Description
p
Distance of highest peak in unfolded histogram from Tonic (DistTonic)
n
n
In unfolded histogram the Hindustani alaps are concentrated near the tonic,
Carnatic alap pitch distribution is closer to the upper octave tonic
Distance of the highest peak from the tonic in the unfolded histogram taken as
feature.
Rashid Khan raga Todi
9/27
Sudha Raghunathan raga
Subhapanthuvarali
Department of Electrical Engineering , IIT Bombay
40
9 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Feature Description
p
Melodic Transitions (MT)
n
n
n
The overall progression of the melody can be characterized by this
Haar wavelet basis function used to represent the melody
Fifth level approximation is used
p
n
10/27
Lower level approximation captures very minute variation whereas higher
level gives a coarse representation.
Normalized size of upward jumps(>1semitone) in concatenated pitch
contour taken as feature.
Department of Electrical Engineering , IIT Bombay
40
10 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Database
p
Choice of alap section used for labeling a track
n
n
Unmetered section always rendered in the start.
Database Description
p
p
p
n
11/27
Widely performed raga pairs of same scale interval from Hindustani and Carnatic
Ragas belonging to different scales chosen
Renowned artists of various schools of music chosen
Total of 120 alap sections equally distributed across both the styles
Scale
Hindustani Raga
(No. of clips)
Carnatic Raga
(No. of clips )
Heptatonic
Todi (12)
Subhapanthuvarali (14)
Pentatonic
Malkauns (18)
Hindolam (12)
Nonatonic
Jaijaiwanti (10)
Dwijavanthy (14)
Octatonic
Yaman and Yaman Kalyan (20)
Kalyani (20)
Department of Electrical Engineering , IIT Bombay
40
11 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Listening Tests
p
Test Design
n
n
Hypothesis : Melody alone is sufficient to carry out style distinction
Audio clips re-syntheisized with constant timbre sound.
p
p
n
Interface Description
p
p
p
p
12/27
Removes bias towards artist identity, pronunciation, voice quality.
Volume dynamics retained (Sum of vocal harmonics).
User information in terms of training of subject as well as familiarity
Audios divided in 5 sets --12 clips of each Hindustani (H)-Carnatic (C) in each set.
Listening of 10 sec audio mandatory with option of pause, Skipping an audio not allowed
Decision label as H, C or NS (Not Sure) asked for each clip
Category
Accuracy (no of participants)
trained <3yrs
74.6%
(10)
trained 3-10yrs
79.7 %
(8)
trained >10yrs
89.6 %
(2)
Amateur
75.2 %
(18)
Listener
77.5 %
(13)
Overall Accuracy
76.9 %
(51)
Department of Electrical Engineering , IIT Bombay
40
12 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Automatic classification and feature selection
p
Quadratic classifier used on feature sets
n
n
n
5 fold CV on 5 different random partitions to avoid any raga bias
Exhaustive search on all possible feature combinations for feature selection
Separability of parameterized distribution taken into account for small dataset
p
p
n
13/27
Distribution of the Log Likelihood ratio (LLR) found for a feature set.
𝐿𝐿𝑅= ​ln⁠(​𝐿(​
𝑥∕H )/𝐿(​
F-ratio computed on distribution
LLR as a confidence measure.
C ) ) 𝐹 −𝑅𝑎𝑡𝑖𝑜 =​(​𝜇↓𝐻 −𝑥∕
​𝜇↓𝐶 ​
)↑2 /​​𝜎↓𝐻 ↑2 +​𝜎↓𝐶 ↑2 Highest accuracy achieved is 96% for SN, GM, DistTonic, and PS feature set.
Department of Electrical Engineering , IIT Bombay
40
13 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Automatic classification and feature selection
p
Confusion matrix
Obs.
True
C
H
NS
Obs.
True
C
H
45
8
12
48
3
4
C
H
(a)
C
H
58
3
2
57
(b)
Confusion matrix for (a) listening tests (b) classifier output
The horizontal labels correspond to true labels.
p
Correlation with the listeners
n
n
n
n
14/27
Subjective test label across all listeners decided by 60% threshold for labeling H,
C or NS
For objective measure region around 10% of LLR values around 0 taken as NS
Cost value of -1,1 or 0 is assigned according to the agreement between
subjective and objective labels.
Highest value of correlation was found to be 0.79 across all the feature
combinations
Department of Electrical Engineering , IIT Bombay
40
14 ofof25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Extention to Turkish Music
Hindustani raga Jaijaiwanti by artiste Rashid Khan
Carnatic raga Dwijavanti by artiste R Vedavalli
Turkish makam Nihavent by artiste Hafiz Kemal Bey
0
15/27
Department of Electrical Engineering , IIT Bombay
15 of 25
STYLE CLASSIFICATION IN INDIAN CLASSICAL VOCAL MUSIC
Extention to Turkish Music
p
Experimental Setup and Results
n
n
n
Obs.
True
H
C
T
60 Taksims considered for classification using the melodic features
The features discussed before were applied to 3 classes.
Highest accuracy of 81.6 % was obtained using SN, GM, MT, DistTonic.
H
C
T
NS
131
21
5
(a)
13
118
25
2
7
114
4
4
6
Obs.
True
H
C
T
(b)
H
C
T
56
1
0
3
48
17
1
11
43
Confusion matrix for (a) Listening test output (b) Classifier output
n
n
16/27
Most confusion occurred for T and C which was validate via subjective
tests
No category of primary shape was discriminating Turkish-Carnatic clips
Department of Electrical Engineering , IIT Bombay
40
16 ofof25
Style Classification In Flute Alap recordings
17/27
Department of Electrical Engineering , IIT Bombay
17 of 25
INTRODUCTION
Example: Gayaki vs Tantrakari
18/27
Department of Electrical Engineering , IIT Bombay
40
18 ofof25
INTRODUCTION
Tantrakari vs Gayaki Style
p
Gayaki vs Tantrakari styles
n
Gayaki : Vocal characteristics incorporated into instrument play.
p
n
Tantrakari : Instrumental characteristics (particularly plucked string)
p
p
p
p
Melodic Leaps eg. Discreteness
Elements synonymous with pluck adapted in Flute : Tonguing
Fast Stroke pattern while playing melody adapted in Flute: Fast Tounging
Ornament Production
n
n
n
Glide : Slowly lifting the finger- Rate
controlling the glide shape
Vibrato : Periodic Pulsations in Air Flow
Fast Tounging : Tongue movement
p
p
n
Using tongue to articulate notes differently in a melody
Synonymous with Sitar-rapid movements of right hand (Tantrakari)
Blow Stop :
p
p
19/27
Singing voice is fluid, glides used often
Blowing with stops while rendering a note
Periodic blowing patterns (1-2 or 1-1-2) with same/different note
Department of Electrical Engineering , IIT Bombay
40
19 ofof25
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Database and Annotation
Artist Raga Duration(mins) Hari Prasad Chaurasia Jhinjhoti (T-Metadata) 18:54 Hansadhwani 1:50 Yaman 2:29 Sindh Bhairav 3:30 Rupak Kulkarni Hemant
Ronu Majumdar Vibhas 5:39 Pannalal Ghosh Yaman 2:18 Shri 2:20 Pilu 1:30 ShuddhaBasant 2:46 Nityanand Haldipur (T-Metadata) 6:50 Four Music Experts unaimously aggred on the two clips as tantrakari
20/27
Department of Electrical Engineering , IIT Bombay
40
20 ofof25
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Annotation Rules
p
Five categories used for annotation
n
Tantrakari
p
p
p
p
p
n
p
p
p
p
21/27
:
:
:
:
:
Blowing same notes with stops
Steady notes with abrupt movements
Discreteness with silences
Discreteness with rapid jumps
Fast Tounging with any pitch movement
Gayaki
p
p
T1
T2
T3
T4
T5
G1
G2
G3
G4
G5
:
:
:
:
:
Oscillatory repetitive movements
Non- Oscillatory repetitive movements
Glides
Non-specific continuous pitch
Vibrato
Final characteristic feature taken as the consensus of the feature agreed
by all the musicians
Department of Electrical Engineering , IIT Bombay
40
21 ofof25
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Database and Annotation
Artist Raga Hari Prasad Chaurasia Jhinjhoti (T-Metadata) 18:54 352/465/267 Hansadhwani 1:50 43/14/33 Yaman 2:29 57/12/40 Sindh Bhairav 3:30 50/20/66 (T-Metadata) 6:50 f/T/G(in s) Rupak Kulkarni Hemant
Ronu Majumdar Vibhas 5:39 47/11/44 Pannalal Ghosh Yaman 2:18 39/3/66 Shri 2:20 45/0/29 Pilu 1:30 133/35/105 ShuddhaBasant 2:46 94/0/41 Nityanand Haldipur 22/27
Duration(mins) q
G: Gayaki
T : Tantrakari
f: Niether Tantrakari
or Gayaki
q
Alap-Jod-Jhala
composition derived
from Sitar. (It mainly
contains Tantrakari
elements)
q
Two different style
artists and their
disciple chosen
124/102/111 Department of Electrical Engineering , IIT Bombay
40
22 ofof25
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Database and Annotation
Presence of Tantrakari and Gayaki elements
Artist Raga Vibrato Fast
Blow -
T3/T4 G3/G4 Tounging Stop Hari Prasad Chaurasia Jhinjhoti(T) P P P P P Hansadhwani P O P P P Yaman O O O P P Sindh Bhairav P O O P P RupakKulkarni Hemant(T) O P P P P RonuMajumdar Vibhas P O O P P PannalalGhosh Yaman P O O P P Shri P O O O P Pilu P O O O P ShuddhaBasant P O O O P NityanandHaldipur 23/27
Department of Electrical Engineering , IIT Bombay
40
23 ofof25
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Production and Acoustic Characteristics
p
Blow-Stop & Fast Tonguing – Two of the most distinctive features
n
n
n
n
24/27
Energy Contour different due to difference in articulation
Higher rate of energy variation due to use of tongue.
Pitch movements do not influence energy movements.
Intensity of Tanpura almost constant
Department of Electrical Engineering , IIT Bombay
40
24 ofof25
STYLE CLASSIFICATION IN FLUTE ALAP AUDIOS
Feature Design
p
Features capturing acoustic characteristics of Fast Tounging
n
Sub-band Peak Ratio (SPR) =
n
Sub-band FFT Ratio (SFR) =
-- 2s segments of data across the two styles taken
as data point
-- Ground Truth obtained using manual
annotation
Classifier
Accuracy
25/27
Quadratic
74.07
Linear
73.15
Department of Electrical Engineering , IIT Bombay
40
25 ofof25
p
Conclusion
n
n
n
p
Future Work
n
n
26/27
Successfully proposed melodic features to distinguish vocal music styles
Importance of energy based features brought out for flute style
classification.
Automatic classification results given with degree of separability
Extension of database to non-alap recordings in flute style classification
study.
Addition of new features to distinguish C and T to improve the
classification accuracy
Department of Electrical Engineering , IIT Bombay
40
26 ofof25
References
1] P. Agarwal, H. Karnick and B. Raj, “A Comparative Study of Indian and Western Music
Forms”, ISMIR 2013
2] S. Kini, S. Gulati and P. Rao, “Automatic genre classification of North Indian devotional
music” , NCC 2010
3] Y. Liu, Q. Xiang, Y. Wang and L. Cai, “Cultural Style Based Music Classification of
Audio Signals,” Acoustics, Speech and Signal Processing, ICASSP, 2009.
4] A. Kruspe et al. “Automatic classification of Music Pieces into Global Cultural Areas ”,
In Proc. of 42nd International Conference on Semantic Audio, AES 2011
5] J. Salamon, B. Rocha and E. Gomez, "Musical Genre Classification using Melody
Features extracted from polyphonic music signals", International Conference on
Acoustics, Speech and Signal Processing, 2012
6] C. Adams, “Melodic Contour Typology” , Ethnomusicology 20.2 : 179-215 (1976)
7] S. Justin, S. Gulati and X. Serra, "A Multipitch Approach to Tonic Identification in
Indian Classical Music", ISMIR, 2012
8] C. Clemments, “Pannalal Ghosh and the Bānsurī in the Twentieth Century” , City
University of New York, New York City, September 2010
27/27
Department of Electrical Engineering , IIT Bombay
40
27 ofof25