FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music
Daniel Müllensiefen, Psychology Dept , Geraint Wiggins, Computing Dept
Centre for Cognition, Computation and Culture
Goldsmiths, University of London
1

M4S: Modelling Music Memory and the
Perception of Melodic Similarity (2006-2009)
Question : How do Western listeners perceive melody?
Domain : Western commercial pop music
Method : Computational modelling
2

1.
Results o Music Cognition o Popular Music Research
2.
Methods o Computing Features with FANTASTIC o Modelling Music Knowledge from a Corpus (2 nd order features)
3.
Background o Similar Approaches/Systems o Questions to be addressed
3

Memory for Melodies:
Are there structural features that make melodies more memorable?
How are listeners using musical knowledge to perform implicit and explicit memory tasks?
4

0.4
Modelling explicit and implicit memory performance in a recognition paradigm (M üllensiefen, Halpern & Wiggins, in prep.)
0.35
0.3
0.25
R^2 0.2
0.15
0.1
0.05
0
1st ordrer features
2nd order features
(testset)
2nd order features (pop corpus)
Expl. memory
(AUC)
Impl. memory
(old-new)
Results: o Memory performance is partially explained by musical features o Implicit memory is better explained by raw features or local context o Explicit memory draws on domain knowledge and features that are distinctive wrt corpus
5

Montreal Battery of Amusia, MBEA,
(Peretz et al., 2003)
:
What makes some test items more difficult than others?
What information do subjects actually use to process tasks?
6

Modelling item difficulty in MBEA
(Stewart, M üllensiefen & Cooper, in prep)
Results: o 70-80% of item difficulty can be explained with as few as three musical features o Relation between item difficulty and features is often non-linear o Some subtests don’t measure what they are believed to measure (e.g. scale)
7

Court cases of music plagiarism:
Are court decisions predictable from melodic structures?
What musical information is used in court decisions?
8

1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Ed it
Di st
.
Model court decisions on melody plagiarism
(M üllensiefen & Pendzich, 2009)
Tv
.e
qu al
Tv
.p
la in tif f
Tv
.d
ef en da nt
Accuracy
AUC
Results: o Court decisions can be closely related to melodic similarity o Plaintiff’s song is often frame of reference o Statistical information about commonness of melodic elements is important
9

Melodic structure and popularity:
Does popularity correlate with certain structural features of a tune?
10


Identify features of commercially successful songs on Revolver
(Kopiez & M üllensiefen, 2008)
Criterion for commercial success: Entered charts as cover version (yes/no) p (chart_entry  1) 
1
1  e  (772.4 + 141.2  pitch_range - 4731.3  pitch_entropy)
Results: o 2 features (pitch range and entropy) are sufficient for fully accurate classification into successful / unsuccessful songs o Plausible interpretation as compositional exercise: Invent a chorus melody such that it has a large range and uses only few pitches much more frequently than the majority of its pitches
11

Two Components
 i .
abs .
std 
 i

 p i
  p
 2
N  1
 2.83
Feature Computation Knowledge from a large corpus of music
12

Implemented features inspired by concepts from o Descriptive statistics o Music Theory o Music Cognition o Music Information Retrieval o Computational Linguistics
13

Pre-requisite: Transformation from notes to numbers
Melody: Sequence of tuples
(notes) with time and pitch information: n
 i
( t i
, p i
)
14

Cognitive Hypothesis: Listeners abstract summary representation of short melodies during listening
Evidence: o Feature-based description and analysis of tunes since Bartok
(1936) in o Folk song research (Steinbeck, 1982; Jesser, 1990; Sagrillo, 1990) o Computational analysis (Eerola & Toiviainen, 2004; McKay, 2005) o Summarising mechanism in visual perception (Chong &
Treisman, 2005)
Feature format: Value that represents particular aspect of melody
15

32 summary features based on: o Pitch o Pitch intervals o Duration ratios o Global extension o Contour o Implicit tonality
Ex. 1: Pitch range (p.range): p .
range  max( p )  min( p )
Ex. 2: Standard deviation of absolute intervals (i.abs.std):
 i .
abs .
std 
 i

 p i
  p
2 
N  1

16

Ex. 3: Relative number of direction changes in interpolated contour representation (int.cont.dir.changes) int .
cont .
dir .
changes 

1 sgn
   sgn
 x i  1
 x i

 x
1 i  1

17

Cognitive Hypothesis: Listeners use literal representation of short subsequences of melody for processing
Inspiration: Text retrieval and computational linguistics
Evidence: o Sequence learning for pitches and music (e.g. Saffran et al.,
1996, 2000; Pearce & Wiggins, 2005, 2006; Loui & Wessel,
2006) o Sequence learning as a general mechanism
Format of mtype: String of digits (similar to “word type” in linguistics) representing pitch intervals and duration ratios
18

• m-types only from within melodic phrases
• Pitch intervals: Classified into 19 classes, preserving diatonic information
• Duration ratios: Classified into 3 classes (Drake & Daisy, 2001), perceptual asymmetric class boundaries (Sadakata et al., 2006)
• Mtype length: Parallel handling of various lengths (1,…,5) m-type of length 2:
“s1e_s1e” m-type of length 4:
“s1q_s1l_s1q_s1l”
19

Format of m-type feature: Number that represents distributions of mtypes of various lengths in melody mean .
Honores .
H  1 n
 100  log N
1.01
 V
 
 

20

 Melodic Contour (M üllensiefen, Bonometti, Stewart & Wiggins, 2009;
Frieler, M üllensiefen & Riedemann, in press; Müllensiefen & Wiggins, under review)
 Phrase segmentation (Pearce, M üllensiefen & Wiggins, 2008; accepted)
 Harmonic content (Mauch, M üllensiefen, Dixon & Wiggins, 2008;
Rhodes, Lewis & M üllensiefen, 2007)
 Melodic accent structure (Pfleiderer & M üllensiefen, 2006;
Müllensiefen, Pfleiderer & Frieler, 2009)
21

The M4S Corpus of Popular Music
(M üllensiefen, Wiggins &
Lewis, 2008)
:
 14,067 high-quality MIDI transcriptions
 Representative sample of commercial pop songs from 1950 - 2006
 Complete compositional structure (all melodies, harmonies, rhythms, instrumental parts, lyrics)
 Some performance information (MIDI patches, some expressive timing)
22

Cognitive Hypothesis: Listeners encode commonness of feature value
Evidence: Statistical learning in music and language (e.g.
Huron, 2006; Pearce & Wiggins, 2006; Rohrmeier,
2009)
Method: Replacing feature values by
 relative frequencies (categorical and discrete features)
 frequency density from kernel smoothing (continuous features)
23

Continuous features
Categorical / discrete features
24


Cognitive Hypothesis: Listeners are sensitive to commonness of mtypes
Method: Use frequency information on m-types from large corpus
Example: Normalised distance of m-type frequencies in melody and corpus (mtcf.norm.log.dist)
=> measures whether uncommon m-types are used rather frequently in melody mtcf .
norm .log .
dist 

 i
 m
T F

 i
 D F

 i
TF
  m with TF

' i
 m

 log
2 log

TF

2
 i
 m
TF
 i
 m


25

Three similarity models:
 Euclidean distance
 Gower’s coefficient
 Corpus-based similarity
26

Cognitive Hypothesis: Similarity perception is based on features of melodies and melody corpus as frame of reference
Idea: Derive similarity from distance on cumulative distribution function
Combine features: By entropyweighted averaging
 k , C
 m i
, m j

 l l

 x i

1 f
C
  l
 l l

 x

1 j f
C
  l
27

Feature ANalysis Technology Accessing STatistics In a Corpus:
FANTASTIC
 Open source tool box for computational analysis of melodies*
 58 features currently implemented
 Different similarity models based on features
 Ideas from: Descriptive statistics, music theory, music cognition, computational linguistics, music information retrieval
 2 feature categories: Summary features and m-type features
 Context modelling via integration of corpus: 2nd order features
* http://www.doc.gold.ac.uk/isms/m4s/?page=Software%20and%20Documentation
28

 Effects of different corpora
 Don’t use raw but cognitively weighted pitches (e.g. perceptual accents)
 Don’t reduce: Use information from m-types directly for
 similarity measurement
 classification
 Create a space / model of feature correlations for corpus
(similar to LSA)
29

Folk Song Research / Ethnomusicology
 Bart ók (1936), Bartók & Lord (1951)
 Lomax (1977)
 Steinbeck (1982)
 Jesser (1992)
 Sagrillo (1999)
Popular Music Research
 Moore (2006)
 Kramarz (2006)
 Furnes (2006)
 Riedemann (in prep.)
Computational / Cognitive Musicology
 Eerola et al. (2001, 2007)
 McCay (2005)
 Huron (2006)
 Frieler (2008)
30

Popular Music Research
Questions: How does melodic structure relate to
 Popularity and selection processes
 Style

Transmission processes
 Specific types of behaviour (e.g. singalongability)
 Value attribution (originality, creativity)
Music Cognition Research
Questions: How does melodic structure relate to
 Memory performance and memory errors
 Similarity judgements

Expectancy
 Preference / aesthetic judgements
31

FANTASTIC: A Feature Analysis Toolbox for corpus-based cognitive research on the perception of popular music
Daniel Müllensiefen, Psychology Dept Geraint Wiggins, Computing Dept
Centre for Cognition, Computation and Culture
Goldsmiths, University of London
32

Computing features (summary and m-type): compute.features()
Computing 2 nd order summary features: compute.corpus.based feature.frequencies()
Computing 2 nd order m-type features: compute.m.type.corpus.based.features()
Computing feature-based similarity: feature.similarity()
33