Music Information Retrieval
and Musical Meaning
Frans Wiering
Department of Information and Computing Sciences
Utrecht University, Netherlands
frans.wiering@cs.uu.nl
What is Music Information Retrieval?

Downie 2004:


aspects




a multidisciplinary research endeavor that strives to develop
innovative content-based searching schemes, novel interfaces, and
evolving networked delivery mechanisms in an effort to make the
world’s vast store of music accessible to all
multidisciplinarity
role of technology
idealistic goal
role of musicology



provider of knowledge?
tool user?
true interdisciplinarity?
www.ismir.net
2
What I’m doing in MIR

Utrecht University -> Department of ICS ->
Multimedia group


C-Minor project



MIR is core research area
YahMuugle search engine

yahmuugle.cs.uu.nl
similarity measures for harmonic patterns
WITCHCRAFT project



with Meertens Institute (Dutch popular culture)
search engine for popular songs
to be integrated in Dutch Song Database


www.liederenbank.nl
melodic similarity measures that can deal with oral
variation
3
Content-based searching
user

user need
similar music
user interface

generic IR model,
applied to music
3 main components

query
ranked list


similarity measure

applicable to:

database
database
user interface
similarity measure:
compares query to
database items


musical audio
music
notation/encoding
metadata/tagging
4
Witchcraft Query By Example
http://www.liederenbank.nl/liedpresentatie.php?zoek=70022&lan=nl&wc=true
5
Results
6
Measuring musical similarity

example uses weight flow distance (models melodic contour)
many other measures proposed (modelling pitch, harmony,
rhythm, metre etc.)

challenge: finding similarity measures that




provide meaningful results
are sufficiently generic
have good computational properties
7
MIR from a musicological perspective

Some benefits




identification
data-rich approaches
allows empirical verification
Issues



never enough (reliable) data
is modelling good or bad?
are these what musicologists need?
8
What I’m unhappy about in MIR

there is a lot to be happy about

but there are unfortunate tendencies

anatomical view of music


type of musical knowledge involved


solution in search of a problem?
bad fit to present-day musicological themes


other candidates: relevance, surprise, etc.
few interesting applications for end-users


traditional, production-oriented
measurement of similarity only


static objects, information extracted
cultural/critical/postmodern musicology
I think these tendencies are related
9
MIR bottom-up: two separate worlds

technology comes first


often developed for other domains
explore using simple domain models






provides basic facts
provides ‘ground truth’ for evaluation
separation inhibits MIR contribution to
musical understanding


test against available data
improve/adapt
invent application scenario
musicology (in the widest sense)


how is the data constructed
e.g. of ground truths
totally understandable, but it’s time for a change
10
What is the alternative?

start from high-level goals, e.g.

make all music accessible to all

why?

music is meaningful




no societies without music
important in social activities (religion,
sports, work)
may define someone’s identity
most people deeply affected by some
kind of music
11
Meaningful music retrieval


Musical meaning as foundation of
MIR
Excellent long-term goal






hard problem
affects nearly all aspects of MIR
strong implications for practical
applications
allows true interdisciplinarity
potential for funding
Two lines of approach


cognition
musicology
12
Music cognition


well-developed science
view of music



not the physical object
process whereby it is interpreted by
the musical mind
gaining influence in MIR research

still at a low level (e.g. KrumhanslKessler ‘profiles’ for key finding)
13
Musical mind

long-term memory for music




meaning generation




developed by exposure and training
constrained by biological limitations
can be seen as a ‘pattern Wikipedia’
perception as pattern decoding game
neurological rewards (meaningful)
secondary association with external meanings
cognitive perspective(s) for MIR


model process of meaning generation
similarity becomes association prediction
14
Musicology

Musical meaning used to be taboo subject



Addressed in ‘new musicology’


positivist musicology: objective study of musical materials
meaning and emotion considered subjective and private
with notable precursors
Lawrence Kramer




new musicology = cultural musicology
aim: understanding musical subjectivity in history
subjectivity: disposition to engage in specific social and
historical practices
first and foremost about musical meaning
Lawrence Kramer. Musicology and meaning. The Musical Times 144 (1883), 6-12. 2003
15
Musical meaning in cultural musiclogy




a product of action rather than
structure
emerges from a negotiation
process involving musical text
and context
musical structure/style offer a
potential for meaning generation
Is this potential meaning what
MIR should try to capture?
16
Towards a synthesis

process of meaning generation


negotiation involving work and context


addressed by MIR
redefine similarity



addressed by cognition and musicology
estimation of the ‘state’ of a specific musical mind
prediction of what is meaningful music
make musical context explicit


allows precise definition of intertextuality
simulate ‘pattern decoding game’
17
Mendelssohn’s Musical Mind

Thought experiment





we have endless documentation of his life
and music
we know what he listened to/
played/studied/wrote
musical context can be reconstructed
over time
what if we could recreate M’s Pattern
Wikipedia?
Some possibilities:





place each composition in proper context
assess precise relationships to patterns
study role of tradition and imagination
restrict potential meanings to
approximation of intention
compare to traditional interpretation
18
Some research challenges

theoretical framework for musical meaning
quantifying musical context
model the long-term memory for music
creating computational models that treat music as a
process in time
focussed models of similarity/relevance
personalized MIR
ecologically valid evaluation methods

...suggestions appreciated...






19
Discussion



Can musical meaning provide the ‘big
story line’ for MIR?
Can it be broken down into realistic
research projects?
Does musical meaning indeed provide an
interdisciplinary common ground?



‘terminological compatibility’
What benefits can be expected, for MIR
and for musicology?
Or should the relationship between MIR
and musicology be a different one?
20