Studying Music Performance:
A Probabilistic Approach
David Temperley
Eastman School of Music
University of Rochester
Four goals of music performance:
1.
2.
3.
4.
Render the notes (and other score information) correctly
Convey the emotional content of the piece
Clarify the structure of the piece
Modulate the flow of information
Four goals of music performance:
1.
2.
3.
4.
Render the notes (and other score information) correctly
Convey the emotional content of the piece
Clarify the structure of the piece
Modulate the flow of information The topic of today’s talk
Part I: Clarification of structure
• Three examples
• The circularity problem (performance  perception or
perception  performance?)
• Ambiguity avoidance
Clarification of structure in music performance
Many authors have observed that one goal of a good
performance is to convey the structure of the piece (Sloboda
1985, Clarke 1988, Gabrielson 1999).
(“Structure” might include many things—meter, harmony,
phrase structure, form, motivic connections, etc.)
Sometimes this point is made indirectly—e.g. when it is
said simply that a good performer must be aware of the
structure of the piece. (Why is it beneficial to be aware of this?
Presumably so that it can be conveyed to the listener.)
Clarification of structure – Example 1: Accentuation of
metrically strong events
The perception of meter is a complex process, involving a
variety of different cues or “phenomenal accents” (Lerdahl
& Jackendoff, 1983): length, loudness, articulation,
harmony, parallelism...
Performers can facilitate this process by aligning
phenomenal accents with strong beats—that is, by
emphasizing metrically strong events.
Clarification of meter, cont’d.
The same notes (pitches and rhythms) in different metrical
contexts will be played quite differently: metrically strong notes
are louder, longer, more legato (Sloboda 1983, 1985).
…And listeners can identify which melody is being performed.
Clarification of meter: Dynamics
Performers tend to play metrically strong notes somewhat louder than
others, though the difference is relatively small.
Data from Drake & Palmer (1993) for piano performance, showing average
“hammer velocity” values for notes on beats 1-4 of a 4/4 measure. (Note:
range for entire piece was 30 to 600 units!)
Clarification of structure – Example 2: Phrase-final
lengthening
Performers tend to decrease tempo at phrase boundaries, and more
markedly at larger sectional boundaries (Todd 1985; Palmer 1996; Windsor
& Clarke 1997).
(smooth curve)
(jagged line)
phrase boundaries
Data from Windsor & Clarke 1997
Clarification of structure – Example 3: Individuation of
melodic lines
The perception of polyphonic music requires the identification of individual
lines—voice separation or “contrapuntal analysis” (Temperley 2001).
Performance teachers (of polyphonic instruments) often urge students to
individuate different lines of the texture in some way—for example, by
emphasizing the melody.
A specific case in point is melodic lead—the tendency of the melody notes to
anticipate other notes (Palmer 1996). This could be an strategy to individuate
melodic notes by making them asynchronous with other notes.
According to the “clarification of structure” view, these
phenomena (accentuation of metrically strong events,
phrase-final lengthening, melodic lead) arose, at least in
part, as ways of clarifying musical structure for the listener.
A possible circularity in this argument:
therefore
Listeners expect metrically
strong notes to be louder / longer
Performers play metrically
strong notes louder / longer
A possible circularity in this argument:
therefore
Listeners expect metrically
strong notes to be louder / longer
therefore
Performers play metrically
strong notes louder / longer
How to address the circularity problem? Three responses...
1. Perhaps the process really is circular. Performance tendencies
gave rise to perceptual tendencies, but were then reinforced
by them.
2. Perceptual tendencies may have come first and may have
originated outside music, e.g. in language:
- Stressed syllables are normally longer and louder
(Fry 1955).
- Phrase-final lengthening is a cue to syntactic boundaries
in speech (Lehiste 1973).
3. The case for perception influencing performance is
strengthened if it can be shown that the performance tendencies
are strongest in cases that would otherwise be ambiguous
perceptually.
3. The case for perception influencing performance is
strengthened if it can be shown that the performance tendencies
are strongest in cases that would otherwise be ambiguous
perceptually.
Ambiguity avoidance is well-documented in language: For
example, speakers exaggerate prosodic marking of phrase
structure (e.g. lengthening at syntactic boundaries) when a
sentence is ambiguous in context (Snedeker & Trueswell 2003).
Ambiguity Avoidance in Music – Example 1: Organ
performance
On instruments where dynamic accentuation is not possible, do
performers increase other kinds of accentuation on metrically strong
notes?
In organ pedagogy, performers are taught to emphasize metrically strong
notes through duration (i.e. with longer inter-onset intervals to the
following note—[A]) or articulation (playing them slightly more
legato— [B]) (e.g. Hurford 1988).
: . : .
A
B
Ambiguity Avoidance – Example 2: Phrase-final lengthening
at ambiguous points
Do performers exaggerate phrase-final lengthening in cases
where the phrase boundary would otherwise be unclear or
ambiguous?
Brahms, Intermezzo Op. 116 No. 6, mm. 1-8
Rubinstein’s
performance
(1959)
Question: Why does Brahms mark sostenuto and a fermata at the phrase
ending in mm. 7-8 – when the performer would normally slow down at a
phrase boundary anyway?
Answer: Given the unusual harmonic context (the sudden move to C#
major), the boundary might not easily be noticed here unless especially
reinforced by the performer.
Schumann, “Von fremden Ländern und Menschen” from Kinderszenen
Question: Why does Schumann mark ritardano before the return of the
opening theme?
Answer: The return happens at an unexpected place in the phrase
structure—after only a six-measure phrase—and with no clear change of
harmony; thus the structural boundary might be missed if it were not
especially marked by phrase-final lengthening.
Ambiguity Avoidance – Example 3: The rise of “swing
tempo” in jazz
Swing tempo: The lengthening of the first half of the beat
in relation to the second.
Maple Leaf Rag, as performed by Scott Joplin (1918) and
Jelly Roll Morton (1938) (my transcriptions)
Excerpt 1
(Joplin)
(Morton)
“Maple Leaf Rag”, Excerpt 2
(Joplin)
(Morton)
Morton’s performance features a great deal more
syncopation than Joplin’s—especially left-hand events on
weak eighth-note beats; this potentially obscures the quarternote level of the meter. But it also features a swing
tempo, which clarifies the quarter-note level of the meter (by
making strong-beat notes longer than weak-beat ones).
And in general, the transition from ragtime to jazz features...
• a shift from the “even-note” rhythms of ragtime to the “swing” feel
of jazz (Schuller, 1967)
• an increase in the degree and complexity of syncopation
(Waterman, 1974; Sales, 1984)
In each of these cases of ambiguity avoidance...
- Accentuation of metrically strong notes through length and
articulation in organ performance
- “Disambiguating” ritards marked at potentially ambiguous phrase
boundaries
- The rise of swing tempo in jazz
...we see increased use of “clarifying” performance cues in
situations that would otherwise be ambiguous—supporting
the idea that these cues arose, in part, as a way of clarifying
structure.
Part II: Modulation of Information Flow
Information is the negative log of probability. So in a
sequence of events, the lower the probability of an event, the
more information it conveys. (Also known as surprisal...)
Information
6
5
4
3
2
1
0
Probability
1.0
The Uniform Information Density Hypothesis (Levy &
Jaeger, 2007): A certain rate of information flow is optimal in
language perception, and language production tends to be
adjusted to maintain this rate.
Uniform Information Density in Language (Levy & Jaeger,
2007)
In prosody, words that are more probable are pronounced more quickly:
- More frequent words are shortened (Jurafsky et al., 2001)
- Words that are more probable in local context (e.g. P(wn | wn–1)) are
shortened (Jurafsky et al. 2001; Aylett & Turk, 2004)
- Words that are more probable given larger context (e.g. that have already
occurred in the discourse) are shortened (Fowler & Jousum, 1987)
Modulation of information flow: What does this predict for
music performance?
One prediction: Less probable events should be performed
more slowly.
Modulation of information flow: What does this predict for
music performance?
One prediction: Less probable events should be performed
more slowly.
What about unexpected harmonies, e.g. chromatic
harmonies?
Sundberg’s (1988) rules for good expressive performance
include the “Harmonic Charge Rule”: Slow down on chords
whose pitches are far from the tonic on the circle of fifths.
(Never really tested with regard to performance.)
Expressive performance as modulation of information flow:
Anecdotal evidence
Brahms, Intermezzo Op. 116 No. 6, mm. 1-8 (again)
Question: Why does Brahms mark sostenuto and a fermata at the phrase
ending in mm. 7-8?
Possible Answer: Given the modulation to a distant key and the rapid
harmonic rhythm, this passage is high in information content; slowing
down smooths out the information flow.
How does harmony affect expressive timing? Do performers
slow down on unexpected harmonies?
Chris Bartlette’s dissertation (Eastman, 2007) examined this
experimentally.
Skilled pianists were given pairs of passages to practice and
perform. Within each pair, the passages were identical except
for one chord, which was diatonic (within the key) in one case
and chromatic (outside the key) in the other case.
Effect of harmony on performance expression was examined
(timing, dynamics, asynchrony).
Examples (Barlette, 2007) [Title]
œœœ œ
#### 6 œ.
œ œ œœœœœœ œ œ œœœ
œ œ(above) and a chromatic
The second
& half of
8 a diatonic passage
J
J
passage (below). (MIDI files generated from experimental trial,
œ.
œ œ
œ œ. of performance.)
œ
œ
œ
œ
reflecting ?actual
timing
and
dynamics
#
œ
## # 6 œ
œœ œ.
œ
œ
œ
8
œ
[Title]
œœœœœ œ œ œ œ œœ
œœœœ œ œ œœœ
###### 6 œ.
œ
œœ œ œ œ œ œ œ Ý. J
# 8œ œ œ œ œ œ œ œ
&
#
J
&
œ.
œ œ œ œ .
œ
œœ œ
œ
.
œ
?? ###### 6œ.œœ œ.
œœ œœ
œ
œ
.
.
.
œ
œ
œ
œ
œ
œ.
œ.
##8
E:
ii6
#### œ œ œ œ œ œ œ œnœnœ œ œ œ œ œ Ý.
œ
œ
œ
&
? #### œ.
E:
œ.
œ.
œ.
bII6
œ.
œ.
œœ
œ œ.
Result: Both the “chord of interest” and the previous chord are
lengthened when the chord of interest is chromatic.
Cn-1
Cn
(“chord of
interest”)
Cn+1
Why? To allow more time for the chord to be processed...
“smoothing the information flow.”
Various sources of evidence...
- Sundberg’s rules
- Bartlette’s dissertation
- anecdotal evidence from score markings
...suggest a preference in performance to slow down at
unexpected harmonies. This can be seen as an example of
modulation of information flow.
Part III. The Syncopation-Rubato Trade-Off
Syncopation - non-alignment between beats and
phenomenal accents
Rubato - irregularity or fluctuation in tempo for expressive
purposes
The Syncopation-Rubato Trade-Off: Music with more
syncopation seems to permit less rubato.
Rhythm in traditional sub-Saharan African music (compared
to Western classical music)
- Greater use of syncopation (Jones 1959, Chernoff 1977, other authors)
- Greater strictness of tempo (less rubato):
“When we Europeans imagine we are beating strict time, the
African will merely smile at the ‘roughness’ of our beating.”
(Jones 1959, p. 38)
Also: rock and jazz - a high degree of syncopation (compared to
classical Western music); very little tempo fluctuation.
An Experimental Study of the Syncopation-Rubato
Trade-Off (with Kelly Francis)
Subjects: Doctoral students in piano, Eastman School
of Music
Procedure: Subjects were given short passages and asked
to perform them (after practicing). The variance in tempo
within each performance was then analyzed.
Materials: Four pairs of musical phrases, with the same
pitches in the same approximate positions; one pattern in each
pair is unsyncopated (1A), the other is highly syncopated
(1B). (Deadpan MIDI performances)
Results
Pianists played unsyncopated excerpts with significantly more
variance in tempo than syncopated excerpts (F(1, 9)=8.17, p=.02).
The effect varied greatly across excerpt pairs:
0.045
Tempo variance
0.04
0.035
0.03
0.025
U ns ync opated
Sync opated
0.02
0.015
0.01
0.005
0
1
2
3
Excerpt pair
4
THE BIG QUESTION:
The syncopation-rubato trade-off—is it clarification of
structure, OR modulation of information flow?
THE BIG QUESTION:
The syncopation-rubato trade-off—is it clarification of
structure, OR modulation of information flow?
Clarification of structure: Could we argue that the
combination of rubato and syncopation obscures the
metrical structure more than either one alone?
The syncopation rubato trade-off as clarification of structure
A. A perfectly regular pattern
B. Syncopation: one event occurs
slightly before the beat
C. Rubato: intervals between
beats are slightly irregular
D. Syncopation and rubato
combined: the last beat is slightly
late, the last event slightly
anticipates it (now the onset
pattern is the same as in A!)
600 ms
600 ms
600 ms
600 ms
600 ms
600 ms
450 ms
600 ms
600 ms
800 ms
600 ms
600 ms
800 ms
600 ms
intended rhythm
beats
note onsets
The syncopation rubato trade-off as clarification of structure
A. A perfectly regular pattern
B. Syncopation: one event occurs
slightly before the beat
C. Rubato: intervals between
beats are slightly irregular
D. Syncopation and rubato
combined: the last beat is slightly
late, the last event slightly
anticipates it (now the onset
pattern is the same as in A!)
600 ms
600 ms
600 ms
600 ms
600 ms
600 ms
intended rhythm
beats
note onsets
450 ms
600 ms
600 ms
600 ms
600 ms
800 ms
800 ms
600 ms
So...syncopation
and rubato in
combination
obscures the
meter.
While there are cases where the combination of
syncopation and rubato obscures the meter, it is difficult
to make a general argument to this effect.
What about modulation of information flow?
The Syncopation-Rubato Trade-Off as Modulation of Information Flow
Imagine a one-second segment of a melody. The listener must predict when
the next note will occur.
q
q
P
The Syncopation-Rubato Trade-Off as Modulation of Information Flow
Imagine a one-second segment of a melody. The listener must predict when
the next note will occur.
q
P
q
P
High information—all locations are
equally likely
P
Low information—one location is
much more likely than all others
In a style with no syncopation or rubato, the location of
each note is highly predictable. (Low information)
q
q
In a style with rubato, the beat location itself is unpredictable
so the note location is as well. (Moderate information)
P
q
q
In a style with syncopation, note locations become less predictable
because a note may occur on a weak beat. (Moderate information)
P
q
q
In a style with rubato and syncopation, the possibility of notes on weak beats
combined with the variation in tempo creates high uncertainty. (High
information)
q
q
P
In a style with rubato and syncopation, the possibility of notes on weak beats
combined with the variation in tempo creates high uncertainty. (High
information)
q
q
Perhaps the combination of syncopation and rubato is
avoided to keep information flow at a moderate level.
P
Conclusions
Two goals of music performance:
- Clarification of structure
- Modulation of information flow
The syncopation-rubato trade-off:
Could be clarification of structure, could be modulation of
information flow; more likely the latter.
Thank you!
References
Aylett, M. and Turk, A. 2004. The Smooth Signal Redundancy Hypothesis: A functional explanation for relationships between redundancy,
prosodic prominence, and duration in spontaneous speech. Language and Speech, 47(1):31–56.
Bartlette, C. 2007. "A Study of Harmonic Distance and Its Role in Musical Performance." Ph.D dissertation, Eastman School of Music,
University of Rochester.
Chernoff, J. M. 1979. African Rhythm and African Sensibility. Chicago: Chicago University Press.
Clarke, E. 1988. Generative principles in music performance. In J. A. Sloboda, (Ed.), Generative processes in music: The psychology of
performance, improvisation, and composition (Oxford: Clarendon), pp. 1–26.
Drake, C., & Palmer, C. 1993. Accent structures in music performance. Music Perception 10, 343-78.
Fowler, C., & Housum, J. 1987. Talkers’ signaling of ‘new’ and ‘old’ words in speech and listeners’ perception and use of the distinction.
Journal of Memory and Language, 26, 489-504.
Fry, D. 1955. Duration and intensity as physical correlates of linguistic stress. Journal of the Acoustical Society of America 27, 765-768.
Gabrielsson, A. 1999. “The Performance of Music”, in D. Deutsch, Ed., The Psychology of Music (San Diego: Academic Press), pp. 501602.
Hurford, P. 1988. Making music on the organ. Oxford: Oxford University Press, 1988.
Jones, A. M. 1959. Studies in African Music. London: Oxford University Press.
Jurafsky, D., Bell, A., Gregory, M., & Raymond, W. 2001. Probabilistic relations between words: Evidence from reduction in lexical
production. In J. Bybee & P. Hopper (Eds.), Frequency and the Emergence of Linguistic Structure (Amsterdam: John Benjamins), pp. 22954.
Lehiste, I. 1973. Phonetic disambiguation of syntactic ambiguity. Glossa, 7, 107-21.
Lerdahl, F., & Jackendoff, R. 1983. A Generative Theory of Tonal Music. Cambridge, MA: MIT Press.
(cont’d on next slide)
References (cont’d.)
Levy, R., & Jaeger, F. 2007. Speakers optimize information density through syntactic reduction. Proceedings of the Twentieth Annual
Conference on Neural Information Processing Systems.
Palmer, C. 1996. On the assignment of structure in music performance. Music Perception 14, 21-54.
Repp, B. 1996. Patterns of note onset asynchronies in expressive piano performance. Journal of the Acoustical Society of America, 100,
3917-3932.
Roederer, J. 1975. Introduction to the physics and psychophysics of music. New York: Springer.
Sales, G. 1984. Jazz: America's Classical Music. Englewood Cliffs, N.J.: Prentice-Hall.
Schuller, G. 1968. Early Jazz. New York: Oxford University Press.
Sloboda, J. 1983. The communication of musical metre in piano performance. Quarterly Journal of Experimental Psychology 35, 377-96.
Sloboda, J. 1985. The Musical Mind. Oxford: Clarendon Press.
Snedeker, J., & Trueswell, J. 2003. Using prosody to avoid ambiguity: Effect of speaker awareness and referential context. Journal of
Memory and Language 48, 103-30.
Sundberg, J. 1988. Computer synthesis of music performance. In J.A. Sloboda (Ed.), Generative Processes in Music, 52–69. Oxford:
Clarendon Press.
Temperley, D. 2001. The Cognition of Basic Musical Structures. Cambridge, MA: MIT Press.
Temperley, D. 2007. Music and Probability. Cambridge, MA: MIT Press.
Todd, N. P. M. 1985. A model of expressive timing intonal music. Music Perception 3, 33-58.
Waterman, G. 1974. Ragtime. In N. Hentoff and A. J. McCarthy (eds.), Jazz: New Perspectives on the History of Jazz by Twelve of the
World’s Foremost Jazz Critics and Scholars, pp. 43–57. New York: Da Capo Press.
Windsor, L., & Clarke, E. 1997. Expressive timing and dynamics in real and artificial musical performances: Using an algorithm as an
analytical tool. Music Perception, 15, 127-152.