Piano touch, timbre,
ecological psychology, and
cross-modal interference
Richard Parncutt
Centre for Systematic Musicology
University of Graz, Austria
ISPS, 28-31 August 2013, Vienna
International Symposium on Performance Science
Abstract
The piano has a wide timbral range, and performance quality is often judged
in timbral terms. Yet despite decades of research, there are still fundamental
disagreements about the nature and origin of piano touch. Scientists
(acousticians) maintain that the timbre of a single tone cannot be varied
independently of its loudness. Performers, humanities scholars and concert
audiences take the opposite for granted: timbre and loudness can be
independently varied by gestural means. Both sides are right, but their
implicit definitions of timbre differ, and both fail to clearly distinguish
between physical measures and descriptions of subjective experience.
Scientists assume that timbre depends only on physical sound parameters;
but experiential parameters generally depend on concurrent input from other
senses, the listener’s relevant knowledge and expectations, and immediately
preceding and following events. The paradox of timbre disappears if we
accept, based on empirical evidence, that timbre generally depends on input
from more than one sensory modality (weak synesthesia). Embodied
corporality and conceptual metaphors are the norm - not the exception.
Gestural and ecological approaches to timbre perception pose existential
challenges to disembodied cognitive orientations.
Timbre in traditional psychoacoustics
American Standards Association (1960):
“that attribute of sensation in terms of which a
listener can judge that two sounds having the
same loudness and pitch are dissimilar”
A negative definition!
• Neither pitch nor loudness
• The multidimensional remainder
Timbre is experience!
• Traditional psychoacoustics: clear distinction
between physical and experiential variables
• But many psychoacousticians talk about
timbre as if it were physical
• I assume that timbre is always experiential
Timbre depends on
Time domain
– temporal envelope
Frequency domain
– spectral envelope
Vocal timbre
Frequency Domain
Time Domain:
Vocal Tract Shape
Articulation
Spectral Envelope
Temporal envelope
Timbre of musical instruments
• Mixture of time and frequency domain
– Spectral flux: Different temporal envelopes of partials
• Absolute frequency
– spectral & temporal envelopes depend on fo
– timbre learned for each absolute frequency
• Spectral interval structure
– Clarinet, bowed violin: exactly harmonic
– Bell: inharmonic
– Piano, guitar: stretched harmonic
Spectral flux in a bass clarinet tone
Grey & Moorer (1977)
McAdams (1993)
Timbral space
Spatial representation of timbral relationships
Method:
•
•
•
•
Choose some timbres
Rate similarity of all pairs
Multidimensional scaling
Result is usually 3-D, axes:
– onset, centroid, roughess…
– depending on tested sounds
Interpretation:
• Results difficult to generalise
• Cognitive representation?
• Distances in space depend on
exposure & familiarity
McAdams et al. (1995), Psychological Research
Ecological psychology of timbre
•
•
Environmental interaction
Experience as byproduct of interaction
(epiphenomenon)
Theory of James J. Gibson:
• Affordances and information pick-up
• Invariances of sound sources
• Perceptual learning
Complementary to the cognitive approach
Relationship to evolutionary psychology
Distal versus proximal stimuli
External versus internal focus of attention
In sport and music, performance may be better if attention
directed to effects of movements - not movements
themselves (Wulf & Prinz, 2001):
• Golf: attention on goal, not movements
• Pianists: attention to sound, not technique
Why?
• Sophisticated unconscious motor control mechanisms
• Perception of goal inseparable from proprioception
Implication
Perception of timbre involves source perception
Ecological-evolutionary foundations
of auditory perception
Assumption: Sounds are only interesting
( consciously perceived) if they carry
information about environmental
interaction that could affect survival or
reproduction.
The affordances of a sound source
• what it means to us
• what we can do with it
 Timbre reflects affordances!
• Human voice
• Lion footsteps, waterfall, car, gun…
Words to describe the jazz voice
pleasant
black
LOOSE
dark
round
easy
BRIGHT
OPEN
CLASSICAL
AIRY
yawning
forward
NASAL
HARSH
CLEAR
SMOKEY
FORCEFUL
SPOKEN
swinging
resonant
BREATHY
VIBRATO
FIXED
soft
NATURAL
held back
PRESSED
support
relaxed
(Prem & Parncutt, 2007)
Affordances of vocal timbre
“I fell in love with his voice. He is African
American, and his voice has this depth, it is
souly, in a wonderful way. He is like a
gentleman, and has this warmth. He gives
his voice space and that arouses a very
balanced feeling in me. His sadness, his
elegance, his joyfulness really come
through, so that I can experience them.”
Data collected by Ella Prem
Timbre is multimodal
• Performer can’t separate timbre (as experience!)
from gestures used to achieve it
• Listeners share the performer’s experience
(empathy)
How?
• audiovisual mirror neurons (Kohler et al., 2002)
• auditory-motor interactions (Zatorre et al., 2007)
Proprioception
Perception of position and movement of body
parts and associated muscular tension
Part of the somatic sense
touch, proprioception, temperature, pain
An aspect of corporality
Aspects of music experience that depend on the
bodies of performers and listeners
Weak synesthesia
cross-modal interference
• All perceptual modalities: seeing, hearing tasting…
• The norm - not the exception
• Ecologically natural - cognitively strange
The limits of subjective analysis
We cannot completely separate…
• input from different modalities: hearing, vision, touch
• sensations within a modality (pitch, timbre).
A non-musical example
When you taste yoghurt with a lighter
(table-) spoon, it seems
• thicker
• sweeter
• more expensive
Harrar & Spence (2013). The taste of cutlery. Flavour, 2: 21.
Weak synaesthesia in music
• Musical pitch “rises” and “falls”
• A piece “moves”
• A voice is “warm”, “round”, “relaxed”
Embodied cognition
in different disciplines
• Psychology: cognition depends on features of
the physical body of an agent (Wilson &
Foglia, 2011)
• Music psychology: dance, rhythm, aesthetics
etc. (Leman, 2008).
• Humanities: conceptual metaphor, ideas in
one domain help us understood in another
(Lakoff & Johnson, 1980).
Timbral range of the piano
One note:
• Louder is brighter
Many notes:
• higher is brighter
• physical interactions among strings,
soundboard, internal resonances
• dependence on timing, dynamics, pedaling
The pianist’s “touch”
A traditional psychoacoustic approach
Physics:
• Hammer hits string in free flight
• Only free parameter is hammer velocity (shank rotation?)
• Fingertips hit the keys differently in legato and staccato touch
Perception:
• Any effect beyond hammer-velocity effect is usually inaudible
due to masking
(cf. “touch precursor”, “early noise”; Goebl et al., 2004)
Piano touch: A synaesthetic approach
For the pianist:
• All aspects of somatic sense: touch, position,
movement, temperature, pain…
For the audience:
• Vision (perception of gesture)
• Projected somatic sense (empathize with performer)
For both:
Multisensory perception of the interaction between the
instrument and the performer
The feel of a piano
“Pianos and rooms are generally interdependent:
anyone who has ever travelled with a piano knows
that the same Steinway or Bösendorfer not only
sounds different in different halls, but also seems to
react differently in its mechanism. Indeed, the
resistance of the key, over and above the
measurable mechanical aspect, is a psychological
factor”
Alfred Brendel (1976)
The feel of a piano
For pianists, the physical resistance or
bounciness of the piano mechanism depends
• physically on temperature and humidity
• psychologically on touch-sound relationship
(to which they are very sensitive)
Multisensory piano timbre
In physics, spectral and temporal envelopes of an isolated piano
tone cannot be changed independent of intensity
In psychology and experience, timbre involves interactions
between:
• performers’ proximal sensations (tactile, auditory, visual,
proprioceptive)
• distal perception and cognition (performance space;
communication with the audience; cultural context)
These statements are not contradictory!
Weak synasthesia in wind/string timbre
• independent control over pitch and timbre,
but:
• intonation and timbre not psychologically
separate (Ely, 1992; Geringer & Madsen, 1981;
Platt & Racine, 1985)
– musicians who play with good timbre are judged
by experts and amateurs to have good intonation
and vice-versa
Weak synesthesia in Renaissance
choral timbre
Listeners, audiences want “authentic” timbre.
Some confuse that with just intonation.
The sound is presumably optimal near 12-tone
equal temperament (Devaney et al, 2011)
Weak synaesthesia in acousmatic music
Acousmatic music is abstract, electronically
synthesized sound from loudspeakers.
Ecological approach to aesthetics:
• Listeners constantly guess and imagine
sources or causes of musical sounds – as in
everyday life
• Electronically generated sounds sound more
“musical” if similar to familiar sounds
Disembodiment of music theory and
cognitive music psychology
• Music theory depends on the score
• Cognitive music psychology depends on abstract
cognitive structures
The ecological solution:
Study our experience of our physical interactions!
Example: complex tones in speech and music
(Terhardt, 1984).
A new definition of timbre
1.
2.
3.
4.
Experiential
Holistic and multimodal
Quantitative and qualitative
Proximal and distal
1. Timbre is experiential
• Like pitch, loudness, and (in vision) color
• Not a physical parameter!
• Corresponds to physical states and events
2. Timbre is holistic and multimodal
Timbre depends generally and intrinsically on
• input from all senses: hearing, vision, touch, gesture
• physical context: acoustics and appearance of listening
space
• temporal context – immediate, global
• empathy: shared or projected knowledge, experience,
expectations
• emotional reactions
• associations
Our ability to separate these components is limited.
3. Timbre is qualitative and quantitative
• complete description: both kinds of data
• both are intrinsically vague and intangible
• quantitative approach:
– multidimensional (axis labels are qualitative)
• qualitative descriptors
– refer to environment, body, speech (Traube, 2004)
4. Timbre is distal and proximal
• A complete description includes both
• Traditional psychoacoustic approach is proximal
• Combine with ecological source-based approach
Why?
Timbre generally depends on imagined visual and
tactile properties of sound sources, and past
experience of those sources
e.g. different clarinet registers are all “clarinet”
Solving the “touch” problem
• Redefine timbre: combine experiential
psychophysics with ecological psychology
• Promote interdisciplinary interaction:
humanities, sciences and performance
• Abandon philosophical materialism:
(artistic) experience exists!