Temporal guidance of musicians’
performance movement is an acquired skill
Rodger, M. W. M.1., 2., O’Modhrain, S.2., 3. and Craig, C. M.1.
1.
School of Psychology, Queen’s University Belfast, Belfast, UK
Sonic Arts Research Centre, School of Creative Arts, Queen’s University
Belfast, Belfast, UK
2.
3.
School of Music, Theatre and Dance, University of Michigan, MI, USA.
Correspondence concerning this article should be addressed to: Matthew W. M.
Rodger, School of Psychology, Queen’s University Belfast, David Keir Building,
18-30 Malone Road, Belfast, UK. BT9 5BN
E-mail: m.rodger@qub.ac.uk
Telephone: +44 (0) 28 9097 5653
Fax: +44 (0) 28 9097 5486
Abstract
The ancillary (non-sounding) body movements made by expert musicians during performance
have been shown to indicate expressive, emotional and structural features of the music to
observers, even if sound of the performance is absent. If such ancillary body movements are a
component of skilled musical performance then it should follow that acquiring the temporal
control of such movements is a feature of musical skill acquisition. This proposition is tested using
measures derived from a theory of temporal guidance of movement, ‘General Tau Theory’ (Lee
1998; Lee et al. 2001), to compare movements made during performances of intermediate-level
clarinetists before and after learning a new piece of music. Results indicate that the temporal
control of ancillary body movements made by participants was stronger in performances after the
music had been learned, and was closer to the measures of temporal control found for an expert
musician’s movements. These findings provide evidence that the temporal control of musicians’
ancillary body movements develops with musical learning. These results have implications for
other skilful behaviors and non-verbal communication.
Keywords: music performance, skill acquisition, tau theory, temporal control of
movement
1
Introduction
Nearly all actions require the skilful control of movement prospectively
through time to realize their goals, whether these are interceptive (e.g. catching a
ball), locomotive (e.g. timing steps in a triple-jump), or expressive (e.g. ballet
dance). Expressive musical performance is amongst the most skilful activities that
humans are able to achieve. Hundreds of hours of practice are required to master
the fine-grain motor control necessary to produce an aesthetically satisfying
sequence of sounds (Sloboda 1996). However, it is not enough to sound the
correct notes in the right order; in addition, the timing and temporal unfolding of
sequences of notes must be carefully controlled to accomplish an expressive,
engaging musical performance (Palmer 1989; Clarke 1999). Making music,
therefore, is only possible by skillfully coupling the temporal control of
movements to unfolding perceptual goals.
In addition to the effective movements required to excite, sustain or
modulate sound from an instrument, such as bowing across violin strings or
striking the surface of a drum, there are also non-sounding, ancillary movements
made by experts during performance that are musically relevant. Ancillary
movements --- such as upper body sway, nodding, toe-tapping, head shaking, and
other non-sounding gestures --- are perceptually communicative to audiences of
expressive, emotional and structural characteristics of the music, even in the
absence of sound (Dahl and Friberg 2007; Davidson 1993; Vines et al. 2006;
Vines et al. 2011). Indeed, non-musicians have been found to be more accurate at
judging the intended expressive characteristics of performance from visual
observance of ancillary movements in the absence of sound (Davidson 1993).
Furthermore, it is not only the expressive intentions of the performer that can be
detected in their ancillary body movements; the skill level of the player can also
be perceived from vision of body motion alone (Rodger et al. 2012), which
suggests that ancillary performance movements change with and reflect musical
skill acquisition.
For movement to be perceived as skillful there must be some quality in the
biological motion signal that relates to the skilled control of the movement
(Schmidt 2012; Brault et al. 2012). Therefore, if musicians’ ancillary movements
can perceptually specify the skill of the player, this must entail some motor
2
control-relevant property of these movements that is available to the perceiver.
Following this line of argument, changes in musicians' body movements in
performance that occur with learning were studied within the context of a general
theory of temporal guidance of movement: General Tau Theory.
General Tau Theory
General Tau Theory (Lee 1998; Lee et al. 2001) puts forward a unified
account of how organisms can skillfully navigate their environments by coupling
the timing of their movements to the dynamics of external sensory information
and/or oscillations of internal neural power. Drawing on both Gibson’s ecological
psychology (Gibson 1979) and Bernstein's work on the coordination of movement
(Bernstein 1967), the theory posits that the control of any purposeful movement
can be considered as the prospective closure of action-gaps, that is, the gap
between the current state of the organism’s effector and a goal state. Examples of
such action gaps include the distance between a fielder's hand and a cricket ball
(displacement gap), the angle between one’s gaze and the direction of a flash of
light in the periphery of vision (angle gap), the pitch gap between middle-C and
F4 when singing (pitch gap), etc. In all cases, gap closure needs to be controlled
prospectively in time in order to execute a successful action. Thus, to understand
the motor control of a given action, there is a need to identify the temporal
information which is used to guide the closure of the motion gap prospectively in
time (Lee 2004).
It is proposed within this theory that a primary informational invariant for
the prospective control of movement is ‘tau’, which is the changing time-toclosure of a motion-gap, defined as the ratio between the magnitude of the gap
and its current rate of closure. The control of movement involves keeping the tau
of movement (τX(t)) coupled to a guide tau (τY(t)) at a constant ratio (k),
Eq. 1: τX(t) = kX,Y × τY(t).
As the tau of external events is geometrically related to the tau of the energy array
on different sensory systems (Lee 1998), this invariant property can be directly
perceived, and has been implicated as a control variable in a number of different
behaviors across different species. It has been shown that bats and hummingbirds use auditory and visual tau respectively to control their approach to land
3
(Lee et al. 1995; Delafield-Butt et al. 2010), while gannets time the tucking of
their wings based on optical tau during dives (Lee and Reddish 1981). In humans,
drivers use tau information in optic flow to control braking (Lee 1976; Bootsma
and Craig, 2003), and long-jumpers time their footfalls during a run up based on
the tau between them and the board (Lee et al. 1982). The ability to couple the tau
of movement to sensory tau for control of action has also been found to develop in
infants, with increased tau coupling developing to improve performance in
catching a moving toy (Kayed and van der Meer 2009).
In many actions there is no temporal information directly specified by an
unfolding event in the environment onto which one can couple the tau of
movement. For example, the unaccompanied solo singer before she begins her
first note does not have an extrinsic guide for the temporal closure of a pitch
action gap. General Tau Theory posits that in such instances the temporal control
of movement is intrinsically coupled to an internal tau guide ‘tauG’ generated in
the neural system. This intrinsic guide may be considered as “a changing
canonical energy gap” (Schogler et al. 2008, p362). Mathematically, the dynamics
of this guide can be described as the tau of an object moving under constant
gravity (Lee 1998),
Eq. 2: τG(t) = 0.5 × (t – TG2/t),
where TG is the period of the closure of this guide gap, and t is time across
the duration of TG. Intrinsic guidance of movement through tauG-coupling can
thus be described by the equation,
Eq. 3: τX(t) = kX,G × τG(t),
Evidence for intrinsic tauG-coupling has been found in golfers guiding
their swings in putting (Craig et al. 2000a), musicians controlling note-producing
actions (Schogler et al. 2008), timekeeping movements to intercept beats (Craig et
al. 2005), and infants controlling sucking pressures (Craig and Lee 1999). As with
extrinsic tau-guidance, evidence suggests that strength of tauG-coupling
corresponds to motor skill development. Measures of tauG-coupling control for
both increasing and decreasing suction in preterm infants' were initially markedly
weaker when compared with term infants (Craig et al. 2000b). As preterm infants
matured and became more adept at feeding, the tau-coupling control measures
were found to approach those values obtained for term infants.
4
In both extrinsically and intrinsically tau guided movements, the k-value
or constant ratio between the two taus determines the velocity and acceleration
characteristics of the movement. As k increases from 0 to 1, the temporal position
of peak velocity moves from near the start of the movement (e.g. for k=0.3)
towards the end of the movement. (e.g. for k=0.8). A k value of 0.5 represents a
bell-shaped velocity profile. Qualitatively, tau-coupled movements with greater kvalues have the quality of greater force at the endpoint, or more “oomph”
(Schogler et al. 2008, p363), than movements with lower k-values. In controlling
pitch and intensity glides, double bass players and singers were shown to vary the
k-values of tauG-coupled notes between performances with different emotional
intentions (Schogler et al. 2008). This indicates that tau-coupling as a control of
movement also shapes the expressive qualities of actions.
In this study, tauG-coupling in musicians’ ancillary body movements was
measured as players learned to play a novel piece of music. The learning of a new
piece constitutes a dimension of musical skill acquisition, as entirely novel
sequences and patterns of actions must be acquired even if individual component
actions have been mastered (Sloboda 1996; Drake & Palmer 2000). As the
ancillary movements of expert musicians have been related to the expressive,
emotional and structural flow of music in performance, these movements
constitute an interesting component of skilful communicative performance. In this
study, we investigated whether temporal control of performers’ ancillary
movement changed with the acquisition of musical skill.
Method
Participants
Three clarinetists, with a playing ability ranging from grade levels 5-8 as
defined by the Associated Board of the Royal Schools of Music1, were recruited
through advertisement, and took part voluntarily. One professional clarinetist took
part in a single session to give benchmark values for tauG-coupling of ancillary
movements. This expert was recruited through an artist-in-residence program
within the School of Creative Arts at Queen’s University Belfast.
1
Details of ABRSM grade definitions can be found at the following online address:
http://www.abrsm.org/regions/fileadmin/user_upload/syllabuses/clarinet0510.pdf
5
Apparatus and Setup
3D motion-capture data were recorded using 4 Qualisys ProReflex infrared
cameras, running on Qualisys Track Manager software, sampling at 120Hz.
Participants wore 6 reflective passive markers: one on each leg above the knee
joint; one on each upper arm, midway between the elbow and shoulder joints; one
on the bell of the clarinet; and one on a cap worn on the head so that the position
of this marker was above the forehead. These marker placements were chosen
because existing research has identified these anatomical locations as key sites of
musically-relevant movement in clarinet performance (Wanderley et al. 2005).
Audio recordings were taken of each performance, using one Rode stereo
condenser microphone positioned 1 m in front of the participant connected to a
TASCAM HD-P2 digital audio recorder unit.
The piece of music used for all trials in this study was `Strange But True'
from the 2008-2013 ABRSM Clarinet Grade 3 performance syllabus [Associated
Boards of the Royal Schools of Music 2007]. This piece was chosen to be
sufficiently easy to be learned by all participants over the course of the trials.
Procedure
The 3 participants each took part in 5 sessions, with sessions consisting of
4 trial performances. Recording sessions were separated by 1-2 days and
participants were told not to practice the piece between sessions. For each trial,
the participants began playing the music from the score positioned in front of
them, after an auditory signal to indicate the beginning of the trial. Recordings
ended when either the participant had finished playing the piece or a predefined
maximum recording time (60 seconds) had elapsed. The expert took part in a
single session, consisting of 3 trial performances of the same piece, with the same
instruction and recording process. Only movements made during the
performances of the music (between the onset of the first note and offset of the
final note) were used for analysis to ensure that all trials contained movements
coinciding with musical sound. Ethical approval for this study was granted by a
local ethics committee.
6
Analysis
All data were filtered using a 10th-order reverse Butterworth filter, with a
cut-off frequency of 20 Hz. Principal Component Analysis (PCA) was carried out
on the 18-vector (6 markers × 3 dimensions) x, y, z position data from motion
capture of each trial performance. PCA is a data reduction technique for
compression of large data sets (Jolliffe 2002), and has been shown to be
appropriate for feature extraction in human movement analysis (Daffertshofer et
al. 2004). This technique allows the extraction of common motional variance
across different movement points and dimensions, by reducing the dimensional
space of the data set on the basis of covariance between signals. Projecting the
original data onto this new dimensional space results in a signal that maximally
represents this covariance. Hence, for example, a swooping motion of the upper
body which would contain motion of a number of markers in different dimensions
would be captured by PCA as the common covariance of that movement across
each of these different position signals. This covariance is then represented by a
single signal which captures the underlying movement. For the motion analysis in
this study, the first principal component was used, which represents the changing
quantity of movement that best described the overall body motion of the
performer in each trial, that is, the greatest common variance from all markers that
contribute to this movement dimension. From the principal component of each
trial, the individual movements – semi-periods – were calculated as the movement
displacement values on the principal component vector between velocity zerocrossings. Movements with durations less than 100 ms were discarded as these
were taken to be too short in duration to be relevant to performance.
Following the analysis method used by Craig et al. (2005) and Schogler et
al. (2008), the changing tau profile within each movement semi-period was
calculated as the displacement divided by the velocity with respect to time
throughout the movement. Then, for each movement semi-period, a corresponding
tauG guide of equal duration was calculated using Equation 2, with TG being equal
to the movement duration. The tau of the movement was linearly regressed
against the hypothetical tauG guide recursively, by successively excluding the first
n samples (n1 = 0), until the r-squared of the regression model exceeded 0.95. The
measure of coupling-strength from the analysis is the percentage of the original
movement that remains in the model after n samples have been excluded to give
7
an r-squared greater than 0.95, with higher values indicating stronger coupling of
the tau of the movement to a tauG guide. A second more qualitative measure is the
slope of the regression line between the tauG guide and the tau of the movement,
which is an estimate of the average k-value in the movement.
In addition to the tau analysis of movement, the jerk of movements were
also analysed for comparison, as the inverse of mean jerk of movement is often
used as a measure of smoothness. A dimensionless measure of jerk was calculated
for each individual movement, to account for variance in movement duration and
amplitude, as suggested by Hogan and Sternad (2009), using the following
equation,
𝑡2
⃛ (𝑡)2 𝑑𝑡)𝐷5 /𝐴2
Eq 4: (∫𝑡1 𝑥
,where D is the duration of an individual movement between t1 and t2, and A is the
movement amplitude.
Musicians’ ratings of audio recordings of performances
To check that the participants had improved in their performance of the piece over
the course of the study, 15 trained musicians, with an average of 17.4 years
playing experience (s.d. = 6.6 years), rated audio recordings of performances
which were selected randomly from each participant in the first, third and final
sessions. Musicians rated performances on two 7-point Likert-scales, technicality
(‘accuracy of pitch, timing, dynamics, timbre’) and expressivity (‘emotion, colour,
flow, personality’). Participants in this rating study were given a copy of the
musical score to assist in making their judgments. The order of presentation of the
audio clips was randomized. Participation was voluntary.
Results
The ratings of audio recordings of performances by trained musicians
(Table 1) revealed that all participants improved in both technical accuracy and
degree of expressivity over the 5 sessions, showing that learning of this piece of
music had taken place. Analysis of improvement in musical performance on each
measure was carried out for each participant by subtracting the ratings given in
the final session from those given in the first and using single sample t-tests to
compare these differences to 0 (using a Bonferroni-corrected of alpha value of
8
.0167 for multiple t-tests). In terms of technicality, ratings in the 5th session were
significantly higher than in the first for all three participants (Participant1: t(14) =
13.23, p < .001, Cohen’s d = 3.41; Participant2: t(14) = 6.10, p < .001, Cohen’s d =
1.57; Participant3: t(14) = 2.75, p < .001, Cohen’s d = 0.71). On expressivity,
significant increases in ratings between session 1 and 5 were found for Participant
1 (t(14) = 3.67, p = .003, Cohen’s d = 0.95) and Participant 2 (t(14) = 5.00, p < .001,
Cohen’s d = 1.29). However, although ratings of expressivity for Participant 3 did
increase, this change was not significant after correcting for multiple t-tests (t(14) =
2.57, p = .022, Cohen’s d = 0.66). It can also be noted from Table 1 that the mean
ratings for the expert performer were higher than those of the participants, as
would be expected.
Table 1. Mean ratings of technicality and expressivity (on 7-point scales) for performances by each
participant in sessions 1, 3 and 5, and for the expert (standard deviations in brackets). Significant
differences in ratings between the first and final sessions are shown,
Technicality
Expressivity
Session 1
Session 3
Session 5
Session 1
Session 3
Session 5
Participant 1
1.3 (0.5)
1.9 (0.5)
2.9 (0.6)*
1.5 (0.6)
2.0 (0.7)
2.3 (0.9)
Participant 2
3.5 (1.0)
4.2 (1.3)
5.3 (0.6)*
3.3 (1.0)
4.7 (1.4)
5.0 (1.0)
Participant 3
4.7 (0.9)
4.7 (1.0)
5.4 (1.1)*
3.5 (1.1)
4.1 (1.1)
4.3 (1.2)
Expert
5.8 (0.9)
5.9 (0.7)
* - significant difference between first and final session after Bonferroni correction of alpha value.
Analysis of the ancillary movements of each of the three participants was
carried out as individual case studies. The outcomes of the principal component
analysis for each of the participants, as well as that of the expert clarinetist, are
shown in Figure 1. The figure shows the variance explained by the first n
principal components in each performance that cumulatively sum up to explain >
90 % of the overall variance in the original movement signals. For all participants,
a large amount of the full body movement is captured by the first principal
component. Analysis of correlation between the first principal component and
each of the original motion capture signals revealed that for participants 2 and 3,
the first component correlated highly with movement of the all 6 markers in their
horizontal axes (mean r’s across performances of markers in horizontal axis: P2
=.89 – .97; P3 = .80 – .90). This indicates that the main source of ancillary
9
movement for these participants was in lateral full body sway. This is similar to
results found for the expert, although the principal component of the expert’s
ancillary movement showed more lateral sway in the upper body (mean r’s for
head and arms in horizontal axis: .83 – .95; mean’s for clarinet bell and legs in
horizontal axis: .70 – .78). For participant 1, the movements that correlated
highest with the principal component varied across the study, coming mostly from
the sagittal movement of the upper body in the first session (mean r’s for head and
arms in sagittal axis: .83 – .98), and from the the horizontal movement of the
upper body in the final session (mean r’s for head and arms in sagittal axis: .78 –
.97). Thus, this participant’s style of ancillary movement changed over the course
of learning to play the piece of music.
Figure 1. Results from principal component analysis averaged over the 4 performances in each
session for each of the three participants and the expert. Bars show percentage variance from
original data captured by the first principal components that cumulatively sum up to > 90 %
variance explained.
Figure 2 illustrates the steps to calculate measures of tauG-coupling for an
example movement taken randomly from a captured performance. Mean values
for these measures were calculated for each trial, and then a mean average was
taken across the trials for each session. Figure 3 shows the means for percentagecoupling in movements for each participant, averaged over the four performances
for each of the five sessions, with the mean for the expert shown as a horizontal
line in each case. Single-subject model statistics (Bates et al. 2004) were used to
compare means of percentage tau-coupling between the first and final session for
each participant. Mean percentages of movement-coupled were found to increase
over the five sessions for all three participants (P1: 85.4 % → 93.0 %, p < .01; P2:
90.9 % → 97.6 %, p < .01; P3: 84.7 % → 92.6 %, p < .05). This indicates a
significant increase in the degree of tauG coupling of body movements in
performance as the skill to play the piece of music was acquired. Figure 3 shows
that, for all participants, this increase in mean percentage tau-coupling values tend
towards the benchmark value of the expert performer (mean = 97.3 %).
Figure 2. Stages of tau analysis with an example movement. (a) Movement velocities were
calculated and individual movements were extracted from points between velocity zero-crossings.
10
(b) The tau of each movement was calculated from X (displacement) / Ẋ (velocity). Corresponding
intrinsic tauG guides for each movement were also calculated. (c) The tau of each movement was
recursively linearly regressed against the tauG guide, until the r-squared of the regression exceeded
0.95. The gradient of the regression line constituted the estimated k-value. In this example: rsquared = 0.987; percentage of movement coupled = 100%; estimated k-value = 0.292.
Figure 3. Mean values for percentage of movement tauG-coupled (percentage of movement
remaining when r > .95 in recursive regression model) for each participant from tau analysis of the
participants’ ancillary body movements over the five recording sessions. The mean percentage taucoupling value for the expert clarinetist from one session is shown as a dashed horizontal line. * difference between means significant, p < .05. ** - difference between means significant, p < .01.
Error lines represent standard error of means across the four performances within each session.
Estimated k-values from the tau-coupling analysis are shown for each
participant as means across the four performances for each of the five sessions in
Figure 4. These were also compared between the first and final session using
single-subject model statistics. For participant 1 and 2, the decreases in the kvalue over the course of the five sessions were statistically significant (P1: 0.851
→ 0.550, p < .05; P2: 0.676 → 0.484, p < .01). Although participant 3 also
showed a decrease in the k-value of movements over the sessions, this did not
reach significance (P3: 0.760 → 0.605, p > .05). This result indicates that, on the
whole, the average abruptness of body movements decreased as learning of the
musical piece progressed. It can be noted (see Figure 4) that this change in value
was in the direction towards the mean k-value of movements in the expert
performances (mean = 0.529).
Figure 4. Mean estimated k-values from tau analysis of ancillary body movements averaged over
the four performances in each of the five sessions, for each of the three participants. The estimated
k-value for the expert clarinetist is shown as a horizontal dashed line in each case. * - difference
between means significant, p < .05. ** - difference between means significant, p < .01. Error lines
represent standard errors of means across the four performances within each session.
Analysis of the mean dimensionless jerk of movements using singlesubject model statistics (Bates et al. 2004) revealed no significant difference
between the first and the final session for any participant (Figure 5). However, it
can be noted that participant 3 did appear to show a decrease in jerk over the five
11
sessions, even though this was not significant. Also, the mean jerk of movements
made by the expert was lower than the means of the participants. This would
suggest that there may be some slight relationship between musical skill and the
inverse of jerk as a measure of smoothness in ancillary movements. Correlation
analysis between jerk and the tau analysis measures revealed no significant
correlations between jerk and either percentage coupling (P1: r(23) = -.04; P2: r(23)
= -.14; P3: r(23) = -.05, all p > .05) or k-value (P1: r(23) = -.04; P2: r(23) = -.14; P3:
r(23) = -.05, all p > .05). This would indicate that the changes observed in the tauGcoupling of musicians’ ancillary movements over the course of learning to play
the piece of music are not related to reduction in movement jerk, which is a more
standard measure of movement smoothness, but rather reveal something different
about the manner with which the movements are controlled in time.
Figure 5. Mean dimensionless jerk of ancillary body movements averaged over the four
performances in each of the five sessions, for each of the three participants. The mean jerk of
movement for the expert clarinetist is shown as a horizontal dashed line in each case. Error lines
represent standard errors of means across the four performances within each session.
Discussion
As participants learned to play a new piece of music, the strength of tauGcoupling of the ancillary movements made while playing the piece of music were
found to significantly increase over the 5 training sessions. This would suggest
that the control of such ancillary movements, which have elsewhere been shown
to be musically relevant in perception and performance (Davidson 1993;
Wanderley et al. 2005), improves over the course of musical skill acquisition.
This trend for greater strength of tauG-coupling of movement after learning is
further supported by comparisons with a professional expert clarinetist, whose
movements in performance revealed the highest degree of tauG guidance.
Coupling strength between the tau of movement and tauG guide have previously
been linked to levels of skill (Craig et al. 2000a) and motor control ability (Craig
et al. 2000b). The results from the present experiment provide evidence that the
performance gestures of musicians (which are generally unintentional, yet related
to the music itself) become more controlled as the music being played is learned
12
and the movements required to play the notes also become more controlled. This
suggests that ancilliary body movements may be an acquired skill in embodied
musical performance.
The results also showed that the average coupling constant, k, of tauGguided body movements decreased between the first (unlearned) and final
(learned) performances. As with the results for the strength of temporal control of
movement, the k-values of movements made by the participants in their final
session performances were closer to the results found in performance movements
made by the expert professional musician. The change in the k-value measure
over the course of the trials may relate to the changing intentional qualities of
movements as the piece of music is acquired. Musicians have been shown to
modulate this property for tauG-guidance of their effective, sound-producing
actions for control of expressive effects (Schogler et al 2008), and so the results
here could reflect a convergence towards a particular form of expressive
movement for performance of this piece of music. On the other hand, in a study of
acquisition of balance control in gait initiation from infant to adult, k-values in
tauG-coupled centre-of-pressure movements were found to decrease with age
(Austad and van der Meer, 2007), which was taken to show that as balance control
developed, movements were controlled to be less collision-like and more touchlike. The present results, therefore, might be taken to show that musicians’
performance body movements became less abrupt while learning the piece of
music, reflecting a greater degree of skilled control.
If the control of ancillary movements in learned music performance is
tauG-coupled, as these results would suggest, then it may be the tau of movement
as it is picked up by observers that is allowing them to accurately judge the skill
level of performers from vision of their movements alone (Rodger et al. 2012). In
rugby, it has been found that expert players tune into the tau of key body
movement segments in detecting deceptive movements made by an attacking
player (Brault et al. 2012). Sensitivity to the tau of movement may also allow
audiences to perceive information about musical performance from musicians’
body movements.
In this study, the sound recordings of performances were judged as both
more technically accurate, and more expressive following musical learning. As
the measures of tau-coupling also increased over the sessions, it appears that
13
improvements in musical technique are accompanied by improvement in the
temporal control of ancillary movements. It is interesting to note that changes in
ratings of expression over the sessions were not significant for Participant 3, and
nor were the changes in k-value, suggesting that this measure of ancillary
movement may be related to musical expressivity.
It would be informative to find a way to relate the tau-coupling of
musicians’ ancillary movements to the relevant higher-order properties of the
sound of musical performance. Schogler et al. (2008) showed that the bowing
movements of cellists and vocal fold movements of singers matched the tau
patterns found in pitch and intensity glides in the resulting musical sound. From
the results in the present study, it could be hypothesized that the temporal patterns
of musicians’ ancillary body movements would correlate with the tau of a higherorder property of the musical sound, such as the temporal envelope of changing
intensity or tempo across multiple notes, which have been shown to be vehicles
for musical expression (Clarke 1999; Palmer 1989). Todd (1992, 1995) has
argued that the temporal shapes of expressive musical timing follow contours
similar to those created by the motion of objects under gravity, and so given that
the formulation of tauG guides is based on such motion, there may be a natural
link between the patterns of timing in musicians’ ancillary body movements and
the patterns of timing found in musical expression. Presenting mismatched visual
recordings of body movements and sound has been shown to disrupt perceivers’
judgment of skill level in music performance (Rodger et al. 2012), which would
suggest some common neural processes are involved in the perception of musical
sound and ancillary body movement. It is possible that the tau of movements and
that of changing properties in the sound are linked neurally in a multi-modal,
embodied perception of music.
The results found here, namely that musicians’ ancillary performance
movements become more tauG-coupled over the course of learning a new piece of
music, have implications for other skilful and communicative behaviors. The
tauG-coupling of movements in sport or dance may provide a useful metric of a
performer’s developing motor control as they acquire new movement-based skill,
which could assist in training evaluation and instruction. In the domain of speech
behavior, the communicative power of non-verbal gesture (Goldin-Meadow 2003;
McNeill 2005), which has parallels with non-sounding gestures in music
14
(Wanderley and Vines 2006), entails that speech accompanying movements
require skilful motor control. Measuring the degree of temporal guidance of
movements and changes in intentional quality using the techniques employed in
this experiment may provide insight into the control of such movements, and
could help to identify disorders or atypical development in communication.
Conclusion
The ancillary body movements made by clarinetists during performances
showed increasing temporal control during the learning of a new piece of music as
measured by tauG-coupling. Moreover, coupling constant k-values, which describe
the unfolding quality of a movement, decreased over the sessions, indicating a
transition toward more smooth movements as a piece is learned. These results
follow other research into changes in tau guidance for motor control that occur
with improved movement ability. It follows that expressive body movements
made by expert musicians during performance require skilful temporal control,
and that this develops during musical learning. The implications of these findings
are not just important for understanding music performance and the observation of
skill on the part of audiences, but are also relevant to other skilful or
communicative behaviors, such as sports performance, or non-verbal gestural
communication in speech.
Acknowledgements
This research was supported by the SKILLS integrated project funded by the European
Commission (FP6-IST, Proposal/Contract no.: 035005) and by the TEMPUS-G project funded by
the European Research Council (210007 StIG). Thanks go to Rob Plane for giving of his time to
record the expert performances used in this study.
References
Associated Board of the Royal Schools of Music (2007) Selected clarinet exam pieces 2008-2013
Grade 3. ABRSM Publishing, London
Austad H, Van der Meer ALH (2007) Prospective dynamic balance control in healthy children and
adults. Exp Brain Res 181:289-295. doi:10.1007/s00221-007-0932-1
Bates BT, James CR, Dufek JS (2004) Single-subject analysis. In: Sterigou N (Ed.) Innovative
Analysis of Human Movement. Human Kinetics, Leeds, pp 3-28.
Bernstein NA (1967) The co-ordination and regulation of movements. Pergamon Press, Oxford
15
Bootsma RJ, Craig CM (2003) Information used in detecting upcoming collision. Percept 32:525544. doi:10.1068/p3433
Brault S, Bideau B, Kulpa R, Craig CM (2012) Detecting deception in movement: The case of the
side-step in rugby. PLoS One 7:e37494. doi: 10.1371/jounral.pone.0037494
Clarke EF (1999) Rhythm and timing in music. In: Deutsch D (Ed.) The psychology of music, 2nd
edition. Academic Press, New York, pp 473-500.
Craig CM, Delay D, Grealy MA, Lee DN (2000a) Guiding the swing in golf putting. Nat 405:295296. doi:10.1038/35012690
Craig CM, Grealy MA, Lee DN (2000b) Detecting motor abnormalities in preterm infants. Exp
Brain Res 131:359-365. doi:10.1007/s002219900227
Craig CM, Lee DN (1999) Neonatal control of sucking pressures. Evidence for an intrinsic (tau)guide. Exp Brain Res 124:371-382. doi:10.1007/s002210050634
Craig CM, Pepping G, Grealy M (2005) Intercepting beats in predesignated target zones. Exp
Brain Res 165:490-504. doi:10.1007/s00221-005-2322-x
Daffertshofer A, Lamoth CJC, Meijer OG, Beek PJ (2004) PCA in studying coordination and
variability: a tutorial. Clin Biomech 19:415-428. doi:10.1016/j.clinbiomech.2004.01.005
Dahl S, Friberg A (2007) Visual perception of expressiveness in musicians' body movements.
Music Percept 24:433-454. doi:10.1525/mp.2007.24.5.433.
Davidson J (1993) Visual perception of performance manner in the movements of solo musicians.
Psychol Music 21:103-114. doi:10.1177/030573569302100201.
Delafield-Butt J, Galler A, Schogler B, Lee DN (2010) A perception - action strategy for
hummingbirds. Percept 39:1172-1174. doi:10.1068/p6708
Drake C, Palmer C (2000) Skill acquisition in music performance: relations between planning and
temporal control. Cogn 74:1-32. doi:10.1016/S0010-0277(99)00061-X
Gibson JJ (1979) The ecological approach to visual perception. Lawrence Erlbaum Associates,
Hillsdale, NJ
Goldin-Meadow S (2003) Hearing gesture: how our hands help us think. Harvard University Press,
Harvard, Belknap
Hogan N, Sternad, D (2009) Sensitivity of smoothness measures to movement duration, amplitude,
and arrests. J Motor Behavior 41:529-534. doi:10.3200/35-09-004-RC
Jolliffe IT (2002) Principal component analysis, 2nd ed. Springer, New York
Kayed NS, Van Der Meer ALH (2009) A longitudinal study of prospective control in catching by
full-term and preterm infants. Exp Brain Res 194:245-258. doi:10.1007/s00221-008-1692-2
Lee DN (1976) A theory of visual control of braking based on information about time to collision'.
Percept 5:437-459
Lee DN (1998) Guiding movement by coupling taus. Ecol Psychol 10:221-250. doi:
10.1207/s15326969eco103&4_4.
Lee DN (2004) Tau in action in development. In: Reisser JJ, Lockman JJ, Nelson, CA (eds) Action
as an organizer of learning. Lawrence Erlbaum Associates, Hillsdale, NJ, pp 3-49
Lee DN, Georgopoulos AP, Clark MJO, Craig CM, Port NL (2001) Guiding contact by coupling
the taus of gaps. Exp Brain Res 139:151-159. doi:10.1007/s002210100725
16
Lee DN, Lishman JR, Thomson JA (1982) Regulation of gait in long jumping. J Exp Psychol:
Hum Percept Perform 8:448-459. doi:10.1037/0096-1523.8.3.448
Lee DN, Reddish PE (1981) Plummeting gannets: a paradigm of ecological optics. Nat 293:293294. doi:10.1038/293293a0
Lee DN, Simmons JA, Saillant PA, Bouffard F (1995) Steering by echolocation: a paradigm of
ecological acoustics. J Comp Physiol 176:347-354. doi:10.1007/bf00219060
McNeill D (2005) Gesture and thought. University of Chicago Press, Chigago
Palmer C (1989) Mapping musical thought to musical performance. J Exp Psychol: Hum Percept
Perform 15:311-346. doi:10.1037/0096-1523.15.2.331
Rodger MWM, Craig CM, O’Modhrain (2012) Expertise is perceived from both sound and body
movement in musical performance. Hum Mov Sci 31:1137-50. doi:10.1016/j.humov.2012.02.012
Schmidt A (2012) Movement pattern recognition in basketball free-throw shooting. Hum Mov Sci.
doi:10.1016/j.humov.2011.01.003
Schogler B, Pepping G-J, Lee DN (2008) TauG-guidance of transients in expressive musical
performance. Exp Brain Res 189:361-372. doi:10.1007/s00221-008-1431-8
Sloboda J (1996) The acquisition of musical performance expertise: deconstructing the "talent"
account of individual differences in musical expressivity. In: Ericsson KA (ed) The road to
excellence: the acquisition of expert performance in the arts and sciences, sports, and games
Lawrence Erlbaum Associates, Hillsdale, NJ, pp 107-126
Todd NPM (1992) The dynamics of dynamics: A model of musical expression. J Acoust Soc Am
91:3540-3550. doi: 10.1121/1.402843
Todd NPM (1995) The kinematics of musical expression. J Acoust Soc Am 97:1940-9
Vines BW, Krumhansl CL, Wanderley MM, Levitin DJ (2006) Cross-modal interactions in the
perception of musical performance. Cogn 101:80-113. doi:10.1016/j.cognition.2005.09.003
Vines BW, Krumhansl CL, Wanderley MM, Dalca IM, Levitin DJ (2011) Music to my eyes:
Cross-modal interactions in the perception of emotions in musical performance. Cogn 118:157170 doi:10.1016/j.cognition.2010.11.010
Wanderley MM, Vines BW (2006) Origins and Functions of Clarinettists' Ancillary Gestures. In:
King E, Gritten A (eds) Music and gesture: new perspectives on theory and contemporary practice.
Ashgate, Hampshire, UK, pp 165-191
Wanderley MM, Vines BW, Middleton N, McKay C, Hatch W (2005) The musical significance of
clarinetists' ancillary gestures. J New Music Res 34:97-113. doi:10.1080/09298210500124208
17
Figure 1. Results from principal component analysis averaged over the 4 performances in each
session for each of the three participants and the expert. Bars show percentage variance from
original data captured by the first principal components that cumulatively sum up to > 90 %
variance explained.
18
Figure 2. Stages of tau analysis with an example movement. (a) Movement velocities were
calculated and individual movements were extracted from points between velocity zero-crossings.
(b) The tau of each movement was calculated from X (displacement) / Ẋ (velocity). Corresponding
intrinsic tauG guides for each movement were also calculated. (c) The tau of each movement was
recursively linearly regressed against the tauG guide, until the r-squared of the regression exceeded
0.95. The gradient of the regression line constituted the estimated k-value. In this example: rsquared = 0.987; percentage of movement coupled = 100%; estimated k-value = 0.292.
19
Figure 3. Mean values for percentage of movement tauG-coupled (percentage of movement
remaining when r > .95 in recursive regression model) for each participant from tau analysis of the
participants’ ancillary body movements over the five recording sessions. The mean percentage taucoupling value for the expert clarinetist from one session is shown as a dashed horizontal line. * difference between means significant, p < .05. ** - difference between means significant, p < .01.
Error lines represent standard error of means across the four performances within each session.
20
Figure 4. Mean estimated k-values from tau analysis of ancillary body movements averaged over
the four performances in each of the five sessions, for each of the three participants. The estimated
k-value for the expert clarinetist is shown as a horizontal dashed line in each case. * - difference
between means significant, p < .05. ** - difference between means significant, p < .01. Error lines
represent standard errors of means across the four performances within each session.
21
Figure 5. Mean dimensionless jerk of ancillary body movements averaged over the four
performances in each of the five sessions, for each of the three participants. The mean jerk of
movement for the expert clarinetist is shown as a horizontal dashed line in each case. Error lines
represent standard errors of means across the four performances within each session.
22