Journal of New Music Research
Fo
ee
rP
User experience with digital musical instruments:
Development and demonstration of a new generalized
method for longitudinal evaluation
Journal: Journal of New Music Research
rR
Manuscript ID NNMR-2023-0145
Manuscript Type: Research Article
digital musical instruments, user experience, user engagement,
longitudinal evaluation, methodology, T-Stick
ev
Keywords:
Classifications: instruments, interfaces, performance
iew
ly
On
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 1 of 54
User experience with digital musical instruments: Development and
demonstration of a new generalized method for longitudinal evaluation
[author(s)]
iew
ev
rR
ee
rP
Fo
On
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
User experience with digital musical instruments: Development and
demonstration of a new generalized method for longitudinal evaluation
We present development and demonstration of a new method for studying user
experience (UX) with digital musical instruments (DMIs) over time. We
introduce DMIs, music interaction, stakeholders, and components of the userinstrument relationship (UIR), and discuss evaluation strategies for study of the
UIR, indicating limitations of current methods. By using a small sample of
individuals with diverse musical backgrounds and a compressed time period, we
Fo
demonstrate the efficacy of this procedure in examining dynamic aspects of the
UIR with a novel DMI. Results from this initial demonstration indicate that: 1)
the method is effective in assessing changes in UX over time; 2) experience of
rP
the DMI user can be characterized by fluctuations between different areas of
concern based on users’ previous musical experience, and; 3) hedonic and
ee
cognitive UX factors play a notable role in learning to play a new DMI.
Furthermore, a key aspect of UIR development and the appropriation process is
rR
the discovery of the unique possibilities offered by a DMI, which may defy a
user’s pre-existing musical expectations. Finally, we discuss limitations of our
ev
study and provide suggestions to refine and expand the method for future
replications using different instruments, groups of participants, and longer time
spans.
iew
Keywords: digital musical instruments (DMIs); user experience (UX); user
engagement; music interaction; longitudinal evaluation; methodology; T-Stick
On
1. Introduction
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 2 of 54
This section introduces digital musical instrument (DMI) design, musicianship, and
evaluation, with a focus on experiential aspects of the user-DMI relationship.
1.1 Digital musical instruments: Design and performance
Digital technology has profoundly influenced the composition, performance, and
consumption of music, an effect particularly observable in DMI design and practice. A
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 3 of 54
DMI is defined as a control interface that associates input gestures with parameters of a
sound synthesizer through a series of gesture-parameter correspondences collectively
referred to as mappings (Miranda & Wanderley, 2006). Research into DMI
development, practice, and evaluation has increased over previous decades (Fasciani &
Goode, 2021; Holland et al., 2019; Jensenius & Lyons, 2017) and matured into a
discipline with an active design, research, and performance community.
Within DMI research, musical practice is an important topic. Due to the intrinsic
Fo
separation between control interface and sound synthesizer, the nature of practice is
rP
notably divergent from that with acoustic instruments. As a result of these differences,
as well as the inherently novel nature of DMIs, it is challenging, if not impossible, to
ee
determine widely-accepted principles for DMI design, practice, and evaluation.
rR
Several works have made recommendations for design of DMIs, mostly
focusing on technical improvements (Cook, 2001; Jordà, 2004; Morreale, 2015;
ev
Wanderley, 2001; Wessel & Wright, 2002). These guidelines, however, often overlook
iew
secondary aspects of design that may not relate to fundamental usability but have
notable impact on user experience (UX). Factors such as visual aesthetics, sound
quality, feel, creative and expressive possibilities, nuance, engagement, and challenge
On
can make an important contribution to the experience of music-making, as well as to a
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
DMI’s ability to support intimate and embodied user-instrument relationships (UIRs)
(Sullivan, Guastavino, & Wanderley, 2021).
While several new DMIs are created and presented every year, primarily at the
International Conference on New Interfaces for Musical Expression (NIME), many are
not adopted by members of the larger musical community and do not establish longevity
beyond initial presentation (Ferguson & Wanderley, 2010; McPherson & Kim, 2012;
Morreale & McPherson, 2017; Morreale, McPherson, & Wanderley, 2018). Barriers to
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
adoption and longevity include idiosyncratic design, technical obsolescence, and lack of
developed technique, existing repertoire, notation, or standardized training methods
(Fukuda et al., 2021; Mamedes et al., 2014; Marquez-Borbon & Martinez-Avila, 2018).
Despite barriers, users adapt themselves to the idiosyncratic nature of DMIs,
developing strategies that capitalize on DMIs’ unique features (Lepri & McPherson,
2021). There exist several examples of DMIs that have stood the test of time, having
been used in performance for 10 or more years (Casciato & Wanderley, 2007; de
Fo
Laubier & Goudard, 2006; Nieva et al., 2018). Thus, longitudinal studies spanning from
rP
a few weeks to several months are necessary to examine how musicians approach
DMIs, learn to interact with them, and develop UIRs that evolve over extended practice.
ee
1.2 Music interaction and the user-instrument relationship
rR
Music interaction refers to the intersection of music and human-computer interaction
ev
(HCI). Focal points include the role of intrinsic motivation in developing high-level
iew
skill over time, the embodied nature of musical activity, the required complex motor
and cognitive skill, and the temporal precision of musical movement (Holland et al.,
2019; Wanderley & Orio, 2002). For convenience, we adopt the abbreviation UIR to
On
refer to this combination of experiential factors, encompassing initial appropriation,
dynamic experiential factors, and extended engagement, and characterized by a
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 4 of 54
complex interplay of skill development, embodiment, emotion, and control.
Appropriation describes user’s adoption and adaptation of technology based on
their personal needs and practices (Dobrian & Koppelman, 2006; Dourish, 2003), as
well as how users transcend basic adoption to personalize their interaction within the
context of their personal knowledge and practices, thereby making technology ‘their
own’ (Bar, Weber, & Pisani, 2016).
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 5 of 54
Aspects of UX change and interact, evolving with development of the UIR, from
initial adoption and appropriation, through the learning process, to prolonged use and
eventual mastery of an instrument. A recent work ([author(s)], 2021) introduced
hedonic and cognitive experiential factors, distinct from utilitarian factors such as
usability and efficiency, to encapsulate many such facets of the UIR. Cognitive factors
describe psychological constructs including expertise, conceptualizations, motivation,
and cognitive load, while hedonic factors relate to affect and emotion. These factors are
Fo
closely related, exert considerable influence on one another, and typically show changes
rP
over time (Triberti et al., 2017).
Embodiment is another element of the UIR important in music interaction
ee
(Holland et al., 2019). Embodied interactions describe a relationship between user and
rR
artefact in which the two become integrated such that the object feels like an extension
of the user’s body and mind (Fels, 2004).
ev
To contextualize and understand experiential aspects of the UIR, we note two
iew
frameworks and associated measurement tools from user-centric HCI and music
interaction research. The first of these, engagement, describes a quality of UX that
characterizes positive interactions with technology and provides an explanatory model
On
for why users are attracted to a technology and how this attraction is cultivated. O’Brien
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
& Toms (2008; 2010) developed a six-factor model of engagement, along with User
Engagement Scales (UES) to assess attributes of focused attention (FA), perceived
usability (PU), aesthetics (AE), endurability (EN), novelty (NO), and felt involvement
(FI). Thus, the importance of engagement underscores the need to provide users with
worthwhile cognitive and affective interactions. Within the realm of music making,
engagement can provide a useful conceptual scaffolding for understanding aspects of
UX that characterize uptake and ongoing use of DMIs.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
The Musicians’ Perception of the Experiential Quality of Musical Instruments
Questionnaire (MPX-Q) is a psychometric tool consisting of 43 items divided across
three subscales (Schmid, 2017). The questionnaire, focused on performer experience
and perceptions, was intended to provide researchers with a standardized and rigorous
method for musical instrument evaluation. In addition to the questionnaire itself, the
three subscales offer a useful three-factor framework for understanding the UIR,
comprising experienced freedom and possibilities (EFP), perceived control and comfort
Fo
(PCC), and perceived stability, sound quality, and aesthetics (PSSQA).
rP
Despite their potential benefits for increasing understanding of DMI interaction,
these frameworks have, to our knowledge, received limited attention in DMI evaluation.
ee
1.3 Digital musical instrument evaluation
rR
While task-based usability evaluation is a hallmark of classical HCI, evaluation in
ev
NIME (and in more recent HCI) highlights the importance of methods focused on the
iew
nature of individuals’ interactive experiences (Brown, Nash, & Mitchell, 2017; O’Brien
& Toms, 2008). In music interaction, user-centric qualitative strategies may be
complementary to task-based usability measurement for assessment of the rich interplay
On
of experiential factors that characterize the UIR (Springett, 2009). The use of multifactor frameworks, such as the UES and MPX-Q, described previously, offer a useful
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 6 of 54
foundation for development of qualitative methods to evaluate this collection of
overlapping factors.
In UX evaluation, consideration of an interface’s target user is essential. In
music interaction, distinction between novices and experts is crucial, as interaction with
a DMI is likely to change over time spent practicing due to the accumulation of
knowledge and skills. Evaluation strategies may need to be tailored based upon
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 7 of 54
characteristics of a specific population of interest, or be sufficiently general to allow for
the study of a diverse range of skill levels and user characteristics ([author(s)], 2021).
Designers and performers are but two of a handful of DMI user perspectives,
commonly referred to as `stakeholders,` characterized by differing priorities and
concerns. While these perspectives are identified as distinct, there are many scenarios in
which these roles overlap, such as cases in which a performer designs their own DMI.
Researchers have suggested up to four stakeholders in DMI evaluation: performer,
Fo
designer, composer, and audience (Brown, Nash, & Mitchell, 2017; O’Modhrain, 2011).
rP
It is important, though, to keep awareness of the overlapping nature of stakeholder roles
(Mamedes et al., 2014) and how different perspectives are prioritized in the evaluation.
ee
Reviews of DMI evaluation literature (Barbosa et al., 2012; Barbosa et al., 2015;
rR
Brown, Nash, & Mitchell, 2017; [author(s)], 2021; Stowell et al., 2009) note patterns of
idiosyncrasy, exploratory approaches, lack of formal structure, use of ad-hoc and
ev
untested methods, prioritization of the performer perspective, and an absence of studies
iew
conducted over time. These patterns suggest current strategies provide little insight with
respect to extended use and how UX and the UIR develop over time.
Several authors (Gelineck & Serafin, 2012; Springett, 2009) identify the need
On
for increased longitudinal study of DMI use, noting research conducted over time is
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
necessary for understanding development of expertise ([author(s)], 2021), evolution of
technique (Moro & McPherson, 2020), and formation of the UIR (Malloch et al., 2019).
Use of existing structured frameworks and assessment tools could increase the extent to
which such evaluations are formal and systematic, facilitating replication and
comparison of different DMIs or iterations thereof (Young & Murphy, 2015).
In summary, reviews of DMI evaluation imply that there are few, if any, general
methods for evaluation of experiential aspects of DMI usage that are capable of both
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
addressing the complex nature of the DMI user and facilitating longitudinal study of
ongoing learning, experiential changes, and UIR development.
To develop and carry out a large-scale longitudinal evaluation of creative
technology is no small feat. It requires thoughtful planning, careful execution, and
rigorous testing to be effective and informative. Issues in longitudinal studies, expected
or unexpected—e.g., technical and availability issues, subject availability and dropout
during the experiment—typically prevent their deployment. Furthermore, without a
Fo
small-scale preliminary execution to estimate the number and type of subjects required,
rP
determine the physical and operational resources required to enable subjects to provide
useful data, and provide researchers with the opportunity to assess the method itself,
ee
longer term evaluations run the risks of participant attrition, accumulating superfluous
rR
data tangential to research targets, and potentially squandering financial and human
resources. It is thus essential that researchers be sure of a longitudinal method’s
ev
effectiveness before applying it at scale.
1.5 The current study
iew
The original research presented herein was carried out in response to the lack of formal,
On
empirical, and replicable long-term evaluations of UX with DMIs. This research
introduces and demonstrates a generalized method for studying dynamic aspects of the
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 8 of 54
UIR over time. By using a small sample of individuals with diverse musical
backgrounds and a compressed time period, we assess the efficacy of this procedure in
examining the development of the UIR with a novel DMI, with hopes to refine and
expand the method for future replications with more participants over longer time spans.
2. Materials and methods
To investigate UIR evolution over time, we developed a new method for structured
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 9 of 54
longitudinal evaluation with the goal of understanding and characterizing changes in the
UIR over a learning period, using interview and survey methods. Participants learned to
play an existing DMI, the sopranino T-Stick, a hardware interface created by
embedding sensors within an ABS plumbing pipe (Malloch and Wanderley, 2007;
Nieva et al., 2018). They practiced a series of tasks of increasing complexity, created
and performed an original musical excerpt, and reported on their experience through
surveys and interviews administered throughout the study.
Fo
2.1 Research questions
rP
Our method was developed to address two related research questions:
ee
(1) To what extent can the devised general method capture changes in experiential
rR
facets of the UIR as individuals learn to play a novel DMI?
(2) How effective is the method in providing information about experiential
ev
phenomena that contribute to the adoption, sustained use, and longevity of
DMIs?
iew
2.2 Participants and research environment
On
Three graduate students at [author(s)] were selected to meet two criteria: 1) they were
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
to have previous experience with and interest in music technology setups, and 2) they
were to have had little or no direct experience with the T-Stick. These criteria were
deemed sufficient for assessment of the methodology, as individuals with existing
knowledge of music technology (including limited exposure to the T-Stick) would
possess vocabulary and knowledge that would allow them to provide useful insight and
critical commentary related to both the musical apparatus and research method.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
All three participants completed the entire study, consisting of seven sessions
spaced over 20 days and were compensated with $80.00 CAD. The research was
conducted at [author(s)].
2.3 Materials and tools
2.3.1 Apparatus
Fo
The instrumental apparatus consisted of sopranino T-Stick, a Pure Data synthesis patch
rP
hosted on the GuitarAMI Sound Processing Unit (SPU; Meneses et al., 2019), and
mapping between the T-Stick’s embedded gestures (Meneses, Fukuda, & Wanderley,
ee
2020) and patch parameters. Mono audio from the SPU was routed through an Allen &
rR
Heath ZED-FX10 mixer to two Genelec 8020A monitors mounted at approximately
shoulder height (when standing). Audio-visual recordings of performances and
ev
interviews were taken using a MacBook Pro webcam and microphone.
iew
INSERT FIGURES 2-1A AND 2-1B HERE
On
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 10 of 54
The T-Stick (Figure 2-1, top) was chosen because of its stable design and relatively
widespread use by multiple individuals in research, composition, workshops, and
performance. Availability was also a factor, as at least 10 sopranino T-Sticks exist,
allowing for redundancy in case of hardware failures. The development of embedded TStick gestures (Meneses, Fukuda, & Wanderley, 2020) allowed for the extraction of
meaningful data from raw sensor output, facilitating the mapping of performer gestures
to sound synthesis parameters. A complete list of embedded gestures available in the T-
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 11 of 54
Stick is available in Appendix A (cf. Table A-1).
A custom subtractive synthesis patch was developed by [author(s)] in Pure Data
and was further developed by [author(s)], an active T-Stick performer, who added
functionality and refined operation for the mapping and task requirements of this
research. The list of parameters available for mapping is also available in Appendix A
(cf. Table A-2).
Mapping development involved consultation and brainstorming sessions with
Fo
active T-Stick performers and researchers. Performance videos, instrument mappings,
rP
and synthesis patches were collected and reviewed to develop a list of potential
ee
mappings and tasks, which was then revised based on what parameters the synthesis
patch created for this research made available (Table A-2). Final mapping choices are
displayed in Table 2-1.
iew
ev
rR
INSERT TABLE 2-1 HERE
2.3.2 Task blocks and cycles
On
Ten tasks (T), divided into four task cycles (C) (Table 2-4), were devised. Tasks were
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
designed to be learnable in a short period of time, to be representative of real-world TStick performance practice, and to allow for tasks to be combined in a manner that
produced useful musical results. The task order was intended to increase in complexity,
beginning with the control over fundamental parameters such as amplitude and
frequency and gradually exposing additional parameters for manipulation.
Table 2-2 shows the ten tasks and their grouping into the four task cycles.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
INSERT TABLE 2-2 HERE
2.4 User study overview
This study consisted of six stages, summarized in Table 2-3 and Figure 2-2.
Stage 1: Prior to being admitted to the study, individuals who had expressed interest
were emailed a pre-screening questionnaire to assess eligibility and to collect
Fo
information about their interest in and experience with music technology setups.
rP
Stage 2: After participants were deemed appropriate, they attended an individual
ee
introductory session consisting of informed consent, entrance interview, open
exploration of the instrument, the first impressions questionnaire, and presentation of
rR
the first task cycle (T01 to T03). The entrance interview consisted of questions related
ev
to participants’ performance experience and general attitudes about DMIs/EMIs.
Following this interview, participants were given five minutes of unguided exploration
iew
using the T-Stick without researchers present. After five minutes, participants were
asked to complete a survey which investigated first impressions of the instrument.
On
Participants rated the extent to which they agreed or disagreed with five survey items
using a 5-point Likert scale ranging from “Strongly Disagree” to “Strongly Agree.”
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 12 of 54
After presentation of the first task cycle, participants were asked to schedule
practice time in the lab for the remainder of the study. We elected not to have
participants take the instrument home with them to ensure technical support was
available during practice sessions. Participants were given the option to practice
immediately before or immediately after each evaluation session. All participants
scheduled their practice before each session.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 13 of 54
Finally, participants were informed that, in addition to the task cycles, they
would be required to prepare and perform a short musical excerpt of their own creation
prior to the final evaluation session. Participants were instructed to incorporate as many
of the learned gestural tasks as possible.
Stage 3: Most data collection occurred during the five task cycles in S3. The cycles
were conducted three to four days apart, and each had a corresponding evaluation
Fo
session in which participants responded to interview and survey questions related to
rP
their experience that cycle. Evaluation sessions ranged in length from 15 to 45 minutes.
Task cycles consisted of task demonstration followed by a period during which
ee
participants were asked to practice tasks for that cycle. Tasks were demonstrated for
rR
participants using videos uploaded to a private YouTube channel. Tasks can be seen by
clicking on the task title in Table 2-2. Each video consisted of a task title, a
ev
demonstration of [author(s)] executing the task, and embedded text instructions
iew
describing how the task is performed. Researchers were present during the
demonstration to answer questions. Participants were subsequently emailed links to
each task video and informed they could watch the videos as many times as desired. For
On
each cycle, participants were given up to 60 minutes of independent practice and were
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
asked to practice until they felt able to comfortably replicate each task. A researcher
was on the premises to answer questions or provide technical support.
Stage 4: Evaluation sessions for each task cycle consisted of three components. First,
participants were asked to perform each task in front of the researchers. Audio and
video of each task was recorded. Second, participants were asked to fill out the
evaluation session questionnaire, which asked participants to rate their ability to
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
perform each task on a 3-point scale (“No”; “Sometimes”; “Yes”) and to rate the extent
to which they agreed with 14 statements on a 5-point scale (from “Strongly Disagree” to
“Strongly Agree”). Finally, an evaluation session interview consisting of three openended questions was conducted. Audio and video of each interview was recorded.
Following the session interview, participants were shown videos for the following
cycle.
Fo
Stage 5: Following their final practice period, participants were asked to perform their
rP
original excerpt for the researchers. Audio and video recordings were taken of each
performance. After their final performance, participants completed an exit questionnaire
ee
and interview covering their experience over the entire study, specific positive and
rR
negative experiences, and suggestions for improvements to the musical apparatus.
ev
Stage 6: The final component of data collection consisted of a four-question survey
iew
distributed to two third-party evaluators, intended to assess the extent to which tasks
could be learned and used over a short period of time by novice users. Evaluators were
members of the [author(s)] involved with T-Stick research. They were provided with
On
videos of each final performance and asked to provide ratings and commentary related
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 14 of 54
to range of gestures used, smoothness of motion, instrument responsiveness, and overall
quality.
INSERT TABLE 2-3 HERE
INSERT FIGURE 2-2 HERE
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 15 of 54
We collected both qualitative data and scale ratings about participant experience
throughout the study. Questions used for the survey and interview components were
either researcher-developed or sourced from one of three existing surveys: the
Electronic Musical Instrument (EMI) Survey (Sullivan, Guastavino, & Wanderley,
2021), the UES Short-Form (O’Brien, Cairns, & Hall, 2018), and the MPX-Q (Schmid,
2017). Table B-1 in Appendix B provides a complete record of question sources.
Fo
Interview guides and questionnaires are available, with all data, at [author(s)].
3. Results
rR
ee
rP
This section describes results collected from questionnaires and interviews. We detail
the process used to analyze interview data, in addition to presenting selected quotations
ev
from participants, illustrative of aspects of experience our method was able to capture.
3.1 Scale Rating Data
iew
This section briefly describes ratings from the first impression (S2) and exit
On
questionnaires (S5), as well as participant ratings of their ability to perform the tasks
(S3) and overall ratings of participant performances by third-party evaluators (S6). Due
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
to spatial limitations, discussion of scale rating data is more thoroughly presented and
explored elsewhere ([author(s)], 2023). This article will focus on the illustrative
qualitative data collected.
Results from the first impressions questionnaire suggested that, during the initial
interaction, the instrument was generally perceived as confusing but appealing to the
senses. Participants provided neutral ratings of the sound quality and aesthetic appeal
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
but rated the intuitiveness of the mapping as low. While P1 and P3 provided very
similar responses, P2 was typically more negative towards the instrument.
Three notable points emerged from the exit questionnaire. First, the T-Stick was
generally not perceived as solid or reliable. Second, despite technical issues,
participants did indicate the instrument could serve a purpose in their musical practice.
Finally, subjects strongly agreed that the T-Stick allowed different possibilities than
previously used acoustic or digital instruments.
Fo
Participant ratings of their abilities to perform the tasks were generally positive,
rP
although filter cutoff and frequency tasks proved difficult for P2 and P3. Additionally,
P3, the hobbyist, experienced more difficulty with the tasks overall, reporting that they
ee
were able to perform the majority (six of ten) tasks only sometimes. Finally, there were
rR
no instances in which any participant indicated being entirely unable to perform a task,
indicating that all tasks were achievable, but posed differing degrees of difficulty.
ev
Data from the multiple evaluation questionnaires, reported in [author(s)], 2023,
iew
demonstrated changes in participant responses over many of the questionnaire items.
Feelings of fun, creativity, time slipping away, expressiveness, and a desire to play the
instrument again generally increased over time, excluding the session in which
On
participants were asked to learn one of the more difficult tasks (frequency) and to
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 16 of 54
practice another difficult task (filter cutoff) from the previous cycle. Items related to
intuitiveness and absorption generally increased over time, while items inquiring about
frustration and how taxing the T-Stick was to use showed more fluctuation,
With respect to the methodological contributions of this research, ratings data
provides evidence that questionnaire items were able to capture changes in the constructs
of interest over the learning period. With regard to theoretical contributions, it is
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 17 of 54
worthwhile to note that the factors of interest do change, even over periods as short as two
to three days.
Concerning participants and third-party evaluator ratings, two data sources
provide supporting evidence for the suitability of the task cycles developed for this
study. First, participants reported being able to perform all the tasks to a moderate or
advanced degree. Second, evaluators rated the quality of the final performances as high
(cf. Figure 3-1), confirming participants were able to play the T-Stick at the end of the
Fo
study. Ratings were remarkably consistent across all performers.
rP
INSERT FIGURE 3-1 HERE
3.2 Qualitative Data – Interviews
ev
rR
ee
This section describes the interview transcription and coding procedure, as well as the
iew
data from the entrance, evaluation session, and exit interviews.
3.2.1 Interview transcription and coding procedures
On
To prepare data for analysis, audio recordings of all interviews were transcribed.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Transcriptions were divided into individual incidents of one or more sentences focused
on a single concept or idea. This resulted in a full set of 277 incidents for all participants
over all interviews. Interview data was coded using an approach similar to thematic
analysis (Braun & Clarke, 2006), a method of qualitative analysis in which data is
interpreted by assigning codes based on 1) specific themes of the research inquiry
(deductive approach), and 2) patterns that emerge from the data (inductive approach).
Coding took place over three passes. The first pass used a deductive approach with pre-
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
defined codes based on the project’s research questions, conceptual frameworks from
the literature, and the survey and interview questions that participants responded to.
Incidents were reviewed one at a time and assigned a relevant code. New codes were
added inductively as they emerged from the data. Modifications were made during the
first pass and fully considered during the second. The final set of interview codes
contained 51 categories divided across nine areas:
1. Interaction Factors – Specific experiential aspects of HCI and music interaction.
Fo
2. Technical Factors – Construction of hardware, implementation of mapping,
rP
sound synthesis, and overall musical apparatus.
ee
3. Hedonic and Cognitive Factors of Interest – Specific affective and cognitive
factors of interest.
rR
4. UES Framework – O’Brien & Toms’ (2008) original six-factor engagement
ev
framework. A seventh code, reasons for disengagement, was added in this area.
5. MPX-Q Framework – The three MPX-Q (Schmid, 2017) subscales.
iew
6. Temporal Factors – Changes in UX and the UIR over time.
7. Personal Factors – Users’ background, prior experience, and preferences.
On
8. Affective Factors – Emotional responses resulting from interaction with the
instrument.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 18 of 54
9. Methodological Factors – Aspects of the instrument, research environment, and
method that impacted participant experience.
The second pass was carried out in the same way as the first but used the above final
coding scheme. Incidents coded in the first pass were reviewed to ensure all possible
codes were considered. Cases where the coder (the first author) was unsure were
flagged and reviewed with the co-authors during the third pass to assign all relevant
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 19 of 54
codes. Each incident was assigned at least one code, although many were assigned
multiple codes.
3.2.2 Interview Data by Category
Figure 3-2 provides an overview of coded interview data for the nine areas over all
interviews, illustrating changes over time and differences between the content of
evaluation session (ES1 to ES5) interviews.
rP
Fo
INSERT FIGURE 3-2 HERE
ee
Two patterns are observable in Figure 3-2. The first is a notable spike in the first
rR
evaluation session interview (ES1). This applies to the Interaction Factors, Technical
Factors, Hedonic and Cognitive Factors of Interest, UES Framework, and Temporal
ev
Factors. This can be attributed to novelty effects. Since this interview took place after
iew
the participants’ first practice session, it is likely they had more items to report based on
this being their first extended exposure to the instrument. It is logical that users would
On
have reported many items related to technical factors and interaction factors as they
experienced them for the first time, and not reported these items in subsequent sessions.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Second, some categories were coded much more than others. Comments related
to Interaction Factors, Technical Factors, and Factors of Interest were most common,
while incidents related to Personal Factors, Affective Response, and Methodological
Factors were less common. A moderate number of comments were coded under the
UES and MPX-Q Frameworks, as well as Temporal Factors. The lack of commentary
related to Personal Factors is explained by the nature of the questions; participants were
directly asked about their background and preferences in entrance and exit interviews,
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
but not in intervening session interviews. The high prevalence of comments related to
Interaction Factors and Factors of Interest suggests the coding scheme was effective in
capturing these aspects of UX and the UIR. The high proportion of technical comments
can be attributed to usability issues participants encountered.
3.2.3 Pre-screening questionnaire and entrance interview
Given both the pre-screening questionnaire and entrance interview (EN) focused on
Fo
participant background and performance preferences, results are presented together.
The pre-screening questionnaire showed that all the participants had some
rP
previous DMI experience. While P1 and P3 identified their primary context of DMI
ee
usage was ‘Personal Practice’ (they performed for public audiences between 0 and 10
rR
times per year, P2’s primary context of use was ‘Live Performance’ (and performed 21
to 50 times per year). P2 indicated that they used a variety of DMIs including modular
ev
synthesizers, keyboards, samplers, drum machines, effects processors and pedals,
iew
computer software, and MIDI controllers. In contrast, P1 and P3 both indicated previous
experience with MIDI controllers and computer software only.
Responses to entrance interview questions further underscored differences
On
between the three participants. These differences notably affected the ways in which
they approached and conceptualized the T-Stick. For example, P1 mentioned being a
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 20 of 54
composer, ‘interested to expand the possibilities of an acoustic instrument.’ (P1, EN).
They indicated recent DMI experience composing for the Karlax, a DMI of note that
has been manufactured and made available to the public in limited quantities.
P2’s performance history, experience with, and thoughtfulness about DMIs was
evident. When asked about factors that might prevent them from adopting or continuing
to use a DMIs, they noted interface design and practical concerns, including complexity,
legibility of internal synthesizer parameters, and portability.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 21 of 54
P3 described themselves as ‘very much a hobbyist’ (P3, EN), indicating they
don’t perform, but do practice with a MIDI keyboard and SuperCollider. In describing
their practice, they noted elements of play, exploration, experimentation, and ‘in-themoment aesthetic’ (P3, EN). P3 mentioned barriers to DMI use, including skill,
reliability, and setup. They compared SuperCollider to their classical guitar practice,
noting sonic limitations of the acoustic instrument and spatial limitations of their DMI
setup:
Fo
The guitar only needs my lap … with [my DMI setup] I need somewhere to lay
rP
everything down and that requires … table space that’s not always available. (P3, EN)
These contrasting participant perspectives are referenced throughout our discussion, as
ee
the capacity of our method to capture divergent aspects of experience illustrates its
rR
effectiveness in accommodating the study of these differing but overlapping roles.
3.2.4 Evaluation session interviews
ev
During the first session, participants explored the T-Stick to assess its possibilities and
iew
limitations. P1 used a tennis metaphor for exploration: ‘The gestures … the shape …
makes me feel to create a piece about tennis …’ (P1, S2). P2 explored the instrument
On
through its gestural limitations: ‘[I] tried to exaggerate the gestures that were done, see
what I could do ...’ (P2, S2). Finally, P3 described more general exploration: ‘I was
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
mainly just trying to explore what I could do with the T-Stick …’ (P3, S2).
Exploration and discovery were ongoing. In some cases, users encountered
states they were unable to get out of. All participants discovered the reverb functionality
before the task videos were shown and P2 found tasks easier to perform with the reverb
stuck on, despite this being a technical error. P1 and P2 both discovered ‘sweet spots’
that allowed them the most control and predictability over specific parameters.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
P1 and P2 were often influenced by their background and personal preferences.
P1 felt they were unable to control basic parameters such as pitch in a manner they were
accustomed to based on experience with acoustic instruments. This affected their
perception of the instrument’s fundamental musicality:
I’m not sure if it can be musical because I have not much control about the elementary
… You need rhythm, pitch … the timbre aspect is quite limited … (P1, S4)
P2 suggested the instrument did not match their sonic aesthetic, which limited their
Fo
motivation to interact with it. Relatedly, P2 had the most experience with the T-Stick,
having previously created a mapping for it, and had the most negative attitude towards it
rP
throughout the study. While P2 stated during S2 that they ‘wouldn’t choose to use [the
ee
T-Stick] in performance,’ this attitude had changed by the final session:
rR
[M]y positioning going into it was like ‘Man, the T-Stick … I’m never going to be able
to get sound out of it I like.’ And that’s not necessarily the case. But it kind of
ev
prevented me from wanting to experiment with it a fair amount. Today I just tried to let
that go and mess around with it. (P2, S6)
iew
P3 also described a change in their attitude towards their interactions with the TStick, providing some insight into their own personal learning process and goals:
On
Whether or not I would be happy recording what I did and showing it to people, I don’t
think it’s quite there yet … but I definitely feel like there’s more movement in that
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 22 of 54
direction and it’s going not just from having the sounds that are separately something I
can see as being pleasing, but I’m starting to put them together in a way where in
combination they’re also somewhat pleasing. (P3, S5)
All participants flagged inconsistent response of the T-Stick as a major
limitation that affected their sense of control and confidence:
[A]s a musician, you need to be more in control to have more secure possibilities,
secure tricks … You can play with it … but can you really perform a piece? (P1, S5)
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 23 of 54
Latency was also flagged as an issue by both P1 and P2 with regard to onset of sound
and their ability to keep time and establish musical rhythm. P2 made several comments
about the perceived fragility of the T-Stick as a major limitation throughout the study,
attributing it to the T-Stick’s lab-based construction: ‘I feel like I have to babysit the
instrument so that I don’t break it. And that limits the ease of use …’ (P2, S2). P1 and
P3 noted the grip required to maintain sustained pressure on to control amplitude led to
pain that persisted outside of the research environment.
Fo
Three specific tasks caused difficulty for multiple participants: filter cutoff (jab),
rP
frequency (touch position), and reverb (shake). Specifically, energetic jabs or shakes
could cause the battery to dislodge, forcing the instrument to power cycle. Participants
ee
also encountered difficulty in finding the borders between each frequency zone.
rR
Affectively, frustration was a common occurrence for participants, often a result
of the T-Stick’s inconsistent response or difficultly in replicating demonstrated tasks.
ev
This was not, however, always perceived as negative:
iew
“[I]t’s a normal thing. When you start playing …, at the beginning, you’re not good …
When you are creative and you want to, it’s interesting to be frustrated.” (P1, S6)
During the learning process, participants also reported cases of appropriation and
On
embodiment, developing a more intimate understanding of the T-Stick and situating it
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
within their own personal knowledge, abilities, and musical practice. For example, P2
noted how their developing understanding of the T-Stick affected their openness to it:
[Y]ou have to see [the T-Stick] as a platform, more so than an instrument in its own
right, ‘cause it can evolve based on the sound layer and the mapping layer, which is
critically important. So this is just an iteration of the instrument as it’s evolving … I’d
say that I have opened up to it more. (P1, S5)
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
3.2.5 Exit interview
The exit interview (EX) provided an opportunity for participants to reflect on their
experience over the entire study and provide suggestions for improvements to the TStick hardware, mapping, sound synthesis, and set of tasks. Several themes emerged.
First, all participants mentioned that the T-Stick offered musical possibilities
that were different from those offered by other acoustic or digital instruments:
I’ve mainly used keyboard-based MIDI controllers or acoustic instruments—mainly
Fo
stringed instruments—and so none of that really aligns quite to what is easy to do here.
So it absolutely is very different from my experience … (P3, EX)
rP
Relatedly, all participants spoke about how they might make use of the T-Stick in the
ee
future based on its perceived possibilities and limitations with respect to their individual
priorities and practice. P1 expressed interest in composing pieces for T-Stick in
rR
combination with acoustic instruments, while P3 expressed interest in exploring the
ev
technical operation of the instrument. P2 considered how it could be used as a controller
for synthesizer parameters in their existing loop-based performances, ‘warping time or
iew
changing filter settings or reverb settings’ (P2, EX).
When reflecting on challenges experienced over the study, P2 and P3 mentioned
On
their ability to control the T-Stick, both in terms of transparency and responsiveness:
I was cursing at it the whole time. I said ‘you piece of s*** f****** [expletives] thing.’
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 24 of 54
When it would reset, when it would stop making sound, when I went from 89 degrees to
90 degrees … I was expecting it to have a certain contour of reliability. It didn’t and
that frustrated me. (P2, EX)
All participants also identified specific breakthroughs that occurred with respect to their
learning, both spontaneously and gradually, over the course of the study. Comments
from P2 illustrate one such breakthrough:
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 25 of 54
I got into it and I was able to just play with it. It felt like it was sonifying the motions of
my body without me having to think too much about it and that was really exciting. … I
felt like my movements were being captured by the instrument and converted into
sound in a way that I enjoy. (P2, EX)
In addition to learning to play the instrument, participants pointed to personal affective
and cognitive aspects of their experience that changed over the learning process,
suggesting that, even over this short period, the UIR was evolving:
Fo
The learning process … has been … particularly positive … [O]ver the course of the
sessions, having the module tasks come together and explain “Oh, that’s what was
rP
happening there. It wasn’t some mysterious unexplainable thing … I was doing this,
and now that I know that’s a thing that matters. Oh, that makes more sense.” (P3, EX)
ee
Participants also provided suggestions to improve the interface and method, including
rR
modifications to mapping or gestures, increased stability and responsiveness, changes to
sound synthesis, and the addition of feedback. These results are reported in the first
author’s MA Thesis ([author(s)], 2023).
On
4. Discussion
iew
ev
In this section, we compare the perspectives of the users in our study, noting the
capacity of our method to accommodate these varied viewpoints. We identify
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
limitations of the demonstration, as well as potential methodological alterations. We
conclude with a brief discussion of methodological, conceptual, and practical
contributions.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
4.1 Three perspectives
In the pre-screening questionnaire and entrance interview, participants described their
musical background and practice preferences. These factors greatly informed how they
approached the T-Stick and affected the nature and extent of their engagement.
4.1.1 The composer
P1 came from a composition background and was concerned with ideas related
to the instrument’s inherent nature. P1 reported feeling that several behaviours of the T-
Fo
Stick were random, but questioned whether this was an intentional aspect of the design
rP
(it was not). They noted that, whether the unpredictability was intentional or not, it was
detrimental to their notion of musicianship. They discussed theoretical ideas of
ee
creativity and inspiration more than other participants. From early on, P1 thought about
rR
the original excerpt they would be asked to present at the study conclusion. They
ev
described, at several points, how the T-Stick might encourage or limit compositional
possibilities. They also addressed notions of musicianship, including the need for an
iew
instrument to offer a level of control sufficient to afford ‘secure tricks’ (P1, S5) and the
limitations of the T-Stick in creating pitch-based music.
4.1.2 The performer
On
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 26 of 54
P2, an active DMI performer, often described practical concerns about reliability, ease
of control, and how the T-Stick might be used in their performance setup:
I’m looking at these instruments as part of a system … not just one instrument in a
vacuum. I’m trying to make them fit into a broader context (P2, EN)
P2 cited concerns about the ability of the instrument to withstand the physical stress of
being jostled in a backpack or moved energetically when a user is absorbed in the act of
performing. Issues related to the battery disconnecting and unexpected power cycling
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 27 of 54
led to perceptions of the T-Stick as unable to withstand normal rigors of real-world
performance. P2 also emphasized latency as a major problem. P2 cited these as notable
reasons for disengagement, as their active performance practice necessitated that an
instrument be ‘gig-ready’ (P6, EX). These concerns, standard considerations for an
individual who performs regularly, were not considered at length by other participants.
Notably, P2 acknowledged negative bias towards the T-Stick at the study’s inception,
having been involved with related research. This bias moderated towards the end of the
Fo
study as P2 overcame challenges they experienced in initial interaction.
4.1.3 The hobbyist
ee
rP
Finally, P3 was a self-identified hobbyist with classical guitar and a simple DMI setup
consisting of a MIDI controller and synthesis patches coded in SuperCollider. Their
rR
concerns related to the flexibility afforded by DMIs and the role of exploration and
ev
experimentation in musical practice. When compared with other participants, P3 was
most open to the instrument as it was presented, and did not have an overarching
iew
trajectory that guided their interaction, such as developing a composition or integrating
the instrument into a real-world performance setup. Relatedly, P3 was conscious of their
On
learning and skill development. They described feeling as though they were improving
with respect to performing the tasks but that they didn’t necessarily feel they would be
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
able to use learned gestures ‘in a musical context’ (P3, S2). While P2 was frequently
critical of the T-Stick, P3 was often critical of their abilities, uncertain whether it was
them or the instrument that was the root cause of frustration.
4.2 Merging perspectives: Similarities and differences
Despite differences in their backgrounds, preferences, and musical goals, participant
experiences showed notable similarities, providing evidence for the overlap and fluidity
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
of these perspectives. Areas in which these viewpoints diverged typically entailed
specific preconceptions or requirements based on individuals’ unique goals.
Results from our demonstration suggest that the handful of stakeholder roles
identified in the literature (O’Modhrain, 2011) may not be inclusive of all individuals
who interact with DMIs, specifically amateurs and hobbyists. Additionally, boundaries
between these perspectives are blurred. As such, these viewpoints may not be
representative of distinct categories but of overlapping approaches to interaction, each
Fo
with differing priorities and processes of engagement. In many ways, our most
rP
informative results related to areas where perspectives overlapped.
Participants often fluidly shifted their thought patterns between describing
ee
performative aspects of their interaction, from providing technical and design-related
rR
suggestions and describing aspects of musical interfaces they liked or disliked to
reflecting on the nature of learning, interaction, and musicianship. In conjunction with
ev
the fuzzy boundaries between DMI stakeholder perspectives, this implies that there is
iew
not simply an overlap between patterns of thought, e.g., technical development vs.
musical performance state of mind (Franco & Wanderley, 2017), but that the DMI
‘user’ may describe a more fluid holistic perspective not well characterized by a
On
categorical framework. Crucially, we note that our method was able to offer insight into
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 28 of 54
each of these perspectives, as well as the similarities and differences between them.
4.2.1 Experiential similarities
A major area of consistency among the participants was their sense that the T-Stick
afforded different interaction styles and musical possibilities than other DMIs or
acoustic instruments they had used in the past. This was closely related to the EFP subscale of the MPX-Q. Conversely, participants also identified limitations of the interface.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 29 of 54
While comments related to exploration were most common in early evaluation
sessions, exploration and discovery of new possibilities were ongoing. Interestingly,
even by the end of the study, participants did not feel they had explored all the
interactive or musical possibilities, showing the potential for further exploration of the
setup (Wessel & Wright, 2002).
Ratings from participants and third-party evaluators provided evidence that the
set of tasks used was learnable and that, by the end of this short study, all participants
Fo
appeared to be somewhat competent T-Stick performers despite minimal experience
rP
with the instrument at the outset and no experience with the specific mapping used.
While frustration with technical and control-related shortcomings of the T-Stick
ee
was prevalent throughout the study, all participants found ways to overcome these
rR
limitations and mitigate frustration, especially in later sessions. For example, P3
avoided frustration related to the filter cutoff task by focusing on tasks they felt more
ev
comfortable performing. Such breakthroughs reflect aspects of appropriation and
iew
suggest that the process of adapting to DMIs’ limitations and idiosyncrasies, may be an
important part of DMI learning (cf. Lepri & McPherson, 2021; Morreale, McPherson, &
Wanderley, 2018).
On
Changes in affect were often accompanied by cognitive changes in how
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
participants conceptualized or approached the T-Stick. For instance, at the outset of the
study, P2’s pre-existing negative attitude towards the instrument was evident: ‘I
wouldn’t choose to use [the T-Stick] in performance or anything’ (P2, S2). By the final
evaluation session, this participant had opened up towards the T-Stick by changing their
conception of it:
[Y]ou have to see [the T-Stick] as a platform more than an instrument in its own right,
‘cause it can evolve based on the sound and the mapping layer, which is critically
important. So this is just one iteration of the instrument as it’s evolving … (P2, S6)
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
Similarly, P1 noted a change in their attitudes towards the T-Stick by recontextualizing
the frustration they experienced during the learning process: ‘When you are creative
and you want to, it’s interesting to be frustrated.’ (P1, S6)
P3’s change in perspective was of a different character, as they noted feeling
like their understanding of the instrument through the accumulation of knowledge and
expertise over the task cycles and practice sessions made the instrument more
approachable and less surprising. They noted feeling that the instrument was less
Fo
‘mysterious’ (P3, EX) and that they required a less conscious effort to interact with the
T-Stick.
ee
rP
4.2.2 Experiential differences
rR
While participants’ reports shared common features, other aspects of their experience
were notably divergent based on their unique goals and requirements. Each participant’s
ev
plan for future use was thoughtful, specific, and reflective of their personal practice. P1,
iew
the composer, expressed a desire to compose mixed pieces for T-Stick and acoustic
instruments. P2 suggested the T-Stick could be a useful controller for parameters of
other DMIs in their loop-based performance setup. P3, the hobbyist, wanted to further
On
explore the technical side, delving into the operation of the hardware and mapping.
These specific plans suggest three different real-world creative uses for the instrument.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 30 of 54
Furthermore, the thoughtfulness of these plans provides evidence of the development of
an intimate relationship with the T-Stick and a conceptualization of the DMI in the
context of distinct artistic practices, important aspects of appropriation and
embodiment.
Participants also referred to the musical limitations of the T-Stick. This area was
often informed by previous musical experience. As such, it was P1, the composer, who
most discussed the musical limitations of the instrument. P1 noted frustration related to
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 31 of 54
the limited range of the T-Stick’s frequency, amplitude, and timbre, feeling as though
the available range placed limits on their compositional ideas. It should be noted that
these reported limitations were reflective of the musical apparatus as a whole, and may
or may not be specifically linked to the hardware, sound synthesis, or mapping used.
P2, a pianist and active DMI performer, echoed these sentiments, though they
did not necessarily see them as restricting the expressive potential of the T-Stick. For
P2, these limitations overlapped with concerns related to control and latency, as the
Fo
responsiveness of the instrument was limited by these factors. They suggested that
rP
simple melodic tasks, relatively easy to accomplish using a traditional acoustic
instrument, were more difficult with the T-Stick:
ee
If I tried to play Jingle Bells on it … I actually did try to play Jingle Bells … I
rR
couldn’t … I mean I could, but it was off. (P6, EX)
P3, who was most interested in the exploration of music technologies, was least
ev
concerned with the ability to replicate acoustic instrumental practice.
iew
4.2.3 Development of the user-instrument relationship
Initial interactions were often informed by musical background, personal preferences, or
On
pre-existing expectations. This had notable influence on how participants developed
relationships with the T-Stick by appropriating it in the context of these personal
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
influences and describing how they might use it in future creative practice.
Participants also reported instances of embodiment, suggesting that they were
developing a more intimate and engaging relationship with the T-Stick. P1 described the
process of finding a stable position that allowed them to maximize the expressive
potential of the instrument, while P2 reported feeling more able to express themselves
by making using of the physical space, allowing them to establish a feedback loop
between the sound of the instrument and their own movement. P3 described the process
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
of playing as becoming less conscious and more automatic over time and feeling as
though their intentions and the resultant sounds were becoming more aligned.
Accidental discoveries also facilitated changes in UIR. For instance, P2
discovered that a glitch, in which reverb did not dissipate when shaking was stopped,
made other tasks easier to accomplish as the reverb obscured abrupt sonic changes. P1
discovered a “rest position” (P1, S6), which they could use as a base from which to
explore other gestures, increasing their sense of the T-Stick’s expressive possibilities.
Fo
Evolution of the UIR over the course of this short study was summarized by P2:
rP
I started out bumbling around just like any new creative user and eventually I felt like I
could just mess around on it a little bit. It’s like learning to play tennis—getting to a
ee
point where you can hit it back over the net—that’s a huge milestone because it means
you can now play the game. You can play, you can participate. So I felt like I went from
rR
someone who could not participate to someone who could. (P2, EX)
4.3 Limitations
iew
ev
We note three specific limitations of the current research related to participants, scope,
and technical aspects of the instrument. First, our sample size was small. Music
On
interaction experience and UIR development are, however, highly individual, meaning
aggregate measures may not yield description of a “standard” UX. Despite the small
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 32 of 54
sample size, we were able to capture data about and characterize three diverse
perspectives on the DMI learning process. Second, this study was limited to a single
DMI and a short timeframe, providing limited time for practice in between sessions.
While we aim to expand this research to longer periods and additional DMIs, we believe
this short demonstration to be indicative of the method’s efficacy to capture dynamic
aspects of interaction. This is especially critical as most existing evaluations are not
conducted over time at all.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 33 of 54
4.4 Contributions
Our most notable contribution was development and demonstration of our longitudinal
evaluation method. We were able to detect changes in participants’ emotional and
cognitive reactions to a novel DMI, observe development of expertise and skill, and
observe initial indications of the evolution of an engaging UIR, illustrating how factorbased frameworks such as the UES and MPX-Q can be employed in evaluation
research.
Fo
While information gleaned from preliminary demonstrations is limited with
rP
respect to hypothesis testing or providing results that can be generalized to larger
populations, initial execution of a method is a critical and necessary step in the process
ee
of realizing such sizeable research enterprises. The methodological demonstration
rR
described herein is illustrative of this:
ev
(1) Even a small group of participants contained a diversity of perspectives, with
each participant emphasizing different aspects of interaction with the T-Stick,
iew
focusing on different aspects of the DMI or their experience. While the
composer and hobbyist prioritized musical possibilities and the ability to
On
experiment, respectively, the user who performed regularly for public audiences
was concerned with predictability of interaction and repeatability of gestures;
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
(2) More specifically, participants’ experiences with the instrument and their
perceptions of the quality of the instrument were informed by these differing
priorities. While P2’s perceptions of the T-Stick ranged from unusable to
acceptable in specific contexts, other participants were more open to
unpredictable aspects of interaction.
(3) Finally, the crucial discovery that the T-Stick using did not behave according to
the same conceptual models as acoustic instruments was a pivotal point in all
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
three users’ experience. A key aspect of UIR development and the appropriation
process was the discovery of the unique musical possibilities offered by a DMI.
This discovery occurred after a relatively short period of interaction, and was
linked with users’ openness towards the instrument as it was presented, as well
as their initial composition and performance goals. Longer replications may
reveal other experiential developments that only become apparent after this
discovery is made.
Fo
These points illustrate the value of conducting small-scale methodological
rP
demonstration prior to embarking on more resource-intensive longitudinal research.
ee
Furthermore, results illustrate that useful and informative data about longitudinal
experience can be collected even through limited research protocols.
rR
A second contribution of this research is practical and actionable information
ev
related to development of the T-Stick. While the instrument was imperfect, it was
generally usable, supported novice learning, and fostered the development of complex
iew
and intimate UIRs. Furthermore, participants in our study were able to overcome many
of the instrument’s idiosyncrasies and technical limitations by modifying their
On
behaviour and increasing their understanding of the instrument’s behaviour, in other
words, they embraced the instrument’s weirdness (Lepri & McPherson, 2021).
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 34 of 54
Finally, we report two important theoretical contributions. First, results indicate
that accepted categorical models of stakeholders with vested interest in DMI design and
evaluation may be overly simplistic. While participants explicitly identified themselves
as performer, composer, and hobbyist, interview responses often shifted between
performance, design, composition, and other concerns not isolated to one specific role.
We recommend further investigation into how the DMI user is characterized.
Specifically, we suggest that models of DMI users should incorporate a wider range of
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 35 of 54
skill levels, musical styles, and patterns of engagement. Furthermore, we encourage a
holistic understanding of the DMI ‘user’ as not just a constellation of overlapping
perspectives, but individuals with knowledge spanning many component disciplines of
music interaction who are capable of fluid ideation that seamlessly transitions between
design, performance, composition, technical, and interaction priorities.
Second, our findings provide initial observational insights into how novices
experience DMIs during initial learning. Such insights can aid designers in creating
Fo
DMIs that are accessible and pleasurable to use for novices. General observations
rP
obtained through use of our method include the importance of exploration and
discovery of a DMI’s musical possibilities during initial learning, the central role of
ee
frustrations and breakthroughs in appropriation and UIR development, and that short-
rR
term, task-based learning is sufficient for DMI users to develop original musical ideas
and extrapolate ideas for use of a DMI in their personal practice. Such qualitative
ev
insights further support the use of our generalized procedure as a method for both
iew
understanding music interaction experience, and for providing inspiration for future
research, as each observation could be examined more thoroughly in more targeted
research.
ly
5. Conclusions and Future Work
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Using frameworks from HCI (UES) and music technology (MPX-Q), as well as
categories contextualizing the role of affect and cognition in the UIR, we developed a
general method for longitudinal evaluation of DMIs. We describe our method in detail
and report on a small-scale demonstration of this method, noting methodological,
practical, and theoretical contributions to DMI interaction research.
Longitudinal research presents many challenges, including attrition, the need for
financial and human resources, and the requirement of well-conceived, thoroughly
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
tested, flexible, and robust methods. While the need for longitudinal research of DMI
interaction is acknowledged, the question of how to execute such research does not have
simple or readily apparent answers; while longitudinal research is needed, we may not
yet know how properly to conduct it. In this respect, the demonstration of a generalized
method for longitudinal evaluation of UX with DMIs represents an important milestone.
Results indicate the procedure, task set, data collection techniques, and analysis
strategy are suitable for capturing and characterizing experiential music interaction
Fo
phenomena. Most importantly, results suggest existing categorical models of
rP
stakeholders in DMI design and evaluation may be limited in characterizing DMI
interaction, failing to fully capture the fluid and multifaceted nature of DMI users.
ee
We intend to replicate and build upon this research in future, though data
rR
collected provides useful insight with respect to interaction between musicians and
DMIs. Future research can gradually expand this research protocol to other hardware
ev
controllers, mappings, synthesizers, tasks sets, and longer spans of time, ideally months
iew
or years. By facilitating long-term study of the UIR with different DMIs in different
contexts using a generalizable method, the DMI community could increase
understanding of dynamic aspects of music interaction, while also increasing the
On
formality and replicability of UX evaluation research.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 36 of 54
Acknowledgments
This research was funded in part by a [author(s)] to the first author and a [author(s)]to the
fourth author. It was approved by the [author(s)] Research Ethics Board. Informed consent was
obtained from all subjects involved in the study. Special thanks to [author(s)] for technical
support and assistance in developing the musical apparatus and set of tasks. The authors report
there are no competing interests to declare.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 37 of 54
References
Bar, F., Weber, M. S., & Pisani, F. (2016). Mobile technology appropriation in a distant
mirror: Baroquization, creolization, and cannibalism. New Media & Society,
18(4), pp. 617-636.
Barbosa, J., Calegario, F., Teichrieb, V., Ramalho, G., & McGlynn, P. (2012).
Considering audience’s view towards an evaluation methodology for digital
musical instruments. In Proceedings of the 2012 International Conference on
New Interfaces for Musical Expression. Ann Arbor, MI, USA.
Barbosa, J., Malloch, J., Wanderley, M. M., & Huot, S. (2015). What does “evaluation”
Fo
mean for the NIME community? In Proceedings of the 2015 International
Conference on New Interfaces for Musical Expression. Baton Rouge, LA, USA.
rP
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 3(2), 77-101.
ee
Brown, D., Nash, C., & Mitchell, T. (2017). A user experience review of music
interaction evaluations. In Proceedings of the 2017 International Conference on
rR
New Interfaces for Musical Expression. Copenhagen, Denmark.
Casciato, C., & Wanderley, M. M. (2007). Lessons from long-term gestural controller
ev
users. In Proceedings of the 4th International Conference on Enactive Interfaces.
Grenoble, France.
iew
Cook, P. (2001). Principles for designing computer music controllers. In Jensenius, A.
R., & Lyons, M. J. (Eds.). (2017). A NIME Reader: Fifteen Years of New
Interfaces for Musical Expression (pp 1-27). Springer. Cham, Switzerland.
On
Dobrian, C., & Koppelman, D. (2006). The E in NIME: Musical expression with new
computer interfaces. In Proceedings of the 2006 International Conference on
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
New Interfaces for Musical Expression. Paris, France.
Dourish, P. (2003). The appropriation of interactive technologies: Some lessons from
placeless documents. Computer Supported Cooperative Work, 12(4), 465-490.
Fasciani, S. & Goode, J. (2021). 20 NIMEs: Twenty years of new interfaces for musical
expression. In Proceedings of the 2021 International Conference on New
Interfaces for Musical Expression. Shanghai, China.
Fels, S. (2004). Designing for intimacy: Creating new interfaces for musical expression.
Proceedings of the IEEE, 92(4), pp. 672-685.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
Ferguson, S., & Wanderley, M. M. (2010). The McGill Digital Orchestra: An
interdisciplinary project on digital musical instruments. Journal of
Interdisciplinary Music Studies, 4(2), pp.17-35.
Franco, I., & Wanderley, M. M. (2017). Prynth: A framework for self-contained digital
music instruments. In Aramaki, M., Kronland-Martinet, R., Ystad, S.,
(Eds.). Bridging People and Sound (pp. 357-370). Springer. Cham, Switzerland.
Fukuda, T., Meneses, E., West, T., & Wanderley, M. M. (2021). The T-Stick music
creation project: An approach to building creative community around a DMI. In
Proceedings of the 2021 International Conference on New Interfaces for
Fo
Musical Expression. Shanghai, China.
Gelineck, S., & Serafin, S. (2012). Longitudinal evaluation of the integration of digital
rP
musical instruments into existing compositional work processes. Journal of New
Music Research, 41(3), pp. 259-276.
ee
Holland, S., Mudd, T., Wilkie-McKenna, K., McPherson, A., & Wanderley, M. M.
(2019). Understanding music interaction, and why it matters. In Holland, S.,
rR
Mudd, T., Wilkie-McKenna, K., McPherson, A., & Wanderley, M. M., (Eds.).
New Directions in Music and Human-Computer Interaction (pp. 1-20). Springer.
Cham, Switzerland.
ev
Jensenius, A. R., & Lyons, M. J. (Eds.). (2017). A NIME Reader: Fifteen Years of New
iew
Interfaces for Musical Expression. Springer. Cham, Switzerland.
Jordà, S. (2004). Instruments and players: Some thoughts on digital lutherie. Journal of
New Music Research, 33(3), pp. 321-341.
On
Laubier, S. de & Goudard, V. (2006). Meta-Instrument 3: A look over 17 years of
practice. In Proceedings of the 2016 International Conference on New Interfaces
for Musical Expression. Paris, France.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 38 of 54
Lepri, G., & McPherson, A. (2021). Embrace the weirdness: Negotiating values
inscribed into music technology. Computer Music Journal, 45(3), pp. 39-57.
Malloch, J., Garcia, J., Wanderley, M. M., Mackay, W. E., Beaudouin-Lafon, M., &
Huot, H. (2019). A design workbench for interactive musical systems. In
Holland, S., Mudd, T., Wilkie-McKenna, K., McPherson, A., & Wanderley, M.
M., (Eds.). New Directions in Music and Human-Computer Interaction (pp. 2340). Springer. Cham, Switzerland. 6_2
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 39 of 54
Malloch, J., & Wanderley, M. M. (2007). The T-Stick: From musical interface to
musical instrument. In Proceedings of the 2007 International Conference on
New Interfaces for Musical Expression. New York, NY, USA.
Mamedes, C. R., Rodrigues, M. G., Wanderley, M. M., Manzolli, J., Garcia, D. H. L., &
Ferreira-Lopes, P. (2014). Composing for DMIs – Entoa, music for
Intonaspacio. In Proceedings of the 2014 International Conference on New
Interfaces for Musical Expression. London, UK.
Marquez-Borbon, A., & Martinez-Avila, J. P. (2018). The problem of DMI adoption
and longevity: Envisioning a NIME performance pedagogy. In Proceedings of
Fo
the 2018 International Conference on New Interfaces for Musical Expression.
Blacksburg, VA, USA.
rP
McPherson, A. P., & Kim, Y. E. (2012). The problem of the second performer: Building
a community around an augmented piano. Computer Music Journal, 36(4), pp.
10-27.
ee
Meneses, E. A. L., Fukuda, T., & Wanderley, M. M. (2020). Expanding and embedding
rR
a high-level gesture vocabulary for digital and augmented musical instruments.
In Proceedings of the 2020 International Conference on Human-Computer
ev
Interaction. Copenhagen, Denmark.
Meneses, E. A. L., Wang, J., Freire, S., & Wanderley, M. M. (2019). A comparison of
iew
open-source Linux frameworks for an augmented musical instrument
implementation. In Proceedings of the 2019 International Conference on New
Interfaces for Musical Expression. Porto Alegre, Brazil.
On
Miranda, E. R., & Wanderley, M. M. (2006). New Digital Musical Instruments: Control
and Interaction Beyond the Keyboard. A-R Editions Ltd. Middleton, WI, USA.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Moro, G., & McPherson, A. P. (2020). Performer experience on a continuous keyboard
instrument. Computer Music Journal, 44(2/3), pp. 69-91.
Morreale, F. (2015). Designing new experiences of music making [Doctoral dissertation,
University of Trento].
Morreale, F., & McPherson, A. P. (2017). Design for longevity: Ongoing use of
instruments from NIME 2010-14. In Proceedings of the 2017 International
Conferences on New Interfaces for Musical Expression. Copenhagen, Denmark.
Morreale, F., McPherson, A. P., & Wanderley, M. M. (2018). NIME identity from the
performer’s perspective. In Proceedings of the 2018 International Conference
on New Interfaces for Musical Expression. Blacksburg, VA, USA.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
Mudd, T., Holland, S., & Mulholland, P. (2019). The role of nonlinear dynamics in
musicians’ interactions with digital and acoustic musical instruments. Computer
Music Journal, 43(4), 25-40.
Nieva, A., Wang, J., Malloch, J., & Wanderley, M. M. (2018). The T-Stick:
Maintaining a 12-year-old digital musical instrument. In Proceedings of the
2018 International Conference on New Interfaces for Musical Expression.
Blacksburg, VA, USA.
O’Brien, H. L., Cairns, P., & Hall, M. (2018). A practical approach to measuring user
engagement with the refined User Engagement Scale (UES) and new UES short
Fo
form. International Journal of Human-Computer Studies, 112(2018), pp. 28-39.
O’Brien, H. L., & Toms, E. G. (2008). What is user engagement? A conceptual
rP
framework for defining user engagement with technology. Journal of the
American Society for Information Science and Technology, 59(6), pp. 938-955.
ee
O’Brien, H. L., & Toms, E. G. (2010). The development and evaluation of a survey to
measure user engagement. Journal of the American Society for Information
rR
Science and Technology, 61(1), pp. 50-69.
O’Brien, H. L., & Toms, E. G. (2013). Examining the generalizability of the User
ev
Engagement Scale (UES) in exploratory search. Information Processing &
Management, 49(5), pp. 1092-1107.
iew
O’Modhrain, S. (2011). A framework for the evaluation of digital musical instruments.
Computer Music Journal, 35(1), pp. 28-42.
[author(s)]
[author(s)]
On
Schmid, G.-M. (2017). Evaluating the Experiential Quality of Musical Instruments: A
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 40 of 54
Psychometric Approach. Springer. Wiesbaden, Germany.
Springett, M. (2009). Evaluating cause and effect in user experience. Digital Creativity,
20(3), pp. 197-204.
Stowell, D., Robertson, A., Bryan-Kinns, N., & Plumbley, M. D. (2009). Evaluation of
live human-computer music-making: Quantitative and qualitative approaches.
International Journal of Human-Computer Studies, 67(11), pp. 960-975.
Sullivan, J., Guastavino, C., & Wanderley, M. M. (2021). Surveying digital musical
instrument use in active practice. Journal of New Music Research, 50(5), pp.
469-486.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 41 of 54
Triberti, S., Chirico, A., La Rocca, G., & Riva, G. (2017). Developing emotional
design : Emotions as cognitive processes and their role in the design of
interactive technologies. Frontiers in Psychology, 8(1773).
Wanderley, M. M. (2001). Performer-instrument interaction: Applications to gestural
control of sound synthesis [Doctoral dissertation, IRCAM—Université Paris 6].
Wanderley, M. M., & Orio, N. (2002). Evaluation of input devices for musical
expression: Borrowing tools from HCI. Computer Music Journal, 26(3), pp. 6276.
Wessel, D., & Wright. M. (2002). Problems and prospects for intimate musical control
Fo
of computers. Computer Music Journal, 26(3), pp. 11-22.
West, T., Caramiaux, B., Huot, S., & Wanderley, M. M. (2021). Making mappings:
rP
Design criteria for live performance. In Proceedings of the 2021 International
Conference on New Interfaces for Musical Expression (NIME 2021). Shanghai,
China.
ee
Young, G. W., & Murphy, D. (2015). HCI models for digital musical instruments:
rR
Methodologies for rigorous testing of digital musical instruments. In
Proceedings of the 2015 International Symposium on Computer Music
ev
Multidisciplinary Research. Plymouth, UK.
iew
On
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Appendix A
This appendix contains the list of T-Stick embedded gestures and the sound synthesis
parameters available in the Pure Data synthesizer.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
INSERT TABLE A-1 HERE
INSERT TABLE A-2 HERE
Appendix B
This appendix contains a summary table of the questions used for all questionnaires and
Fo
interviews, as well as the construct targeted by each item and its original source.
rP
INSERT TABLE B-1 HERE
iew
ev
rR
ee
On
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 42 of 54
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 43 of 54
100x23mm (144 x 144 DPI)
iew
ev
rR
ee
rP
Fo
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
ev
rR
ee
rP
Fo
100x69mm (144 x 144 DPI)
iew
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 44 of 54
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 45 of 54
ee
rP
Fo
149x67mm (144 x 144 DPI)
iew
ev
rR
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
rP
Fo
81x69mm (72 x 72 DPI)
iew
ev
rR
ee
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 46 of 54
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 47 of 54
ev
rR
ee
rP
Fo
149x99mm (144 x 144 DPI)
iew
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
Input Data
Mapping
Touch – Position Five frequency zones are positioned along the length of the T-Stick. A more
percussive Rain Stick Mode is activated when touch is detected at both ends.
Touch – Pressure
Force-sensitive resistors sense pressure applied to the surface which is
mapped to the synthesizer’s energy parameter.
Jab
Sets the low pass filter cutoff. A higher energy jab allows more high
frequency content through the filter.
Shake
Changes the reverb parameter based on the vigorousness of shaking.
YPR – Pitch
Resonance parameter of the low pass filter.
YPR – Roll
Timbre parameter of the synthesizer.
Fo
Table 2-1. Mappings between T-Stick data and synthesis parameters.
iew
ev
rR
ee
rP
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 48 of 54
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 49 of 54
Task ID
T01
Task Name
Framing
Cycle
C1
Task Instructions
To hold the T-Stick without produced any sound, the
instrument can be grasped by the top cap, the bottom cap,
or both. Sound will be produced when the surface of the
T-Stick between the two caps is contacted.
T02
Filter Cutoff
C1
The cutoff of the low pass filter sets the frequency below
which sound will be passed through. If the cutoff is set too
low, no audible sound will be produced. The value is set
using the T-Stick’s jab gesture. More physical effort will
result in a higher acceleration value, and thus, a higher
filter cutoff.
T03
Amplitude
C1
The volume of the output can be set by applying varying
levels of pressure along the surface of the instrument
between the two caps.
T04
Frequency
T05
Timbre
T06
Resonance
C2
T07
Infinite Reverb
C3
Reverb is controlled by shaking the T-Stick. A more
energetic shake gesture will produce more reverberation.
When the maximum amount of reverb is attained through
continuous shaking, an infinite reverb can be produced by
maintaining this energy.
T08
Low Level Reverb
C3
Reverb is controlled by shaking the T-Stick. A low to
moderate level of reverb can be maintained by shaking
with a small to medium amount of energy. This amount of
reverb can be sustained by continuously applying this
same amount of energy to the shaking gesture.
T09
Rain Stick Mode
C4
This mode changes the sound of the T-Stick from the
drone synthesizer to a more percussive granular sound.
This mode can be activated by grasping the T-Stick at each
end of the touch-sensitive surface (beyond the caps).
T10
Mode Switching
C4
This tasks consists of shifting fluidly between the drone
synthesizer and the rain stick mode.
Fo
C2
Five frequency zones are distributed along the length of
the T-Stick. Touching each of these zones will produce a
different frequency, the amplitude of which can be
controlled by the amount of pressure applied.
rP
C2
Timbre controls the mix between two voices of the
synthesizer. Timbre is mapped to the roll parameter of the
accelerometer and can be controlled by moving the T-Stick
through space in the roll dimension.
rR
ee
The resonance of the low pass filter is mapped to the pitch
parameter of the accelerometer and can be controlled by
moving the T-Stick through space in the pitch dimension.
iew
ev
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Table 2-2. Task (T) and task cycle (C) list and instructions. Task names link to each
demonstration video.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
Stage Number
Stage Name
State Components
S1
S2
Pre-Screening
Questionnaire
Introductory Session
Eligibility screening
Participants’ background and experience
Informed consent
Entrance interview
Open exploration
First impressions questionnaire
S3
Task Cycle (x4)
Task demonstration . Participants were provided with
demonstrated tasks for the first four practice sessions.
The fifth session was used to prepare their original
excerpt.
Practice session (x5)
Task performance (x4)
S4
Third-Party
Evaluation
Evaluation session questionnaire (x5)
Evaluation session interview (x5)
The final performance of an original musical excerpt
Exit questionnaire
Exit Interview
Third-party ratings of participants’ final performances
rR
ee
S6
Evaluation Session
(ES; x5)
Final Evaluation
Session
rP
S5
Fo
Table 2-3. List of the six study stages (S) names and components.
iew
ev
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 50 of 54
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 51 of 54
Gesture
Sensor
Task Instructions
Touch
Capacitive Sensors
Senses the amount of surface area being touched
Brush/MultiBrush Capacitive Sensors
Senses movement along the T-Stick surface in two
directions parallel to the length
Rub/MultiRub
Capacitive Sensors
Senses continuous bidirectional movement applied
along the length
Shake
Gyroscope
Senses the “energy” of rapid continuous movement in
three dimensions (XYZ)
Jab
Gyroscope
Senses the “energy” of short, impulsive movements in
three dimensions (XYZ)
YPR
(Yaw/Pitch/Roll)
Inertial
Measurement Unit
Senses orientation, rotation, and tilt of the T-Stick in
three dimensional (XYZ) space
Fo
Table A-1. T-Stick embedded gestures.
iew
ev
rR
ee
rP
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Journal of New Music Research
Parameter
Description
Energy
Amplitude and brightness. Sounds harsh at high parameter values.
Frequency
Fundamental frequency of two oscillators, limited to a baritenor range.
Timbre
Crossfade between the synthesizer’s two voices
Cutoff
Cutoff frequency of the synthesizer’s low-pass filter. Can be used to dampen
excessive brightness resulting from high energy values.
Resonance
Q value of the low-pass filter. High values result in self-oscillation.
Reverb
Crossfade and time of the synthesizer’s reverberation. High values result in
infinite reverb.
Table A-2. Synthesis patch parameters.
iew
ev
rR
ee
rP
Fo
ly
On
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Page 52 of 54
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 53 of 54
Data Collection
Pre-Screening
Questionnaire
Entrance Interview
Construct of Interest
ResearcherDeveloped
Potential participants’ level of Q1, Q4, Q6
interest in and experience with
DMIs.
Q2, Q3,
Q5
MPX-Q UES
N/A
N/A
N/A
N/A
Q1 to
Q3
N/A
rP
Q5
N/A
Q4
N/A
N/A
N/A
N/A
N/A
Q5 to Q1 to
Q14
Q4
N/A
N/A
N/A
N/A
N/A
N/A
N/A
Q1 to
Q3
N/A
N/A
N/A
N/A
On
First Impressions
First impressions of musical
Questionnaire
apparatus.
Evaluation Session Task Participants’ ability to perform
Performance (x4)
tasks from each block.
Evaluation Session Participants’ perceptions of the
Questionnaire (x4)
instrument while learning to
perform tasks from each block.
N/A
N/A
Participants’ experience
Q1 to Q3
learning to perform the tasks
from each block.
Final Performance
Participants’ ability to recall,
N/A
perform, and combine tasks
from each block in a larger
musical structure.
Exit Questionnaire
Participants’ overall
Q4 to Q5
perceptions of the instrument
after learning all tasks and
their interest in ongoing use of
the instrument.
Exit Interview
Participants’ experience over Q1 to Q10
the entire study and
relationship with the
instrument.
Third-Party Evaluator Evaluator perceptions of the
Q1 to Q4
Questionnaire
range of gestures, smoothness
of motion, and overall quality
of participants’ final
performance.
iew
ev
rR
ee
Evaluation Session
Interview (x4)
EMI
Use, performance, and general Q1, Q3, Q5, Q2, Q4,
attitudes with respect to DMIs; Q9 to Q11 Q6 to Q8
Previous experience with the
T-Stick and other music
technology setups
Fo
N/A
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Journal of New Music Research
Table B-1. Data collection: Target constructs and question sources.
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk
Page 54 of 54
Journal of New Music Research
Figure 2-1 Caption: Top: Sopranino T-Stick. Bottom: GuitarAMI Sound Processing Unit
(SPU).
Figure 2-1 Alt-Text: Above: An image of the T-Stick hardware, a length of ABS plumbing
pipe wrapped in heat shrink. Below: An image of the GuitarAMI Sound Processing Unit
(SPU), a rectangular black metal box with a small display screen and five footswitches.
Figure 2-2 Caption: Diagram of the six study stages (S).
Figure 2-2 Alt Text: A visual depiction of the six stages of the study with arrows to indicate
Fo
the order of each stage and the repeated task cycles of stages three and four. The data
collected during each stage is indicated below the flow diagram.
rP
Figure 3-1 Caption: Ratings of each participant’s final performance by third-party evaluators.
ee
A rating of 1 corresponds to “Low quality.” A rating of 10 corresponds to “High quality.”
rR
Figure 3-1 Alt Text: A bar chart depicting ratings of the overall quality of participants’ final
performances, provided by two third-party evaluators.
ev
Figure 3-2 Caption: Total incidents (y-axis) in nine categories summed for all participants
over evaluation session interviews (ES1 through ES5).
iew
Figure 3-2 Alt Text: A histogram of the number of coded interview incidents, combined for
all participants, in each of nine coding categories. Frequency bars are provided individually
On
for each of the five evaluation session interviews, illustrating how the frequency of
occurrence for each category changes over time.
ly
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
URL: http://mc.manuscriptcentral.com/nnmr Email: nnmr-peerreview@journals.tandf.co.uk