Bar Songs
Evaluating an Aural Analog to Simple Data Visualizations
IS 805 - Dr. Wayne Lutters
Karim Said
2016-02-09
9 Page(s)
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
Introduction
Data visualizations have become invaluable tools for making sense of the deluge of information we are
faced with daily in our increasingly wired world. Fortunately, the innate drive to get from “data to
understanding” (Fry B. , 2008) has prompted the design and production of progressively complex and
expressive visualizations (McCandless, 2010). With their dynamic presentation, interactivity, and striking
fidelity, contemporary visualizations appeal to our need to operate in data-rich environments as well as
our sense of aesthetics (Fry B. J., 2004); however, data visualizations are by definition limited to evoking
only one of the senses, sight, and ignore the possibility conveying information through other modalities.
The notion of conveying information using sounds has been raised (Norman, 2002), but attempts have
mainly focused on strict acoustic (and often cacophonous) translations of data. A true aural analog to
today’s beautiful data visualizations remains to be heard and, befitting a challenge rooted in humancentered computing, promises to test the abilities of both artists and engineers.
This study seeks to investigate the reception among listeners of common bar charts (a very simple but
pervasive form of data visualization) translated into musical compositions. Fundamentally, the study is
driven by the following questions:
1. What emotional responses does data represented musically (DRM) evoke?
2. Is DRM easily and/or intuitively understood?
a. How well can listeners convey their understanding of DRM?
3. Is DRM memorable?
a. Can DRM become a hummable earworm?
The study leverages a prototype application for the generation of Bar Songs (i.e., short musical
compositions based on simple two-axis data sets). Bar Songs follow several basic rules for the translation
of data into music (described in Table 1). The application was developed using the Processing
programming language (Processing.org) and a library for sound synthesis called SoundCipher
(SoundCipher - Music and Sound for Processing). All source code for the prototype is available via a
public repository hosted by GitHub (egyptiankarim/BarSongs - GitHub).
Data Characteristic
Musical Referent
Independent Variable Percussion; Single unit progressions of the independent variable are represented
as strikes of a snare drum. Every fifth independent variable value is represented
Value (e.g., Time)
as a bass drum strike.
Dependent Variable String (Piano); Prior to production of the musical composition, Bar Songs
Value (e.g., Stock processes all dependent variable values and maps them to various notes in the
key of C. The higher the value the higher the pitch of the translated note.
Price)
Running Averages of String Ensemble (Synthesized Violin); The running average of dependent
Dependent Variable variables over the following ten independent variable intervals is compared to
the average of the entire data set to determine whether a random C-chord should
Values
be played as a major or minor.
Table 1 - The Bar Songs prototype application mapped several characteristics of simple data sets into musical referents.
While the Bar Songs application was the principal tool used to demonstrate the notion of DRM to
participants, the focus of the study is not to evaluate the prototype’s efficacy (in fact, study participants
Bar Songs
1/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
were never even shown or asked to use the Bar Songs prototype, but instead were only asked to respond
to the music the application generated). Rather, the intent of the study, as reflected by its core questions
and overall design, is to evaluate the general idea of DRM.
Figure 1 - Bar Songs was never shown to study participants, rather they responded only to the music generated by the application.
Study Design + Rationale
During the weeks of 2011-11-13 and 2011-11-20, a series of semi-structured interviews were conducted
with seven convenience-sampled participants (detailed in Table 2). Interview stations were arranged in
various locations across the UMBC campus (e.g., The AOK Library, the Residential Life Office, and a study
lounge in Harbor Hall), and passerby were asked to participate in a short study in exchange for peanut
M&M’s (a surprisingly effective strategy for garnering interest). During interviews, participants were
asked a variety of survey style questions regarding their age, academic concentration, and musical
training experience. Subsequently, participants were asked to listen to short segments of Bar Songs based
on a variety of data sets (e.g., trigonometric functions, stock prices of various technology companies, and
random data), and their general affect was observed. Following the Bar Songs, participants were asked to
describe their experience in a variety of ways including verbal dialogs, drawings, and humming exercises.
All interviews were video recorded for subsequent analysis.
In considering the design of this study, several issues became apparent. Specifically, the eventual study
design would have to address the effects of small-scale study logistical limitations on data richness, the
inherent difficulty in evaluating general notions of technology usability based on prototypes, and the risk
related to selecting a convenience sample from amidst a population of peers.
Bar Songs
2/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
Mundane factors related to schedules and budgets (though seldom explicitly mentioned) can often
influence the robustness of qualitative research. These types of hindrances are unfortunate, especially
when a researcher’s success hinges on the richness of the data they collect. In both the Tsujita, et al.
paper, “InPhase”, and the Tollmar, et al. paper, “A Picture is Worth a Thousand Keywords”, the
researchers balance seemingly short data collection windows and relatively small participant sample
sizes throughout their prototype-centric investigations of broad technology usage themes (ambient
communication systems, and mobile image-based search, respectively). In both cases, the researchers
rely on observations, surveys, and interviews to collect information from their convenience sampled
participants. While perhaps not as robust as would ideally be desired, both studies make an intriguing
case for their respective areas of interest, despite the external factors that may have been constraining
their studies.
Bar Songs was similarly constrained from its outset by schedule and scope, and so a semi-structured
interview data collection method was selected in the hope that prompts would assist participants in fully
expressing their experiences while still allowing the study to remain focused. Additionally, observation of
participant affect during interviews was determined early on to be essential to the overall robustness of
the study, and as such was incorporated into the study design. Specifically, the convenience sample of
UMBC students raised a concern that verbal responses to interview prompts may be unduly positive (in
an attempt by participants to appease a peer interviewer), and so it was believed that observations at
several points during the interview would help uncover more genuine responses to Bar Songs, without
excessively complicating management of the study within a short timeframe. Finally, survey like prompts
were incorporated at the beginning of all interviews for their efficiency in garnering meta-data about
participants that would prove useful during the data analysis that would follow. In all, this weak-form
triangulation of data collection techniques proved, as with Tsujita, et al. and Tollmar, et al., to be an
effective strategy for quickly developing a general understanding of user reception of a broader
technological concept in a prototype-centric study.
It is worth noting that the typical strategy (at least in commercial settings) for collecting information
about a technology prototype involves using focus groups; however, because the Bar Songs study sought
to garner information on the unique perceptions of participants to DRM, it was believed that the public
nature of a focus group would interfere with the earnestness of responses. Moreover, the semi-structured
interview provided the opportunity for focused exploration of individual experiences that might have
been overlooked in a focus study where participants are inherently competing for attention. Finally, there
was a concern that any focus group would be dominated by participants with deeper musical
backgrounds, and the study sought to focus equally on participants of all skill levels.
In their paper, “A Picture is Worth a Thousand Keywords”, Tollmar, et al. inadvertently highlight the
difficulties of leveraging a prototype application (especially ones of novel import) in order to draw
conclusions regarding general technology usage themes. The authors begin with a discussion of the
nascence of the field of mobile image-based search, and go on to describe their implementation of a
smartphone-based application that leverages the smartphone’s camera to capture images and create
search queries based on their composition. Throughout the article, the authors describe several features
of their prototype (e.g., the “Nonmode” and “Shopping Mode” application search settings), but neglect to
recognize how these idiosyncrasies bias their investigation of mobile image-based search in general.
Recognition of the bias that a prototype may introduce in a study of general technology usage themes was
central to the Bar Songs study. Fortunately, the semi-structured interview format of the study leant itself
Bar Songs
3/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
well to the detachment of study participants from the prototype, and allowed a clearer focus on the
general idea of DRM. Specifically, the chosen study design allowed the interviewer to completely handle
usage of the Bar Songs prototype, requiring participants only to listen to the application’s output (indeed,
the interview protocol rejected letting participants interact with Bar Songs’ visual and control interfaces).
Despite efforts to detach participants from the quirks and foibles of the particular implementation used
to demonstrate the notion of DRM, however, it is critical to realize the bias inherent in using any
prototype (even sparingly) to generalize on broader technology themes. In as much as this study is a
pilot, these residual biases are palatable; however, a more robust study would have to further account for
them (discussed in greater length in Conclusion).
In the Tsujita, et al. paper, “InPhase”, a convenience sample is selected from amongst what can be
inferred to be the principal researcher’s cohort and familial relations. While a strictly interpretivist
worldview recognizes no inherent harm in selecting study participants in such a manner, ignoring
potential biases introduced by such non-probabilistic sampling methodologies is detrimental to a study’s
face validity and is best addressed at the outset.
Considering the logistical constraints of the Bar Songs study, selection of participants was limited to the
immediately available population (i.e., the UMBC community). In order to account for the potential biases
introduced by such a sampling strategy, however, several rules were employed within the study design to
enhance the face validity of any eventual findings. Specifically, all participants were passerby in welltrafficked areas of the UMBC community (e.g., the library, offices, and dormitory study halls), and people
with close personal or professional relationships to the interviewer were eschewed in favor of casually
familiar and previously unknown participants. Moreover, as described earlier, recognizing that even an
unknown peer group may be prone to unduly positive interview responses (in the hopes of appeasing a
fellow student interviewer, perhaps), elements of observation were incorporated into the interview
protocol in attempt to capture participant affect more truthfully than may have been verbally described.
In all, by taking cues from other successful prototype-centric investigations of general technology usage
themes, the design selected for this study balances the challenges of logistical limitations in a small-scale
study, biases arising from leveraged prototypes, and sample selection from amongst a population of
peers.
Participant
P1
P2
P3
P4
P5
P6
P7
Age
18
19
22
22
21
21
20
Sex
Female
Female
Male
Male
Female
Male
Female
Major
Pre-Nursing
Bio., Psych., Pre-Med
Financial Economics
Political Science
GIS
Mech. Engineering
Psych., Pre-Law
Musical Background
~6 Years (Piano)
N/A
~1 Year (Banjo/Guitar [Self-Taught])
~10 Years (Violin, Choir)
~3 Years (Clarinet)
N/A
~5 Years (Saxophone, Choir)
Table 2 - Convenience sample participants were all students of UMBC.
Bar Songs
4/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
Initial Analysis
Analysis of the data collected during this study’s seven semi-structured interviews largely followed an
analytic induction approach. Specifically, several iterations of content and discourse analysis were
conducted to assess study participant affect, the ease of understanding DRM, and the memorability of
DRM.
In the first phase of content analysis, interview sessions were reviewed and listener responses to Bar
Songs were tallied for number of smiles, grimaces, brow furrows, and other affective displays. This
particular phase of content analysis was intended to counteract the potential biases (i.e., unduly positive
responses) that may be expressed by study participants being interviewed by a peer. The rationale for
using this sort of analysis hinges on the idea that participant affect can be more truthfully discerned from
facial expressions and body language than from verbal responses.
In a similar vein, participant’s drawn responses to various Bar Song prompts were chosen for analysis of
affect when it was observed that several participants were using abstract images to convey their thoughts
about Bar Songs.
Generally, this first phase of analysis reflects the analytically inductive nature of the overall approach, as
the initial set of codes (i.e., listener smiles and grimaces) was quickly augmented when brow furrowing
and looks of concentration were observed. Excel spreadsheets were created to record this data during
playback of video-recorded interview sessions.
In order to assess how easy and intuitive participants experienced Bar Songs to be, video-recorded
interview sessions were reviewed to capture responses to prompts following each of three Bar Song
segments played for participants. Additionally, drawn responses were compared to the Bar Songs that
inspired them, to discern how effectively participants were able to express visually the data they had just
heard musically. Again, Excel spreadsheets were used to record notable quotes along with video clip
information (i.e., file name, and timestamp).
The last phase of analysis consisted of yet another review of the video-recorded interview sessions with a
focus on assessing the memorability of Bar Songs. Specifically, prompts for participants to recall
characteristics of Bar Songs, to compare and contrast Bar Songs against one another, and to attempt to
hum Bar Songs were reviewed. Notable quotes were recorded in an Excel spreadsheet along with video
clip file names and timestamps.
In all, the coding strategy employed reflects a middle ground between using pre-formed “etic” categories
and a more grounded approach. This approach allowed the formation of initial categories based on the
study’s core research questions, while still offering the flexibility to recognize emergent patterns in the
collected data.
Preliminary Findings
Early on during the study, several themes quickly became apparent and persisted throughout most of the
data collection and analysis stages. Regarding the general affect of participants listening to Bar Songs, the
most apt description appears to be boredom. Further, based on participant response, Bar Songs did not
appear to be easy or intuitive to understand, or particularly memorable.
Bar Songs
5/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
Perhaps the most difficult of the research questions proposed by this study regards the emotional
responses evoked by DRM. Study participants, almost unanimously, echoed P1’s sentiment that “[the Bar
Song] was nice… It was a nice little tune”, regardless of the Bar Song played. The almost universally
positive response of study participants was a key indicator that verbal interview responses alone may be
unreliable as a true measure of listener affect. In fact, observation of listener facial expressions and body
language challenges the idea that Bar Songs were emotionally evocative at all. Open coding tallied no
smiles, or grimaces for any of the Bar Songs played and the general demeanor for almost every
participant is best described as flat. The two counter examples to this theme of boredom seemed to occur
with P4 and P5 who both closed their eyes, with brow furrowed, and appeared to be deeply
concentrating during each of the Bar Songs; however, when asked to describe their experiences both
remarked they were just trying to maintain their attention. Possibly the most (inadvertently) telling
response resulted from P1 who, trying to pinpoint where she had heard a Bar Song before, decided it was
“kind of like an elevator [song]”.
It is worth noting that during analysis of participants’ drawn responses to various Bar Song prompts
another theme did emerge regarding affect; P4, P5, and P6 all drew significantly abstract images which
did seem to speak to their perception of serenity in the Bar Songs. Specifically, P4 drew a full-page
mountain-lined lake scene. When prompted for an explanation of the scene, P4 expressed that the
“tranquil” image represented the calm and measured nature of the Bar Song. Similarly, P5 drew an image
of a sunset and sunrise to reflect the falling and rising pitch of notes in the trigonometric Bar Song. P6
sketched a bow-wielding piano player with a sheepish (literally) audience, commenting on the simplistic
nature of the Bar Songs and their similarity to melodies a person learning an instrument might practice.
Whether, these pleasant-leaning images represent a more positive affect than boredom among listeners
is unclear (not enough participants chose to convey their Bar Song experiences in such an abstract
manner); however, either explanation is possible as the drawings could just as easily reflect idle doodling
by an uninterested participant as they could the earnest satisfaction of a Bar Song fan.
Regarding the intuitiveness of DRM there was, again, an almost universally supported conclusion that Bar
Songs were not easily or naturally understood beyond the recognition of basic patterns. Specifically, in
response to the Bar Songs based on trigonometric functions all participants were able to identify either
verbally or through drawn responses (e.g., P2, P3, and P7 all drew distinct sinusoidal forms to represent
the music they had just heard) that the data was moving “up a scale, [and] down a scale” (P3);
additionally, these gross generalizations were often repeated for Bar Songs based on stock information
that also slightly matched the “up and down” (P1) pattern. Similarly, several participants recognized that
Bar Songs based on random data sounded “erratic” (P2) or “all kind of mixed together” (P5), but none
were able to more closely describe what they were hearing. Moreover, prompts to compare and contrast
various Bar Songs were met with the common conceit that they all “sounded similar” (P5), with P3
explicitly remarking that “[he was] trying to differentiate between the [Bar Songs], and [was] having
difficulty”.
Overall, the findings for the intuitiveness of DRM certainly warrant deeper investigation (perhaps even
more so that this study’s principal research question). The idea that even general patterns in DRM are
regularly discernible, despite a lack of more detailed understanding of the underlying data, may call for a
study in and of itself.
Again nearly universally, the findings for the final research question addressed by this study, which
regarded memorability of DRM, seemed to show that Bar Songs were not memorable beyond their basic
Bar Songs
6/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
characteristics. Although all participants were successful in recalling the distinct musical referents of the
Bar Songs and general characteristics of the compositions as a whole, most encountered difficulty
humming back the Bar Songs because, as P6 stated, “it didn’t seem like [the musical referents] were
playing together”. In fact, many participants rejected even trying to hum back the Bar Songs, remarking
that they didn’t remember enough of the short pieces they were played to recreate them.
Discouraging as the results of this study may seem, it would be inappropriate to conclude that the notion
of DRM is a wash. Despite the efforts of the study design to detach participants from the shortcomings of
the Bar Songs application, it seems fairly obvious that doing so fully is impossible (though not without
opportunity for mitigation, as discussed in Retrospective Reflection and Conclusion). The data collected
and analyzed, then, potentially speaks more to the shortcomings of the prototype and cannot be
genuinely interpreted as a comprehensive reflection of the larger technology usage theme. Moreover, the
diversity of drawn responses to various Bar Song prompts, and the emergent notion that general patterns
in DRM are easily recognizable beg deeper investigation and new questions.
Retrospective Reflection
Note: Excuse the sudden shift to first-person parts of speech in this section. It’s the only place where it
seemed appropriate and even necessary.
Having concluded my study, I found it useful to consider all of the positive and negative aspects of my
chosen approach. In particular, I had the dual benefit and disadvantage of sharing a similar study
framework with the two most panned papers of the semester, the challenge of developing a prototype
with elements of an artistic discipline in which I am a total novice, and all of the miscellaneous difficulties
facing a first-time academic interviewer.
It was a bit rattling to realize that from a methodological perspective my proposed study matched the
Tollmar, et al. and Tsujita, et al. papers (i.e., a prototype-centric qualitative investigation of a broader
technology usage theme), considering they were the two most critically received studies of the semester.
Fortunately, I was able to leverage some of the lessons learned from reviewing “A Picture is Worth a
Thousand Keywords” and “InPhase”, and was able to avoid some of the pitfalls (e.g., balancing prototype
usage with the scope of conclusions regarding general technology usage themes, and managing
convenience sampling techniques in a small-scale study). On the other hand, some of those same pitfalls
faced by Tollmar and Tsujita did prove to be unavoidable. Specifically, it was difficult to fully detach the
shortcomings of my specific prototype from the results of my data collection, which impacts the veracity
of any findings in light of the broad intended scope of the study.
The Bar Songs prototype itself was a source of several challenges including the fact that I am myself a
musical novice (I’ve had no formal training in any instrument, and have only casual experience in the
theoretical underpinnings of the art form). In retrospect, my study could have been repurposed as an
exercise in participant design had I been able to recruit some musicians and a steady group of users, but
such a change to the study design would’ve been significant and it’s unclear if such a design would even
be viable considering the logistical limitations of a semester-long pilot study. The benefit of a participant
design approach is clear, however, as iteratively improved prototypes would help to further detach study
participants from the shortcomings of any one specific prototype and more directly focus on the broader
notion of DRM. Moreover, working with continually improved versions of a prototype provides a sort of
“built-in” triangulation that would enhance the believability of any eventual findings.
Bar Songs
7/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
Finally, the tribulations of an inexperienced academic interviewer are all well known to the initiated;
faulty recording devices, anemic interview guides, and a dearth of willing participants are always a
challenge. Fortunately, a few choices, some lucky, addressed a lot of first-timer issues; in short order: an
interview guide with preformatted areas to record certain critical data and quotes, and a supply of
M&M’s.
Conclusion
Looking ahead, there are certainly a number of opportunities for this study to be improved upon. In
particular, the Bar Songs prototype can stand to be enhanced (a process which in and of itself could be
the subject of a participant design study), sample sizes could be augmented and diversified, and
quantitative methods could be introduced to help explore more statistically-oriented notions of usability.
As mentioned earlier, the development of Bar Songs occurred devoid of any actual musical expertise. A
participant design approach to further development of the prototype application, especially if trained
musicians could become involved, would provide a great deal of insight of how musical referents could be
optimally used and arranged. Improved musical expression of different patterns of data (e.g., dramatic
stings for precipitous drops in data, and leitmotifs for common patterns) may improve the usability and
maybe even memorability of Bar Songs. Moreover, the availability of multiple versions of prototype
applications for DRM would provide a form of triangulation to the data collection process and contribute
to the overall rigor of any future studies. As stated earlier, the source code for the Bar Songs prototype is
currently being hosted publically through GitHub, and it is hoped that the application will find coadoption by other developers for further advancement.
Core to any study seeking to investigate broader technology usage themes, is the notion that eventually
the technology will be broadly used; this idea demands generalizability of study findings. Given more
time, a larger participant pool, selected using a probabilistic sampling approach, could contribute to a
more robust study. Considering the technology at the core of this study could potentially be developed for
widespread use (maybe even as a commercial product), a random sample from a much larger population
may be more reflective of usability and memorability of DRM.
Alternatively, a purposive sampling of potential key user communities (e.g., blind users with an interest
in using this sort of technology for assistive purposes) would also add an interesting dimension to future
study efforts. One unfortunate circumstance of this study was its lack of participants with previous
experience using assistive technologies or other sound-based interfaces. A targeted sampling of such
participants could add a great deal of insight to any future studies, especially one leveraging participant
design for improvement of a prototype.
Finally, this study could also be improved through the introduction of quantitative methods.
Unfortunately, when gauging the correctness and memorability of user interpretations of DRM,
interviews and other qualitative approaches lack the definitiveness of statistical models. Specifically, an
experiment could be arranged to identify the effects of changing different musical referents on the
correctness of user interpretations of various Bar Songs. Similarly, measures of a Bar Song’s
memorability could be tested and measured according to a rubric, and the resulting scores used to track
the effectiveness of various prototype implementations. The results of such quantitative investigations
could be used to make more confident claims as to the general usability of DRM.
Bar Songs
8/9
IS 805 - Dr. Wayne Lutters
Karim Said
Bar Songs
2016-02-09
Bibliography
egyptiankarim/BarSongs
GitHub.
(n.d.).
Retrieved
https://github.com/egyptiankarim/BarSongs
12
09,
2011,
from
GitHub:
Fry, B. J. (2004). Computational Information Design. Massachusetts Institute of Technology, School of
Architecture and Planning.
Fry, B. (2008). Visualizing Data. O'Reilly Media.
McCandless, D. (2010). Information is Beautiful. Collins.
Norman, D. A. (2002). The Design of Everyday Things. New York, NY, USA: Doubleday/Currency.
Processing.org. (n.d.). Retrieved 12 09, 2011, from Processing: http://www.processing.org
SoundCipher - Music and Sound for Processing. (n.d.). Retrieved 12 09, 2011, from SoundCipher:
http://explodingart.com/soundcipher/
Bar Songs
9/9