Are Rock Legends Random Phenomena? : Testing the patterns
of missed notes to determine degrees of randomness.
I: Introduction
Guitar Hero is a popular video game where rock enthusiasts can act as Slash,
Hendrix or Clapton playing their favorite songs with a guitar-shaped controller.
Players attempt to hit sequences of notes at specific times as dictated by the game.
If the player hits a wrong note, plays at the incorrect time, or misses the note
altogether, the note doesn’t count. As more notes are missed, the in-game
spectators respond unfavorably, and the player risks getting booed off the stage
before the end of the song. From experience it seems to be the case that some parts
of songs are more difficult than others, and lead to a higher proportion of missed
notes. We wanted to test this hypothesis by looking at the distributions of misses
through different songs. Our null hypothesis is that notes are missed at random, and
our alternative is that misses follow some sort of pattern.
To test these hypotheses, we invited undergraduate students and professors
to play Guitar Hero one afternoon in the math department, and recorded their
efforts while playing different songs. There was a broad range of skills, and we
encouraged players to pick a challenging difficulty level in order to give us data with
many misses.
After collecting the data, the next step was to transfer the data from a video
recording to an electronic series of 0’s and 1’s to symbolize hits and misses,
respectively, in the series. This required us to play back the videos slowly, and
record by hand the sequence of notes for each song. It was difficult to ensure
accuracy in our transcription of notes, so it is very possible that there are
inaccuracies that may exist in our data. We attempted to overcome this by having
multiple students transcribe the data, in hopes that we would have sequences that
matched. This was rarely the case, and as a result, there may exist some
inaccuracies within our data.
Our data took the form of something like this:
(0,1,0,0,0,1,0,0,1,1,1,0,0,0,0,1,0,0,0,0). This shows a song with 20 total notes, 6 of
which were misses and 14 of which were hits. We would assume this song to have a
relatively high degree of non-randomness, due to the grouping of misses midway
through the song. We developed our estimators to try to pick out these particular
types of non-randomness.
In section II, we will describe the estimators we developed as tools to using
during our bootstrap hypothesis testing. Section III gives the results of our power
study, whereas section IV gives a description of how we applied our methodology to
the actual songs. We finalize our discussion in section V.
II: Methodology
In order to determine if our songs had misses that followed a type of pattern
(not random), we needed to create estimators that would allow us to obtain a test
statistic for each song. This test statistic could then be compared to the test
statistics of songs generated using a bootstrap hypothesis test to give us the p value.
The first estimator we developed is called “consecutive.” To generate the test
statistic for “consecutive,” we create a point value that is based on the number and
size of miss streaks throughout the song. For each miss in a song, we add points to
the total depending upon how long the streak of consecutive misses is. If there is
only one missed note following the first miss, we add one point to the point total. If
there are two missed notes following the first miss, we add 1+2=3 to the point total.
For any length of misses following the first miss, we add 1+2+…+n to the point total,
where n is the number of misses until the next hit. After calculating the point total,
we divide that by the number of misses in the song to obtain the test statistic. See
below for a visual calculation:
Note
Song
1
0
2
0
3
0
4
1
0
5
0
6
0
7
0
8
0
9
0
10
0
11
0
12
1
13
1
14
1
15
1
16
0
17
0
18
0
19
0
20
0
0
+1 +2 +3 = 6
0 +1 +2 = 3
0 +1 = 1
0
Test Statistic = Point Total/Misses = (0+6+3+1)/5 = 10/5 = 2
It is important to note that this 1+2+…+n needs to be added for the number
of misses after every miss. So for a streak of four misses, like above, we would add
1+2+3 for the first miss, 1+2 for the second miss, and 1 for the third miss. The final
miss is followed immediately by a hit, so we don’t add any points for the final miss.
This estimator was developed with the idea that long miss streaks should be
indicative of non-randomness. Longer miss streaks will add more points to the
point total, so higher test statistics represent a higher degree of non-randomness.
Our second estimator was called “the spread.” This estimator is very similar
to “consecutive” in that we calculate a point total and divide it by the total number of
misses. The point total is calculated again by assigning point values for misses
following other misses, but in a slightly different fashion. At each miss, we add three
points to the point total if next note is also a miss, two points if the 2nd note is a miss,
and one point if the 3rd note is a miss. We sum all the points, and then divide by the
total number of misses, just like in the first estimator. A major difference of this
estimator compared to the first one is that our first estimator is dependent upon
consecutive misses, whereas this one does not require misses to be in a row. We
still add to the point total, even if there are hits scattered with our misses in the
three notes following each miss. Also, we are now limiting the correlation effect to
only 3 notes after each miss. In other words, whereas before we were penalizing
longer miss streaks more severely as they grow in length, this estimator is only
dependent upon the three notes directly following each miss. See below for a visual
calculation:
Note
Song
1
0
2
0
3
0
4
1
5
0
6
0
7
1
8
1
9
0
10
1
11
0
12
1
13
0
14
0
15
0
16
0
17
0
18
0
19
0
20
0
X +0 +0 +1
X +3 +0 +1
X +0 +2 +0
X +0 +2 +0
X +0 +0 +0
Test Statistic: Point Total/Misses = 9/5 = 1.8
*Where X represents the positions of missed notes, and thus the evaluation of the next three notes
following it.
This estimator uses a lot of the same ideas behind the consecutive estimator.
Our notion of non-randomness was still based on the idea of misses being closely
grouped together. However, this estimator did not require misses to follow directly
after one another, but could allow for small numbers of hits to be scattered within a
group of misses. Again, we thought that songs with higher degrees of nonrandomness would have greater point totals, indicating higher test statistics should
represent greater degrees of non-randomness.
Our third estimator is called “streaks” and looks for long streaks of hits. To
calculate the test statistics for “streaks,” we took the length of the two longest hit
streaks, added them together, and multiplied this by the proportion of misses within
the song. See below for a visual calculation:
Notes
Songs
1
0
2
1
3
1
4
0
5
0
| 1 2
6
0
7
0
3 4
8
0
9
0
5 6 |
10
1
11
0
12
0
13
1
14
0
| 1
15
0
16
0
17
0
18
0
2
3
4
5 |
19
1
20
0
Test Statistic: (6+5)*(.2) = 2.2
This estimator was developed with the idea that long hit streaks should also
be indicative of non-randomness. As longer hit streaks appear, it should also be true
that missed notes are more closely packed together. Thus, larger test statistics
should show a higher degree of non-randomness.
We tested these three estimators against different scenarios of non-random
misses, and wanted to see how successful they were at detecting non-randomness.
The results of this test can be seen in section III.
III: Power Study
Consecutive
Scenario/α level
S – Negative Pair Corr
L – Negative Pair Corr
S – Positive Pair Corr
L – Positive Pair Corr
S- p(0.1)AR1(0.5)
L- p(0.1)AR1(0.5)
S- p(0.1)AR1(0.3)
L- p(0.1)AR1(0.3)
S- p(0.2)AR1(0.5)
L- p(0.2)AR1(0.5)
Random p(0.1)
0.01
0.00
0.00
0.75
0.76
0.97
1.00
0.78
1.00
1.00
1.00
0.02
0.05
0.00
0.00
0.96
0.96
1.00
1.00
0.93
1.00
1.00
1.00
0.08
Spread
0.1
0.00
0.00
0.98
0.98
1.00
1.00
0.97
1.00
1.00
1.00
0.16
0.01
0.00
0.00
0.66
0.66
0.98
1.00
0.76
1.00
0.99
1.00
0.01
Streaks
0.05
0.00
0.00
0.91
0.89
1.00
1.00
0.85
1.00
1.00
1.00
0.07
0.1
0.00
0.00
0.95
0.94
1.00
1.00
0.92
1.00
1.00
1.00
0.15
0.01
0.00
0.00
0.08
0.09
0.43
0.63
0.19
0.28
0.64
0.82
0.02
0.05
0.00
0.00
0.30
0.30
0.65
0.86
0.30
0.58
0.79
0.96
0.07
0.1
0.00
0.00
0.41
0.42
0.77
0.93
0.40
0.69
0.87
0.99
0.14
Table 1: Power outputs for each estimator on 11 different scenarios.
To test the effectiveness of each of our three estimators at recognizing nonrandomness, we tested each estimator against 11 different scenarios, all with
differing patterns of non-random misses. The first two rows show songs with a
negative pairwise correlation, which are short in length (200 notes) and long in
length (600 notes), respectively. This type of scenario could be represented within
the game by a player who is alternating back and forth between two fret buttons,
and due to the quickness of the song and the distance between the fret buttons, is
only able to get one fret button and not the other, and thus hits every other note.
We also tested positive pairwise correlations, which would be the result of having a
greater chance of missing a note following another miss. We also tested our
estimators against 6 time series-based scenarios, in which the correlation between
notes decreased as the space between the notes increased. Our final test was
against a complete random series of hits and misses, and was used to determine the
actual type I error rate of estimators.
All three of our estimators failed to recognize the non-randomness
associated with negative pair correlation. At the onset of this project, we
determined non-randomness to mean the same thing as grouping with respect to
the locations of hits and misses. A negative pair correlation is certainly nonrandom, as notes follow some correlation with the preceding note, but a negatively
paired correlation will not creating groupings the way we were anticipating. Thus,
all three of our estimators failed completely to pick up this type of non-randomness.
Our estimators fared much better against the positive pairwise correlation as
well as the time series scenarios. These variations of non-randomness fell much
closer to the type of patterns we were anticipating, and our estimators performed
much better. Many times our estimators had powers very close to 1.00 for many
scenarios. This indicated a strong chance of avoiding a type II error, and thus a high
probability of detecting a false null hypothesis.
We tested our estimators against completely random sequences of hits and
misses, in order to determine how effective our estimators were. Ideally, the power
of the estimators would match the value of significance level. In each of our three
estimators, our powers were greater than those of alpha, telling us our estimators
were more powerful than they should be, and our powers in general were too
strong. This means that if we were to adjust the tests so that our type I error rate
was what it should actually be, then our powers would decreases.
IV: Application
All Along The
Watch Tower
Estimator
Consecutive
Spread
Streaks
P-Value
0.000*
0.000*
0.023*
Hurt So Good
Consecutive
Spread
Streaks
0.039*
0.116
0.152
American Girl
Estimator
Consecutive
Spread
Streaks
P-Value
0.000*
0.000*
0.000*
Play That Funky
Music
Consecutive
Spread
Streaks
0.000*
0.000*
0.001*
A Perfect Circle
Estimator
Consecutive
Spread
Streaks
P-Value
0.000*
0.000*
0.473
Ring of Fire
Consecutive
Spread
Streaks
0.000*
0.000*
0.003*
Table 2: P-Values for each song and estimator.
* Significant at the α = 0.05 level
Using a bootstrap hypothesis test, we were able to calculate the proportion of
bootstrapped songs with a greater test statistic than our observed songs. Since
bootstrapped songs had misses at random, we are able to conclude that songs with
very low p-values have a high chance of having missed notes follow some pattern.
Using an α=0.05 level, we can see that all but three song-estimator combinations
show significance, indicated by an asterisk. For these songs, we can reject the null
hypothesis and conclude that for this particular playing of this particular song,
misses were not random. This can be concluded on 15 of our 18 tests.
Notice that we cannot make a conclusion about missed note arrangements
for all players and all songs in Guitar Hero. If we wanted to do this, we would need
to create many independent tests, and then see the proportion of how many show
significance levels indicative of non-randomness, and compare this to the α level we
chose.
V: Discussion
We were able to conclude that in most instances, for most players on most
songs, missed notes followed some pattern and were not random. We are only able
to make a conclusion on a song-to song basis, and would need further tests to make
a more general conclusion about missed notes in Guitar Hero.
Overall, our estimators did a fair job at recognizing non-randomness,
particularly for the time series scenarios. This is due to how we thought of nonrandomness at the onset of the project. We defined non-randomness as missed
noted and hit notes being somewhat grouped together. This resulted in estimators
that looked solely at grouping of notes to establish non-randomness. When faced
with the negative pair correlation scenario, our estimators were terrible at picking
up on the non-random nature of the missed notes, due to our inability to think of
non-randomness in this way.
If we were to continue our research, we would need to modify our estimators
to look for scenarios of non-randomness other than grouping. For example, we
might be able to modify our “streaks” estimator to better recognize the negative pair
correlation scenario. In that scenario, it would be likely that maximum hit streaks
and maximum miss streaks should be small, and we could look for small streaks to
be more indicative of non-randomness.
Also, we could create a new test to make a more general statement about the
nature of misses in Guitar Hero. By running independent tests, most likely with
many different players, many different songs, and one estimator, we could compare
how many observed songs have a significance level showing non-randomness to the
alpha level at which we test. If a significantly greater number of songs pass than we
expect, we could make the broader conclusion that songs played on Guitar Hero
have missed notes that are not random and follow some sort of pattern.