Caffeine Junkies: Effects of caffeine on
word-search task performance of Dartmouth
Students
• “Coffee: creative lighter fluid.”
--Floyd Maxwell (1957-)
Research Rationale
•
We find that many of our friends
consume caffeinated beverages prior
to engaging in academic tasks, exams,
or other performance evaluations with
the belief that the stimulant will
improve performance and enable them
to focus to a higher degree on the task
at hand. We have designed an
experiment in which we examined the
actual effects of caffeine on the
performance of a simple verbal/spatial
task (a word search test). The
experimental topic may have
implications on the effects of caffeine
in a college academic testing
environment.
Our Hypotheses
•
•
•
•
Null Hypothesis:
Consumption of a tall (355mL) caffeinated coffee will have no effect of
the number of words, or have a negative effect on the number of words,
found in a simple word search test in a two minute time period.
Alternate Hypothesis:
Consumption of a tall (355mL) caffeinated coffee will improve
performance, i.e. increase the number of words found in a simple word
search test in a two minute time period.
"Coffee falls into the stomach ... ideas begin to move, things remembered arrive at full
gallop ... the shafts of wit start up like sharp-shooters, similies arise, the paper is
covered with ink ..." -- Honore de Balzac
Parameter
•
•
If you choose a subject at random from the
Dartmouth College undergraduate student
body, then there is some degree to which
caffeine will improve his/her performance on
a verbal/spatial task. This mean score is the
average number of words found by
participants after drinking caffeinated coffee;
-caff. The parameter that we tested is the
difference between -caff and the average
number of words found by participants after
drinking non-caffeinated coffee, -non.
Under the null hypothesis, caffeine will not
increase the number of words found in the
simple word search task; hence the null
hypothesis is that -caff = -non, while the
alternate hypothesis is that -caff > -non.
Please note that the subjects were NOT in
fact chosen at random for our initial testing;
Participants were selected on the basis of
their presence at Collis student center on
the evening of our testing, and whether or
not they agreed to be a part of our study (i.e.
whether they had sufficient time to take our
test, and had the desire to drink coffee
specifically).
Experimental Equipment 1
– Our materials were as follows:
– 1 testing site. This was a large, common room, with a certain degree of noise
and student traffic. This was NOT a controlled environment.
– 17 identical coffee cups
– 3 pens available for students taking the word search test
– 1 blue highlighter
– 9 cups of caffeinated coffee brewed by Collis Café (all from one batch of coffee)
– 8 cups of decaffeinated coffee brewed by Collis Café (all from one batch of
coffee)
– 17 copies of a simple word-search task. NOTE: All subjects completed identical
word searches. The word search used for experimentation was chosen due to
its unusual subject matter, astronomy; we hope to avoid any advantage in word
recognition, i.e. the words to be found should be equally familiar to most college
students (not derived from any specific movie, hobby, or area of expertise that
some students are more likely to be familiar with than others).
– 1 watch, to time the “wait period” as well as the word search testing period
Coffee should be black as hell, strong as death, and as sweet as love.
--Turkish proverb
The task...
Experimental Equipment 2
•
Our experiment required the
following personnel:
–
1 Pourer. Pourer duties
include: pour coffee (caff and
non) into cups, label the bottom
of the decaffeinated cups with a
blue pen.
–
2 Distributors. Distributor
duties include: give subjects the
coffee cups. The distributors
are blind to the coffee
condition, with no knowledge
of which color marking
corresponds with which cup.
The distributor also hands out
the word search task, and keeps
time; the initial 15-minute
“wait” period, as well as the 2minute testing period.
Experimental Procedure
Experimental Procedure 1
•
Our Experimental Protocol was as follows: One experimenter approached
potential participants in Collis student center, and asked them if they would
be willing to be a part of a brief, 17-minute experiment in which they would
drink a full cup of coffee and engage in a short word search task. Once
students were recruited for participation, the pourer filled and marked as
caff or non 17 cups (total). The distributors then handed out the cups of
coffee to willing subjects, and timed a waiting period of 15 minutes.
Participants were instructed to drink the coffee but to keep their cups.
“Decaf: Used to sober up after a night on non-alcoholic beer!” - Unknown
Experimental Procedure 2
•
•
17 subjects total participated in
the experiment (9 caffeinated
experimental subjects, 8 noncaffeinated control subjects).
Research has shown that caffeine
reaches its densest concentration
in the blood 15 to 45 minutes after
consumption; this is the reason we
imposed a 15-minute wait period
after participants finish the coffee.
Participants remained in the same
seat during this period (although
they were permitted to talk to
others, read, do homework, etc.)
Experimental Procedure 3
•
•
•
After the 15-minute “wait period” delay, the distributor handed out the word
search task, as well as pens, to the participants. Subjects were instructed
to find and circle as many words as possible during a two minute time
period.
At the end of this 2-minute period (measured by the Distributors with
watches), subjects were instructed to put their pens down and the
Distributors collected the word search tests and the cups.
The distributors and pourer then recorded the number of words found on
each puzzle. Subjects’ identity was anonymous, with all data coded
according to experimental status (caff or non).
"It is inhumane, in my opinion, to force people who have a genuine medical
need for coffee to wait in line behind people who apparently view it as some
kind of recreational activity."
-- Dave Barry
Caffeine: is it helping students or
just an addiction?
“It is caffeine alone that sets my mind in motion. It is through beans of
java that thoughts acquire speed, that hands acquire shakes, that
shakes become a warning...I am...IN CONTROL...OF MY
ADDICTION!” -- From the Minicon Graffiti Wall, 1989
Statistical Analysis: Preliminary test
statistic
•
•
•
Our primary analysis goal was to
determine whether or not the
difference between the average
number of words found for each
independently sampled group is
significant.
Due to the fact that our sample size
was extremely small (n-caff = 9, n-non
= 8) we implement a t-test for the
significance of the difference between
means.
We assume that the two populations
(people who ingest caffeinated vs. noncaffeinated beverages) are normally
distributed, and that they have equal
standard deviations. (σ-caff = σ-non)
Descriptive Statistics
•
•
•
After collecting data, we found the
following average scores for our
experimental (caffeinated) and control
(non) groups:
• X-caff = 6.33
• X-non = 5.38
We also calculated variance with the
following equation:
• s2 = (x – x)²
(n – 1)
This gave us the following resulting
variances for each group:
• s2 -caff = 6.945
• s2 -non = 2.839
Statistical Analysis: Significance
level
Since we wanted to be able to report these results as statistically significant,
we chose the significance level to be 0.05. In other words, we left only a
5% chance of a type 1 error. In this setting, a type 1 error corresponds to
concluding that caffeine improves Dartmouth students’ performance on a
word search test, when in fact there is no improvement. The equation we
used to determine whether or not the difference in average number of words
found from the two groups is significant is:
t=
x-caff – x-non
_
√[(s2²/n-caff) + (s2/n-non)]
when s2 =
(n-caff – 1)s-caff2 + (n-non – 1)s-non2
n-caff + n-non – 2
t-score
• For s2 = (9 – 1)(6.945) + (8 – 1)(2.839)
9+8–2
•
Therefore
• t=
s2 = 5.029
6.33 – 5.38
√[(5.029/9) + (5.029/8)
• t = 0.877
Degrees of Freedom
•
Using the t-table, we must take into account the degrees of freedom. Due to the
fact that we have two samples, we find that
• df = n-caff + n-non – 2
• df = 9 + 8 – 2
•
df = 15
Statistical Analysis: Critical region
•
•
•
Because our Alternate Hypothesis is that -caff will be significantly greater
than µ-non, we are only concerned with the critical region under the rightsided tail of our distribution, at the .05 significance level. Therefore, to find
the critical region, we use degrees of freedom (dF= 15) and the t-table
(see Appendix A of our class text) to find the area under the standard
normal curve equal to the significance level of 0.05.
The critical region for our experiment, given the following:
• Null hypothesis: -caff = -non
• Alternate hypothesis:  -caff >  -non
• Significance level:  = 0.05
Thus the Critical region is: t > 1.753
INsignificant results!
•
Because our calculated t value of 0.877 does not fall within the above critical
region, the difference between the means of our two groups (caffeinated and
non-caffeinated) is NOT significant at the 0.05 level.
•
As we discussed in our proposal, we expected not only that caffeinated subjects
would find a significantly higher average number of words in their word search test,
but also that the power of our hypothesis would be approximately 40% (i.e.
caffeinated subjects would find at least 40% more words than would subjects given
a non-caffeinated beverage). Given our results, the likelihood that caffeine does in
fact help subjects on a word search task, to the extent that we believed it would, is
extremely low.
Confounding factors 1
•
•
Our experiment dealt with the question of
how caffeine affects performance amongst
Dartmouth College students.
Unfortunately, we were unable to conduct
an experiment in which all variables were
controlled, or which was extensive
enough, to truly draw any conclusions on
this topic.
Our primary difficulty is rooted in our
extremely small sample size. The
optimum size for a two-independentsample test is N-1 + N-2 > 60. We had
hoped to include at least 30 participants in
each group (caffeinated and noncaffeinated subjects); however, we were
unable to recruit an adequate number of
participants.
Confounding factors 2
•
•
A second confounding factor in the
implementation of our experiment was our
inability to control the participants’ caffeine
consumption prior to testing. We would have liked
to recruit subjects at least 24 hours before testing,
and to have instructed them to refrain from
ingesting caffeinated substances. However, due to
time constraints, our experiment was conducted
without knowledge of our subjects’ blood caffeine
content. This may have affected our results.
Additionally, we did not recruit a random sample
for our testing. Participants were selected on the
basis of their presence at Collis student center on
the evening of our testing, and whether or not they
agreed to be a part of our study (i.e. whether they
had sufficient time to take our test, and had the
desire to drink coffee specifically).
Confounding factors 3
•
Finally, although our test was double-blind (one
student poured the coffee, and differentiated
between caffeinated and non-caffeinated, while the
other two students distributed the coffee) in order
to avoid all experimenter bias we would have liked
to have a non-affiliated math 5 student to keep
time, pass out the tests, and collect the tests and
coffee cups. We also would have preferred making
our own coffee, so that we could have measured
caffeine concentration. Also, we did not measure
the volume of coffee given to each participants,
although we made a visual approximation to the
best of our skill.
Conclusion
•
In conclusion, although we were
disappointed that our experiment did
not yield significant results, we believe
that the topic of caffeine consumption
and effects on student health and
academic welfare should be further
explored in more controlled studies.
Some possible areas of research
include: what are student perceptions
of the effects of caffeine consumption,
and are these perceptions correct?
What are the effects of continued
caffeine use across the college years?
And finally, why does Dirt Cowboy
close at like 4pm when I need my
coffee at 11pm? :-)