Pilot: Customizing a Commercially Available Digital Game to Assess Cognitive Function
William C. M. Grenhart, John F. Sprufera, Jason C. Allaire, & Anne Collins McLaughlin
Department of Psychology, North Carolina State University, Gains Through Gaming Lab
BACKGROUND
METHODS
RESULTS
 The activities involved in the game play of almost any
commercial digital game requires reaction time, decision
making, reasoning, spatial ability, memory, coordination, and
planning, to name a few.
 There is a strong link between cognitive abilities and
commercial digital game
 People who play digital game have faster reaction times, can
track more objects simultaneously, and can multitask better
Intervention studies have shown that playing digital games
improves cognition.
 However, it is difficult to isolate specific tasks the require
specific cognitive abilities because you cannot typically
experimentally control the game play of commercial games
 The ability to do so would allow for:
1. Digital game tasks that can be used to assess a specific
cognitive ability.
2. Develop interventions using commercially available games
that target specific cognitive abilities.
 Using StarCraft 2, the current study developed and validated
specific in game tasks that test one of four cognitive abilities
(i.e. working memory, selective attention, processing speed,
visual discrimination).
Procedure
Testing took place in a testing room computers and paper and pencil.
Participants began by filling out a demographic questionnaire followed by
the cognitive tests. The StarCraft 2 tests were counterbalanced with the
traditional tests. Each test was preceded with a practice round during
which additional instruction was given if needed.
Measures
Test
Ability Tested
Traditional:
Processing Speed
Number Comparison (paper-and- pencil)
Digit -Symbol Substitution (paper-and-pencil) Processing Speed/Memory
Working Memory
Inquisit N-Back (N1, N2, N3)
Attention
Useful Field of View (UFOV, touch screen)
StarCraft 2:
Unit Comparison
Processing Speed
Processing Speed/Memory
Unit-Building Substitution
Working Memory
N-Back (N1, N2)
Attention
UFOV
Specific Aim 2: Correlation patterns between
StarCraft 2 cognitive tests and traditional cognitive
tests.
SPECIFIC AIMS
1) Determine if performance on the StarCraft 2 cognitive tests
correlates with performance on the traditional cognitive tests.
2) Determine if the similarities exist between the StarCraft 2 and
traditional formats in the pattern of relationship between the
different tests
3) Determine whether video game usage had an effect on
performance on the StarCraft 2 tests.
RESULTS
 Specific Aim 1: Compared performance between
StarCraft 2 cognitive tests and traditional cognitive
tests.
Table 1. Correlations Between
StarCraft 2 Number Comparison
and Traditional.
StarCraft 2
Traditional
Total Speed
.24
Response Time
-.25
Accuracy
.12
Note: ^p < .10. *p < .05. **p < .01. ***p <
.001
PARTICIPANTS
The sample consisted of 31 undergraduate psychology students
Mean age of 19 years (SD = 1.37; range = 18 - 23 years)
35.5% Female
Participants were given course credit for participating in the
study
DEVELOPMENT OF STARCRAFT 2
TASKS
 The StarCraft 2 tasks were
developed using the StarCraft 2 Map
Editor.
 Using editor nearly every aspect of the game experience can be
altered.
 The editor contains every art assets, animations, and game
functions used in the official game and many that are not.
 The programing in editor resembles a very high level and
simplified object oriented programing language.
 Design for the StarCraft2 tests closely mimicked the traditional
tests, differences include:
Number Comparison: Units act in the place of numbers.
Participants choose either “same” or “different” rather than
only choosing “different” when appropriate in order to prompt
the next trial.
Digit-Symbol Substitution: In place of numbers and symbols,
buildings and units were used. Rather than drawing symbols,
participants selected the corresponding unit from an array of
the available options that appeared beneath the trial building.
N-Back: Air ships acted as the target stimuli. They appeared,
flew across the screen and then disappeared after a standard
duration.
UFOV: After choosing the direction of the peripheral target,
there was a short animation to reveal the correct location.
Table 2. Correlations Between
StarCraft2 Digit-Symbol Substitution
and Traditional.
StarCraft 2
Traditional
.43*
Total Speed
-.42*
Response Time
Total Correct
Accuracy
.43*
.11
Note: ^p < .10. *p < .05. **p < .01. ***p <
.001
Table 5. Correlations of Cognitive Tests Within Format
StarCraft 2 Tests
Num.
DSS N-Back hits N-Back F/A N-Back Ratio
Comp.
DSS
.35^
N-Back hits
.34^ .38*
N-Back F/A
.51** .04
.24
N-Back Ratio
-.27
.20
UFOV 2 + 3
.24
.15
.18
Traditional Tests
DSS
N-Back hits
N-Back F/A
Num.
Comp.
.50**
-.16
-.23
-.07
-.29
.38*
N-Back Ratio
.14
.26
.06
-.90***
UFOV 2 + 3
-.01
.16
-.03
-.02
DSS
.40*
-.79***
.31
-.18
N-Back hits N-Back F/A N-Back Ratio
-.03
Note: ^p < .10. *p < .05. **p < .01. ***p < .001. Highlight =similar significance between
formats.
Specific Aim 3: Impact of video game usage on
performance on StarCraft 2 cognitive tests.
Table 6 .Performance Differences Between Gamers and Non-Gamers.
Variable
Gamer
Non-Gamer
t-test
M (SD)
M (SD)
t
Num. Comp.
15.14 (5.27)
17.61 (6.53)
-.91
DSS
26.57 (8.75)
18.75 (14.35)
1.36
N-Back
-5.29 (1.80)
-6.35 (2.47)
1.05
UFOV
0.25 (.11)
0.20 (.06)
1.46
Note: ^p < .10. *p < .05. **p < .01. ***p < .001.
Table 3. Correlations Between StarCraft 2 N-Back and Traditional.
SC2 N-Back
Hits
False Alarms
Ratio
Hits
False Alarms
Ratio
.20
.34^
-.20
-.11
.33^
-.38*
.23
-.20
.34^
Note: ^p < .10. *p < .05. **p < .01. ***p < .001
Table 4. Correlations between StarCraft 2 UFOV and Traditional.
SC2 UFOV
Task 1
Task 2
Task 3
Task 1 Threshold
-.05
.01
-.03
Task 1 Score
.07
.04
-.04
Task 2 Threshold
.21
-.07
-.06
Task 2 Score
-.34
-.16
.06
Task 3 Threshold
-.12
.11
-.02
Task 3 Score
.10
.14
.07
Note: ^p < .10. *p < .05. **p < .01. ***p < .001.
Table 7. Correlations Between Performances and Hours per Week of
Video Games Played.
Variable
Hours
M(SD)
5.94 (7.52)
Num. Comp.
-.29
DSS
.07
N-Back
.20
UFOV
.13
Note: ^p < .10. *p < .05. **p < .01. ***p < .001.
CONCLUSIONS & LIMITATIONS
 Performance was similar for between the StarCraft 2 and
traditional digit-symbol substitution and N-back tasks.
 Intra-ability patterns were similar between the StarCraft 2 and
traditional formats.
 Video game usage did not impact performance on the StarCraft
2 tests.
 Having only 30 participants limited the power of this study to
find more significant findings
 The minimum display time StarCraft 2 can show a stimuli is
1/16 sec. (62.5 ms) and most people can correctly identify a
single target (UFOV task 1) with it displayed for only 17 ms (1
frame). This created a floor effect for the StarCraft 2 UFOV.
 Some of the participants had difficulty understanding or
properly seeing the stimuli on some of the StarCraft 2 test.
This was more a failing of design than an hazard of the game
itself.
 Overall, testing cognition with customized, commercially
available digital game was validated.
ACKNOWLEDGEMENTS
- Gains Through Gaming Lab at North Carolina University
- GainsThroughGaming.org