Becoming a Gamer: Cognitive
Effects of Real-Time Strategy
Gaming
Brian Glass
The University of Texas at Austin, 2011
Overview
• Real-time strategy (RTS) gaming
• Why study gaming?
• Blitzy results from 40-hour training with novices
• Novel modeling technique
Real-Time Strategy Games
Games
Video Games
Action
Most Existing
Research
e.g., First-person
shooter
Adventure
Simulation
Strategy
Real-Time Strategy Games
Games
Video Games
Action
Real-Time
Strategy
Adventure
Simulation
• God-like view / command
• Rapid, fluid action
• Small window into larger world (literal spotlight)
• Resource management
• War themes, many “fronts”
Strategy
Why video games?
1. Popular
2. Lead to generalized learning
3. We can control them
Why video games?
Popular
• 72% of Americans reported playing
video games in 2008
• Females are fastest growing
demographic
•
In 2010, 40% of gamers were female
•
Of all gamers, 33% were adult women > 20%
who were males under 17
• Striatal dopamine during video game was
on par with methamphetamine injection
(Koepp, et al., 1998)
• DSM-V committee found insufficient
evidence for video game addiction (as of
2007)
Entertainment Software
Association, 2010 Essential
Facts Guide
Why video games?
Lead to generalized learning
• Transfer to new tasks and new contexts
• Prior work demonstrates that long-term
training in attention-based tasks can lead to
performance enhancement
• Most training procedures tap specific skill
 specific system
Green & Bavelier (2008)
Existing Research: Action Games
Players vs. Non-Players
•
•
Self-report as avid action gamers (5+ hours per week)
Tested on various speeded-choice RT tasks
Dye, Green, &
Bavelier (2009)
Existing Research: Action Games
Training Non-Players
•
Pretest – 50 hours of play over 9 weeks – Posttest
Dye, Green, &
Bavelier (2009)
Existing Research: Action Games
Training Non-Players
• Multiple-object tracking task
• Useful field of view task
• Attentional blink (10 Hz)
• Crowding tasks (identification among tightly packed distractors)
• Mental rotation
• Piloting training (flight simulator)
Feng, Spence & Pratt (2007)
Dye, Green, & Bavelier
(2009)
Myers, et al. (2000)
StarCraft: “Gold Standard” RTS
Video
Attention and StarCraft
Does RTS gameplay have more of an attentional component?
Strains on attention:
 Multi-agent system
 Resource management
 Multiple “fronts”
Attention and StarCraft
StarCraft
• Alerting
• Sensitivity to incoming stimuli
Sensitivity to enemy
units which may appear
on the screen
• Orienting
• Selection of information from sensory input
Selecting which game
elements to attend to
• Executive
• Monitoring and resolving conflicts
Determining priority of
which enemy units to
attack
(Posner & Rothbart, 2007)
Training Novices
3. We can control them
1. Control Game (The Sims)
2. Low-Attention Starcraft
3. High-Attention Starcraft
Used in: Dye, Green, &
Bavelier, 2009
Training Novices
High-Attention
(Two Bases)
Low-Attention
(One Base)
Training Novices
Intake /
Screening
Pretest
20 Hours of
Gaming
Midtest
1. Control (The Sims)
2. Low-Attention StarCraft
3. High-Attention StarCraft
20 Hours of
Gaming
Posttest
Training Novices
Battery of cognitive / perceptual tasks – Expect Change?
Stroop – Directed attention
Task Switching
Multitask Switching
Attentional Network Test (ANT)
Visual Search
Filtering
Digit Span – Working Memory
Operating Span (Ospan)
Balloon Analogue Risk Taking Task
Multimedia Multitasking Index
Yes
Maybe
No
Pretest
only,
possible
covariate
Training Novices
Attentional Network Test (ANT)
Fan, McCandliss,
Fossella, Flombaum,
Posner (2005)
Training Novices
Attentional Network Test (ANT)
ANT, Time3-Time1
0.30
100.0
0.25
50.0
0.20
0.0
RT
Accuracy
ANT, Time3-Time1
0.15
-50.0
0.10
-100.0
0.05
-150.0
0.00
-200.0
SC-Low SC-High
SC
Sims
SC-Low SC-High
SC
All groups faster, but StarCraft-High group more accurate
Sims
Training Novices
Multitask Switching
0
0
3
Switching between
locations, remembering
states
4
Odd?
Odd?
1
7
2
6
Odd?
8
Training Novices
Multitask Switching
Switching between
locations, remembering
states
Multitask Switching, Time3-Time1
0.4
0.3
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4
RT - Switch Trials
ACC - Switch Trials
Multitask Switching, Time3-Time1
SC-Low SC-High
SC
Sims
400.0
200.0
0.0
-200.0
-400.0
-600.0
-800.0
-1000.0
-1200.0
*
SCLow
SCHigh
SC
Sims
Training Novices
Task Switching
4
3
Even / Odd?
5
1
p
u
Vowel / Cons.
j
o
E/O E/O E/O V/C
Switch
V/C E/O E/O V/C V/C V/C V/C E/O V/C
Switch
Switch
Switch Switch
Ophir, Nass, Wagner
(2009)
Training Novices
Task Switching
E/O E/O E/O V/C
V/C
Switch
Task Switching, Time3-Time1
Task Switching, Time3-Time1
0.10
0.08
0.04
RT
Accuracy
0.06
0.02
0.00
-0.02
-0.04
-0.06
SC-Low
SCHigh
Switches
SC
Sims
Non-Switches
0
-50
-100
-150
-200
-250
-300
-350
-400
-450
-500
SC-Low
SCHigh
Switches
SC
Sims
Non-Switches
Ophir, Nass, Wagner
(2009)
Training Novices
Filtering
Information Filtering, Time3-Time1
Information Filtering, Time3-Time1
0.150
40
20
Reaction Time
Accuracy
0.100
0.050
0.000
-0.050
0
-20
-40
-60
-80
-0.100
-100
-0.150
-120
SCLow
SCHigh
SC
Sims
SC-Low
SCHigh
SC
Sims
Ophir, Nass, Wagner
(2009)
Training Novices
Balloon Analog Risk Taking (BART)
• Sensation seeking
• Impulsivity
• Constraint deficiencies
• Self-reported:
• Addictive Risk Bx
• Health Risk Bx
• Safety Risk Bx
Lejuez, et al. 2002
Training Novices
Balloon Analog Risk Taking (BART)
0.12
0.10
0.08
0.06
0.04
0.02
0.00
-0.02
-0.04
Riskier
Accept Length
BART, Time2-Time1
SC-Low
SCHigh
SC
Sims
Lejuez, et al. 2002
Results Summary
Battery of cognitive / perceptual tasks
Stroop – Directed attention
Task Switching*
Sims
SC-Lo
SC-Hi

+
+
-A / -RT +A / -RT +A / -RT
Multitask Switching*
-RT


Attentional Network Test (ANT)*
-RT
-RT
+A / -RT
Visual Search – Target/Distractors
H<E
-RT
H<E
-RT
H~E
-RT
Filtering
-RT
-A / -RT +A / -RT
Digit Span – Working Memory*

+
+
Operating Span (Ospan)*
-
+

+Risk
++Risk
Risk
Balloon Analogue Risk Taking*
*NHST p<0.05
3
2
Continuing Analyses
StarCraft Measures
• Games won
 Duration
 Final difficulty levels
• Feature Modeling
 70+ features are being captured during game play
Feature Modeling with RADAR
Feature Recording for Each Unit
ID
unittype
isselected
isabuilding
unit ID
type of unit
is selected by the player
is a building
isidle
idle = 1, not idle = 0, being built = 2, newly built = 3
isloaded
disttocc
hasgas
hasminerals
isrepairinganother
loaded into a transporter
distance to nearest command center
scv carrying gas
scv carrying minerals
is repairing another unit
tickssincelastunderattack
milliseconds since last under attack
prophealth
killcount
velocity
healedbymedic
blindedbymedic
targetDist
islifted
cntfriendly
cntenemy
distfriendly
distenemy
cntfriendlybldg
cntenemybldg
distfriendlybldg
distenemybldg
proportion health remaining
other units killed
movement velocity
being healed by a medic
being blinded by a medic
distance to target
is lifted (building)
friendly units in view
enemy units in view
distance to nearest friendly unit
distance to nearest enemy unit
# friendly buildings in view
# enemy buildings in view
distance to closest friendly building
distance to closest enemy building
cntfriendlyunderattack
# friendly units under attack in view
cntenemyunderattack
# enemy units under attack in view
distfriendlyunderattack
distenemyunderattack
cntfriendlybldgunderattack
propvisible
proportion of possible tiles in view which are unobstructed and viewable
propbuildable
proportion of possible tiles in view on which buildings can be built
ispatrolling
istraining
remaintraintime
isresearching
remainresearchtime
screenMoved_int
mmDragged_int
is patrolling
is being trained
remaining time to be trained
is a building which is researching
remaining time to research
has the screen moved since last recording
has the minimap been dragged since last recording
sX
sY
x
y
critterinview
angle
xdivy
ydivx
mousedist
distscreentop
distscreenleft
distscreencorner
cntfriendlyunderattackodd
screen x position on map
screen y position on map
x position
y position
critter in view
angle of rotation
x position divided by y position
y position divided by x position
distance from the mouse cursor
distance from unit to top of screen
distance from unit to left of screen
distance from unit to screen corner
# friendly units under attack is odd
cntenemyunderattackodd
# enemy units under attack is odd
cntfriendlybldgunderattackodd
# friendly buildings under attack is odd
distance to closest friendly unit under attack
cntenemybldgunderattackodd
xodd
yodd
angleodd
xgreatery
# enemy buildings under attack is odd
x position is odd
y position is odd
angle of rotation is odd
x position is greater than y position
distance to closest enemy unit under attack
cntfriendlyunderattackmult3
count of friendly units under attack is multiple of 3
# friendly buildings under attack in view
cntenemyunderattackmult3
count of enemy units under attack is multiple of 3
cntenemybldgunderattack
# enemy buildings under attack
cntfriendlybldgunderattackmult3 # friendly buildings under attack is multiple of 3
distfriendlybldgunderattack
distance to clsoest friendly building under attack
distenemybldgunderattack
distance to closest enemy building under attack
cntgas
cntmineral
distgas
distmineral
# of geysers in view
# of mineral patches in view
distance to closest gas
distance to closest minerals
cntenemybldgunderattackmult3
xmult3
ymult3
anglemult3
random1
# enemy buildings under attack is multiple of 3
x position is a multiple of 3
y position is a multiple of 3
angle of rotation is a multiple of 3
random number
Feature Modeling with RADAR
Feature Recording for Each Unit
unittype
isselected
isabuilding
isidle
isloaded
disttocc
hasgas
hasminerals
isrepairinganother
tickssincelastunderattack
prophealth
velocity
healedbymedic
distenemy
cntfriendlybldg
cntenemybldg
distfriendlybldg
cntenemybldgunderattack
distfriendlybldgunderattack
cntgas
cntmineral
propvisible
yodd
xgreatery
cntfriendlyunderattackmult3
cntenemyunderattackmult3
ymult3
anglemult3
type of unit
is selected by the player
is a building
idle = 1, not idle = 0, being built = 2, newly built = 3
loaded into a transporter
distance to nearest command center
scv carrying gas
scv carrying minerals
is repairing another unit
milliseconds since last under attack
proportion health remaining
movement velocity
being healed by a medic
distance to nearest enemy unit
# friendly buildings in view
# enemy buildings in view
distance to closest friendly building
# enemy buildings under attack
distance to clsoest friendly building under attack
# of geysers in view
# of mineral patches in view
proportion of possible tiles in view which are unobstructed and viewable
y position is odd
x position is greater than y position
count of friendly units under attack is multiple of 3
count of enemy units under attack is multiple of 3
y position is a multiple of 3
angle of rotation is a multiple of 3
Feature Modeling with RADAR
Response Adaptive Display
Anticipates Request (RADAR)
Feature Modeling with RADAR
• StarCraft – Correctly predicts how likely a
unit is to be selected – 75% of the time.
Compare Novice vs. Expert
Cross-validate over games to measure
consistency of game play
Summary
• RTS gaming represents unique opportunity for long-term
cognitive training
• Train 36 novices over three conditions
• High-Attention RTS
• Low-Attention RTS
• Control (The Sims)
• StarCraft vs. Control (Sims)
• When accuracy is high, RT improvement
• When accuracy is moderate, accuracy + RT improvement
• However, Control faster at location switching task
•StarCraft-High vs. StarCraft-Low
• StarCraft-Low group is riskier on time 3
• StarCraft-High group > SC-Low at information filtering
• Game factors such as success, and some offerings from RADAR
to be taken into account
• 2-months follow up
Thank You!
Todd Maddox
Brad Love
Marc Tomlinson
Devon Greer
Rusty Gomez
Attention and StarCraft
•
Alerting
• Sensitivity to incoming stimuli
• Fronto-Parietal
• Norepinephrine system, locus coeruleus
•
Orienting
• Selection of information from sensory input
• Cholinergic system, basal forebrain
• Reduce neural activity and RT for invalid cues
•
Executive
• Monitoring and resolving conflicts
• Usual suspects: ACC and LatPFC
• Targets of mesocortical dopamine system
Training Novices
Operating Span (OSpan)
Ospan, Time3-Time1
O-Span Score
10
5
0
-5
-10
-15
-20
SC-Low SC-High
SC
Sims
Task Switching
Task Switching, Time2-Time1
Task Switching, Time2-Time1
0.10
0.08
0.04
RT
Accuracy
0.06
0.02
0.00
-0.02
-0.04
-0.06
SC-Low
SCHigh
Switches
SC
200
150
100
50
0
-50
-100
-150
-200
-250
-300
Sims
SC-Low
Non-Switches
Switches
SC
Sims
Non-Switches
Task Switching, Time2-Time1
Task Switching, Time2-Time1
200
0.100
150
0.080
100
RT - Switch Cost
ACC - Switch Cost
SCHigh
0.060
0.040
0.020
0.000
50
0
-50
-100
-150
-0.020
-200
-0.040
-250
-0.060
-300
SCLow
SCHigh
SC
Sims
SC-Low
SCHigh
SC
Sims
Multitask Switching
Multitask Switching, Time2-Time1
Multitask Switching, Time2-Time1
600.0
ACC - Cost
RT - Cost
400.0
200.0
0.0
-200.0
-400.0
-600.0
SCLow
SCHigh
SC
Sims
0.4
0.3
0.2
0.1
0.0
-0.1
-0.2
-0.3
-0.4
SC-Low SC-High
SC
Sims