The Effects of Latency
on User Performance in
Warcraft III
Nathan Sheldon, Eric Gerard, Seth
Borg, Mark Claypool, Emmanuel Agu
Computer Science Department
Worcester Polytechnic Institute
Worcester, MA, USA
http://www.cs.wpi.edu/~claypool/papers/war3/
Why Study Network Games?
• In 2000, U.S. economy grew by 7% while
computer game industry grew by 15%
“Economic Impacts of the Demand for
Interactive Entertainment Software”, Interactive
Digital Software Association, 2001.
• 60% of Americans age 6+ play computer
•
games
221+ million computer games sold in 2002
– 2 games for every household in America
“Top Ten Industry Facts”, Interactive
Digital Software Association, May 2003.
2
NetGames'03, Redwood City, CA, USA
May 2003
Why Study Warcraft III?
• Top selling computer game genres
–
–
–
–
Strategy (27.4%)
“Top Ten Industry Facts”, Interactive
Digital Software Association, May 2003.
Children’s (15.9%)
Shooter (11.5%)
Family Entertainment (9.6%)
• Warcraft III set sales record
– Fastest to sell 1 million copies
“Warcraft III - Shatters Sales Records Worldwide...”,
Blizzard Press Release, October 2002
3
NetGames'03, Redwood City, CA, USA
May 2003
Network Games and Latency
•
Latency degrades performance of interactive
applications
– Web-browsing – seconds
– Audioconference – 100’s of milliseconds
– First Person Shooters (FPS) – 100’s of milliseconds
•
• Real-Time Strategy (RTS)?
Knowing effects of latency useful for
– Building better network games
– Building better networks to support games (QoS)
 Effects of Latency on Warcraft III (RTS)
4
NetGames'03, Redwood City, CA, USA
May 2003
Outline
• Introduction

• Experiments

• Analysis
• Conclusions
5
NetGames'03, Redwood City, CA, USA
May 2003
Warcraft III Overview
RTS User Interaction
Components:
• Exploration
• Building
• Combat
6
NetGames'03, Redwood City, CA, USA
May 2003
Exploration Map
Performance?
• Time
(to reach end)
7
NetGames'03, Redwood City, CA, USA
May 2003
Building Map
Performance?
• Time
(to build technology tree)
8
NetGames'03, Redwood City, CA, USA
May 2003
Combat Map
Performance?
• Games Won
• Unit Scores
9
NetGames'03, Redwood City, CA, USA
May 2003
Controlling Latency
• Warcraft III uses client-server
– Set computer B as server (also a client)
– Set computer C or D as client
• NIST Net on computer A
– Induce latency [0 ms to 3500 ms]
10
NetGames'03, Redwood City, CA, USA
May 2003
Outline
• Introduction

• Experiments

• Analysis
– Application Level
– Network Level
– User Level

• Conclusions
11
NetGames'03, Redwood City, CA, USA
May 2003
Building and Latency
Build Time vs. Latency
9:36
Linear (Time To Build)
R2 = 0.0516
9:21
Build Time (m:s)
Time To Build
9:07
8:52
8:38
8:24
8:09
7:55
7:40
0
500
1000
1500
2000
2500
3000
3500
Latency (ms)
12
NetGames'03, Redwood City, CA, USA
May 2003
4000
Exploration and Latency
Time To Complete
Linear (Time To Complete)
Explore Time vs Latency
5:31
Explore Time (m:s)
2
R = 0.6334
5:16
5:02
4:48
4:33
4:19
4:04
3:50
0
200
400
600
800
1000
Latency (ms)
13
NetGames'03, Redwood City, CA, USA
May 2003
1200
Combat and Latency (1)
Unit Score Difference
Unit Score Difference vs. Latency
3000
2000
1000
0
-1000
-2000
-3000
R2 = 0.0138
0
200
400
600
800
1000
1200
1400
Latency (ms)
14
NetGames'03, Redwood City, CA, USA
May 2003
1600
Combat and Latency (2)
15
NetGames'03, Redwood City, CA, USA
May 2003
Outline
• Introduction

• Experiments

• Analysis
– Application Level
– Network Level
– User Level


• Conclusions
16
NetGames'03, Redwood City, CA, USA
May 2003
Bandwidth
3.8 Kbps
4.0 Kbps
6.8 Kbps
17
NetGames'03, Redwood City, CA, USA
May 2003
Inter-Packet Times
18
NetGames'03, Redwood City, CA, USA
May 2003
Payload Distributions
19
NetGames'03, Redwood City, CA, USA
May 2003
Bandwidth and Latency
• Battle.net games had ~100 ms latency
• Trace combat maps with induced latency
20
NetGames'03, Redwood City, CA, USA
May 2003
Payload Distributions and Latency
21
NetGames'03, Redwood City, CA, USA
May 2003
Commands and Latency
• Pilot studies suggest 6 bytes of overhead
•
•
22
per command
Remove 6 bytes from each packet payload
Add up remaining command payloads
NetGames'03, Redwood City, CA, USA
May 2003
Outline
• Introduction

• Experiments

• Analysis
– Application Level
– Network Level
– User Level
• Conclusions
23



NetGames'03, Redwood City, CA, USA
May 2003
User-Level Analysis
• 0-500 ms latency, users could easily adjust
• 800+ ms, game appeared erratic
– Degradation in gaming experience
• 500-800 ms degradation depended upon
– User
• More skilled were more sensitive
– Strategy
• Micro managers were more sensitive
• Combat managers were more sensitive
24
NetGames'03, Redwood City, CA, USA
May 2003
•
•
Conclusions
Typical Internet latencies do not significantly
affect user performance in Warcraft III
– Some effect on exploration
– No statistical effect on building or combat
•
RTS game play emphasizes “strategy” (which takes
10s of seconds or minutes), not “real-time”
RTS games less sensitive to latency than are FPS
•
At the network level:
25
– RTS in QoS class similar to that of Web browsing
– Small packets with low bandwidth
– Command aggregation at higher latencies
NetGames'03, Redwood City, CA, USA
May 2003
Future Work
• Effects of latency on user strategies
• Other RTS games
– Age of Empires
– Command and Conquer
• Effects of latency on other genres
– First Person Shooter
– Role Playing Game
• Effects of loss
26
NetGames'03, Redwood City, CA, USA
May 2003
The Effects of Latency
on User Performance in
Warcraft III
Nathan Sheldon, Eric Gerard, Seth
Borg, Mark Claypool, Emmanuel Agu
Computer Science Department
Worcester Polytechnic Institute
Worcester, MA, USA
http://www.cs.wpi.edu/~claypool/papers/war3/