A Provider-side View of
Web Search Response Time
YINGYING CHEN, RATUL MAHAJAN,
BASKAR SRIDHARAN, ZHI-LI ZHANG (UNIV. OF MINNESOTA)
MICROSOFT
Web services are the dominant way
to find and access information
Web service latency is critical to
service providers as well
 Google
Latency
+0.5 sec
revenue
-20%
 Bing
Latency
+2 sec
revenue
-4.3%
Understanding SRT behavior is challenging
SRT (ms)
300+t
t
W
T
Th
F
200+t
SRT (ms)
M
t
peak
off-peak
S
Su
Our work
 Explaining
 Identify
 Root
systemic SRT variation
SRT anomalies
cause localization
Client- and server-side instrumentation
𝑇𝑓𝑠
𝑇ℎ𝑒𝑎𝑑
𝑇𝑓𝑐
𝑇𝑠𝑐
𝑇𝑏𝑟𝑎𝑛𝑑
𝑇𝑖𝑛𝑡𝑐ℎ𝑘1
𝑇𝑟𝑒𝑠𝐻𝑇𝑀𝐿
𝑇𝑡𝑐
𝑇𝐵𝑂𝑃
𝑇𝑖𝑛𝑡𝑐ℎ𝑘2
𝑇𝑒𝑚𝑏𝑒𝑑
𝑇𝑟𝑒𝑓
𝑇𝑠𝑐𝑟𝑖𝑝𝑡
Referenced
content
Impact Factors of SRT
server
𝑇𝑓𝑠
network
browser
query
𝑇ℎ𝑒𝑎𝑑 𝑇𝑏𝑟𝑎𝑛𝑑 𝑇𝑖𝑛𝑡𝑐ℎ𝑘1 𝑇𝑟𝑒𝑠𝐻𝑇𝑀𝐿 𝑇𝐵𝑂𝑃 𝑇𝑖𝑛𝑡𝑐ℎ𝑘2 𝑇𝑒𝑚𝑏𝑒𝑑 𝑇𝑠𝑐𝑟𝑖𝑝𝑡 𝑇𝑟𝑒𝑓 𝑇𝑓𝑐 𝑇𝑠𝑐 𝑇𝑡𝑐 𝑇𝑛𝑒𝑡
Primary factors of SRT variation
 Apply
Analysis of Variance (ANOVA) on the time intervals
𝑉𝑎𝑟 𝑆𝑅𝑇 =
SRT
variance
𝑘 𝑉𝑎𝑟
𝑇𝑘 , 𝑆𝑅𝑇
+ ƞ
Variance explained Unexplained
variance
by time interval k
Explained
variance (%)
60
40
20
0
𝑇𝑛𝑒𝑡
server
𝑇𝐵𝑂𝑃
𝑇𝑟𝑒𝑓
𝑇𝑠𝑐𝑟𝑖𝑝𝑡
network
𝑇𝑟𝑒𝑠𝐻𝑇𝑀𝐿
𝑇𝑓𝑐
𝑇ℎ𝑒𝑎𝑑
browser
𝑇𝑠𝑐
𝑇𝑡𝑐
query

Primary factors: network characteristics, browser speed, query type

Server-side processing time has a relatively small impact
RTT
Variation in network characteristics
Explaining network variations
 Residential
networks send a higher fraction of
queries during off-peak hours than peak hours
 Residential
networks are slower
residential
enterprise
unknown
RTT (ms)
1.25t
25%
t
residential enterprise
Residential networks are slower
Residential networks send a higher fraction of
queries during off-peak hours than peak hours
Variation in query type

Impact of query on SRT
 Server
processing time
 Richness

of response page
Measure: number of image
Explaining query type variation
Peak hours
Off-peak hours
Browser variations

Two most popular browsers: X(35%), Y(40%)

Browser-Y sends a higher fraction of queries during off-peak hours

Browser-Y has better performance
Javascript
exec time
1.82t
82%
t
Browser-X Browser-Y
Summarizing systemic SRT variation
 Server:
Little impact
 Network:
 Query:
Poorer during off-peak hours
Richer during off-peak hours
 Browser:
Faster during off-peak hours
Detecting anomalous SRT variations
 Challenge:
interference from systemic variations
Week-over-Week (WoW) approach
𝑆𝑅𝑇 = Long term trend + Seasonality + Noise
Comparison with approaches that do not
account for systemic variations
WoW
False negative
10%
False positive
7%
One Gaussian Change point
model of SRT
detection
35%
40%
17%
19%
Conclusions
 Understanding
SRT is challenging
 Changes
in user demographics lead to systemic
variations in SRT
 Debugging
 Must
SRT is challenging
factor out systemic variations
Implications
 Performance
 Should
understand performance-equivalent classes
 Performance
 Should
management
consider the impact of network, browser, and query
 Performance

monitoring
debugging
End-to-end measures are tainted by user behavior changes
Questions?