Next Stop, the Cloud:
Understanding Modern Web Service
Deployment in EC2 and Azure
Keqiang He, Alexis Fisher, Liang Wang,
Aaron Gember, Aditya Akella, Thomas Ristenpart
University of Wisconsin-Madison
1
Move to IaaS clouds
Private
Datacenters
Co-location
Centers
2
Incomplete view of cloud use
Want a global, in-depth understanding
of IaaS cloud usage patterns
3
Fundamental questions
1. Who is using public IaaS clouds?
→ Traffic patterns; network design/traffic engineering
2. How are these services using the cloud?
Impact of failures; ways to improve availability
→ Design of new systems/services
→
3. How can quality of experience be improved?
→
Deployment design (e.g., ideal region/zone usage)
4
First in-depth empirical study
examining IaaS cloud usage patterns
and identifying ways tenants could
better leverage IaaS clouds
5
IaaS cloud environments
DNS
Server
VM
VM
Platform-as-a-Service
LB
(PaaS)
Proxy
VM
PaaS
Node
CDN
6
Datasets
• University packet capture
– Deep, but possibly atypical, perspective
IP Ranges
Campus
Traffic
1.4TB Capture
Conn
records
• Alexa subdomains DNS records
– Broad, high-level perspective
Zone transfer
Top 1
million
Subdomains
DNS
records
7
Fundamental questions
1. Who is using public IaaS clouds?
→ Traffic patterns; network design/traffic engineering
2. How are these services using the cloud?
Impact of failures; ways to improve availability
→ Design of new systems/services
→
3. How can quality of experience be improved?
→
Deployment design (e.g., ideal region/zone usage)
8
How many popular web
services use the cloud?
43% in top
250K domains
5.1%
0.7%
Both
40,333
Cloud-using domains
94.2%
Cloud availability and performance
issues may impact over 4%
2 of 68
9.
of popular web services
Non-cloud-using domains
959,667
13.
29.
35.
36.
3 of 142
4 of 181
18 of 24
14 of 89
9
Which (locally) popular
web services use the cloud?
Traffic Volume (Bytes)
18.3%
81.7%
1.
2.
3.
4.
5.
68.2%
1.7%
1.1%
0.7%
0.6%
1.
2.
3.
4.
5.
atdmt.com
3.1%
2.4%
msecnd.net 2.3%
1.5%
s-msn.com
1.4%
10
What type of content is served by
(locally) popular web services?
2.4%
Other (TCP)
0.3%
Other (UDP)
0.1%
24.2%
DNS (UDP)
HTTP (TCP)
HTTP Content Type
73.0%
Compression could be employed
1.
HTTPS (TCP)
text/html
to save WAN bandwidth
2.
text/plain
Top 100 cloud-using
and improve delivery
latency
3.
image/jpeg
domains account for
≈80%
≈100%
of HTTP flows
24.1%
23.4%
10.6%
4.
app/flash
5.
app/octet-stream 7.8%
8.7%
11
Fundamental questions
1. Who is using public IaaS clouds?
→ Traffic patterns; network design/traffic engineering
2. How are these services using the cloud?
Impact of failures; ways to improve availability
→ Design of new systems/services
→
3. How can quality of experience be improved?
→
Deployment design (e.g., ideal region/zone usage)
12
IaaS cloud environments
DNS
Server
VM
LB
Proxy
PaaS
PaaS
VM
VM
Node
CDN
13
Which front ends are used?
DNS
Server
VM
sub.abc.com
A
?
72.44.32.1
sub.abc.com
CNAME
abc.elb.aws.com
abc.elb.aws.com
A
72.44.42.1
VM
LB
Proxy
VM
sub.abc.com
CNAME
proxy.heroku.com
proxy.heroku.com
A
72.44.62.1
DNS
records
PaaS
Node
?
?
14
Which front ends are used?
PaaS front end
LB front end
VM front end
3%
8%
Elastic
Beanstalk
4%
72%
There is limited uptake
0%
50%
100%
of value-added
features
15%
3+ VMs
35%
2 VMs
50%
1 VM
97%
≈60% of ELB-using subdomains
resolve to 2+ ELB IPs
≈4% of ELB IPs are shared
by 10+ subdomains
15
Which front ends are used?
LB
PaaS
9.
LB
PaaS
13.of value-added
Failures
features
CDN
29.
will impact
the
most
popular
VM
35.
EC2-using
subdomains
36.
VM
LB
16
Which locations are used?
DNS
records
17
How many and which regions are used?
Virginia
40 548
Ireland
California
116 366
Oregon
Singapore
521 681
Tokyo
Sao Paulo
18
How many and which regions are used?
632
502
Virginia
862
558
California
Illinois
1 205
Texas
Ireland
2 071
1 035
Netherlands
Singapore
1 395
Hong Kong
19
How many and which regions are used?
8%
3%
2+ regions
2+ regions
1 region
1 region
Single region
97% failures can
take down a large fraction
1
9.
7.
of cloud-using2subdomains
13.
29.
35.
36.
1
1
2
18.
20.
31.
92%
3
5
1
5
How many and which
availability zones are used?
• Latency measurements
– Low latency => instances are in same zone
– High coverage, low accuracy (noisy)
• IP address correlation
– IPs are in the same /16
subnet => instances
are in the same zone
– Higher accuracy,
low coverage
21
How many and which
availability zones are used?
22.3%
3+ Zones
33.2%
1 Zone
Single zone failures can take
44.5% down 16% of subdomains
2 Zones
and cripple many9.others
13.
29.
35.
36.
4
5
1
3
5
22
Fundamental questions
1. Who is using public IaaS clouds?
→ Traffic patterns; network design/traffic engineering
2. How are these services using the cloud?
Many
popular
web
services
are
not
→ Impact of failures; ways to improve availability
deployed optimally for their users
→ Design of new systems/services
3. How can quality of experience be improved?
→
Deployment design (e.g., ideal region/zone usage)
23
Future measurements
•
•
•
•
Use of other clouds (e.g., Rackspace)
Use of other EC2 and Azure features
Back end deployments
Packet captures from other vantage points
24
Summary
• Systematic method for studying IaaS cloud use
• 4% of popular web services use EC2/Azure
• VMs are the most popular EC2 front end, but
value-added features are used by top domains
• Limited region and zone use (and diversity)
makes web services vulnerable to failures
25
Are web services deployed near users?
• Compare a domain’s customer country
(according to Alexa) and region usage
• 47% of subdomains are not deployed
in the same country as users
• 32% subdomains are not deployed
on the same continent as users
26
What is the user perceived network
performance for different regions/zones?
Regions should be selected
based on users’ locations
27