Content Distribution Networks
Outline
Implementation Techniques
Hashing Schemes
Redirection Strategies
Spring 2003
CS 461
1
Design Space
• Caching
– explicit
– transparent (hijacking connections)
• Replication
– server farms
– geographically dispersed (CDN)
Spring 2003
CS 461
2
Story for CDNs
• Traditional: Performance
– move content closer to the clients
– avoid server bottlenecks
• New: DDoS Protection
– dissipate attack over massive resources
– multiplicatively raise level of resources needed to attack
Spring 2003
CS 461
3
Denial of Service Attacks (DoS)
client
client
server
client
attacker
Spring 2003
CS 461
4
Distributed DoS (DDoS)
zombie
attacker
client
client
server
slave attacker
zombie
client
zombie
Spring 2003
zombie
CS 461
5
Redirection Overlay
Geographically distributed server clusters
R
R
R
R
Internet Backbone
R
R
R
R
clients
Spring 2003
Distributed request-redirectors
CS 461
6
Techniques
• DNS
– one name maps onto many addresses
– works for both servers and reverse proxies
• HTTP
– requires an extra round trip
• Router
– one address, select a server (reverse proxy)
– content-based routing (near client)
• URL Rewriting
– embedded links
Spring 2003
CS 461
7
Redirection: Which Replica?
• Balance Load
• Cache Locality
• Network Delay
Spring 2003
CS 461
8
Hashing Schemes: Modulo
svr0
URL (key)
%
• Easy to compute
• Evenly distributed
• Good for fixed number of
servers
• Many mapping changes after a
single server change
svrN
Spring 2003
CS 461
9
Consistent Hashing (CHash)
url-0
• Hash server, then URL
svr0
• Closest match
svr1 • Only local mapping changes
after adding or removing
servers
• Used by State-of-the-art CDNs
svrN
url-1
svr2
Unit circle
Spring 2003
CS 461
10
Highest Random Weight (HRW)
high
URL
weight0
RH
svr0
svr1
svr2
svr
svr
svrN
Spring 2003
sort
weight1
weight2
• Hash(url, svrAddr)
• Deterministic order of access set of
servers
• Different order for different URLs
• Load evenly distributed after server
changes
weight0
weightN
low
CS 461
11
Redirection Strategies
• Random (Rand)
– Requests randomly sent to cooperating servers
– Baseline case, no pathological behavior
• Replicated Consistent Hashing (R-CHash)
– Each URL hashed to a fixed # of server replicas
– For each request, randomly select one replica
• Replicated Highest Random Weight (R-HRW)
– Similar to R-CHash, but use HRW hashing
– Less likely two URLs have same set of replicas
Spring 2003
CS 461
12
Redirection Strategies (cont)
• Coarse Dynamic Replication (CDR)
–
–
–
–
Using HRW hashing to generate ordered server list
Walk through server list to find a lightly loaded one
# of replicas for each URL dynamically adjusted
Coarse grained server load information
• Fine Dynamic Replication (FDR)
– Bookkeeping min # of replicas of URL (popularity)
– Let more popular URL use more replicas
– Keep less popular URL from extra replication
Spring 2003
CS 461
13
Simulation
• Identifying bottlenecks
– Server overload, network congestion…
• End-to-end network simulator prototype
–
–
–
–
–
Models network, application, and OS
Built on NS + LARD simulators
100s of servers, 1000s of clients
>60,000 req/s using full-TCP transport
Measure capacity, latency, and scalability
Spring 2003
CS 461
14
Network Topology
C
S
S
R
C
C
WA
C
C
R
R
R
C
R
C
R
R
PA CA
R
MA
IL
C
R
R
MI
S
S
NE
CO
R
C
R
S
DC
R
S
R
SD CA
S
S
GA
TX
R
C
R
C
S
R
C
C
S – Server, C – Client, R - Router
Spring 2003
CS 461
15
Simulation Setup
• Workload
– Static documents from Web Server trace, available at
each cooperative server
– Attackers from random places, repeat requesting a
subset of random files
• Simulation process
– Gradually increase offered request load
– End when servers very heavily overloaded
Spring 2003
CS 461
16
Capacity: 64 server case
Normal Operation
Throughput req/s
35000
30000
25000
20000
15000
10000
5000
0
Rand
CDR
R-CHash
FDR
R-HRW
FDR-Ideal
A single server can handle ~600 req/s in simulation
Spring 2003
CS 461
17
Capacity: 64 server case
Under Attack (250 zombies, 10 files, avg 6KB)
Throughput req/s
50000
40000
30000
20000
10000
0
Rand
CDR
R-CHash
FDR
R-HRW
FDR-Ideal
A single server can handle ~600 req/s in simulation
Spring 2003
CS 461
18
Latency: 64 Servers Under Attack
Random’s Max: 11.2k req/s
R-HRW
CDR
R-CHash
Rand
R-CHash Max: 19.8k req/s
R-HRW
FDR
FDR
100
CDF of Response
CDF of Response
100
R-CHash
CDR
80
60
40
20
80
60
40
20
0
0
0.1
1
10
100
Response Time in Logscale (Seconds)
Spring 2003
CS 461
0.1
1
10
100
Response Time in Logscale (Seconds)
19
Latency At CDR’s Max: 35.1k req/s
FDR
CDR
CDF of Response
100
80
60
40
20
0
0.1
Spring 2003
1
10
100
Response Time in Logscale (Seconds)
CS 461
20
Capacity Scalability
Under Attack (250 zombies, 10 files)
45000
40000
35000
30000
25000
20000
15000
10000
5000
0
Throughput req/s
Throughput req/s
Normal Operation
0
16
32
48
64 80
50000
45000
40000
35000
30000
25000
20000
15000
10000
5000
0
96 112 128
16
32
48
64
80
96 112 128
Num of Servers
Num of Servers
CDR
R-HRW
Rand
Spring 2003
0
CDR
CS 461
R-HRW
Rand
21
Various Attacks (32 servers)
1 victim file, 1 KB
10 victim files, avg 6KB
30000
30000
Throughput req/s
Throughput req/s
35000
25000
20000
15000
10000
5000
0
25000
20000
15000
10000
5000
0
100
200
300
400
500
600
700
800
Num of Zombies (slave attackers)
Rand
Spring 2003
R-HRW
100
200
300
400
500
600
700
Num of Zombies (slave attackers)
CDR
Rand
CS 461
R-HRW
CDR
22
800
Deployment Issues
• Servers join DDoS protection overlay
– Same story as Akamai
– Get protection and performance
• Clients use DDoS protection service
– Same story as proxy caching
– Incrementally deployable
– Get faster response and help others
Spring 2003
CS 461
23