BLE
SCALA
ON
Ankit Singla, P. Brighten Godfrey
Kevin Fall, Sylvia Ratnasamy, Gianluca Iannaccone
ROUTING STATE/NODE
Scalable Routing
SHORTEST-PATH ROUTING
NOT VERY SCALABLE!
?
NUMBER OF NODES = n
Scalable Routing
STRUCTURED
TOPOLOGIES
6, 20
But, what about
general topologies?
4, 11
3
Scalable Routing
HIERARCHY
• Inefficient
• Location-dependent
addressing
t
s
4
Why Not DNS/DHTs?
A
RIGHT NEXT TO YOU,
SILLY!
WHERE IS A?
Arbitrarily long paths!
5
Local Name Resolution
• Why local name-resolution matters
– Google: 400ms delay = 0.59% searches/user
– Time for name-resolution is a significant fraction
of latency for short flows (which are the majority)
– Fate sharing property
6
Flat Names
• Arbitrary names – independent of location
• Useful primitive
– Mobility
– Security
– Simplified management
7
Another Way to Scale
• Compact Routing
– [Thorup & Zwick ’01, Abraham et al. ’04]
– Idea: accept (slightly) longer paths, get lower state
(stretch =
– State
route length
)
shortest path length
=> Stretch
8
ROUTING STATE/NODE
Compact Routing
Θ 𝑛 state; stretch = 1
Θ
.
.
.
NUMBER OF NODES = n
Θ
𝑛 state; stretch = 3
3
𝑛 state; stretch = 5
Θ 𝑘𝑛2/(𝑘+1) state; stretch = k
9
Algorithm != Protocol
NOT DISTRIBUTED
NOT DYNAMIC
10
Disco
Disco: Distributed Compact Routing
• Scalable: 𝑂 𝑛 state* per node
• Efficient: 𝑂(1) stretch
• Flat-names: Route over arbitrary names
*Assumption:
Θ
𝑛 ;3
..
partial source routes are of size O(polylog(n))
11
Disco
Local Name Resolution
Name Resolution
NDDisco: Name-Dependent
Distributed Compact Routing
12
NDDisco
𝑂( 𝑛) random landmarks (enable approximately shortest paths)
L(t)
b
Addr (t) = (L(t), b, t)
s
t
13
NDDisco
For nearby nodes, the approximation can be poor!
L(t')
t'
s
14
NDDisco
𝑂( 𝑛) node vicinity
𝑂( 𝑛) random landmarks
L(t)
t'
s
t
15
NDDisco
Destination can help!
L(t)
s
t
16
Disco
Local Name Resolution
Name Resolution
NDDisco: Name-Dependent
Distributed Compact Routing
17
Name Resolution
• Name -> Address
mappings stored in DHT
over landmarks
• Similar to prior work
[BVR, S4]
t
s
Arbitrarily high stretch for address lookup!
18
Disco
Local Name Resolution
Name Resolution
NDDisco: Name-Dependent
Distributed Compact Routing
19
Local Name Resolution
• Scatter t’s address to
𝑂( 𝑛) random nodes
• Then each node s has
t’s address somewhere
in its vicinity w.h.p.
• ‘Scattering’ addresses?
• Finding addresses?
L(t)
?
s
t
Smells like hash functions!
20
Sloppy Groups
EVERYONE AGREES
ON THIS PART
t
G(t)
• G(t) = nodes w for which first k bits of h(w) match h(t)
• Set k such that |G(t)| = Θ( 𝑛)
• Groups are sloppy because nodes’ estimates of ‘n’ vary!
21
Address Dissemination
• Requirements:
– Low diameter for quick dissemination of addresses
– Low messaging overhead
– Fault tolerance
22
Address Dissemination
• Predecessor, successor and small O(1) number of fingers
• Distance-vector style dissemination of addresses
23
1: Overlay Topology
• Predecessor, successor and small O(1) number of fingers
– Fault tolerance: Use landmarks to fix overlay when broken
– Miserly messaging: each node hears each address O(1) times
– We don’t care about routing, only diameter, which is small!
24
2: Dissemination Protocol
• Disseminate new addresses learnt to overlay neighbors
• Record where each address was learnt from (distance vector-like)
• Always forward away from the direction received from
– Avoids count-to-infinity problem because of ordered space
25
Handling Failures: Example
WITHDRAW ADDRESSES
LEARNT FROM X
X
GET NEW NEIGHBOR
FROM LANDMARK
No soft-state required!
Update only on changes
26
Routing in Disco
L(t)
w(t)
s
t
• s computes h(t)
• w(t) = Longest-prefix
match for h(t) in Vicinity(s)
First : s
Later: s
w(t)
L(t)
L(t)
t
t
27
Protocol Messages
• Estimating ‘n’: synopses
exchanged with neighbors
• Landmark and Vicinity
updates: path vector style
• Sloppy-group bootstrap
through landmarks
• Address-dissemination:
distance-vector style
s
28
Summary
• State at a node
– Vicinity: 𝑂( 𝑛) closest nodes
– Landmarks: 𝑂( 𝑛) random landmarks
– Address-Groups: 𝑂( 𝑛) overlay neighbors
• Stretch
– For first packet <= 7
– For later packets <= 3
29
Evaluation
• Protocols
–
–
–
–
Disco
NDDisco
S4 [Mao et al ‘07]
VRR [Caesar et al ‘06]
• refer to paper
• Topologies
– Router-level Internet
topology (n = 192,244)
– AS-level Internet topology
(n = 30,610)
– Geometric random
graphs, mean degree 8
– G(n, m), mean degree 8
30
State
SHORTEST PATH
PROTOCOL
GEOMETRIC RANDOM GRAPHS OF INCREASING SIZE
31
State
S4
DISCO
LONG TAIL
NDDISCO
ROUTER-LEVEL INTERNET TOPOLOGY
32
First-Packet Stretch
DISCO
S4
LIMITED BY SMALL
PATH-LENGTHS IN
TOPOLOGY
ROUTER-LEVEL INTERNET
TOPOLOGY
DISCO
LARGE FIRST-PACKET
STRETCH
GEOMETRIC RANDOM GRAPH
16384-NODES
33
Later-Packets Stretch
DISCO
NEAR-IDENTICAL
ROUTER-LEVEL INTERNET
TOPOLOGY
S4
DISCO < 1.2
GEOMETRIC RANDOM GRAPH
16384-NODES
34
Are the Landmarks Clobbered?
DISCO
~4 IN EVERY 10,000
EDGES SEE MORE
CONGESTION
AS-LEVEL INTERNET TOPOLOGY
35
Conclusion
• Disco routes over flat names with O(1) stretch
and 𝑂 𝑛 state
• What next?
– Policy support?
– Applications
36
Thank You
Questions?