Scalable Network
Proximity Estimation
Puneet Sharma (puneet.sharma@hp.com)
HP Labs, Palo Alto
Joint Work with Sujata Banerjee, Sujoy Basu, Rodrigo Fonseca,
Sung-Ju Lee, and Zhichen Xu
Self-Managing Networks Summit, June 1-2, 2005
© 2005 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice
Network Proximity Estimation
•
Proximity estimation is key to finding “best”
resources
− closest game server, closest media service etc.
− overlay neighbor selection, building distribution trees etc.
•
Challenges:
− O(N2) probing overhead
• Millions of globally distributed resources => 1012 probe
messages
− Dynamic environment => Faster re-computation
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
2
Related Work
•
Infrastructure support based
− IDMaps, Dynamic Distance Map
− King --- recursive DNS lookup
− M-Coop – tracers link to each other that mimics BGP
•
Landmark based:
− Landmark ordering
• Cannot distinguish nodes with same (or similar) landmark order
− GNP
• Pre-computation of the landmark nodes
• Sensitive to the measurement errors
− Lighthouse
• Quality depends on the choice of lighthouses
− Vivaldi
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
3
Netvigator: Network Proximity Estimation
•
Efficient computation of the nodes in proximity to a
given node
− Find the “k closest nodes to a given node”
•
Proximity estimation not Distance estimation
•
Landmark based scheme: O(NL) v/s O(N2)
measurement overhead
•
Uses widely deployed tools: ping, traceroute
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
4
Methodology: Landmarks-based Scheme
d1
d1
d2
d2
d3…
dn
C1, C2, C3
d1d1
C1, C2, C3
d2
d2
…d3
dn
d1
d2d2
… d3
C1, C2,d1
C3
dn
Using Landmark Vectors
Clustering
instead
Embed nodes
in a of
Global
Embedding
Cartesian
Space
using Landmark Vectors
d1
d1
d2
d2
…
d3
dn
Leverage Route Information
C1, C2, C3traceroute instead of ping
Host
7/27/2016
Landmark
C1, C2, C3
d1d1
d2d2
…d3
dn
Landmark
Vector
Router (Milestone)
Copyright © 2005 HP corporate presentation. All rights reserved.
5
NetVigator: Clustering Algorithms
•
min_sum
minc  C: l  L(n,c) (dist(n,l) + dist(c, l))
•
max_diff
c
maxc  C: l  L(n,c) ABS(dist(n,l) - dist(c, l))
•
inner_product
l l  L(n,c) ((1/(dist(n,l)2) X (1/dist(c, l)2))
max∑c  C:
n
min_sum: Minc  C: l  L(n,c) (dist(n,l) + dist(c, l))
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
6
Netvigator Evaluation
•
Tested using large scale simulations, and
implementation on the HP Intranet and
PlanetLab.
•
Evaluation Metrics
−Accuracy: Finding the best candidate in top “n”
−Precision: How many correct top “n” ?
−Penalty: How bad is the best found candidate?
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
7
Visualization using ZoomGraph:
Top 5 closest nodes to planetlab1.cse.nd.edu
planetlab1.cse.nd.edu
NetVigator Result
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
8
Visualization using ZoomGraph:
Partial Topology
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
9
Evaluation with PlanetLab data
NetVigator performance
•High accuracy: Over 90% accuracy
•Low overhead: 15% measurement overhead
•Robust to bad measurements
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
10
Distributed Querying of Internet
Position Information
Position Information Partitioning
query specific search expansion
A
Query: closest node to me?
Query: closest node to me?
R(A)
I.N.
Distributed
Infrastructure
nodes
Postition updates
I.N.
I.N.
NetVigator:
Landmark Vector
Respository
I.N.
I.N.
Hosts
7/27/2016
I.N.
Position
Updates
Copyright © 2005 HP corporate presentation. All rights reserved.
•Overloaded Central Server
•Single Point of Failure
•Query Latency
11
Distributed NetVigator
•
Closest Heuristic Partitioning
− Simple idea: assign each node to its closest
infrastructure node
− infrastructure nodes know each other’s ‘positions’
− redirect position update/query to the closest
infrastructure node
− query-specific search expansion
•
Provides
− load-balancing
− efficient querying: low latency, low search expansion
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
12
Distributed Querying: Evaluation
•
Schemes Compared:
− Closest mapping
• Map each node to its closest infrastructure node
− DHT based mapping
• Map ‘position information’ to DHT geometry (we only looked at
1d initially)
− Geometric  Hilbert Mapping
− Distance Vector  Hilbert Mapping
− Order Vector  Recursive Partitioning
•
Evaluated on three datasets
− PlanetLab: all-pairs-ping data
− King Dataset: 1740 DNS servers
− Transit-Stub synthetic topology
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
13
Results – King Dataset
Closest Heuristic Partitioning
• High hit ratio at Root
•4-6 times lower latency to Root
•Lower search expansion
•Load-balanced
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
15
Results: Hops in search
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
18
Results: Load Balance
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
19
Concluding Remarks
•
NetVigator: computation of network proximity that:
−
−
−
−
•
is highly scalable as well as accurate
is robust to bad measurements and choice of landmarks
allows incremental computation
has been tested using large scale simulations, and implementation
on the HP Intranet and PlanetLab
Next Steps:
− Deploy as a PlanetLab service
− Application to DARPA CHART (Control for High-throughput Adaptive
Resilient Transport) project
− Explore n-D DHT mappings
− Scalable inference of other network/path properties such as
bandwidth, loss etc.
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
20
More information…
•
Netvigator: Scalable Network Proximity Estimation, Zhichen Xu,
Puneet Sharma, Sung-Ju Lee and Sujata Banerjee, HP Labs
Technical Report, HPL-2004-28
•
Distributed Querying of Internet Distance Information, Rodrigo
Fonseca, Puneet Sharma, Sujata Banerjee, Sung-Ju Lee and Sujoy
Basu, presented at IEEE Global Internet 2005 Symposium
http://www.hpl.hp.com/research/mmsl/projects/net/
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
21
7/27/2016
Copyright © 2005 HP corporate presentation. All rights reserved.
22