Structuring Unstructured Peer-to-Peer Networks
Stefan Schmid
Roger Wattenhofer
Distributed
Computing
Group
HiPC 2007
Goa, India
Networks…
Internet Graph
DISTRIBUTED
Web Graph COMPUTINGNeuron Networks
Different properties:
• Natural vs. Man-made
• Robustness
• Diameter
• Routability
• ...
Public Transportation Networks
Social Graphs
Stefan Schmid @ HiPC 2007
2
An Interesting Network: Peer-to-Peer Network
•
•
Network of peers, e.g., to share files
•
Desirable properties:
- Scalability
- Low degree, low network diameter
- Fast routing
- etc.
•
Important: p2p accounts for
much Internet traffic today!
(source: cachelogic.com)
Popular Examples:
- File sharing: BitTorrent, eMule,
Kazaa, ...
- Streaming: Zattoo, Joost, ...
- Internet telefony: Skype, ...
- etc.
Stefan Schmid @ HiPC 2007
3
Some Own Applications
•
Wuala online storage system
- Student project, start-up, http://wua.la
•
Pulsar streaming
- tilllate.com, DJ events, ...; pstreams.com
- cheap infrastructure at content provider
•
BitThief BitTorrent downloads
•
Distributed Computations
- BOINC client for ECC discrete
logarithm challence
Stefan Schmid @ HiPC 2007
4
Structured vs. Unstructured Topologies
•
Old „p2p“ systems such as Napster were based on server
- Server stores index: search for contents is simple
- Problem: single point of failure
- Legacy issues...
•
Unstructured systems, e.g., Gnutella, allow arbitrary topologies
and arbitrary data placement
- Peers just connect to an arbitrary set of other peers
- No single point of failure
- But often inefficient: routing based on flooding or random walk
•
Structured systems, e.g., eMule‘s Kad network, give guarantees
- Proactive maintenance of topology
- Provable network diameter and peer degree
- Routing possible, look up, e.g., in log(n) hops
(maybe also low stretch)
Stefan Schmid @ HiPC 2007
5
What is „better“?
•
Really?
Unstructured systems have less maintenance overhead
- Peers can join and leave wherever they want
Really?
Flooding
always
possible!
•
Unstructured systems allow for a richer set of queries
- e.g., range queries, Boolean queries
•
Most importantly: despite the interesting properties (and large body of
research) of structured networks, today‘s predominant networks are still
unstructured (e.g., Gnutella, BitTorrent, etc.)
•
But unstructured systems often have scalability problems
- When Napster was unplugged, Gnutella went down.
Discussion needs to be continued...!
Stefan Schmid @ HiPC 2007
6
Routing in Arbitrary Topologies?
•
How to find a file in an arbitrary network?
•
Option 1: Flooding (up to a certain hop radius r)
- Robust, but does not scale.
- Does not find the „needles“, but does a good job finding popular files.
•
Option 2: Random Walks
- Less messages, but no lookup performance guarantee.
- Potentially large delay (solution: many parallel „walkers“)
- Walkers can be lost...
- Analysis difficult.
- Again: Good to find popular contents, bad to find needles.
Stefan Schmid @ HiPC 2007
7
Flooding
•
This talk considers search operations by flooding.
•
Efficiency of flooding?
Flooding efficiency depends on network topology!
Very efficient on trees!
Many redundant transimissions...
Stefan Schmid @ HiPC 2007
8
Clustella
•
We propose Clustella
- a new P2P client for unstructured peer-to-peer systems
- based on flooding, but with „smart neighbor selection“
- allows for more efficient flooding!
Stefan Schmid @ HiPC 2007
9
Vision
•
Clustella Vision:
unstructured p2p network
Normal client
Clustella client
By connecting to peers in far-away parts of the network, small cycles in
the topology are avoided, and flooding is more efficient. Not only
Clustella clients do benefit, but also all other clients in the network.
Stefan Schmid @ HiPC 2007
10
Flood Coverage
•
Main open question: How to connect to remote peers?
•
Given a set of potential neighbors, it would be useful to know the hop
distance to each of those!
•
Then, we could connect to the one furthest away...
•
Goal: Maximize flood coverage, i.e., maximize minimum number of
nodes reached by a r-hop flooding – locally and despite dynamics
Stefan Schmid @ HiPC 2007
11
Hop-Estimation With Clustering
•
Main idea: Use clustering!
- Divide network into different clusters.
- Peers in different clusters belong to different network regions and can
safely be connected without creating small cycles.
•
How to achieve such a clustering? Introduction of beacons!
- Two parameters: radius Rd and radius Rb (Rd < Rb)
- If a peer has no beacon in Rd neighborhood, it becomes a beacon itself.
- A peer knows all beacons in its Rb neighborhood.
- Rb roughly equals the flooding radius R
Stefan Schmid @ HiPC 2007
12
Clustella Mechanism (1)
•
•
•
One beacon in radius Rd
Beacon known in radius Rb
Flooding radius R
•
Beacons append their ID to all packets
(piggy-back)
•
If packet expires before, other peers
(here: π‘‘) forward beacon information
•
Entire Rb neighborhood will know
beacon π‘
•
Peers try to connect to peers which
have no beacons in common!
Stefan Schmid @ HiPC 2007
13
Clustella Mechanism (2)
•
Edges are undirected
•
All peers have degree d or d+1
•
If connection is accepted if own
degree is d or smaller; otherwise, a
neighbor may have an open slot, or a
connection is broken down
•
Invariant quickly reestablished!
•
Neighbors of existing neighbor are
also good candidates, as they are
located in the same network region.
Stefan Schmid @ HiPC 2007
14
Two Challenges
•
Evaluation of current neighbors
- Existing neighbors are always in the same network region
- Evaluating their quality and comparing them to alternative neighbors is
difficult
- Include routes in packets! Exclude beacons known from a neighbor only
•
Dynamics
- Clustella must be robust to churn, i.e., frequent joins and leaves
- E.g., node crash: Clustella peer p stores some neighbors for each of its
neighbors q; these neighbors are good candidates as they are in the same
network region as q
Stefan Schmid @ HiPC 2007
15
Evaluation
•
Simulation of three different neighbor selection strategies
- Gnutella-like (unfair?): Peers join at some well-known entry point and ask
for their neighbors‘ neighbors until they reach full degree
- Random walk (more interesting?): Peers find new peers by a random walk
of length L
- Clustella: Peers find new neighbors by exploring the network using a walk
of length L and by taking beacon information into account
•
Results
- Gnutella-like topologies result in very inefficient flooding operations
- Clustella yields higher flood coverage than random walk
Stefan Schmid @ HiPC 2007
16
Future Work
•
Hierarchical clustering (beacons with different radii)
- Already a small hierarchy can yield better flood coverage
- However, maintenance of hierarchy can be expensive under churn!
- Moreover, fairness must be guaranteed: High-level beacon peers should not
have more work to do!
•
Smaller messages
- Reducing the message sizes for large radii is important!
- Idea: Use of Bloom filters instead of sending beacon IDs directly
Stefan Schmid @ HiPC 2007
17
Conclusion
•
We believe that structuring topologies can be benefitial to
peer-to-peer systems!
•
Clustering with beacons is simple and probably also useful
in other applications, e.g., in music graph
•
Implementation must ensure fairness and use small
message sizes.
•
A good choice of parameters important for both efficiency
and stability.
•
Incorporation into Gnutella??
Stefan Schmid @ HiPC 2007
18
Thank you.
Thank you for your interest.
Stefan Schmid @ HiPC 2007
19