www.kom.tu-darmstadt.de
www.httc.de
The Future of the
Peer-to-Peer Paradigm
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
Prof. Dr.-Ing. Ralf Steinmetz
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
TU Darmstadt - Darmstadt University of Technology,
Dept. of Electrical Engineering and Information Technology, Dept. of Computer Science
KOM - Multimedia Communications Lab, Tel.+49 6151 166151,
Merckstr. 25, D-64283 Darmstadt, Germany, Ralf.Steinmetz@KOM.tu-darmstadt.de Fax. +49 6151 166152
9. Conclusion
httc - Hessian Telemedia Technology Competence-Center e.V
Merckstr. 25, D-64283 Darmstadt, Ralf.Steinmetz@httc.de
P2P-Oslo-030516.fm 1 16.May.03
L2
Data Link
Layer
(Sicherung)
L1
Physical Layer
(Bitübertragung)
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
WAN:
ISDN
&
ATM
Web
LAN, MAN
High-Speed
LAN
Transport
Network
Medium-Term
Routing&Traffic
Engineering
Short-Term
Congestion
Control
Basics
Other Lectures of “ETiT” & Computer Science
9. Conclusion
Introduction
Legend:
P2P-Oslo-030516.fm 2 16.May.03
Telnet
Files
Internet:
IP
RT(C)P
KN I
KN II
KN III
Time Scale
L3
Network Layer
(Vermittlung)
Long-Term
Network
Planning &
Engineering
Trafficmgmnt
4. Some Applications and
Systems: 2nd Generation
Internet:
TCP, UDP
Media
Data Flow
Security
3. Some Applications and
Systems: 1st Generation
L4
Transport
Layer
(Transport)
MM COM - QoS specific
2. Notion of Peer-to-Peer
Mobile Communications
1. Motivation
IP-Tel:
Signal.
H.323
SIP
Moblie IP
Overview
Application
Layer
(Anwendung)
Email
L5
P2P
Applications
Inst. Msg.
Complementary Courses: Distributed Multimedia Systems (MM I and MM II),
Telecooperation II,III,.., Embedded Systems, Mobility in Comm. Networks
Transitions & Addressing
www.kom.tu-darmstadt.de
www.httc.de
Overview
www.kom.tu-darmstadt.de
www.httc.de
Overview
(2)
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and Systems: 1st Generation
4. Some Applications and Systems: 2nd Generation
Overview
1. Motivation
2. Notion of Peer-to-Peer
5. Peer-to-Peer & Related Mechanisms
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
6. Peer-to-Peer Intrinsic Issues
5. Peer-to-Peer & Related
Mechanisms
7. Search: Distributed Hash Tables
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P Networking
8. Major Issues in P2P
Networking
9. Conclusion
9. Conclusion
Note: some pictures (section client-server and DHT) from Klaus
Wehrle, thanks!
P2P-Oslo-030516.fm 3 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
1. Motivation
freenet
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
One of the newest buzzwords in networking is Peer-to-Peer (P2P)
Is P2P a hype?
• initially 40 million Napster users in 2 years
• strong presence at international networking conferences
• major traffic source, e.g. at universities more than ~40% P2P traffic
• initially mp3 audio: (free?) music
• later also video (and any data) exchange
• enabled by relatively high bandwidth access of end systems
P2P-Oslo-030516.fm 4 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Evolution of Internet Computing Paradigms
1st generation (since the beginning of the Internet):
• permanent IP addresses, always connected
• static domain name system (DNS) mapping
• limited specialized applications, protocols: Telnet, FTP, Gopher, ....
⇒ WORLD WIDE ACCESS
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
2nd generation (since 90ties):
• WWW & graphical browsers
• dynamic IP addresses / NAT* (network address translation) / roaming users
• heterogeneous applications, asymmetric server based services
• protocol: HTTP, ..
⇒ WORLD WIDE WEB
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
3rd generation (since 2000):
• more collaboration and personalized applications
• powerful edge devices (peers), instant networking
• protocols/applications:
• Napster, Gnutella (Limewire/Bearshare/Morpheus since 2.0)
• Edonkey/MLDonkey, Fasttrack (Kazaa), Freenet, ..
⇒ WORLD WIDE PEERING
P2P-Oslo-030516.fm 5 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
2. Notion of Peer-to-Peer
Just many interconnected systems or applications?
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
another view reveals
• P2P means
• self-organizing,
• decentralized,
• scalable,
• fault-tolerant,
• ad-hoc,
• unpredictable performance, ...
P2P-Oslo-030516.fm 6 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
2.1 A first Definition
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
Definition of P2P Networking (C. Shirkey):
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 7 16.May.03
"Peer-to-peer (P2P) is a class of applications that takes
advantage of RESOURCES - storage, cycles, human presence available AT THE EDGES of the Internet. Because accessing these
decentralized resources means operating in an environment of
UNSTABLE CONNECTIVITY AND UNPREDICTABLE IP ADDRESSES, peer-to-peer
nodes must operate outside the DNS and have significant or total
autonomy from central servers”
www.kom.tu-darmstadt.de
www.httc.de
Some Preliminary Answers
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
1. File sharing, but..
2. Internet of the future, but ..
3. realization that client server can not solve in an optimal way everything
• server is a single point of failure
• traffic: asymmetric and concentrated at server
• scalability
• unused resources at the clients
Client Server
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
but ..
P2P-Oslo-030516.fm 8 16.May.03
...
Peer-to-Peer
www.kom.tu-darmstadt.de
www.httc.de
2.2 Application Domains
Overview
1. Motivation
2. Notion of Peer-to-Peer
(classical) File Sharing: music, video and other data
Napster, Gnutella, FastTrack (KaZaA, Morpheus, ...), etc.
Distributed Storage
• Anonymous File Sharing
Freenet, PAST, OceanStore, etc.
Collaboration
• P2P groupware, P2P content generation, Online Games
• P2P instant messaging
ICQ (PC-based Chat-System - I Seek You), AIM (AOL Instant
Messenger)
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
Distributed Computing
• P2P CPU cycle sharing
• GRID Computing, ..., distributed simulation
SETI@home: Search for Extraterrestrial Intelligence, Popular
Power: former battle to the influenza virus
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
Distributed Search Engines, Intelligent Agents
Pastry, CAN, Chord
9. Conclusion
Security and Reliability
Resilient Overlay Network (RON), Secure Overlay Services (SOS)
Charging for Content and Services
Mojo Nation: battling free rider
P2P-Oslo-030516.fm 9 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
2.3 Characteristics
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Resources (location, sharing)
1. relevant resources located at NODES AT THE EDGES (“peers”)
2. Peers share (one type of) their resources
3. RESOURCE LOCATIONS
• widely distributed
• most often largely replicated
P2P-Oslo-030516.fm 10 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Characteristics
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Networking
4. VARIABLE CONNECTIVITY is the norm
• support of dial-up users with variable IP addresses
• operating outside the domain name system (DNS)
P2P-Oslo-030516.fm 11 16.May.03
(2)
www.kom.tu-darmstadt.de
www.httc.de
Characteristics
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Interaction of Peers
5. combined CLIENT AND SERVER functionality
• “SERVer + cliENT = SERVENT”
• minimal demands for the infrastructure
• services provided by end systems
6. DIRECT INTERACTION (DATA TRANSFER) between peers
• e.g. (more-or-less) no central control
P2P-Oslo-030516.fm 12 16.May.03
(3)
www.kom.tu-darmstadt.de
www.httc.de
Characteristics
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Management
7. peers with significant AUTONOMY and mostly having SIMILAR RIGHTS
• e.g. storage / processing done by autonomous end-systems
8. NO CENTRAL control or centralized usage of a service
9. SELFORGANIZEING system (ad-hoc like)
P2P-Oslo-030516.fm 13 16.May.03
(4)
www.kom.tu-darmstadt.de
www.httc.de
3. Some Applications and Systems: 1st Generation
Approaches for
• Distributed Collaboration/Communication
• P2P groupware, P2P content generation
• P2P instant messaging (like Jabber, ICQ), Online games
• Distributed Storage
• P2P file sharing
Overview
3.1 File Sharing with Central Server (like Napster)
3.3 Decentral File Sharing (like Gnutella)
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
• (online) backups
3.4 Anonymous File Sharing (like Freenet)
• Distributed Computing
• P2P CPU cycle sharing, distributed simulation
3.2 Processing Sharing with Central Server
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 14 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
3.1 File Sharing with Central Server (like Napster)
see e.g. http://www.napster.com/
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
History
• first famous P2P file sharing tool
• for downloading mp3 files only
• first: 1999-2001
• free content access, violation of copyright of artists
• later: since 2001
• by introduction of chargeing: users abandonned service
• April 2001: OpenNap appr. 45.000 users, 80 (interconnected) directory
servers, more than 50 TByte of data
Characteristics
• users as providers and consumers
• napster user namespace instead of
• fixed IP addresses
• domain name system (DNS)
P2P-Oslo-030516.fm 15 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
File Sharing with Central Server (like Napster)
(2)
Download
Search
Overview
1. Motivation
Central
Napster
Server
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
• central P2P network
• decentral storage (content at the edges)
• central server (file index, address of peers, search engine)
• file transfer between clients
• decentralization
• as tool, not as goal
P2P-Oslo-030516.fm 16 16.May.03
Napster
Client
www.kom.tu-darmstadt.de
www.httc.de
File Sharing with Central Server (like Napster)
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Phases:
1. Directory update
• Peers provide data to central server
• stored files - mp3 tags, related metadata, own IP-address
2. Search
• requesting peer addresses central server
• central server delivers related IP addresses
3. Test of quality
• requesting peer
• sends ping (gets pong) from all potential peers
• select the best
4. Download - data transfer
• requesting peers
• sets-up connection with best peer
• retrieves data
P2P-Oslo-030516.fm 17 16.May.03
(3)
www.kom.tu-darmstadt.de
www.httc.de
3.2 Processing Sharing with Central Server
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
see e.g. http://setiathome.ssl.berkeley.edu/
SETI = Search for ExtraTerrestrial Intelligence
• first famous P2P network for sharing processing power
• for the massively parallel analysis of radio signals
• started 1998, until october 2000 4x1020
• floating point operations performed (largest computation ever performed)
Principle
• about 50 GB of data coming in from the Arecibo telescope per day
• distributed via central server to millions of processing-clients
• client = screensaver (using idle CPU cycles)
• uses unused processing power of desktop computers
Characteristics
• architecture similar to napster
• similar idea: GRID computing
P2P-Oslo-030516.fm 18 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
3.3 Decentral File Sharing (like Gnutella)
see e.g.
http://dss.clip2.com/GnutellaProtocol04.pdf
http://gnutella.wego.com/
http://www.gnutella.co.uk/
Overview
1. Motivation
2. Notion of Peer-to-Peer
GNU (GNUs not Unix) + Nutella
SERVENT A
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
decentral file
sharing tool
main
differences to
Napster
• no central
server
• all kinds of
files are
shared
P2P-Oslo-030516.fm 19 16.May.03
Download
Gnutella
Messages
Servent B
Servent
www.kom.tu-darmstadt.de
www.httc.de
Gnutella: Architecture
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Characteristics
• each node keeps a database of known and connected nodes
• bootstrapping
• any peer may work
• host cache provide list of entry point to the network
• initially part of the servent
• otherwise published at diverse webpages
• each message identified
• by a pseudo globally unique ID (GUID)
• 128 bit random eg. consisting of
• time code + network MAC address + random number
• answers routed back to origin based on GUID of requesting message
• PONG, QUERY HIT, PUSH
• message broadcast for node discovering and search requests
• flooding (to all connected nodes) is used to distribute information
• nodes recognize message they already have forwarded
• by their GUID and
• do not forward them twice
• hop limit by TTL
• usually TTL = 7
• same for all peers
P2P-Oslo-030516.fm 20 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Phases of the Protocol (Version 0.4)
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
1. connecting
• PING message: Actively discover hosts on the network
• PONG message: Answer to the PING messages, includes information about
one connected Gnutella servent
2. search
• QUERY message: Searching the distributed network
• QUERY HIT message: Response to a QUERY message
• can contain several matching files of one servent
3. data transfer
• HTTP is used to transfer files (HTTP GET)
• PUSH message: To circumvent firewalls
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 21 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Phase 1: Gnutella - Connecting
In order to connect to a Gnutella network the servent must initially know (at
least) one member node of the network and connect to it
• these first member nodes must be found by other means (Web, ...)
• nowadays host caches are usually used
Overview
1. Motivation
servent
1
ping
servent
2
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
ping
servent
1
pong
servent
2
ping
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
pong
9. Conclusion
servent
1
P2P-Oslo-030516.fm 22 16.May.03
servent
2
www.kom.tu-darmstadt.de
www.httc.de
Phase 1: Gnutella - Connecting - The Gnutella Net
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
servent
1
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Gnutella connection (TCP on specified Port)
servent connects to a number of nodes
• it got PONG messages from
⇒ thus it becomes part of the Gnutella net
P2P-Oslo-030516.fm 23 16.May.03
(2)
www.kom.tu-darmstadt.de
www.httc.de
Phase 2: Gnutella - Searching
servent
1
query
servent
2
Overview
query
1. Motivation
2. Notion of Peer-to-Peer
servent
1
3. Some Applications and
Systems: 1st Generation
servent
2
query
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Flooding: QUERY message is distributed to all connected nodes
1. a node that receives a QUERY message
• increases the HOP count field of the message and
• forwards it to all nodes
• except the one he received it from
• if
• HOP <= TTL (Time To Live)
• A QUERY message with this GUID was not received before
P2P-Oslo-030516.fm 24 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Phase 2: Gnutella - Searching
servent
1
query hit
servent
2
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 25 16.May.03
2. the node also checks whether it can answer to this QUERY
• with a QUERYHIT message
• if e.g. available files match the search criteria
3. QUERYHIT message
• contains
• the IP address of the sender
• information about one or more files that match the search criteria.
• information passed back the same way the QUERY took.
(2)
www.kom.tu-darmstadt.de
www.httc.de
Phase 3: Gnutella - Data Transfer
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
peer sets-up connection
• data transfer is not part of the Gnutella protocol
• HTTP GET is used
• special case:servent with the file is behind a firewall
• the downloading servent cannot initiate a connecting
• it can instead send the PUSH message,
• asking the other servent to instead initiate a HTTP connection to him and
• to push the file via it
• but, does not work if both servents are behind firewalls
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 26 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Gnutella Problem: Scalability Issues
Overview
Gnutella in its original version V 0.4 suffers from a range of scalability issues
• due to
• decentral approach and
• flooding of messages.
Frequency Distribution of Gnutella Messages (Portmann et. al.):
1. Motivation
QUERY
54.8%
PONG
26.9%
PING
14.8%
QUERY HIT
2.8%
PUSH
0.7%
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 27 16.May.03
⇒ 41.7% of the messages just for network discovery
www.kom.tu-darmstadt.de
www.httc.de
Gnutella Problem: Scalability Issues
A TTL (Time To Live) of 4 hops for the PING messages
• leads to a known topology of rougly 8000 nodes
• TTL in the original Gnutella client was 7
Average bandwidth usage (kbps) per node for peer discovery :
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
8. Major Issues in P2P
Networking
9. Conclusion
PING rate
500 Nodes
4000 Nodes
8000 Nodes
1/min
4.8
68.2
194.9
1/sec
288
4090.4
11694.5
Average bandwidth usage (kbps) per node for search messages:
4. Some Applications and
Systems: 2nd Generation
7. Search: Distributed Hash
Tables
(2)
Search rate
500 Nodes
4000 Nodes
8000 Nodes
1/min
2.5
36.8
127.0
1/sec
151.0
2211.0
7617.3
⇒ Low bandwidth peers easily use up all their available bandwidth
• for passing on PING and QUERY messages of other users,
• leaving no bandwidth for up- and downloads
⇒ reason for the breakdown of the Gnutella network in August 2000.
Hierarchy introduced: Superpeers (like in FastTrack..)
• to cope with load of low badwidth peers
P2P-Oslo-030516.fm 28 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Gnutella Problem: Free Riders
Free Rider
• exist in big anonymous communities
• selfish individuals that opt out of a voluntary contribution
• to the community social welfare
• i.e. by not sharing any files but downloading from others
• ... and get away with it
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Study results (Adar/Hubermann 2000):
• 70% of the Gnutella users share no files
• 90% answer no queries
Solutions
• incentives for sharing
• servents only accept connections / forward messages from servents that
share content
• but: how to verify? quality of the content?
• micropayment
• see Mojo Nation
P2P-Oslo-030516.fm 29 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Gnutella Problem: Overlay Topology Design
Overview
search for german movie
german movies
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
japanese
movies
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 30 16.May.03
british movies
Gnutella connection
Gnutella Node
IP Route
IP Node
www.kom.tu-darmstadt.de
www.httc.de
Gnutella Problem: Overlay Topology Design
(2)
search for german movie
german movies
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
japanese
movies
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
british movies
9. Conclusion
This overlay network has a better topology
• Gnutella network
• only 2-5% of the connections are within the same autonomous system
P2P-Oslo-030516.fm 31 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Gnutella: How Scalability Issues are tackled
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
Modern Gnutella Versions (Limewire, Bearshare) use a number of
mechanisms to overcome the performance problems addressed before:
• Optimization of connection parameters
• number of hops etc
• PING/PONG opimization
• e.g. C. Rohrs und V. Falco. Limewire: Ping Pong Scheme, April 2001.
• http://www.limewire.com/index.jsp/pingpong
• Dropping of low-bandwidth connections
• moves modem/ISDN users to the edge of the Gnutella Network
• Ultra-Peers
• similar to KaZaA super nodes
• File Hashes to identify files
• similar to eDonkey
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 32 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Gnutella: Some Related Work
Overview
Scalability
• M. Portmann et. al.: The Cost of Peer Discovery and Searching in the
Gnutella Peer-to-peer File Sharing Protocol
• J. Guterman: Gnutella to the Rescue? Not so Fast, Napster fiends.
• K. Sripanidkulchai, Carnegie Mellon University: The popularity of Gnutella
queries and its implications on scalability
• Clip2.com: Bandwidth Barriers to Gnutella Network Scalability
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
Free Riders
• E. Adar, B. Hubermann: Free Riding on Gnutella
• P. Golle et.al.: Incentives for Sharing in Peer-to-Peer Networks
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
Overlay Topology
• G. Pandurangan et.al.: Building Low-Diameter P2P Networks
• G. Pandurangan et.al.: Building P2P networks with good topological
properties
9. Conclusion
P2P-Oslo-030516.fm 33 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
3.4 Anonymous File Sharing (like Freenet)
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
e.g. Freenet
see http://freenet.sourceforge.net/
• from Ian Clarkt atUniversity of Edinburgh
Principle:
• Documents are
• encrypted
• cut into several pieces and
• stored on several machines
Characteristics
• provides anonymity for users
• author of document cannot be identified
• prohibits censorship of documents
• as document pieces are copied at several machines
• removes any single point of failure or control
• decentralized storage
• provide plausible security wrt. node operators
• as they cannot read the content on their disks
P2P-Oslo-030516.fm 34 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
4. Some Applications and Systems: 2nd Generation
Approaches for
• Distributed Search Engines (and some Intelligent Agents)
Performance Improvements
• decentralized by networks with supernodes / distributed servers
4.1 Decentral File Sharing with Distributed Servers (like
eDonkey 2000)
Overview
1. Motivation
4.2 Decentral File Sharing with Super Nodes (like KaZaA)
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
Charging for Content and Services (battling free riders)
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 35 16.May.03
4.3 File Sharing with Charging (like Mojo Nation)
www.kom.tu-darmstadt.de
www.httc.de
4.1 Decentral File Sharing with Distributed Servers
(like eDonkey 2000)
For example: eDonkey 2000
see e.g.
http://www.edonkey2000.com/
http://www.emule-project.net/
http://savannah.gnu.org/projects/mldonkey/
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Principle
• Distributed server is set up and run by power-users
• thus it is nearly impossible to shut down all servers
• servers interchanged their server lists
• using UDP as transport protocol
• Client application connects to one random server and stays connected
• using TCP
• this server manages file indices
• searches are directed to the server
• clients can also extend their search by sending UDP search messages to
additional servers
P2P-Oslo-030516.fm 36 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Decentral File Sharing with Distributed Servers (like
eDonkey 2000)
• additionally:
• New servers
(2)
send their port +
IP to other
servers (UDP)
Download
Search
• Servers send
Server Lists
server lists (other
TCP
servers they
know) to the
Extended Search
clients
UDP
• Server lists can
also be
Server
downloaded on
various websites
Client
• Files are identified
by unique MD4 file
hashes and not by filenames, this helps in
• resuming a download from a different source
• downloading the same file from multiple sources at the same time
• verification that the file has been correctly downloaded
P2P-Oslo-030516.fm 37 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
4.2 Decentral File Sharing with Super Nodes (like KaZaA)
Overview
1. Motivation
2. Notion of Peer-to-Peer
see
www.fasttrack.nu, www.kazaa.com
www.grokster.com, gift.sourceforge.net
System
• Developer: Fasttrack
• Clients: KaZaA (Grokster, GiFT/OpenFT)
Properties:
• currently most successful P2P network
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P system
# users
# files
terabytes
# downloads
(from
download.com)
Fasttrack
2,6 Mio.
472 Mio.
3550
4 Mio
eDonkey
230.000
13 Mio.
650-2600
600.000
Gnutella
120.000
28 Mio.
105
ca. 525.000
• Numbers are from 10’2002
Architecture: neither completely central nor decentral
Also
• today Gnutella clients use supernodes to reduce communication overhead
P2P-Oslo-030516.fm 38 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Decentral File Sharing with Super Nodes (like KaZaA) (2)
Download
Search
Overview
1. Motivation
SUPERNODE
Client
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
normal
Client
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
peers
• connected only to some super nodes
• send IP address and file names only to super peers
super nodes - super peers:
peers with high-performance network connections
• take the role of the central server and proxy for simple peers
• answer search messages for all peers, i.e. reduction of load of peers
• one or more supernodes can drop out without problems
additionally the communication between nodes is encrypted
P2P-Oslo-030516.fm 39 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
4.3 File Sharing with Charging (like Mojo Nation)
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
e.g. Mojo Nation
see http://www.mojonation.net/
• extends Freenet
defines electronic money (mojos)
• mojos are earned by offering resources to the Mojo Nation network
• storage space
• processing power
• bandwidth
• mojos can be spent on
• searching
• downloading
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 40 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
5. Peer-to-Peer & Related Mechanisms
Overview
A deaper understanding of the relationship
• peer-to-peer and
• client server
• GRID
may lead to new common solutions
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 41 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
5.1 Peer-to-Peer and Client Server - Revised
Client Server
...
Peer-to-Peer Mechanisms
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Hybrid Peer-to-Peer Mechanisms
• decentralized
• shared usage of resources
• interaction (data transfer) between peers
• Use of server(s) / super peers for coordination
P2P-Oslo-030516.fm 42 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
5.2 GRID Computing
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
http://www.gridforum.org
Similar idea, similar concept as in P2P
High performance data processing centers
• needed for scientific applications
• but
• expensive to provide
• often do not offer enough performance
Solution: GRID
• to interconnect the existing data processing centers
and operate it as distributed processing center (the GRID)
5. Peer-to-Peer & Related
Mechanisms
P2P
GRID
6. Peer-to-Peer Intrinsic
Issues
History
Sharing MP3 files &
illegal content
Saving costs for data
processing centers
7. Search: Distributed Hash
Tables
Users
Mainly private users
Scientists
Typical Transfer
Volume
Small (MP3) to medium
(video)
Huge (often terabytes)
Typical Service
File Sharing
Processing Sharing
Hugh number of users
cause scalability issues
Transfering huge
amounts of data
8. Major Issues in P2P
Networking
9. Conclusion
Typical Problems
P2P-Oslo-030516.fm 43 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
6. Peer-to-Peer Intrinsic Issues
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Selected topics
• 6.1 Overlay Phenomenon
• 6.2 Small World Networks
P2P-Oslo-030516.fm 44 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
6.1 Overlay Phenomenon
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P networks form OVERLAY NETWORK
• on top of the Internet IP network
IP network form an OVERLAY NETWORK
• (politically and technically) over the
underlying telecom infrastructure
• introduced their own addressing
scheme
• e.g. user names
overlay network node
• emphasized fault-tolerance
basic network node
• based on the end-to-end principle
• as much intelligence as possible to the end nodes
⇒ P2P can learn a lot from the according Internet solutions
• routing algorithms etc
⇒ P2P and the Internet are based on very much the same principles
• but on different layers
Other (non P2P) overlay network
• e.g. VPNs (virtual path networks)
P2P-Oslo-030516.fm 45 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
6.2 Small World Networks
Overview
1. Motivation
Clustered sparseness
• Small World Networks comprise few edges
• Network set-up by many interconnected clusters
Small Diameter
• minimal distance between the most apart peers is small
• path length grows logarithmically with size of network
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
Power Law characteristics
• independent form network size
• depends on network specific constant value > 0
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Building - growing process
• networks continuously grow
• new node preferably attached to well interconnected nodes
• e.g. Gnutella
• nodes with many attached peers respond to many PING requests
• this nodes are preferred wrt setting up new connections
• host caches also prefer to comprise higly interconnected nodes
P2P-Oslo-030516.fm 46 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Small World Networks
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
(2)
Fault tolerance
• robust again random faults (and attacks)
• uncertainty with faults at highly connected nodes
Examples
• energy supply at the US
#N(L) ~ L-4
• cross-references in science:#N(L) ~ L-3
• WWW:
#N(Lout) ~ L-2.45,
• Max. 19 clicks between any 2 pages
• Peer-to-Peer Overlays: Gnutella #N(L) ~ L-2
• telephone calls (AT&T): called numbers
P2P use
• optimization of search mechanism
• random walk, high degree search, ...
9. Conclusion
P2P-Oslo-030516.fm 47 16.May.03
#N(Lin) ~ L-2.1
www.kom.tu-darmstadt.de
www.httc.de
7. Search: Distributed Hash Tables
?
Location of
„Matrix.avi“?
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
Issue in any P2P system
1. Location of data
• where shall the data be stored?
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
publish ("reference to contents", ..)
2. Locating of data
• how shall the query mechanism work?
LookUp ("refernce to contents", ...
9. Conclusion
Constraints
• low communications and storage compelxity
• robustness
P2P-Oslo-030516.fm 48 16.May.03
I have
„Matrix.avi“?
www.kom.tu-darmstadt.de
www.httc.de
Alternative 1: Central Server - Client Server
X has it!
S
d
c
Lookup(“contents”)
Publish(“Contents”,X)
Overview
X
1. Motivation
Get(“Contents”)
2. Notion of Peer-to-Peer
e
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Principle
• simple
• server comprises location data
Comments
• all issues like client server
• scalability, single point of failure, cost of management, ...
• excellent for simple/small environments
P2P-Oslo-030516.fm 49 16.May.03
Search Query
Download data
www.kom.tu-darmstadt.de
www.httc.de
Alternative 2: Flooding
Get(“Contents”)
Lookup
(“Contents”)
here
here
Overview
1. Motivation
Search Query
Download Data
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Principle
• simple
• location known only at actual data storage
Comments
• all issues like broadcasting & flooding
• network load, scalability
• excellent if high robustness needed
P2P-Oslo-030516.fm 50 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Comparison: Trade-off
Suchaufwand
O(N)
Overview
1. Motivation
2. Notion of Peer-to-Peer
Fluten
Flaschenhals:
• Kommunikation
?
O(log N)
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
Zentraler
Server
O(1)
5. Peer-to-Peer & Related
Mechanisms
O(1)
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
O(log N)
O(N)
Speicheraufwand
Approaches
• central server: but
• scalability, storage needs O(N),
• flooding: but
• amount of messages, network load
P2P-Oslo-030516.fm 51 16.May.03
Flaschenhals:
• Speicher, CPU, Netz
• Verfügbarkeit
Wehrle
www.kom.tu-darmstadt.de
www.httc.de
Comparison: Trade-off
y Skalierbar: O(log N)
Suchaufwand
O(N)
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
f Ausfälle
Flaschenhals:
• Kommunikation
f Attacken
f Kurzzeitige Nutzer
Verteilte
HashTabellen
O(log N)
Zentraler
Server
O(1)
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
O(1)
7. Search: Distributed Hash
Tables
9. Conclusion
y Resistent gegen Änderungen
Fluten
4. Some Applications and
Systems: 2nd Generation
8. Major Issues in P2P
Networking
(2)
O(log N)
Flaschenhals:
• Speicher, CPU, Netz
• Verfügbarkeit
O(N)
Speicheraufwand
Distributed Hash Table Principle
• using HASH function: uniform distribution to many locations of
• actual contents
• information where content is located (better)
• Issues
• uniform distribution & continuous updating (leave and join of nodes)
P2P-Oslo-030516.fm 52 16.May.03
Wehrle
www.kom.tu-darmstadt.de
www.httc.de
1. Step: Mapping of Contents to linear Range of Values
Overview
1. Motivation
2. Notion of Peer-to-Peer
All contents (i.e. name of file) mapped to range of integer values
• N: max. amount of files
• N << 0, ..., 2m-1
Hash function
• H (string of characters) mod 2m
• e.g.
• H (movie/matrix/divx/en) = 2313
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 53 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
2. Step: Arrangement of Range of Values
Example
hash values (reflecting data description)
01000
Overview
10012000
20014000
40017000
2. Notion of Peer-to-Peer
4. Some Applications and
Systems: 2nd Generation
10.00121.000
21.00140.000
nodes to store data or location information
1. Motivation
3. Some Applications and
Systems: 1st Generation
700110.000
Values most often arranged as circle
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 54 16.May.03
2m-1 0
40.00165.535
www.kom.tu-darmstadt.de
www.httc.de
2. Step: Arrangement of Range of Values
Distributed
Hash-Table
Arrangement
Overview
Real Topology
1. Motivation
2. Notion of Peer-to-Peer
40.00165.536
3. Some Applications and
Systems: 1st Generation
21.00140.000
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
(2)
Nodes reponsability
• part of the range of values
• often redundancy incorporated
P2P-Oslo-030516.fm 55 16.May.03
10012000
40017000
01000
10.00121.000
I
administer
2001-4000
www.kom.tu-darmstadt.de
www.httc.de
3.A Step: Location of Ressource - at "Directory Peer"
40.00165.536
21.00140.000
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
at directory peer
• storage of data there
• but, not flexible at all
⇒ in general not used
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 56 16.May.03
40017000
01000
7001-10.000
Overview
10012000
10.00121.000
H(“contents”)=2313
20014000
www.kom.tu-darmstadt.de
www.httc.de
3.B Step: Location of Ressource - Remote
10012000
40.00165.536
01000
X
21.00140.000
Overview
1. Motivation
40017000
7001-10.000
H(“Contents”)=2313
pointer to location
(2313 Æ X)
10.00121.000
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Nodes
• store tupel (KEY, VALUE)
• Key
•
= H (movie/matrix/divx/en) i.e. 2313
• Value
• real address information
•
= (IP, Port) i.e. (141.12.158.3, 4711)
Characteristics
• more flexible
• addition step to reach contents requiered
P2P-Oslo-030516.fm 57 16.May.03
20014000
www.kom.tu-darmstadt.de
www.httc.de
4. Step: Location of Remote Ressource
Lookup(H(“Content”))
Æ 2313
Knoten verwaltet Schlüssel
2001-4000, also auch 2313
Overview
Key = H(“Content”)
1. Motivation
2. Notion of Peer-to-Peer
(2313, (ip, port))
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
Einstiegsknoten
(beliebig)
Value = Verweis auf
Lokation der Daten
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Content based routing
• calculate HASH function
• location of directory resource
• location of actual RESSOURCE WITH CONTENTS
Issues
• effective routing to searched contents from any initially addressed node
P2P-Oslo-030516.fm 58 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Examples using DHT
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
Pastry
• Microsoft Research, Rice University
Chord
• UC Berkeley, MIT
CAN - Content Addressable Network
• UC Berkeley, ICSI
Tapestry
• UC Berkeley
etc.
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 59 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
8. Major Issues in P2P Networking
Overview
1. Motivation
Performance & Scalability
• performance predictability
• see Gnutella example
• much more communication overhead than in client-server systems
• bandwidth is usually scarce
• if a peer is unreachable,
• TCP/IP can take up to several minutes to time out the connection
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
Free Riders
• see Gnutella example
• incentives for sharing content, for not misbehaving ....
• any fair cost share
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Copyright Management
• how can I be sure that I am not downloading illegal stuff?
• how can the rights of the owners of intellectual property be enforced?
Interoperability
• there are a number of incompatible protocols
• there are no standards and there is no dominating protocol (yet)
P2P-Oslo-030516.fm 60 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Major Issues in P2P Networking
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
Trust
• how can I be sure that the document/software/movie... I am downloading is
the one that it was announced as
• how can I be sure that it is unchanged, does not contain viruses etc.?
• which community members can I trust?
• trust can be increased by
• decreasing the number of people that must be trusted
• reducing the risk
• components: message digest functions, digital certificates etc.
⇒ Potential Solution: Reputation Mechanisms
• see eBay, PGP web of trust, ...
• trade-off with anonymity
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
(2)
Other security aspects
• authentication?
• protection against denial of service attacks?
• protection against tampering of files, search messages etc.?
• virus protection?
• P2P protocols are built to circumvent firewalls
P2P-Oslo-030516.fm 61 16.May.03
(3)
www.kom.tu-darmstadt.de
www.httc.de
Major Issues in P2P Networking
Overview
1. Motivation
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
Lack of
• common middleware
• in general at w3c, global grid forum
• dedicated at EU MMAPPS - Market Management of Peer to Peer Services
• network management
• overlay network, i.e. inefficiency
• dynamics, i.e. unpredictable
P2P-Oslo-030516.fm 62 16.May.03
www.kom.tu-darmstadt.de
www.httc.de
Major Issues in P2P Networking
(4)
eLearning & telemedia education: domain specific metadata
eBusiness: consideration of business processes
AVAILABILITY:
appropriate search, retrieval, placement, ...
• proactive fine-grained availability:
Ava il Service = Avail Data × ( Avail P2Pdynamics Node × Avail intrinsics Node × Avail Link )
Overview
Demanding Availability
1. Motivation
Supplying Availability
2. Notion of Peer-to-Peer
3. Some Applications and
Systems: 1st Generation
user1
A2
4. Some Applications and
Systems: 2nd Generation
user2
A1
A2
5. Peer-to-Peer & Related
Mechanisms
user3
A0
A0
6. Peer-to-Peer Intrinsic
Issues
user4
P2P Service
A0
A1
Data
A1
System
Node
Link
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
A2: extremely high available
A1: high available
A0: as high as possible
Differentiated Availability
P2P-Oslo-030516.fm 63 16.May.03
P2P Node
dynamics
Node
intrinsics
Fine-Grained Availability
just a hype? ... no, P2P e.g. also as integral part of curricula
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
L1
Physical Layer
(Bitübertragung)
Web
Files
LAN, MAN
High-Speed
LAN
RT(C)P
Transport
Network
Medium-Term
Routing&Traffic
Engineering
Short-Term
Congestion
Control
Basics
Other Lectures of “ETiT” & Computer Science
Introduction
Legend:
P2P-Oslo-030516.fm 64 16.May.03
KN I
KN II
KN III
Time Scale
L2
Data Link
Layer
(Sicherung)
6. Peer-to-Peer Intrinsic
Issues
Internet:
IP
Long-Term
Network
Planning &
Engineering
Trafficmgmnt
5. Peer-to-Peer & Related
Mechanisms
WAN:
ISDN
&
ATM
Media
Data Flow
Security
L3
Network Layer
(Vermittlung)
4. Some Applications and
Systems: 2nd Generation
Internet:
TCP, UDP
MM COM - QoS specific
L4
Transport
Layer
(Transport)
Mobile Communications
3. Some Applications and
Systems: 1st Generation
IP-Tel:
Signal.
H.323
SIP
Moblie IP
2. Notion of Peer-to-Peer
Telnet
1. Motivation
Email
L5
Application
Layer
(Anwendung)
Inst. Msg.
Applications
P2P
Overview
Complementary Courses: Distributed Multimedia Systems (MM I and MM II),
Telecooperation II,III,.., Embedded Systems, Mobility in Comm. Networks
Transitions & Addressing
www.kom.tu-darmstadt.de
www.httc.de
9. Conclusion
www.kom.tu-darmstadt.de
www.httc.de
10. Annex: Some References
http://openp2p.com
http://www.oreilly.com/catalog/peertopeer
http://peer-to-peerwg.org/index.html.
http://www.mmapps.org
• Homepage EU-Project "Market Management for Peer-to-Peer Services";
Overview
1. Motivation
2. Notion of Peer-to-Peer
Peer-to-Peer, Ed. Andy Oram, O'Reilly, 2001.
3. Some Applications and
Systems: 1st Generation
4. Some Applications and
Systems: 2nd Generation
5. Peer-to-Peer & Related
Mechanisms
6. Peer-to-Peer Intrinsic
Issues
7. Search: Distributed Hash
Tables
8. Major Issues in P2P
Networking
9. Conclusion
P2P-Oslo-030516.fm 65 16.May.03