A short overview of p2p technologies Marcelo Dias de Amorim Laboratoire LIP6 Université Pierre et Marie Curie http://www.lip6.fr/ Caen – Le 11 Juin 2003 Plan Introduction Models Example of applications Some systems References IPv6 – Caen, 11 juin 2003 2 Definitions P2P is a class of applications that take advantage of resources – storage, cycles, content, human presence – available at the edges of the Internet. Clay Shirky (www.shirky.com) P2P refers to a class of systems and applications that employ distributed resources to perform a critical function in a decentralized manner. Milojicic et al. (HP) IPv6 – Caen, 11 juin 2003 3 Some controversy Is p2p a new approach? Problems is peer-to-peer systems are neither new nor unique; they make us look for solutions to old problems that we all worked around or tried to ignore before. Andy Oram (O'Reilly & associates) speech at Free and Open Source Software Developers's Meeting Brussels, BE, Feb. 2002 IPv6 – Caen, 11 juin 2003 4 Curiosity (traffic) One year ago… … and today! Others 31% 41% Unidentified P2P 18% 48% 51% 11% IPv6 – Caen, 11 juin 2003 source netflow.internet2.edu 5 Properties No central control, no central database No hierarchy Every node is both a client and a server The communication between peers is symmetric No global view of the system Scalablity Availability for any peer Peers are autonomous System globally unreliable Robustness and security issues IPv6 – Caen, 11 juin 2003 6 Examples of p2p usage File-sharing applications Distributed databases Distributed computing (grid?) Collaboration Distributed games Instant messaging Ad hoc networks Application-level multicast Etc. IPv6 – Caen, 11 juin 2003 7 Overlay networks Overlay IP IPv6 – Caen, 11 juin 2003 8 Overlay networks Overlay IP IPv6 – Caen, 11 juin 2003 9 Centralized model (Napster) File-sharing system Almost distributed system The location of a document is centralized The "transfer" is peer-to-peer Problems Robustness Scalability (?) IPv6 – Caen, 11 juin 2003 10 Centralized model (Napster) location server register INTERNET Document x! OK: Document Peer Z x? IP = a.b.c.d x IPv6 – Caen, 11 juin 2003 11 Non-structured system (Gnutella-like) Two phases (like Napster) Localization + exchange No server Open source gnutella.wego.com Distributed search The query is flooded Loop avoidance Limited TTL (not all nodes are visited) IPv6 – Caen, 11 juin 2003 12 Gnutella IPv6 – Caen, 11 juin 2003 13 Structured systems (DHTs) Based on distributed hash tables (DHTs) No flooding Exact matches Overhead Gnutella-like O(n) DHT O(log n) Examples CAN, Pastry, Chord, Kademlia, Tapestry, etc. IPv6 – Caen, 11 juin 2003 14 Content-Addressable Networks (CAN) Provides a large scale distributed hash table Keys are mapped into values CAN defines a d-dimensional virtual space No relationship with the physical space Query O(n1/d) Neighbors O(d) The virtual space is completely distributed among the peers Each peer is responsible for one share of the space The peer that is responsible for region R is also responsible for the values inside R Documents must be uniquely identified IPv6 – Caen, 11 juin 2003 15 Example IPv6 – Caen, 11 juin 2003 16 Example 1 IPv6 – Caen, 11 juin 2003 17 Example 1 2 IPv6 – Caen, 11 juin 2003 18 Example 1 2 3 IPv6 – Caen, 11 juin 2003 19 Example 4 1 2 3 IPv6 – Caen, 11 juin 2003 20 Example 4 4 1 5 2 3 IPv6 – Caen, 11 juin 2003 21 Example 1 4 6 5 2 3 IPv6 – Caen, 11 juin 2003 22 Example 1 7 4 6 5 2 3 IPv6 – Caen, 11 juin 2003 23 Association ID node 1 7 4 6 5 2 Ex: Node 3 holds this document 3 IPv6 – Caen, 11 juin 2003 24 Association ID node 1 7 4 6 5 2 3 IPv6 – Caen, 11 juin 2003 25 Application-layer multicast Native multicast not yet completely deployed ALM easier/faster to implement Scalability states at end-systems High-level support IPv6 – Caen, 11 juin 2003 26 Application-layer multicast source IPv6 – Caen, 11 juin 2003 27 Taxonomy P2P Applications Parallel Content and file mngt Compute Compo- Content Intensive nentized exchange File system Collaboration Filtering, Instant mining message IPv6 – Caen, 11 juin 2003 Shared apps Games 28 Many works and projects Gnutella and Napster @sourceforge.net IPv6 – Caen, 11 juin 2003 29 JXTA (Sun) Open platform for p2p cooperation Interoperability Any system/peer/application Platform independency Languages (C, Java, etc) Systems platforms (Unix, Windows, etc) Networking platforms (802.11, Bluetooth, TCP/IP, etc) Ubiquity Sensors, PDAs, routers, desktops, laptops, storage systems IPv6 – Caen, 11 juin 2003 30 JXTA (Sun) Objectives Find peers and resources Share files with anyone across the network Create a particular group of peers across different networks Communicate securely with peers across public networks Projects Applications (24 projects) Core (13 projects) Demos (3 projects) Forge (15 projects) Other (12 projects) Services (24 projects) IPv6 – Caen, 11 juin 2003 31 JXTA (Sun) Protocols Peer discovery protocol Peer resolver protocol Peer information protocol Rendezvous protocol Pipe binding protocol Endpoint routing protocol IPv6 – Caen, 11 juin 2003 32 JXTA (Sun) JXTA Community Applications JXTA Community Services Peer Groups JXTA Shell Sun JXTA Services Peer Pipes Peer Commands Peer Monitoring Security J X T A Peer (Desktop, cell phone, PDA, etc.) IPv6 – Caen, 11 juin 2003 33 C O R E JXTA applications IPv6 – Caen, 11 juin 2003 34 Final remarks P2P implies a very large spectrum of areas High interest in both academicals/industrials Much has already been done, but no conclusions are definitive IPv6 and P2P NAT, firewalls, IPv6 as an overlay Many open issues Trust, security, scalability, QoS, etc. IPv6 – Caen, 11 juin 2003 35 Working groups et al. A generic site on p2p from O'Reilly www.openp2p.com P2P working group www.peer-to-peerwg.org/ Internet2 p2p working group p2p.internet2.edu Peer-to-peer development (p2p-hackers) zgp.org/mailman/listinfo/p2p-hackers Interesting meeting www.codecon.org IPv6 – Caen, 11 juin 2003 36 References Distributed Computing Distributed (www.distributed.net) SETI@home (www.seti.org) Genome@home (gah.stanford.edu) Folding@home (www.stanford.edu/group/pandegroup/folding) Global Grid Forum (www.globalgridforum.org) Globus Project (www.globus.org) File sharing Napster (www.napster.com) Gnutella (gnutella.wego.com) Kazaa (www.kazaa.com) IPv6 – Caen, 11 juin 2003 37 References Distributed hash tables CAN (www.acm.org/sigs/sigcomm/sigcomm2001/p13ratnasamy.pdf) Pastry (research.microsoft.com/~antr/Pastry) Chord (www.pdos.lcs.mit.edu/chord) Tapestry (www.cs.berkeley.edu/~ravenben/tapestry) Freenet (freenet.sourceforge.net) Kademlia (kademlia.scs.cs.nyu.edu) Ad hoc networking AODV (www.ietf.org/internet-drafts/draft-ietf-manetaodv-13.txt) OLSR (www.ietf.org/internet-drafts/draft-ietf-manet-olsr10.txt) Tribe (rp.lip6.fr/site_rp/_publications/350-79Viana.ps.gz) IPv6 – Caen, 11 juin 2003 38 References Platforms JXTA (www.jxta.org) .NET (www.microsoft.com/net) Collaboration Groove (www.groove.net) Endeavors (www.endeavors.com) IPv6 as a p2p overlay Working Groups p2p.internet2.edu www.openp2p.com IPv6 – Caen, 11 juin 2003 39