Networks and Distributed Systems
a.k.a. G22.3033-010
Lakshmi Subramanian
http://cs.nyu.edu/~lakshmi
Jinyang Li
http://cs.nyu.edu/~jinyang
Class goals
• Help you
– critically appreciate networks & systems research
– learn creative problem solving (i.e. doing research)
• How?
– Lectures/readings: discuss state-of-art work
– Programming labs: play with real systems
– A semester-long research project
Syllabus, grading etc.
• http://www.cs.nyu.edu/courses/fall06/G22.3033-010
• Class participation (20%)
– Read assigned papers before class!
• Two labs (10%)
• One project (70%)
– Team of 2-3 people (<= 1 Ph.D. student per group)
– Start next week
– Weekly (or once every two weeks) meetings
Who should take the class?
• Grad-level class
– Satisfy M.S. requirement of a “project” course
• Pre-requisite:
– Basic knowledge on networks
• Computer Networks (L. Peterson)
• An engineering approach to computer networking
(S. Keshav)
– Programming experience
• TCP/IP Illustrated (R. Stevens)
Misc.
• Office hours:
– Jinyang: 715 Broadway Rm 705, Tue 5-6pm
– Lakshmi: Rm 706 Mon 5-6pm
– TA: Ja Chen (jchen@cs.nyu.edu)
Next Generation Networks
Jinyang Li
Emerging networks
•
•
•
•
•
Wireless networks
Sensor networks
Overlays and P2P
Delay tolerant networks (DTNs)
…
Wireless networks
Wireless networks: why now?
• Proliferation of wifi-enabled devices
• Faster, cheaper radios and more powerful boxes
Wireless apps: urban mesh
• Provide cheap, ubiquitous Internet connectivity
MIT Cambridge Roofnet
http://pdos.lcs.mit.edu/roofnet
Google Mountain View pole top network
http://wifi.google.com
Wireless apps:
connecting rural villages
Intel/UC Berkeley/NYU Tier project
http://tier.cs.berkeley.edu
Wireless apps: mobile, ad-hoc
communication
MIT CarTel
http://cartel.csail.mit.edu
Wireless networks: challenges
1.
2.
3.
4.
Crappy links
Contention and self-interference
Frequent node/link failures
Many parameters
Goal: Robust, high performance designs
•
•
•
MAC layer
Routing layer
Transport layer
Challenge #1: crappy links
• Many asymmetric, lossy links
Challenge #2: contention
• Many nodes access the medium  collisions
• No way to explicitly detect collisions
Challenge #2: self-interference
• A multi-hop flow interferes at successive hops
1
• At
2
3
4
most every third node can transmit
5
Challenges #3: dynamism
Time (sec)
• Links/nodes fail and recover frequently
• Link qualities change over time
Challenge #4: (too) many
tunable parameters
•
•
•
•
•
Transmission power
Transmission rate
Directional vs. omni antennas
Static vs. dynamic channel assignment
One vs. multiple radios
Current state-of-art
# hops
latency
(ms)
throughput
(kbps)
1
2
3
4
5
6
7
14
26
45
50
60
100
83
2451
771
362
266
210
272
181
MIT Roofnet pair-wise node throughput
(11Mbps 802.11b radios)
Sensor networks
Beyond host-to-host communication
Sensor networks: why now?
• Technology is ready
– Cheaper, smaller, more powerful sensors
– Sense light, temperature, vibration, humidity,
location, pulse, motion, vital sign etc.
• Monitor environment, collection information
UCB Telos
Intel Dot
Xbow MicaZ
Sensor apps: understanding
redwood forests
UC Berkeley/Intel Research
Sensor apps:
real-time patient tracking
Harvard
CodeBlue
Sensor-net challenges
• Different communication paradigm
– host-to-host is the wrong fit
– Data-centric
• Limited resources
– Low radio bandwidth
250Kbps advertised, ~80Kbps in real life
– Slow processor, tiny storage
8MHz CPU, 8K RAM
– Limited energy
Overlays and P2P
Distributed systems meet the Internet
Why p2p/overlay?
Internet
• A distributed system architecture:
– No (minimal) centralized control
– Nodes are symmetric in function
• Enabled by technology improvements
Large scale wide-area systems
• Unmanaged (open p2p systems):
– BitTorrent: >1M nodes
– Skype: >5M users
• Managed
– PlanetLab: 700 nodes over 336 sites
– Akamai CDN: >10K nodes
What’s new here?
• Opportunities:
– Huge aggregate capacity
Network, storage, processing…
– Geographic diversity
• Many apps:
–
–
–
–
–
–
File sharing
CDNs
VoIP
Streaming multicast
Usenet news
…
Challenges
• How to find data?
• How to deal with failures?
– Nodes fail and recover
– Network outage and partition
• (Open networks only) How to deal with selfish or
malicious nodes?
– provide data integrity
– provide privacy or anonymity
Challenge #1: resource discovery
Case study: file sharing
• Where is the file named “Hamlet”?
Challenge #2: churn
• What if the node with “Hamlet” goes down?
Challenge #3: selfish nodes
I do NOT have
Hamlet
• Selfish nodes do not want to upload “Hamlet”
Challenge #4: malicious nodes
I HAVE junk
named Hamlet
• Malicious nodes lie about their contents
Next week
• Naming and addressing
• Project ideas
Distributed systems in a
data-center
• Connected by LANs
low loss and delay
• Provide infrastructural
services for apps
– Network file systems
– Databases
– Distributed data processing
Check out the Spring class
“distributed storage systems”