SatelliteLab: Adding Heterogeneity to
Planetary-Scale Network Testbeds
Marcel Dischinger
MPI-SWS
© 2008 Andreas Haeberlen, MPI-SWS
Andreas Haeberlen
MPI-SWS / Rice University
Ivan Beschastnikh
University of Washington
Krishna P. Gummadi
Stefan Saroiu
MPI-SWS
University of Toronto
SIGCOMM 2008
1
Why are testbeds so popular?
vs.
PLANETLAB
Your cluster
Testbeds

Testbeds provide wide-area network paths

Realistic network paths are crucial for evaluating
the performance of distributed systems
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
2
Access networks and heterogeneity
Internet Backbone
Access
networks

Internet bottlenecks are in access networks


Access networks have high heterogeneity


Studies show that most queueing, loss etc. happens there
Cable, DSL, GPRS, UMTS, EVDO, 802.11, Satellite, Dial-up...
Not represented in existing testbeds
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
3
Inter-AS links covered
Growing the testbed is not enough
27 end nodes
in U.S. and Europe
280 PlanetLab nodes
in U.S. and Europe
Node index

Can we simply add more nodes?



More nodes from acad. networks do not improve heterogeneity
Nodes from the commercial Internet are much more effective
Need nodes from the commercial Internet
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
4
Challenges
1)
Make it easy for volunteers to contribute



2)
Will not allow arbitrary code to run on their machines
Do not want to spend time managing software
Do not want to deal with complaints
Keep testbed easy to use for experimenters



© 2008 Andreas Haeberlen, MPI-SWS
Resources may vary widely (cell phone vs. workstation)
Nodes may have different OSes, hardware, software...
Nodes may be behind middleboxes (NATs, firewalls, ...)
SIGCOMM 2008
5
A new testbed architecture
Tier 1
"Planets"
Tier 2
"Satellites"

Idea: Separate computation and communication


Tier 1: Powerful, well-connected nodes (from existing testbed)
Contribute computation power, memory, storage...
Tier 2: Light-weight nodes
Contribute "just their access-network links"
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
6
Outline





Motivation
SatelliteLab
Evaluation
Applications
Conclusion
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
7
Delegation
Planet A
Network path
Planet B
Adds some
latency
Application
Satellite A

Satellite B
Delegate code execution to the planets



Network path
Problem: Changes the network path
Misses the access links
Solution: Send traffic through the satellites
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
8
Detour routing
Planet A
Planet B
Firewall
Satellite B
Satellite A

Two challenges remain:



Satellites are sending traffic that may trigger complaints
NATs and firewalls may prevent connectivity
Solution: Detour traffic through the planets
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
9
The SatelliteLab detour path
Planet A
Satellite A

Actual path
"Ideal" path
Planet B
Satellite B
To send a packet from planet A to planet B:
1.
2.
3.
4.
5.
© 2008 Andreas Haeberlen, MPI-SWS
A intercepts the data packet
A exchanges a probe packet with its satellite
A sends the data packet to B
B exchanges a probe packet with its satellite
B delivers the data packet
SIGCOMM 2008
10
Challenges revisited
1)
Make it easy for volunteers to contribute




2)
No
No
No
No
complaints: Satellites only communicate with their planet
hassle: Sufficient to install the satellite helper once
CPU slowdown: Planets do all the heavy lifting
security risk: Satellite is just 118 lines of Java code
Keep testbed easy to use for experimenters




All application code runs on the planets
No need to deal with middleboxes
No resource limitations due to satellites
Works with unmodified binaries
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
11
Outline





Motivation
SatelliteLab
Evaluation
Applications
Conclusion
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
12
Evaluation




Is it feasible to recruit edge nodes? Yes.
Is the availability sufficient for experiments? Yes.
Can satellites find a nearby planet? Yes.
Does SatelliteLab preserve...




the
the
the
the
© 2008 Andreas Haeberlen, MPI-SWS
path capacity? Yes.
queueing delay? Yes.
jitter? Yes.
packet loss rates? Yes.
SIGCOMM 2008
13
Is it feasible to recruit edge nodes?
#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

Location
Canada
Canada
Canada
Canada
Canada
Germany
Germany
Germany
Germany
Germany
Germany
Germany
Germany
Hungary
Portugal
UK
NAT
no
no
no
no
yes
yes
no
yes
yes
yes
yes
yes
no
yes
no
no
Type
Desktop
Desktop
Laptop
PDA
Laptop
Desktop
Desktop
Desktop
Laptop
Laptop
Desktop
Desktop
Laptop
Laptop
Laptop
Laptop
Mob.
no
no
yes
yes
no
no
no
no
yes
yes
no
no
no
yes
no
yes
#
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Location
CA, USA
CA ,USA
CO, USA
IL, USA
LA, USA
MA, USA
MD, USA
MD, USA
NJ, USA
NJ, USA
TX, USA
WA, USA
WA, USA
WA, USA
WA, USA
WA, USA
Access link
DSL+Wi-Fi
EVDO
Cable+Wi-Fi
Cable
DSL
Cable+Wi-Fi
Uni
Cable+Wi-Fi
DSL+Wi-Fi
Cable+Wi-Fi
Cable+Wi-Fi
Cable
Cable
Cable+Wi-Fi
Cable+Wi-Fi
DSL
NAT
yes
no
yes
yes
yes
no
no
yes
yes
yes
yes
yes
yes
yes
yes
yes
Type
Laptop
Laptop
Laptop
Desktop
Desktop
Laptop
Desktop
Laptop
Laptop
Laptop
Desktop
Desktop
Desktop
Desktop
Laptop
Desktop
Mob.
no
no
no
no
no
yes
no
yes
no
no
no
no
no
no
yes
no
Built a prototype testbed


Access link
Cable
DSL
Uni+Wi-Fi
Uni+Wi-Fi
Cable+Wi-Fi
DSL+Wi-Fi
Cable
DSL
DSL+Wi-Fi
Cable+Wi-Fi
DSL
DSL+Wi-Fi
ISDN+BT
DSL
Cable
DSL
Asked friends and family to install satellite software
Increased heterogeneity with very little effort


Ten U.S. states, six countries, 15 commercial ISPs
Technologies: Cable, DSL, Ethernet, Wi-Fi, ISDN, BT, EVDO
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
14
SatelliteLab preserves path capacity
PA
SA

Direct path
PB
SB
Direct path vs. interplanetary path


Interplanetary
SatelliteLab path
path
Large difference because bottlenecks are in access networks
Detour routing preserves the path capacity
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
15
SatelliteLab preserves queueing delay
PA
SA

Direct path
PB
SB
Same comparison for queueing delay


Interplanetary
path
SatelliteLab path
Almost all the queueing is in the access networks
Detour routing preserves the queueing delay
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
16
Outline




Motivation
SatelliteLab
Evaluation
Applications




Network coordinate system
Overlay multicast
TCP throughput over cellular links
Conclusion
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
17
Application: Network coordinates


Need to measure RTT between nodes
Surprising observation (in previous work):




Discarding outliers improves network coordinates on
PlanetLab [NSDI'06] but not when deployed with Azureus
BitTorrent client [NSDI'07]
Reason: Huge RTT variance
PlanetLab did not predict this observation
Could SatelliteLab have predicted it?
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
Figures taken from [NSDI'07]
18
Application: Network coordinates

We ran an experiment with several Azureus
nodes on SatelliteLab

We observed same effect as in [NSDI'07]

SatelliteLab could have predicted this outcome!

Can we also use SatelliteLab to explain it?


Yes - Measurements revealed extremely long queues on
the broadband access links
Azureus downloads fill up the queues  high RTT variance!
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
19
Summary


Existing testbeds lack heterogeneity
SatelliteLab: A new testbed architecture for
heterogeneous testbeds



© 2008 Andreas Haeberlen, MPI-SWS
Introduces satellites, a new class of light-weight nodes
Enables experimenters to improve heterogenity by
recruiting end nodes as satellites
Builds on existing testbeds and preserves many of their
useful properties: Ease of use, ease of management...
SIGCOMM 2008
20
SatelliteLab is for YOU!


We have a new testbed
that you can use for
your experiments
To run an experiment, you need:



some planets (e.g., nodes from PlanetLab)
some satellites (e.g., nodes from friends or colleagues)
the SatelliteLab software
http://satellitelab.mpi-sws.org/
© 2008 Andreas Haeberlen, MPI-SWS
SIGCOMM 2008
21