Internet Indirection
Infrastructure
Ion Stoica
UC Berkeley
Nov 14, 2005
Motivations
• Today’s Internet is built around a unicast
point-to-point communication abstraction:
– Send packet “p” from host “A” to host “B”
• This abstraction allows Internet to be highly
scalable and efficient, but…
• … not appropriate for applications that require
other communications primitives:
–
–
–
–
Multicast
Anycast
Mobility
…
2
Why?
• Point-to-point communication  implicitly
assumes there is one sender and one receiver,
and that they are placed at fixed and well-known
locations
– E.g., a host identified by the IP address 128.32.xxx.xxx
is located in Berkeley
3
IP Solutions
• Extend IP to support new communication
primitives, e.g.,
– Mobile IP
– IP multicast
– IP anycast
• Disadvantages:
– Difficult to implement while maintaining Internet’s
scalability (e.g., multicast)
– Require community wide consensus -- hard to achieve
in practice
4
Application Level Solutions
• Implement the required functionality at the
application level, e.g.,
– Application level multicast (e.g., Narada, Overcast,
Scattercast…)
– Application level mobility
• Disadvantages:
– Efficiency hard to achieve
– Redundancy: each application implements the
same functionality over and over again
– No synergy: each application implements usually
only one service; services hard to combine
5
Key Observation
• Virtually all previous proposals use indirection,
e.g.,
– Physical indirection point  mobile IP
– Logical indirection point  IP multicast
“Any problem in computer science can
be solved by adding a layer of indirection”
6
Our Solution
Build an efficient indirection layer
on top of IP
• Use an overlay network to implement this layer
– Incrementally deployable; don’t need to change IP
Application
Indir.
TCP/UDP
layer
IP
7
Internet Indirection Infrastructure (i3)
• Each packet is associated an identifier id
• To receive a packet with identifier id, receiver R
maintains a trigger (id, R) into the overlay
network
data id
Sender
Receiver (R)
data R
id R
trigger
8
Service Model
• API
– sendPacket(p);
– insertTrigger(t);
– removeTrigger(t) // optional
• Best-effort service model (like IP)
• Triggers periodically refreshed by end-hosts
• ID length: 256 bits
9
Mobility
• Host just needs to update its trigger as it moves
from one subnet to another
Sender
id R2
R1
Receiver
(R1)
Receiver
(R2)
10
Multicast
• Receivers insert triggers with same identifier
• Can dynamically switch between multicast and
unicast
data id
Sender
data R1
id R1
Receiver (R1)
id R2
data R2
Receiver (R2)
11
Anycast
• Use longest prefix matching instead of exact
matching
– Prefix p: anycast group identifier
– Suffix si: encode application semantics, e.g., location
data p|a
Sender
data R1
p|s1 R1
p|s2 R2
Receiver (R1)
Receiver (R2)
p|s3 R3
Receiver (R3)
12
Service Composition: Sender Initiated
• Use a stack of IDs to encode sequence of
operations to be performed on data path
• Advantages
– Don’t need to configure path
– Load balancing and robustness easy to achieve
Transcoder (T)
data idT,id
Sender
data id
data T,id
idT T
data R
id R
Receiver (R)
13
Service Composition: Receiver
Initiated
• Receiver can also specify the operations to be
performed on data
data id
Sender
Firewall (F)
data R
data F,R
idF F
Receiver (R)
data idF,R
id idF,R
14
Outline
 Implementation
• Examples
• Security
• Applications
• Legacy application support
15
Quick Implementation Overview
• i3 is implemented on top of Chord
– But can easily use CAN, Pastry, Tapestry, etc
• Each trigger t = (id, R) is stored on the node
responsible for id
• Use Chord recursive routing to find best
matching trigger for packet p = (id, data)
16
Routing Example
• R inserts trigger t = (37, R); S sends packet p = (37, data)
• An end-host needs to know only one i3 node to use i3
– E.g., S knows node 3, R knows node 35
S
2m-1 0
S
3
send(37, data)
20
[8..20]
7
7
[4..7]
3
[40..3]
Chord circle
37 R
[21..35]
41
41
20
37 R
35
[36..41]
trigger(37,R)
35
send(R, data)
R
R
17
Optimization #1: Path Length
• Sender/receiver caches i3 node mapping a
specific ID
• Subsequent packets are sent via one i3 node
[8..20]
[4..7]
data 37
[42..3]
Sender (S)
cache node
[21..35]
[36..41]
data R
37 R
Receiver (R)
18
Optimization #2: Triangular Routing
• Use well-known trigger for initial rendezvous
• Exchange a pair of (private) triggers well-located
• Use private triggers to send data traffic
[8..20]
[4..7]
37
data
[2] 30
2 S
Sender (S)
[42..3]
S
[30]
[21..35]
[36..41]
[2] R
37 R
data
R
2
[30]
30 R
Receiver (R)
19
Outline
• Implementation
 Examples
–
–
–
–
Heterogeneous multicast
Scalable Multicast
Load balancing
Proximity
• Security
• Applications
• Legacy application support
20
Example 1: Heterogeneous
Multicast
• Sender not aware of transformations
S_MPEG/JPEG
send(R1, data)
send(id, data)
id_MPEG/JPEG S_MPEG/JPEG
Sender
(MPEG)
Receiver R1
(JPEG)
send((id_MPEG/JPEG, R1), data)
id
(id_MPEG/JPEG, R1)
id
R2
send(R2, data)
Receiver R2
(MPEG)
21
Example 2: Scalable Multicast
• i3 doesn’t provide direct support for scalable multicast
– Triggers with same identifier are mapped onto the same i3 node
• Solution: have end-hosts build an hierarchy of trigger of
bounded degree
(g, data)
g
x
g g
R1 R2
R2
(x, data)
x
R3
R3
x
R4
R1
R4
22
Example 2: Scalable Multicast
(Discussion)
Unlike IP multicast, i3
1. Implement only small scale replication  allow
infrastructure to remain simple, robust, and
scalable
2. Gives end-hosts control on routing  enable endhosts to
– Achieve scalability, and
– Optimize tree construction to match their needs, e.g.,
delay, bandwidth
23
Example 3: Load Balancing
• Servers insert triggers with IDs that have random suffixes
• Clients send packets with IDs that have random suffixes
send(1010 0110,data)
S1
1010 0010 S1
A
S2
1010 0101 S2
1010 1010 S3
S3
send(1010 1110,data)
B
1010 1101 S4
S4
24
Example 4: Proximity
• Suffixes of trigger and packet IDs encode the
server and client locations
S2
S3
S1
send(1000 0011,data)
1000 0010
1000 1010 S2
1000 1101 S3
S1
25
Outline
• Implementation
• Examples
 Security
• Applications
• Legacy applications support
26
Some Attacks
Eavesdropping
Loop
id1 id2
id R
id A
S
R
id4 id1
Attacker (A)
id2 id3
id3 id4
Attacker
Confluence
Dead-End
id2 id3
Attacker id1 id2
id2 id3
id2 id3
id3 V
Victim
(V)
Attacker
id1 id2
id2 id3
27
Constrained Triggers
• hl(), hr(): well-known one-way hash functions
• Use hl(), hr() to constrain trigger (x, y)
Right constrained
must match
ID: prefix
64
key
128
y.key = hr(x)
suffix
64
x
y
Left constrained
x.key = hl(y.key)
x.key = hl(y)
x
y
x
y
end-host address
28
Attacks & Defenses
Attack
Defense
Trigger
constraints
Pushback
Trigger
challenges
Public ID
constraints
Eavesdropping&
Impersonation
Loops &
Confluences
Dead-ends
Reflection &
Malicious triggerremoval
Confluences
on i3 public nodes
29
Outline
• Implementation
• Examples
• Security
 Applications
 Protection against DoS attacks
– Service composition platform
• Legacy application support
30
In a Nutshell
• Problem scenario: attacker floods the incoming
link of the victim
• Solution: stop attacking traffic before it arrives
at the incoming link
– Today: call the ISP to stop the traffic, and hope for
the best!
• Our approach: give end-host control on what
packets to receive
– Enable end-hosts to stop the attacks in the network
31
Why End-Hosts (and not
Network)?
• End-hosts can better react to an attack
– Aware of semantics of traffic they receive
– Know what traffic they want to protect
• End-hosts may be in a better position to
detect an attack
– Flash-crowd vs. DoS
32
Some Useful Defenses
1. White-listing: avoid receiving packets on arbitrary
ports
2. Traffic isolation:
– Contain the traffic of an application under attack
– Protect the traffic of established connections
3. Throttling new connections: control the rate at
which new connections are opened (per sender)
33
1. White-listing
• Packets not addressed to open ports are
dropped in the network
– Create a public trigger for each port in the white list
– Allocate a private trigger for each new connection
34
2. Traffic Isolation
• Drop triggers being flooded without affecting
other triggers
– Protect ongoing connections from new connection
requests
– Protect a service from an attack on another
services
Transaction server
Legitimate client
(C)
ID1 V
ID2 V
Victim (V)
Web server
Attacker
(A)
35
2. Traffic Isolation
• Drop triggers being flooded without affecting
other triggers
– Protect ongoing connections from new connection
requests
– Protect a service from an attack on another
services
Transaction server
Legitimate client
(C)
ID1 V
Victim (V)
Web server
Attacker
(A)
Traffic of transaction server
protected from attack on web server
36
3. Throttling New Connections
• Redirect new connection requests to a gatekeeper
– Gatekeeper has more resources than victim
– Can be provided as a 3rd party service
Gatekeeper (A)
IDC C
Client (C)
X S
Server (S)
IDP A
37
Outline
• Implementation
• Examples
• Security
• Architecture Optimizations
 Applications
– Protection against DoS attacks
 Service composition platform
• Legacy application support
38
Service Composition
• Goal: allow third-parties and end-hosts to easily
insert new functionality on data path
– E.g., firewalls, NATs, caching, transcoding, spam
filtering, intrusion detection, etc..
• Why i3?
– Make middle-boxes part of the architecture
– Allow end-hosts/third-parties to explicitly route through
middle-boxes
39
Example
• Use Bro system to provide intrusion detection for
end-hosts that desire so
Bro (middle-box)
M
(idM:idBA, data)
(idBA, data)
idM M
client A
idAB A
(idAB, data)
i3
idBA B
server B
(idM:idAB, data)
40
Outline
• Implementation
• Examples
• Security
• Applications
 Legacy Application Support
41
The Problem
• New network architectures & overlays provide
new features
– RON : resilience to path failures
– i3 : mobility, NAT traversal, anycast, multicast
–…
• But still no widespread deployment
• How take advantage of new functionality?
• Enable popular applications (ssh, Firefox, IE) to
benefit from new features
42
Solutions
• Approach 1: rewrite/port apps for each new
overlay
– time-consuming, tedious, impossible for closed
source apps
• Approach 2: enable support for legacy
applications over multiple overlays
• We chose approach 2!
43
Overlay Convergence Architecture for
Legacy Applications (OCALA)
Interpose an Overlay Convergence Layer between
transport layer and overlay networks
Legacy Applications
(ssh, firefox, explorer, …)
Transport Layer
(TCP, UDP, …)
IP Layer
44
Overlay Convergence Architecture for
Legacy Applications (OCALA)
Interpose an Overlay Convergence Layer between
transport layer and overlay networks
Legacy Applications
(ssh, firefox, explorer, …)
Transport Layer
(TCP, UDP, …)
Overlay Convergence (OC) Layer
Overlay
(DOA, DTN, HIP, i3, RON, …)
OC Independent
(OC-I) Sublayer
OC Dependent
(OC-D) Sublayer
45
Simultaneous access to multiple
overlays
Host B
Host C
ssh
Host A
IRC
OC-I
Firefox
IRC
ssh
RON
i3
OC-D
OC-I
IP
i3
…
…
OC-I
RON
RON
i3
…
www.cnn.com
Internet
46
Which overlay to use?
• IP address and port number :
– E.g.: Forward all packets sent to 128.32.132.223
port 22 over i3
• DNS name:
– E.g.: Forward all packets sent to odu.edu.ron over
RON (Resilient Overlay Networks)
– E.g.: Forward all packets sent to odu.edu.i3 over
i3
47
Bridging Multiple Overlays
• Communication across overlays
• Stitch together functionality
Host A
Host C (foo.ron)
Appl.
OC-I
OC-I
OC-D
Appl.
Host B (bar.i3)
i3
i3
i3
RON
OC-I
RON
RON
tunnel
tunnel
path
48
Legacy Client Gateways – Demo
• Clients need not run OCALA locally
• Gateway has special Legacy Client IP (LCIP) module
Overlay server (dilip.i3)
Legacy gateway
Appl.
OC-I
Legacy Client
OC-I
Internet
LCIP
OV
OV
i3
DNSreq(ionhome.pli3.ocalaproxy.net)
49
Legacy Client Gateways – Demo
Access the following URL using your web browser:
http://ionhome.pli3.ocalaproxy.net:8000/ifconfig.html
50
Status
• i3 available as a service on Planetlab
• OCALA available on Linux, Window/XP; enable
legacy applications to take advantage of
–
–
–
–
i3
RON
HIP (Host Identity Protocol)
…
• Current applications
– Mobility
– Transparent access to machines behind NATs
– Secure and transparent access to services behind firewalls
51
Conclusions
• Indirection – key technique to implement basic
communication abstractions
– Multicast, Anycast, Mobility, …
• Internet Indirection Infrastructure
– Platform to deploy new services in the Internet
– Highly flexible forwarding infrastructure: allows endhosts and third-parties to choose their own routes!
i3: http://i3.cs.berkeley.edu
OCALA: http://ocala.cs.berkeley.edu
52
Other Architecture Optimizations
• Shortcuts:
– Sender can cache receiver instead of an i3 server
– Send subsequent packets directly to the receiver
• Private trigger aggregation
– Have all private triggers share the same prefix
– Insert only an anycast trigger with that prefix in i3
53
Conclusions
• i3: tremendously flexible infrastructure
– Allow end-hosts to control forwarding and
replication points in the infrastructure
– Abstract away the behavior of hosts (e.g., mask
mobility) form each other
– Provide a global and flexible identifier space
• IDs can identify virtually anything, end-hosts, services,
sessions, humans, devices, etc..
54
Enable Many Applications
• Access to machines behind NATs
• Transparent access to services/machines
behind firewalls
– Like VPNs but more secure and flexible
•
•
•
•
Protection against DoS attacks
Anycast
Transparent switching between two interfaces
…
55
Status
• Code publicly available:
http://i3.cs.berkeley.edu
– Supports Linux & Windows XP/2000 legacy
applications
– Includes a light weight Chord implementation
56
Private Trigger Aggregation (Problem)
• Each host maintains a private trigger in i3 for each other
host it communicate with
• Too many private triggers
– E.g., can reach 100 after browsing for 5 min!
• Every packet goes through an i3 server even if users
don’t want this
(idAB,data)
A
idAB A
(idAC,data)
idAC A
B
C
idAD A (idAD,data)
D
57
Private Trigger Aggregation (Solution)
• Chose all private triggers to have the same
prefix A
• Keep only an anycast trigger with prefix A in
infrastructure
(idAB,data)
A
(idAC,data)
idA* A
B
C
(idAD,data)
D
58
Private i3 Nodes (Problem)
• The resilience and truthworthiness of i3
infrastructure may not be good enough for
various users
– E.g., a company using i3 to provide transparent
and secure access for remote workers
59
Private i3 Nodes (Solution)
• Allow customers to add their i3 nodes to infrastructure
using anycast and shortcuts
ACME Inc
S
(idAS, data)
cache
idA* A
idAS A
ACME i3 server
60
Service Composition: Research
Questions
• How to locate and compose services?
• How to distribute state across hosts and
middle-boxes?
• How to transparently recover when a middlebox fails? How is the state recovered?
61
Shortcuts (Problem)
• Each packet is traversing an i3 server no
matter whether the user wants/needs to or
not
62
Shortcuts (Solution)
• Associate an “allow-caching” bit with both packets
and triggers
• If both the packet and trigger have the bit set 
sender can cache receiver’s address
Host A
LA
Host B (IPB)
(idBA,data)
LB
idBA A
1
1
cache=IPB
2
3
63
Shortcuts: Discussion
• Shortcuts are allowed only if both the sender
and receiver agree
• i3 used as a lookup infrastructure
• Indirection still needed for
– Mobility
– Symmetric NAT & Firewall traversal
– Anonymity
64