Route Control Platform
Making an AS look and act like a router
Aman Shaikh
AT&T Labs - Research
IEEE CCW 2004
Matt Caesar (UC Berkeley)
Don Caldwell (AT&T Labs – Research)
Nick Feamster (MIT)
Jennifer Rexford (AT&T Labs – Research)
Kobus van der Merwe (AT&T Labs – Research)
Route Control Platform – IEEE CCW 2004
1
You have heard the news
• BGP is broken
–
–
–
–
–
It converges slowly
At times it does not converge at all
It causes routing loops and deflections inside an AS
It’s misconfigured frequently
Traffic engineering is hard with BGP
• Fixing BGP is hard
– Incremental fixes
• Makes BGP even more complicated
– New architectures and inter-domain protocols
• Deployment is almost impossible
Route Control Platform – IEEE CCW 2004
2
What are the fundamental problems?
• AS is a logical entity for inter-domain routing
(i.e. BGP) and yet BGP state and logic are
decomposed across routers inside an AS
– No router has complete BGP state
– Each router makes routing decision based on partial
and incomplete view
• BGP interacts in odd ways with other protocols
– Most notably with the IGP (Interior Gateway
Protocol) running inside an AS
Route Control Platform – IEEE CCW 2004
3
Fixing the fundamental problems:
“Route Control Platform”
• Represents an AS as a single logical entity
– Complete view of AS’s routes
– Computes routes for all routers inside an AS
• Routers no longer have to compute routes
• Exchanges routing information with RCPs in
other ASes
RCP
RCP
Inter-AS Protocol
RCP
iBGP
AS 1
Physical
peering
AS 2
Route Control Platform – IEEE CCW 2004
AS 3
4
The rest of this talk: the case for RCP
• Principles for inter-domain routing
– Treat the AS as a whole and compute routes using AS-wide
state
• Example: high-level policy expression
– Control routing protocol interactions
• Example: interaction between BGP and IGP
• Potential dealbreakers
– Backwards compatibility and incentives
– Scalability and reliability
• Related work (or…haven’t we seen this before?)
– Route reflection and route servers
– Overlay networks
Route Control Platform – IEEE CCW 2004
5
Decomposed configuration state
10.0.0.1
A
C
192.168.0.1
Sprint
UUNet
Simple policy: “Don’t advertise routes learned from UUNet to Sprint”
Configuration is decomposed, so routes must carry state
neighbor 192.168.0.1 route-map IMPORT-C in
route-map IMPORT-C permit 10
set community 0:1000
Assign routes
“From UUNet”
tag at router C
ip community-list 1 permit 0:1000
neighbor 10.0.0.1 route-map EXPORT-A out
route-map EXPORT-A deny 10
match community 1
Don’t send route with
“From UUNet” tag to
Sprint at router A
Route Control Platform – IEEE CCW 2004
6
RCP: Centralize configuration
10.0.0.1
Sprint
A
C
192.168.0.1
RCP
UUNet
• RCP implements policies for entire AS
– Knows about sessions to all other ASes
– Implements policies in terms of relationship with
ASes
• Benefits
– Simpler configuration
– Do not have to tag routes with state
Route Control Platform – IEEE CCW 2004
7
BGP interacts with underlying protocols
d
RR2
RR1
3
1
C1
1
1
3
C2
C1 learns BGP route to destination from RR1
C2 learns BGP route to destination from RR2
C1 sends packets to RR1 via its IGP shortest path which traverses C2
C2 sends packets to RR2 via its IGP shortest path which traverses C1
Persistent forwarding loop 
Route Control Platform – IEEE CCW 2004
8
RCP: Compute routes with complete info
d
RR2
RR1
“RR2” RCP
1
1 “RR2”
3
3
C1
1
C2
• RCP learns all externally learned routes
• Computes consistent router-level paths
• Benefits:
– Intrinsic loop freedom and convergence
– RCP does not have to stick to BGP decision process
• Can “pin” paths for traffic engineering and other purposes
Route Control Platform – IEEE CCW 2004
9
Getting from here to there: three easy steps
eBGP
iBGP
RCP
RCP
Inter-AS Protocol
RCP
iBGP
AS 1
Physical
peering
AS 2
AS 3
• Two key issues
– Backward compatibility
– Deployment incentives
Route Control Platform – IEEE CCW 2004
10
Phase 1: Control over Protocol Interactions
Before: conventional iBGP
eBGP
iBGP
After: RCP gets “best” iBGP routes (and IGP topology)
eBGP
RCP
iBGP
Only one AS has to change its architecture!
Route Control Platform – IEEE CCW 2004
11
Phase 1 Application: Controlling Path Changes
BGP routes take “nearest exist” (shortest IGP path)
Failures or maintenance can change IGP (path) weights
Exit point can also change
Traffic shifts, convergence delay, congestion
A
C
B
D
Route Control Platform – IEEE CCW 2004
12
Phase 1 Application: Controlling Path Changes
BGP routes take “nearest exist” (shortest IGP path)
Failures or maintenance can change IGP (path) weights
RCP can “pin” exit points as IGP weights change
A
C
B
RCP
D
Route Control Platform – IEEE CCW 2004
13
Phase 2: AS-wide Selection and Policy
Before: RCP gets “best” iBGP routes (and IGP topology)
eBGP
RCP
iBGP
After: RCP gets all eBGP routes from neighbors
eBGP
RCP
iBGP
Route Control Platform – IEEE CCW 2004
14
Phase 2 Application: Efficient Aggregation
Aggregation curbs routing table growth
Routers can’t know which routers need more specific routes
192.168.0.0/23
192.168.1.0/24
192.168.0.0/23
192.168.0.0/24
192.168.0.0/23 (??)
iBGP
192.168.0.0/23 (??)
eBGP
Route Control Platform – IEEE CCW 2004
15
Phase 2 Application: Efficient Aggregation
Aggregation curbs routing table growth
RCP can determine which routers need more specific routes
and which routers can do away with less specific routes
192.168.0.0/23
192.168.0.0/24
192.168.0.0/23
192.168.1.0/24
192.168.0.0/23
RCP
192.168.0.0/23
192.168.0.0/24
192.168.0.0/23
192.168.0.0/23
192.168.1.0/24
Route Control Platform – IEEE CCW 2004
16
Phase 3: All ASes have RCPs
Before: RCP gets all eBGP routes from neighbors
eBGP
RCP
iBGP
After: ASes exchange routes via RCP
RCP
RCP
Inter-AS Protocol
RCP
iBGP
AS 1
Physical
peering
AS 2
Route Control Platform – IEEE CCW 2004
AS 3
17
Phase 3 Application: More Flexible Routing
• Better network management
– Diagnostics and trouble-shooting
– Routing co-located with other information (e.g.
traffic)
– Ability to reason about an AS as a single entity
• Protocol Improvements
– Attaching prices to routes
– Inter-AS negotiation of exit points
– Overlay routing informed by IP-layer information
• Your application here…
Route Control Platform – IEEE CCW 2004
18
Scalability and Robustness
• Can RCP scale?
– We have a prototype implementation
• Single-box RCP can handle AS-wide BGP load
• OSPF changes can be troublesome
– Centralized != unable to scale
• Is RCP a single point of failure?
– RCP can be implemented using distributed system
insights
– Consistency (mostly) a non-isssue
• Guarantee from OSPF/IS-IS operation:
– “Either an RCP replica has a complete view of network
(partition) or no view; but never a partial view”
Route Control Platform – IEEE CCW 2004
19
Is RCP basically a Route Reflector?
• Yes, but it’s a better route reflector
• “Customized” routing decisions for clients
– Route reflectors do not compute routes from client’s
perspective
– Route reflectors do not emulate a “full mesh”
• Routing decisions based on complete visibility
– Guaranteed correct routes
– Replication is dictated by system issues
Route Control Platform – IEEE CCW 2004
20
RCP also looks a lot like…
• A “route server”
– Route arbiter: looked at applying policy at exchange
points
– AS agents
• RCP can act as an AS agent; can answer queries for the AS
• An overlay network
– Most previous work is in data overlays
– RCP is a control overlay
• Hierarchical routing is about control overlays
– RCP could give more information and control to data
overlays
• RCP has AS-wide information and direct control over paths
taken through the AS
Route Control Platform – IEEE CCW 2004
21
Conclusions
• RCP embodies two principles for inter-domain
routing
– Treat an AS as a single logical entity
• Compute consistent routes using complete AS-wide view
– Control routing protocol interactions
• Benefits
– Simpler, more expressive configuration
– Intrinsic robustness: no loops, faster convergence
– Enable new applications and innovations
• Opportunity for new traffic engineering applications
Route Control Platform – IEEE CCW 2004
22
More information
FDNA 04 paper
“The Case for Separating Routing from Routers”
Nick Feamster, Hari Balakrishnan, Jennifer Rexford, Aman Shaikh,
Kobus van der Merwe
http://www.research.att.com/~ashaikh/publications.html
Route Control Platform – IEEE CCW 2004
23