Putting BGP on the Right Path:
A Case for Next-Hop Routing
Michael Schapira
Joint work with Yaping Zhu
and Jennifer Rexford
(Princeton University)
Once Upon a Time…
Internet Inter-Network Routing:
• Small network
• Single administrative entity
 NSFNET
• Shortest-path routing
 distance-vector routing
• Then....
Interdomain Routing
Over 35,000 Autonomous Systems (ASes)
Sprint
AT&T
Comcast
Qwest
Interdomain routing = routing between ASes
– Border Gateway Protocol (BGP)
Today’s Path-Based Routing
With BGP
• Complex!
– configuration errors, software bugs, …
• Bad convergence!
– persistent route oscillations, slow convergence, …
• Vulnerable to attacks!
– malicious, economically-driven, inadvertent, …
• and more, and more, and more …
– bad performance, clumsy traffic engineering, …
How Can We Fix
Interdomain Routing?
• One approach: add mechanisms to
an already complex protocol
– route flap damping, S-BGP, …
• Another approach: redesign
interdomain routing from scratch
– HLP, NIRA, pathlet routing, consensus routing, …
• Our approach: simplify BGP!
Background: Today’s PathBased Routing With BGP
• AS i’s routing policy:
 ranking of simple routes from i to each destination d
 export policy
• BGP is a path-vector protocol
Receive
route updates
from neighbors
Choose single
“best”
route
(ranking)
Send route
updates to
neighbors
(export policy)
Background: Today’s PathBased Routing With BGP
3, I’m
using 1d
32d > 31d
1
3
1, 2, I’m
available
d
2
Don’t export
2d to 3
a stable state is reached
AS-PATH = the Route of All Evil
• AS-PATH: list of all ASes on path
– originally meant for loop-detection
• The AS-PATH is to blame!
–
–
–
–
error-prone, software bugs
no/slow convergence
large attack surface
bad scalability, clumsy traffic engineering, bad
performance, …
Getting Off the AS-PATH
• No way back to shortest-path
routing…
• Our proposal: next-hop routing
– make routing decisions based solely
on the “next hop”
– relegate the AS-PATH to its
original role
Wish List
•
•
•
•
•
•
•
•
•
Loop freedom
Fast Convergence
Security
Incentive compatibility
Business policies
Good performance
Traffic engineering
Scalability
Simplicity
Expressiveness vs. Complexity
complexity
BGP’s
path-based
routing
too complex
shortest-path
routing
next-hop
routing
simple
expressiveness
not expressive
enough
sufficiently
expressive
extremely
expressive
Next-Hop Routing Rules!
• Rule 1: use next-hop rankings
541d >
53d
4
> 3>
542d
1
5
4
d
2
3
Next-Hop Routing Rules!
• Rule 1: use next-hop rankings
• Rule 2: prioritize current route
– to minimize path exploration
[Godfrey-Caesar-Hagen-Singer-Shenker]
2=3
2=3
Prioritize
Break
ties in
current
favor
of lower
route
AS
number
2
d
1
3
Next-Hop Routing Rules!
• Rule 1: use next-hop rankings
• Rule 2: prioritize current route
• Rule 3: consistently export
– to avoid disconnecting upstream nodes [Feigenbaum-S-Ramachandran]
2,
11 >> 2,
Export
Export32d,
31dbut
not 31d,
to 4 to 4
4
3
1
2
d
Next-Hop Routing Rules!
• Rule 1: use next-hop rankings
• Rule 2: prioritize current route
• Rule 3: consistently export
– Defn: Node i consistently exports w.r.t.
neighbor j if there is some route R s.t. each
route Q is exportable to j iff R ≤i Q.
– Defn: Node i consistently exports if it
consistently exports with respect to each
neighboring node j.
Next-Hop Routing Rules!
• Rule 1: use next-hop rankings
• Rule 2: prioritize current route
• Rule 3: consistently export
• 3 deployment schemes
– Configure today’s routers
– Create new router configuration interface
– Build new router software
Wish List Revisited
• Loop freedom
•
•
•
•
•
•
•
Security
Incentive compatibility
Business policies
Good performance
Traffic engineering
Scalability
Simplicity
Wish List Revisited
•
•
•
•
•
•
•
•
•
Loop freedom
Fast convergence?
Security
Incentive compatibility
Business policies
Good performance
Traffic engineering
Scalability?
Simplicity
[Feigenbaum-S-Ramachandran]
Existence of Stable State
• Existence of stable state not
guaranteed even with next-hop
rankings (Rule 1) [Feamster-Johari-Balakrishnan]
• Thm: If the next-hop routing rules
hold, then a stable state exists in
the network.
• What about (fast!) convergence?
BGP Oscillations
BGP not guaranteed to converge even
with next-hop routing! [Griffin-Shepherd-Wilfong]
2 > d
1
2
d
1 > d
The Commercial Internet
• ASes sign long-term contracts.
• Neighboring pairs of ASes have:
– a customer-provider relationship
– a peering relationship
peer
providers
peer
customers
Gao-Rexford Framework
• 3 simple conditions that are naturally
induced by the AS-business-hierarchy.
– Topology condition, Preference condition, Export condition
• If the Gao-Rexford conditions hold,
then BGP is guaranteed to converge
to a stable state. [Gao-Rexford]
• But, this might require exponentiallymany forwarding changes! [Syed-Rexford]
Fast BGP Convergence
• Thm: In the Gao-Rexford framework,
next-hop routing convergence to a stable
state involves at most O(L2) forwarding
changes (L = # links).
– all network topologies
– all timings of AS activations and update message
arrivals
– all initial routing states
– all initial “beliefs”
– implications for routing changes and number of BGP
updates
Simulations
• C-BGP simulator. Cyclops AS-level topology, Jan
1st 2010 (33,976 ASes, ~5000 non-stubs)
• Protocols: BGP, Prefer Recent Route (PRR), next-hop
routing
• Metrics: # forwarding changes, # routing changes,
# updates, AS-PATH length
• Events: prefix up, link failure, link recovery
• Methodology: 500 experiments, 10,000 vantage points
(all non-stubs, 5000 stubs)
Simulation Results
(# Forwarding Changes)
maximum number of
routing changes in
next-hop routing = 3
maximum number of
forwarding changes
in PRR = 10
maximum number of
BGP forwarding
changes > 20
Simulation Results
(# Routing Changes)
maximum number of
routing changes in
next-hop routing < 20
maximum number of BGP
routing changes > 160
maximum number of
routing changes
in PRR > 40
Simulation Results
(# BGP Updates, Non-Stub ASes)
maximum number of
updates in next-hop
routing < 300
maximum number of
updates in PRR > 1000
maximum number of BGP
updates > 6000
Simulation Results
(# Routing Changes, The 0.1% Position)
Incentive Compatible
Routing Configurations
3 > d > 1
2
2
1
3
d > 2
d
Each node is getting its best feasible next-hop
Next-Hop Routing is
Incentive Compatible
• Thm [Feigenbaum-Ramachandran-S]: In the
Gao-Rexford framework, next-hop
routing is incentive compatible.
(each node is guaranteed its best
feasible next-hop)
Wish List Revisited
•
•
•
•
•
•
•
•
•
Loop freedom
Fast convergence
Security?
Incentive compatibility
Business policies
Good performance?
Traffic engineering?
Scalability
Simplicity
Limitations of Next-Hop Routing
• AS-PATH length
• AS-avoiding policies
• AS-name prepending
• AS-PATH-based traffic
engineering
Security, Performance,
Traffic Engineering
• Still open research questions.
• Handled mostly outside the routing
protocol.
• We argue that next-hop routing
makes things mostly better.
Performance
• Faster/better convergence than BGP.
• much more scalable.
• But…potential increase in path lengths.
•
b
– loosely correlated with performance
(# routers, physical distance… throughput…).
– still, significant increase clearly undesirable!
– Simulation results: same path length for 9799% of ASes; big increase only for ~0.1%.
Security
• Reduces BGP’s attack surface (AS-PATH
length plays no role in routing decisions).
• More resilient to economically-driven
attacks (incentive compatible).
• More resilient to misconfigurations (in
progress)
• But… AS-avoiding policies impossible.
– come with no guarantees. e2e?
Traffic Engineering
• We discuss how traffic engineering
can be done without relying on the
AS-PATH.
– using different next-hop rankings for different
(groups of) prefixes
– using the BGP communities attribute
–…
Multipath Routing
• Performance, security and traffic
engineering can greatly benefit from
multipath routing.
– multiple working paths
– immediate response to failures
– load balancing among multiple next-hops
– …
• Next-hop routing lowers the barrier for
making this a reality (work in progress).
Multipath Routing
• Exploiting path diversity to
– realize the AS’s own objectives
– customize route selection for
neighboring ASes
• But... multipath routing is not
scalable!
– disseminate and store multiple routes
Multipath Routing is Not Scalable!
I’m using P1
and P2
P1
1
I’m using P1, P2,
Q1 and Q2
4
P2
d
3
Q1
I’m using Q1
and Q2
2
Q2
From AS-PATH to AS-SET
• Next-hop routing is more
amenable to multipath
– nodes don’t care about entire paths
– … other than for loop detection
• Don’t announce routes,
announce sets!
– set = union of ASes on all routes
– BGP route aggregation
Neighbor-Specific
Next-Hop Routing
• Customizing route selection for
neighbors
– operational motivation [Kushman-Kandula-Katabi-Maggs]
– economic motivation [Wang-S-Rexford]
Secure!
Short!
Cheap!
C1
C2
C3
?
z
R1
R2
R3
d
Neighbor-Specific
Next-Hop Routing
• Neighbor-Specific BGP
[Wang-S-Rexford]
– implementable using existing tools
• Results for convergence and
incentive compatibility extend to
multipath!
Conclusions and
Future Research
• BGP is far too complicated!
• New approach: simplify BGP
– without compromising global and local goals!
• Directions for future research:
– getting rid of the AS-PATH?
– software / configuration complexity
– more theoretical and experimental work
Thank You