Why OSPF paths aren’t always shortest
David Applegate
Carsten Lund
Aman Shaikh
AT&T Labs (Research)
NANOG 54
February 06, 2012
Introduction
OSPF is widely used for intra-AS routing
OSPF routes packets along shortest paths
• In terms of weights configured on links
For scalability, OSPF divides domains into areas
• Areas are widely used as well
Areas make OSPF routing complicated
• Paths are no longer shortest
• Other tweaks to areas have increased complexity
– Multi-Area Adjacencies (RFC 5185)
– Multi-Area Routers (RFC 3509)
2
NANOG 54: OSPF Routing with Areas
Outline
Introduction to OSPF
• Basics of OSPF routing
Areas
• How routes are computed with areas
Tweaks to areas
• Multi-area adjacency (RFC 5185)
• Multi-area routers (RFC 3509)
AS Border Routers (ASBRs)
Quiz
3
NANOG 54: OSPF Routing with Areas
OSPF as a Link-State Protocol
With a link-state protocol, every router …
• learns entire network topology
– represents topology as a weighted graph
• computes Shortest Path Tree rooted at itself
OSPF follows this with some tweaks
• Nodes are of two types:
– Transit: routers, subnets
– Stub: prefixes advertized by routers
– Example: /32 for loopbacks
• Path is computed from a source router to a stub
– Via one or more transit nodes
4
NANOG 54: OSPF Routing with Areas
Example
source
451
back1
50
1 9 2 .1 6 8 .0 .1
back2
50
1
50
1 9 2 .1 6 8 .0 .2
50
50
50
375
350
50
50
375
1 9 2 .1 6 8 .0 .3
50
50
access1
access2
1
1 9 2 .1 6 8 .0 .4
451
101
back3
1 9 2 .1 6 8 .0 .5
back4
50
1
1 9 2 .1 6 8 .0 .6
100
50
150
1
350
1
451
50
150
50
access3
1 9 2 .1 6 8 .0 .7
100
100
100
access4
1
1 9 2 .1 6 8 .0 .8
dest
5
1
401
Router
Stub IP,
weight
Link,
weights
NANOG 54: OSPF Routing with Areas
1
151
51
1
Areas
For scalability, OSPF domain is divided in areas
• Areas are numbered 0, 1, 2, …
• Conceptually a hub-and-spoke
– Area 0 is hub, non-zero areas are spokes
Each link and stub is assigned to a single area
• A router can have links in multiple areas
– Such a router is called an area border router (ABR)
– An ABR must have a link in area 0 (RFC 2328)
With areas, every router learns …
• entire topology of areas it has links to
• distance of from ABRs to stubs in remote areas
6
NANOG 54: OSPF Routing with Areas
Example of OSPF Areas
ABR
back1
50
1 9 2 .1 6 8 .0 .1
1
1 9 2 .1 6 8 .0 .2
50
50
50
350
50
50
375
1 9 2 .1 6 8 .0 .3
back3
1 9 2 .1 6 8 .0 .5
50
50
access1
access2
1
1 9 2 .1 6 8 .0 .4
50
350
1
1 9 2 .1 6 8 .0 .6
100
50
150
50
access3
1 9 2 .1 6 8 .0 .7
Color indicates area
(Links and Stubs)
100
100
100
access4
1
ABR
back4
50
1
150
7
1
50
375
ABR
ABR
back2
50
1 9 2 .1 6 8 .0 .8
Black is area 0
NANOG 54: OSPF Routing with Areas
1
1
Path Calculation with Areas
A router ….
• Calculates SPTs for all attached areas
– Leads to intra-area paths to stubs
• Calculates paths to all remote stubs
– Minimize the total distance from itself to the stub
• Total distance = dist(router, ABR) in an attached area +
advertized dist(ABR, stub)
– Leads to inter-area paths to (remote) stubs
8
NANOG 54: OSPF Routing with Areas
Example: Path Calculation with Areas
ABR
d=51
back1
50
1 9 2 .1 6 8 .0 .1
1
50
1 9 2 .1 6 8 .0 .2
50
1
ABR
d=51
1
ABR
d=401
back2
50
50
50
375
350
50
50
50
access1
375
1 9 2 .1 6 8 .0 .3
50
access2
1
1 9 2 .1 6 8 .0 .4
350
1
dest
ABR
d=426
back3
1 9 2 .1 6 8 .0 .5
50
back4
50
1
150
1 9 2 .1 6 8 .0 .6
100
50
150
50
access3
1 9 2 .1 6 8 .0 .7
100
100
access4
1
1 9 2 .1 6 8 .0 .8
source
9
100
NANOG 54: OSPF Routing with Areas
1
Intra-area v/s Inter-area Paths
When a router has to choose between intraarea and inter-area paths, it always chooses
intra-area path
Leads to …
• Sub-optimal paths (within an area)
– Packet takes a longer intra-area path over a shorter
inter-area path
• Area hijacking at ABRs
– Actual path (and the distance) differs from the path
(and distance) calculated by the source router
10
NANOG 54: OSPF Routing with Areas
Area Hijacking Example
ABR
d=51
?
back1
50
1 9 2 .1 6 8 .0 .1
1
50
1 9 2 .1 6 8 .0 .2
50
1
ABR
d=151
d=51
1
ABR
d=451
d=401
back2
50
50
50
375
350
50
50
50
access1
375
1 9 2 .1 6 8 .0 .3
50
access2
1
1 9 2 .1 6 8 .0 .4
350
1
dest
ABR
d=426
back3
1 9 2 .1 6 8 .0 .5
50
back4
50
1
150
1 9 2 .1 6 8 .0 .6
100
50
150
50
access3
1 9 2 .1 6 8 .0 .7
100
100
access4
1
1 9 2 .1 6 8 .0 .8
source
11
100
NANOG 54: OSPF Routing with Areas
1
Multi Area Adjacencies (MADJ)
RFC 5185 allows a link to be in multiple areas
• Stubs can only be in a single area
• Links have a primary area for their interface stubs
Protects against some hijacking cases, but not
all
12
NANOG 54: OSPF Routing with Areas
Area Hijacking Example with MADJ
ABR
d=51
back1
50
1 9 2 .1 6 8 .0 .1
back2
50
1
50
1 9 2 .1 6 8 .0 .2
50
50
1
ABR
d=101
50
375
350
50
50
50
access1
375
1 9 2 .1 6 8 .0 .3
50
access2
1
1 9 2 .1 6 8 .0 .4
350
1
dest
ABR
d=426
back3
1 9 2 .1 6 8 .0 .5
50
back4
50
1
150
1 9 2 .1 6 8 .0 .6
100
50
150
50
access3
1 9 2 .1 6 8 .0 .7
100
100
access4
1
1 9 2 .1 6 8 .0 .8
source
13
100
NANOG 54: OSPF Routing with Areas
1
1
ABR
d=451
Multi Area Routers
dest
b ack 1
1 9 2 .1 6 8 .0 .1
2 0 0
b ack 2
2 0 0
1
1 9 2 .1 6 8 .0 .2
2 0 0
2 0 0
1
b ack 3
1 9 2 .1 6 8 .0 .3
2 0 0
2 0 0
1
b ack 4
1 9 2 .1 6 8 .0 .4
5 0
5 0
5 0
5 0
5 0
5 0
5 0
5 0
1
ABR!!!
ag g 1
1 9 2 .1 6 8 .1 .5
2 0 0
ag g 2
2 0 0
1
1 9 2 .1 6 8 .1 .6
5 0
5 0
5 0
5 0
access1
1 9 2 .1 6 8 .1 .9
ag g 3
1
1 9 2 .1 6 8 .2 .7
1 0 0 0 0
1 0 0 0 0
1
ag g 4
1 9 2 .1 6 8 .2 .8
5 0
5 0
5 0
5 0
5 0
5 0
5 0
access2
1
2 0 0
5 0
1 9 2 .1 6 8 .1 .1 0
source
14
2 0 0
NANOG 54: OSPF Routing with Areas
1
access3
1
1 9 2 .1 6 8 .2 .1 1
1
Multi Area Routers (MAR)
RFC 3509 allows a router to be in multiple nonzero areas without being in area 0
• Protects against dropping traffic
• Specifies subtly different behavior of Cisco and
IBM routers (only)
Leads to more opportunities for area hijacking
15
NANOG 54: OSPF Routing with Areas
Area Hijacking Example with MAR
b ack 1
1 9 2 .1 6 8 .0 .1
2 0 0
b ack 2
2 0 0
1
1 9 2 .1 6 8 .0 .2
2 0 0
2 0 0
1
b ack 3
1 9 2 .1 6 8 .0 .3
2 0 0
2 0 0
1
b ack 4
1 9 2 .1 6 8 .0 .4
5 0
5 0
5 0
5 0
5 0
5 0
5 0
5 0
1
MAR!!!
ag g 1
1 9 2 .1 6 8 .1 .5
2 0 0
ag g 2
2 0 0
1
1 9 2 .1 6 8 .1 .6
5 0
5 0
5 0
5 0
access1
1 9 2 .1 6 8 .1 .9
ag g 3
1
1 9 2 .1 6 8 .2 .7
1 0 0 0 0
1 0 0 0 0
1
ag g 4
1 9 2 .1 6 8 .2 .8
5 0
5 0
5 0
5 0
5 0
5 0
5 0
access2
1
2 0 0
5 0
1 9 2 .1 6 8 .1 .1 0
source
16
2 0 0
access3
1
1 9 2 .1 6 8 .2 .1 1
dest
NANOG 54: OSPF Routing with Areas
1
1
More Area Hijacking
(Stub Area Matters)
ABR
d=351
back1
500
500
back2
19 2.1 68.0.1 1
19 2.1 68.0.2 1
350
200
350
200
agg1
dest
50
50
19 2.1 68.1.3 1
?
d=751
agg2
19 2.1 68.1.4 1
50
50
50
50
50
50
50
50
d=501
access1
access2
19 2.1 68.2.5 1
19 2.1 68.1.6 1
source
17
NANOG 54: OSPF Routing with Areas
ABR
d=251
Importing External Information
External routes can be imported into OSPF
• Example: static routes
Router importing external routes is called an
ASBR (AS Border Router)
• Route contains distance from ASBR to prefix
A router ….
• Calculates SPTs for all the attached areas
• Calculates paths to all remote stubs
• Calculates paths to all external stubs
– Calculates path (and distance) to ASBR and
combines that with dist(ASBR, stub)
18
NANOG 54: OSPF Routing with Areas
Routing to an ASBR
Unusual, because ASBR is not in an area
• ASBR could be reachable in multiple areas
So a router has to calculate per-area path to an
ASBR, and then choose the best path
• Tie-breaking rules depend on whether
RFC1583compatibility is set to disabled
– RFC 1583 is the older OSPF RFC
• Ties broken by
– Least cost
– Highest area number of the link
19
NANOG 54: OSPF Routing with Areas
Example of ASBR routing
source
back1
1 9 2 .1 6 8 .0 .1
200
200
1 9 2 .1 6 8 .0 .2
1
50
50
50
50
50
50
50
50
1 9 2 .1 6 8 .1 .3
1
access2
access1
20
back2
1 9 2 .1 6 8 .1 .4
1
NANOG 54: OSPF Routing with Areas
1
dest
(ASBR)
(stub)
Quiz: What’s the path from source to dest?
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
21
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas
Answer
Depends on …
• What ‘dest’ refers to…
– Choices: r5:192.168.5.0, r5:192.168.5.3 or r5:ASBR
• Whether routers are RFC-3509 compliant or not
– i.e., can r1, r2, and r3 act as true MARs or not
• When RFC-3509 compliant:
– Whether r3 advertises routes learned in green area
into red and blue areas or not
• Vendor dependent
– Cisco and Junipers behave differently
22
NANOG 54: OSPF Routing with Areas
Quiz: What’s the path from source to dest?
dest 192.168.5.0, r1 not RFC3509
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
23
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas
Quiz: What’s the path from source to dest?
dest 192.168.5.0, RFC3509
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
24
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas
Quiz: What’s the path from source to dest?
dest 192.168.5.3, RFC3509, Cisco
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
25
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas
Quiz: What’s the path from source to dest?
dest 192.168.5.3, RFC3509, Juniper
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
26
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas
Quiz: What’s the path from source to dest?
dest r5:ASBR, RFC3509, Cisco
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
27
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas
Quiz: What’s the path from source to dest?
dest r5:ASBR, RFC3509, Juniper
source
r1
1 9 2 .1 6 8 .1 .1
r4
1
1 9 2 .1 6 8 .4 .0
1
5 0
5 0
1 0 0
1 5 0
1 0 0
5 0
r5
1 9 2 .1 6 8 .5 .0
1 9 2 .1 6 8 .5 .3
5 0
1 0 0
1 5 0
5 0
r2
1 9 2 .1 6 8 .2 .1
1
1 9 2 .1 6 8 .3 .1
1 0 0
28
5 0
r3
1 0 0
1
dest
5 0
1 0 0
5 0
1
1
Black = area 0
Red = area 1
Blue = area 2
Green = area 3
NANOG 54: OSPF Routing with Areas