Internet Routing (COS 598A)
Today: Router Configuration
Jennifer Rexford
http://www.cs.princeton.edu/~jrex/teaching/spring2005
Tuesdays/Thursdays 11:00am-12:20pm
Outline
• Individual router
– Router CPU and interfaces
– Links, adjacencies, and sessions
– Filtering and injecting routes
• Network routing design
– Case study of an ISP backbone
– BGP community to convey state
• Current state of the art
– Problems with today’s languages
– Static analysis to detect mistakes
– Template-based automation
Configuring the Router CPU
• Loopback interface
– IP address for accessing the CPU
• Access control for router CPU
– Command-line interface and SNMP set/get
– Id and password, and levels of access permission
• Default parameters
– E.g., TCP parameters
• Services
– E.g., logging, telnet, TFTP, etc.
Access Interfaces: The Basics
• High-level information
– Media (e.g., Serial, Packet-Over-Sonet, and ATM)
– Location (e.g., slot 10, port adaptor 1, port 0)
– Description (i.e., a comment field)
– Capacity (i.e., bandwidth)
• Diverse communication media at layer 2
– Serial link, ATM, packet over SONET, etc.
– Various low-level, media-specific parameters
• Addressing and routing
– IP address and network mask
– Static route to map destination prefix to interface
Access Interfaces: Layer-3 Resource Allocation
• Access control
– Filtering of IP packets
• Based on packet-header bits (e.g., IP addresses, typeof-service bits, port numbers, and protocol)
• Buffer management
– Maximum queue size
– Parameters for Random Early Detection (RED)
• Link scheduling
– Mapping of packets to queues
• Based on packet-header bits
– Allocation of bandwidth resources
• E.g., weights for class-based queuing
Example of Statically-Routed Customer
hostname big-fat-router
interface Loopback0
ip address 12.123.5.240 255.255.255.255
!
interface Serial10/1/0/12:0
description Static Customer
Static route
for 12.34.158.0/23
ip address 12.7.35.1 255.255.255.252
ip access-group 666 in
!
ip route 12.34.158.0 255.255.254.0 Serial10/1/0/12:0
access-list 666 permit 12.34.158.0 0.0.1.255
access-list 666 permit 12.7.35.0 0.0.0.3
Implicit “deny” at the end…
Allow incoming
packets with source
in 12.34.158.0/23
or 12.7.35.0/30
Link Between Two (or More) Routers
• A link is just a (very small) network
– Single subnet (e.g., 12.7.35.0/30)
• IP addresses for each interface
– One on each router
• Special IP addresses
– Broadcast address
– Network address
12.7.35.0/30
12.123.5.240
12.7.108.3
12.7.35.1
12.7.35.2
Intradomain Routing Protocol
• Interface participation
– Enable interface participate in OSPF
– Assign an OSPF weight (and area)
• Link participation
– All interfaces enabled in OSPF, in same area
– Triggers them to establish an adjacency
• Network of OSPF-speakers
– Routers flood the link-state advertisements
– Each router constructs view of the OSPF topology
BGP Session With a Neighbor
• BGP session
– AS number of the neighbor
– TCP connection between two IP addresses
• Reaching the remote end of the connection
– Local router must be able to reach the neighbor router
– Need to be able to route to establish the session!
• Diversity of configurations
– One link vs. multiple links
– Direct connection vs. intermediate components
LR
NR
single access link
LR
NR
multiple access links
LR
NR
firewall,
NAT, etc.
Session With Directly-Connected Interface
• BGP session with the other end of the link
– Single link directly to the neighbor’s router
– Local router knows how to reach the address
AS 7018
interface Serial10/1/0/12:0
LR
ip address 12.7.35.1 255.255.255.252
!
router bgp 7018
12.7.35.1
BGP
session
neighbor 12.7.35.2 remote-as 18585
neighbor 12.7.35.2 <bgp command>…
!
12.7.35.2
NR
AS 18585
Static Routes to Remote BGP Speaker
• BGP session associated with static routes
– Multiple access links to the neighbor’s router
– Or, intermediate components on the path
interface POS7/0
ip address 12.7.35.1 255.255.255.252
LR
!
interface POS8/0
ip address 12.7.45.1 255.255.255.252
!
POS7/0
BGP
session
POS8/0
router bgp 7018
neighbor 12.7.108.3 remote-as 18585
neighbor 12.7.108.3 <bgp command>…
!
ip route 12.7.108.3 255.255.255.255 POS7/0
ip route 12.7.108.3 255.255.255.255 POS8/0
NR
12.7.108.3
Filtering BGP Route Advertisements
• E.g., filter routes not belonging to neighbor
router bgp 7018
neighbor 12.7.108.3 remote-as 18585
neighbor 12.7.108.3 distribute-list CUSTOMER in
!
ip prefix-list CUSTOMER seq 5 12.34.158.0/23
ip prefix-list CUSTOMER seq 15 135.207.0.0/16
• E.g., filter routes for martian IP addresses
router bgp 7018
neighbor 137.39.3.128 remote-as 701
neighbor 137.39.3.128 prefix-list MARTIAN in
!
ip prefix-list MARTIAN seq 5 0.0.0.0/0
ip prefix-list MARTIAN seq 15 10.0.0.0/8 le 32
…
Introducing Routes Into BGP
• Introducing routes into BGP
– Explicit announcement (“network”)
– Redistribution from other protocols
interface Serial10/1/0/12:0
description Static-routed customer
ip address 12.7.35.1 255.255.255.252
ip access-group 666 in
!
router bgp 7018
network 12.34.0.0 mask 255.255.0.0
redistribute static
!
ip route 12.34.158.0 255.255.254.0 Serial10/1/0/12:0
Network Routing Design
Network Routing Design: ISP Backbone
• Inside the network
– OSPF
• Compute paths between routers
– Internal BGP
• Distribute BGP routes inside
• Periphery of the network
– Packet filters
• Block incoming packets at entry points
– Static routes
• Learn how to reach non-BGP customers
– External BGP
• Exchange reachability information with BGP neighbors
Inside the Network: Interior Gateway Protocol
• OSPF: all internal links participate
– Area structure driven by Point-of-Presence
• Backbone area (area 0): inter-PoP links
• Non-backbone areas: intra-PoP links
– Weights driven by topological properties
• Physical distance
• Link bandwidth
20 10
15
10
PoP
Inside the Network: Internal BGP
• iBGP: all routers participate
– Route-reflector hierarchy driven by PoP
• Route reflectors: two RRs per Point-of-Presence
• RR clients: connect to both RRs in their PoP
– Full mesh of top-level route reflectors
PoP
Periphery of the Network: Packet Filters
• Permit valid customer to send (prevent spoofing)
– Permit source addresses belonging to customer
• Permit routing protocols and management to work
– Permit source address of remote BGP speaker
– Permit customer to “ping” your end of the link
• Prevent bogus IP addresses
– Deny source addresses of “martians”
– Deny source address of your own services
• Prevent access to your own equipment
– Deny packets destined to routers from unexpected source
• Prevent unwanted applications/services
– Deny SNMP port number or multicast addresses
– Deny BGP port number from unexpected source
Periphery of Network: External BGP
• External BGP sessions
– Session to each BGP-speaking neighbor
– Import and export policy for each session
• Policy mechanisms
– Filtering: discard route announcements
– Preference: assign high/low preference
– Tagging: mark routes with extra state
• Policy goals
– Business relationships
– Traffic engineering
– Route aggregation
Business Relationships: Assigning Preference
• Prefer-customer policy
– Session with peer
• Import policy: assign local-pref of 80
– Session with customer
• Import policy: assign local-pref of 90
route-map INPEER permit 100
set local-preference 80
!
route-map INCUST permit 100
set local-preference 90
!
Business Relationships: Tagging for Memory
• Tag to remember the type of route
– Session with peer: community 0:1000
– Session with customer: community 0:2000
route-map INPEER permit 100
set local-preference 80
set community 0:1000
!
route-map INCUST permit 100
set local-preference 90
set community 0:2000
!
Business Relationships: Filtering Based on Tag
• Export policy based on tag
– Export all routes to customers
– Export only customer-learned routes to peers
route-map INPEER permit 100
set local-preference 80
set community 0:1000
!
route-map OUTPEER permit 100
match community 0:2000
!
route-map INCUST permit 100
set local-preference 90
set community 0:2000
!
route-map OUTCUST permit 100
match community 0:1000 0:2000
!
Filter routes
from peers
Periphery of Network: Static Routes
• Static route for destination prefix
– Reaching destinations of a non-BGP customer
– Reaching the remote end of eBGP session
• Tagging to limit export of static routes in BGP
– Local to the router: no injection in BGP
• Prefix contained in block allocated to provider’s router
• … and customer connects to Internet in only one place
– Just inside the AS: inject with “no export”
• Prefix contained in provider’s address space
• … and customer connects to only one provider
– Both in and out of AS: inject with no restriction
• Prefix not contained in provider’s address space
• … or customer connects to multiple providers
Complexity of BGP Policies
• Remembering from one router to another
– Business relationship for export policy
– Scope of prefix for route-aggregation
– Geographic location
• Side information unknown to routers
– Business relationship with neighbor AS?
– Customer’s prefix falls in provider’s supernet?
– Customer has multiple access links or providers?
• Intrinsic complexity and diversity of policies
– Business relationships, traffic engineering, route
aggregation, security, etc.
Current State of the Art
Manual Configuration
• Dangerous
– Typo in a routing policy: black hole
– Interfaces in different OSPF areas: no traffic on link
– Missing a packet filter: denial-of-service vulnerability
• Expensive
– Delays in deploying equipment
– Hiring and training skilled engineers
– Lock-in to the router vendor
• Disruptive
– Half of network outages (Yankee Group)
– Failures of Internet services (USITS’03)
– BGP routing anomalies (SIGCOMM’02, NSDI’05)
The Problem With Configuration Languages
• Heterogeneity
– No standard language for the industry
– Differences across products and versions by a vendor
– Change over time due to new protocols and features
• Low-level language
–
–
–
–
Thousands of different commands
Complex inter-relationships between commands
Very little abstraction or nesting
Sometimes no public grammar or parser
• Poor commit semantics
– Command-line interface, one command at a time
– Often no support for atomic actions
Problem With Configuration Languages
• Router-level configuration
– Configuration of individual routers, not the network
– E.g., configure each end of a link separately
• Complexity of distributed protocols
– Some things are hard to do in distributed fashion
– E.g., ensuring complete visibility of routes in iBGP
– E.g., the need for BGP communities to pass state
• Lack of abstractions
– Research emphasis on speed and features
– … not on simplicity, managability, and clean abstractions
– Config languages are no better (though often worse!) than
our abstractions for mechanisms, protocols, and practices
Reducing Configuration Errors
• Configuration checking tools
– Dump the running configuration of all routers
– Parse, join, and query the data
– Generate reports of problems
• Automated configuration
– Templates and rules for generating configuration
– Database for storing the values of variables
– Software to fill templates and apply commands
• Better configuration languages
– Vendor neutral languages for routing policy (RPSL)
– Research on network-wide configuration
Configuration Checking
Configuration Checking: Types of Checks
• Errors: clear mistakes
– Variables used but not defined
– Mismatch between two ends of link or session
• Warnings: risky behavior
– Variables defined but not used
– Dependence on default configuration values
– Violations of best common practices
• Inconsistencies: pattern mismatches
– Same variable defined differently on two routers
– Variables with different names defined same way
• Local policy violations:
– Mismatch with the operator’s explicit intent
Configuration Mistakes: BGP Prefix Filtering
• Martian prefixes
– Goal: block announcements for bogus prefixes
– Variable: prefix-list define martian prefixes
– Instantiation: eBGP sessions apply the prefix-list
• Configuration mistakes
– Error: MARTIAN instantiated but not defined
– Warning: MARTIAN defined but not instantiated
– Inconsistency: MARTIAN defined differently on
different routers
– Local policy violation: MARTIAN not defined as
0.0.0.0/0, 10.0.0.0/8-32, 127.0.0.0/8-32, etc.
Configuration Mistakes: iBGP Configuration
• Internal BGP sessions
– Goal: disseminate reachability info inside the AS
– Mechanism: iBGP sessions between routers
• Configuration mistakes
– Error: Router A has iBGP session to Router B, but
B does not have an iBGP session to A
– Warning: iBGP session between non-Loopback
addresses
– Inconsistency: all routers have two route
reflectors (RRs), except one
– Local policy violations: all client routers should
have two RRs; route RRs should form a full mesh
Automated Configuration
Automated Configuration System
Technical Questions (TQ)
What is your AS
number?
What export policy do
you want?
Do you want a dynamic
default?
What are your address
blocks?
Do you need to receive
communities?
DB
interface <name>
description <cust name>
ip address <addr> <mask>
ip access-group <acl> in
!
router bgp 7018
neighbor <ip> remote-as <asn>
neighbor <ip> route-map CUST-FACE in
neighbor <ip> route-map <outmap> out
neighbor <ip> distribute-list <racl> in
neighbor <ip> soft-reconfiguration-inbound
[neighbor <ip> send-community]
!
query
R
U
L
E
S
interface Serial10/1/0/12:0
description CBB Customer
ip address 12.7.35.1 255.255.255.252
ip access-group 666 in
!
router bgp 7018
neighbor 12.7.35.2 remote-as 18585
neighbor 12.7.35.2 route-map CUST-FACE in
neighbor 12.7.35.2 route-map FULL-ROUTES out
neighbor 12.7.35.2 distribute-list 13 in
neighbor 12.7.35.2 soft-reconfiguration-inbound
!
configlet
template
• Automation through template and database
• … but doesn’t raise the level of abstraction
• So building these kind of systems is challenging
router
Automation Stages
• Initializing a new router
– Base configuration to load on the router
– E.g., access control for the router CPU
– E.g., enabling of services (e.g., telnet, logging)
– E.g., defining MARTIAN lists and routing policies
• Changing the configuration
– Use cases for different activities
– … with TQ, database variables, and template
– E.g., “add link,” “add BGP-speaking customer,” or
“move static-route customer to BGP customer)
– Generates a configlet applied to the router
Conclusion
• Router configuration is hard
– Low-level configuration languages
– Wide variety of protocols and mechanisms
• Improving on manual configuration
– Configuration-checking tools
– Automated configuration systems
– Research needed on languages and abstractions
• Possibility of auto-configuration?
– Specify high-level network design
– Ship a router and plug it in to the network
– Router auto-configured from a server
Next Time: Removing Routing from Routers
• Three short papers
– “Routing as a Service”
– “The Case for Separating Routing From Routers”
– “Network-Wide Decision Making: Toward a Wafer-Thin
Control Plane”
• Review just of the first paper
–
–
–
–
Summary
Why accept
Why reject
Directions for future work
• Plan for a discussion-driven class on Thursday
• Reminder: no class next week!!!