IP Network Traffic Engineering
Albert Greenberg
Internet and Networking Systems Research Lab
AT&T Labs - Research; Florham Park, NJ
See http://www.research.att.com/~jrex/papers/ieeenet00.ps (to appear in
IEEE Network Magazine, special issue on Internet Traffic Engineering,
March 2000).
Joint work with Anja Feldmann, Carsten Lund, Nick Reingold and Jennifer
Rexford.
1
AGG-NANOG-02-00
IP Network Traffic Engineering


Goal? In operational IP networks, improve performance and make more efficient
use of network resources, by better matching the resources with traffic demands
How? By integrating
– traffic measurement
– network modeling
– selection and configuration of network management and control mechanisms.


Time Scale? Tens of minutes, hours, days, …
Applications?
– Troubleshooting performance problems.
» Why is this link congested?
– Incremental load balancing
» How to tune intradomain (OSPF, IS-IS) routing weights, or interdomain (BGP) import
policies?
– Capacity planning and optimization
» How to estimate facilities cost from forecasted demands and optimal design?


Focus of this talk: ISP backbone networks
(See framework draft of new IETF, Traffic Engineering Working Group)
2
AGG-NANOG-02-00
Traffic Engineering in IP Networks
 Topology
– Connectivity and capacity of routers and links
 Demands
– Expected load between points in the network
 Routing
– Tunable rules for selecting a path for each traffic flow
 Performance objective
– Balanced load, low latency, service level agreements, …
 Question:
Given the topology and traffic demands in an IP
network, how do you decide which routes to use?
3
AGG-NANOG-02-00
Short Answer?




The desired detailed, up to date, network-wide views of the topology
are unavailable
The prevailing traffic demands are unknown
The network doesn’t adapt path selection to the load
The static routes aren’t necessarily optimized to the traffic
These challenges arise because IP networks are
 Decentralized
 Self-configuring
 Connectionless
 Operating in loose confederation with peers
Attributes that contributed to success and dominance of IP
4
AGG-NANOG-02-00
Example: Congested Link
 Detecting that a link is congested
– Utilization statistics every five minutes from SNMP
– Active probes suffer degraded performance
– Customers complain
 Reasons why the link might be congested
– Increase in demand between some set of source-destination pairs
– Failed router/link in our network causes change in our routes
– Failure or policy change in another ISP changes traffic flow
 How
to determine why the link is congested?
 How to relieve the congestion on the link?
5
AGG-NANOG-02-00
Long Answer!




Derive topology from network configuration information
Compute traffic demands from edge measurements
Model path selection achieved by IP routing protocols
Build a query and visualization environment for “what-if” analysis
Reporting
Network
Evolution
Configuration
Debugging
Measurements
Configuration
Performance
Debugging
Information Model
Capacity Planning
Provisioning
6
AGG-NANOG-02-00
Toolkit Architecture
Analysis/Visualization
Important to separate models
from methods and data used to
populate models
Routing Model
Info Model
Configuration
Measurements
7
AGG-NANOG-02-00
Configuration
 Information
– Backbone topology, link capacities, and router locations
– Layer 2 and layer 3 links (e.g., ATM PVCs)
– Intra-domain and inter-domain routing (e.g., OSPF weights)
– Customer location and IP addresses; external IP addresses
– Administrative policies, conventions
 Construct
– Unified views of the network topology, and of customer and peer reachability
– Main sources: router configuration files, forwarding tables
8
AGG-NANOG-02-00
Measurements
 Performance statistics
– Impact of traffic demands on the network
» delay, loss, throughput from active probes between edge systems
» Utilization, loss statistics from passive monitoring of links, nodes
– Mapping of statistics onto the network topology
 Traffic Demands
– An accurate view of the demands themselves is extremely useful for effective
traffic engineering
– A large fraction of the traffic is interdomain, and a large number of customers
are multihomed
» Model traffic demands as loads from an edge interface to a set of candidate
edge interfaces
9
AGG-NANOG-02-00
Information Model
 Abstraction of IP networks
– Different views
» router complexes, router, physical (layer 2), abstract (for routing)
– Objects representing
» routers, links, and traffic demands
– Methods for manipulating objects
» finding and selection of objects
» linkage of objects, e.g., router complexes to routers
» statistics: histogram, tables, etc.
 Salient features
– Captures important global network properties
– Supports routing simulation (e.g., change of OSPF weights)
– Trade off between accuracy and simplicity of model
10
AGG-NANOG-02-00
Visualization of Link Utilization and Delay in
Backbone
Utilization (from passive measurement): link color (high to low)
Delay (from active probes): link width (high to low)
11
AGG-NANOG-02-00
Routing Model

Capture: selection of shortest paths to/from (multihomed) customers
and peers; splitting of traffic across multiple shortest paths;
multiplexing of layer 3 links over layer 2 trunks
Y1
X1
Backbone
Y2
X2
X3
X4
peering
links
Y3
access
links
Y4
Y5
12
AGG-NANOG-02-00
Routing Model (continued)
 Intradomain (OSPF) routing emulator
– Extract backbone topology and link weights
– Compute all shortest paths (Dijkstra’s algorithm)
– Split load evenly along all shortest paths
– Emulates Cisco-style use of multiple routes
0.25
0.25
0.5
1.0
0.25
0.5
1.0
0.25
0.5
0.5
13
AGG-NANOG-02-00
Visualization of Traffic Flow in Backbone
Color/size of node: proportional to traffic to this router (high to low)
Color/size of link: proportional to traffic carried (high to low)
14
AGG-NANOG-02-00
Systems

Configuration
– construction of network topology: layer 2, 3 connectivity, capacity, OSPF weights,
customer and peer IP addresses, router locations

Measurements
– Performance (active – delay, loss, throughput; passive – link and node utilization)
– Traffic demands

Information model
– physical level, IP level, router-complex level, abstract level
– router attributes, link attributes

Routing model
– shortest-path routing, OSPF tie-break, multi-homing, interdomain routing
– bookkeeping to accumulate traffic load on each link

Visualization/analysis environment
– querying to subselect links and nodes; histograms; what-if capabilities
– coloring and sizing to illustrate link and node statistics
15
AGG-NANOG-02-00
Key Ideas

data (configuration, routing, measurement) | models (topology,
demands, routing) | analysis
– Generate accurate global views of the network, and provide mechanisms to infer
network-wide implications of changes in traffic, configuration and control
– Architecture—separate systems for measurement, models, methods to populate models,
analysis
» Can and must evolve with change to underlying infrastructure and network
architecture
– Interfaces for modules above
» E.g., design and optimization (e.g., Bernard Fortz and Mikkel Thorup, "Internet
Traffic Engineering by Optimizing OSPF Weights," Proc. IEEE INFOCOM, March
2000. http://www.ieee-infocom.org/2000/papers/165.ps)

… | (informed) provisioning and reconfiguration
– Closing the loop…
– Improving performance and making more efficient use of network resources
16
AGG-NANOG-02-00