SDN in Carrier Networks
Saurav Das, Guru Parulkar, Nick McKeown
Broadcom
27th October, 2011
Outline
• Problem Statement – 2 networks
• Proposed Solution: Unified Control Architecture
• Prototype & Demonstration to validate

Simplicity & Extensibility compared to existing solution
• Problem Statement – MPLS
• Proposed Solution: SDN based MPLS
Wide Area IP Network
3
4
Logical Link between two Routers over the Wide-Area
Other Clients
Physical
Router Link
Physical
Router Link
TDM Switch
40-160
wavelengths
channels
WDM Line
System
Each channel runs at
10 or 40 Gbps.
100 Gbps coming soon!
Optical Fiber
Other
Clients
WDM Switch
5
IP Network
Transport Network
6
Problem Statement
• Today, IP and Transport networks are separate
• planned, designed and operated separately
• by separate teams
• Owning and operating two separate networks:
inefficient!
• Is there a way to run one network instead of two
separate ones?
7
Eliminate Circuit Switching
All Services
Enterprise
Private -Lines
Private-Nets
Cellular
INTERNET
INTERNET
PSTN
TRANSPORT Network
Is there a need for circuit switching in the Transport Network?
Eliminate Circuit Switching
Fundamental
Packet switching
is more expensive
than Circuit switching
Circuit Switch
Control
Scheduler
Input Linecard
Output Linecard
(λ, t, Port)
(λ’, t’, Port’)
Phy
O/E
Framing
Coding
Err det/corr.
TSI/
(DE) MUX
Phy
Switching Fabric
Circuit Switch
Control
Scheduler
Input Linecard
(λ, t, port)
TSI/
(DE) MUX
Phy
Phy
(pkt., port)
Parse
Look
up
O/E
Framing
Coding
Protocol
Err det/corr.
Output Linecard
(λ’, t’, port’)
Phy
MOD
QoS
Set
Push
Pop
Decr
etc.
Queuing,
Queuing
Sampling
Policing
Mirroring
Phy
(pkt.’, port’)
Scheduler
Hashing
ACLs, Routing,
Policy- Routing
QoS – WFQ, pQ, FIFO
Congestion - RED
Control
Packet Switch
Packet and Circuit Switches
Fiber Switch
WDM Switch
TDM Switch
Packet Switch
Fabric
Mux/Demux
Phy
Phy
Fabric
TSI
Parsing
Fabric
Lookup
Modifications
Fabric
ACLs
Queuing
Policing
Policy Routing
Congestion Avoidance
QoS
Sampling & Mirroring
Hashing
Packet and Circuit Switches
B/w
Glimmerglass IOS600
Fujitsu Flashwave 7500
Ciena CoreDirector
Cisco CRS-1
Fiber Switch
WDM Switch
TDM Switch
Packet Switch
1.92 Tbps
1.6 Tbps
640 Gbps
640 Gbps
Packet and Circuit Switches
Glimmerglass IOS600
Fujitsu Flashwave 7500
Ciena CoreDirector
Cisco CRS-1
Fiber Switch
WDM Switch
TDM Switch
Packet Switch
B/w
1.92 Tbps
1.6 Tbps
640 Gbps
640 Gbps
Power
85 W
360 W
1440 W
9630 W
Volume
7” x 17” x 28”
23” x 22” x 22”
84” x 26” x 21“
84” x 24” x 36”
Price
< 50
110.38
83.73
884.35
Packet and Circuit Switches
Glimmerglass IOS600
Fujitsu Flashwave 7500
Ciena CoreDirector
Cisco CRS-1
Fiber Switch
WDM Switch
TDM Switch
Packet Switch
B/w
1
1
1
1
Power
1 W/Gbps
5
51
332
Volume
1 in3/Gbps
4
41
65
1
3
5
53
Price
$/Gbps
Capex Results
1
59%
Convergence
`
17
Outline
• Problem Statement: want one network, not two!


convergence makes sense.
but packets and circuits must work together
• Proposed Solution: Unified Control Architecture
1.
2.
Common Flow Abstraction
Common Map Abstraction
The Flow Abstraction
Common
Dest
Flow
End – to – End Flow
Flow Identifiers
L4: TCP src/dst port
L3: IP dst
src/dst
prefix
addr,
for IP
China
proto
L2.5:
L2:
19
The Flow Abstraction
Common
Web
traffic
Srcfrom
Flowa Handset
All packets
between 2 routers
Flow Identifiers
What is a Flow?
• Classification of packets that have a logical association
• Action & Maintaining Flow State
• Flow based Accounting & Resource Management
L4: TCP dst port 80
L3: IP src
proto
prefix for branch
L2.5: MPLS Label ID
L2: MAC src
20
1. Common Flow Abstraction
Flow Identifiers
L1:
L0: (p2, p5,
λ5),p7,
(p5,p9)
λ8),
(λ5,
λ5(p7,
λ8,λ3)
λ3)
21
1. Common Flow Abstraction
Flow Identifiers
L1: p3, ts6, num3
L0: p4, ts3, num3
p7, ts9, num3
L0:
22
Circuit Switch
Control
Scheduler
Cross-Connect
Table
(λ, t, port)
(λ’, t’, port’)
TSI/
(DE) MUX
Phy
Phy
Lookup
Phy
Parse
MOD
QoS
(pkt., port)
Phy
(pkt.’, port’)
Lookup
Table
Scheduler
Control
Packet Switch
1. Common Flow Abstraction
L4
L3
L2.5
L2
L1
L0
Packet
Switch
Wavelength
Switch
Multi-layer
Switch
Time-slot
Switch
Packet
Switch
Outline
• Problem Statement: want one network, not two!


3 possible options
But really only one (convergence) makes sense.
• Proposed Solution: Unified Control Architecture
1.
2.
Common Flow Abstraction
Common Map Abstraction
2. Common Map Abstraction
routing, access-control, mobility, traffic-engineering,
guarantees, recovery, bandwidth-on-demand …
Unified Control Plane
Unified Control Architecture
Network Functions
routing, access-control, mobility,
traffic-engineering, guarantees,
recovery, bandwidth-on-demand …
Network - API
2. Common Map
Abstraction
State Collection
State Dissemination &
Application Isolation
Unified
Control
Plane
Built for Performance
Scale & Reliability
Switch - API
1. Common Flow
Abstraction
IP
Router
L4
L3
L2.5
L2
L1
L0
Tables for identifiers and actions
Wavelength
Switch
Multi-layer
Switch
TDM
Switch
Ethernet
Switch
Flow is any
combination
Outline
• Problem Statement: want one network, not two!


3 possible options
But really only one (convergence) makes sense.
• Proposed Solution: Unified Control Architecture
1. Common Flow Abstraction
2. Common Map Abstraction
• Prototype & Demonstration to validate

Simplicity & Extensibility compared to industry-solution
Unified Control Architecture
Network Functions
routing, access-control, mobility,
traffic-engineering, guarantees,
recovery, bandwidth-on-demand …
Network - API
2. Common Map
Abstraction
State Collection
State Dissemination &
Application Isolation
Unified
Control
Plane
Built for Performance
Scale & Reliability
Switch - API
1. Common Flow
Abstraction
IP
Router
L4
L3
L2.5
L2
L1
L0
Tables for identifiers and actions
Wavelength
Switch
Multi-layer
Switch
TDM
Switch
Ethernet
Switch
Flow is any
combination
Implementation of the Architecture
2. Common Map
Abstraction
Unified
Control
Plane
NOX
Interface: OpenFlow Protocol
1. Common Flow
Abstraction
Packet &
Circuit
Switches
Converged Network
30
Prototype
Packet switches
NOX
Hybrid Packet-Circuit Switches
31
Prototype – Emulated WAN
NOX
OpenFlow Protocol
NEW YORK
SAN
FRANCISCO
GE links
OC-48 links
(2.5 Gbps)
HOUSTON
32
Implementation of the Architecture
Application across
packet and circuits
2. Common Map
Abstraction
Unified
Control
Plane
NOX
Interface: OpenFlow Protocol
1. Common Flow
Abstraction
Packet &
Circuit
Switches
Converged Network
33
Example Network Application
Control Function: Treat different kinds of traffic differently
Traffic-type
Delay/Jitter
Bandwidth
Recovery
VoIP
Lowest Delay
Low
Medium
Video
Zero Jitter
High
Highest
Web
Best-effort
Medium
Lowest
Function Impl.: Use both packets and circuits,
at the same time.
VOIP
VOIP
VIDEO
HTTP
HTTP
Video of a Demonstration
of Packet-Circuit Control with OF/SDN
www.openflow.org/videos
35
Why is it Simpler?
Application across
packet and circuits
2. Common Map
Abstraction
NOX
Unified
Control
Plane
1. Common Flow
Abstraction
4700 lines of code
Interface: OpenFlow Protocol
Packet and
Circuit
Switches
Converged Network
36
Why is it Simpler?
GMPLS Control Plane
NOX
OSPF-TE
RSVP-TE
EMS
UNI
EMS
Proprietary Interface
IP/MPLS Control Plane
Interface:EMSOpenFlow Protocol
OSPF-TE
RSVP-TE
Proprietary Interface
Vendor Islands
Transport Network
Converged Network
IP Network
37
Why is it Simpler?
∑ = 175,000+
LOC
GMPLS Control Plane
OSPF-TE
RSVP-TE
EMS
15000!
35000^
UNI
45000^
EMS
Proprietary Interface
IP/MPLS Control Plane
OSPF-TE
RSVP-TE
EMS
35000*
45000#
Proprietary Interface
Vendor Islands
IP Network
Transport Network
Sources: * Quagga
#
Tequila
!
MUPBED
^
DRAGON
38
Why is it Simpler?
4726
175,800 +
Aggr.
68,870
~ 13.5
million
Map
& Bw
Rec.
NOX
Linux kernel
OSPF
RSVP
logic
Quagga base
Linux kernel
OSPF
RSVP
logic
51,828
~ 13.5
million
~ 20
million
IOS or JUNOS
Why isWhy
it the
is Right
it Simpler?
Abstraction?
Application across
packet and circuits
2. Common Map
Abstraction
NOX
Unified
Control
Plane
1. Common Flow
Abstraction
4700 lines of code
Interface: OpenFlow Protocol
Packet and
Circuit
Switches
Converged Network
40
Why is it the Right Abstraction?
∑ = 175,000+
LOC
GMPLS Control Plane
OSPF-TE
RSVP-TE
EMS
15000!
35000^
UNI
45000^
EMS
Proprietary Interface
IP/MPLS Control Plane
OSPF-TE
RSVP-TE
EMS
35000*
45000#
Proprietary Interface
Vendor Islands
IP Network
Transport Network
Sources: * Quagga
#
Tequila
!
MUPBED
^
DRAGON
41
Why is it the Right Abstraction?
∑ = 175,000
LOC
GMPLS Control Plane
OSPF-TE
RSVP-TE
EMS
EMS
Proprietary Interface
15000
35000
45000
IP/MPLS Control Plane
UNI
OSPF-TE
RSVP-TE
EMS
35000
45000
Proprietary Interface
Gold
Silver
Bronze
Vendor Islands
Transport Network
Can’t Specify :
- route,
- or delay,
- or recovery mechanism
- or monitoring/stats
- or priorities
Diffserv based TE +
Policy Based Routing
IP Network
42
Why is it the Right Abstraction?
Extensibility
2. Common Map
Abstraction
NOX
Unified
Control
Plane
1. Common Flow
Abstraction
1.
2.
Full View
Control Function not tied to
Distribution Mechanism
Interface: OpenFlow Protocol
Packet and
Circuit
Switches
Converged Network
43
Outline
• Problem Statement: want one network, not two!


3 possible options
But really only one (convergence) makes sense.
• Proposed Solution: Unified Control Architecture
• Prototype & Demonstration to validate

Simplicity & Extensibility compared to existing solution
• Problem Statement - MPLS
MPLS Services
Why do Service Providers use MPLS?
Really about 2 services
MPLS VPNs
MPLS - TE
Motivation
Motivation
Highly profitable
Deterministic Behavior
No easy way
Efficient Resource Utilization
Older ways not used
Older ways not used
Motivation
MPLS has Flow Abstraction
Flow state in
Head-end LER
Incoming
packets
Classification
Into FECs
Label Edge Router (LER)
LSPs
Label Switched Path (LSP)
MPLS network
IP network
Label Switch
Router (LSR)
Motivation
1. MPLS additional feature on complex core-routers
2. IP/MPLS control exceedingly complex
OSPF-TE
RSVP-TE
LDP
I-BGP
LMP
MP-BGP
Label Switched Path (LSP)
IP/MPLS Control Plane
Distributed
Network
Functions
State Distribution
Mechanisms
PE
Label
Distrib
ution
E-BGP
learned
Route
Advert
VPNIPv4
Route
Advert
TE
Label
Distrib
ution
IGPRoute
Advert,
LinkState
LDP
I-BGP
+ RR
MPBGP
RSVPTE
OSPF
v2
Switch Operating
System
Distributed Network Functions
each with their own
State Distribution Mechanisms
MPLS lacks Map Abstraction
Introducing Map Abstraction in MPLS
Services
TE
Network
Applications
Routing
Discovery
Recovery
NETWORK OPERATING SYSTEM
Simpler
Control Plane
OSPF-TE
RSVP-TE
Simpler
Data Plane
Label
Distribution
LDP
OpenFlow
LMP
I-BGP
MP-BGP
Provide the Services without the Complexity!
Label Switched Path (LSP)
PUSH
SWAP
POP
What is Traffic Engineering?
Steering traffic to where the bandwidth is…
• good for the traffic - less congestion
• good for the network - better resource utilization
MPLS Solution:
• Create tunnels routed over under-utilized parts
of the network
• Route traffic through the tunnels
TE-LSP Features
1. Auto-route
2. Auto-bandwidth
3. Priorities
4. Load-share
5. Diffserv aware Traffic Engineering (DS-TE)
6. MPLS FRR
7. Explicit Routes
8. Re-optimization timers
SDN Approach
Basic Idea
• Retain MPLS data-plane operations
• Replace IP/MPLS control plane
• Demonstrate TE & its features
• All made simpler – some greatly (eg. AutoRoute)
• Some made possible only with SDN (eg. global-optimization)
AutoRoute
R3
R4
R2
R6
R5
R1
IP routing
(SPF)
Link-state: cost, up/down
Static-routes,
PBR/FBF,
Autoroute
TE-LSP
routing
(CSPF)
Link-state: cost, up/down
TE-Link-state: weight,
attributes, reservations
AutoRoute
R3
R4
R2
R5
Automated but
unwieldy – stuck
with decision.
R6
R1
Other approaches
flexible but not
automated
Destination Router
Next-Hop
Total-Cost
Destination Router
Next-Hop
Total-Cost
R4
R4, OutIntf 12
10
R4
R4, OutIntf 12
10
R6
R6, OutIntf 9
10
R6
R6, OutIntf 9
10
R2
R4, OutIntf 12
20
R2
R2, OutIntf T1
20
R2
R6, OutIntf 9
20
SDN based AutoRoute
IP routing
(SPF)
Link-state: cost, up/down
Static-routes,
PBR/FBF,
Autoroute
TE-LSP
routing
(CSPF)
Link-state: cost, up/down
TE-Link-state: weight,
attributes, reservations
Default
SPF
Routing
VoIP
traffic
Routing
Customer
traffic
Routing
IP network
with TE
tunnels
TE-LSP Routing
(CSPF)
Flexibility + Automation =
Programmability
IP network
Controller Internals
Controller
Traffic-type Aware
Routing
Default SPF Routing
Load Sharing
Packet-flow Routing
Applications
Network API
TE-LSP Configuration
Bw. Res. & Priorities
TE-LSP Routing
(CSPF)
TE-LSP Statistics &
Auto-Bandwidth
TE Applications
Network API
GUI
(ENVI)
GUI
API
(LAVI)
SwitchAPI
IP Topology
Link Discovery
Label DB
TE tunnel DB
Packet-flow DB
Map
Abstraction
NOX core
(Connection Handler, Event engine)
To switches..
OpenFlow
protocol
Prototype System
Auto – route; Auto – bandwidth
Traffic – aware LSPs; Priorities
TE-LSP configuration
MPLS-TE
MPLS GUI
GUI (Envi)
showing real-time
network state
MPLS API
CSPF Routing
MPLS Stats
Network Operating System (NOX)
OpenFlow
Open vSwitch
Open
vSwitch
with
standard
Open
vSwitch
Open
vSwitch
MPLS
dataMPLS)
plane
(with
Open
vSwitch
(with
MPLS)
Open
vSwitch
(with
MPLS)
Open
vSwitch
(with
MPLS)
Open
vSwitch
(with
MPLS)
Open
vSwitch
(with
MPLS)
Open
vSwitch
Open
vSwitch
(with
MPLS)
(with
withMPLS)
standard
(with
MPLS)
MPLS data plane
Mininet Environment
Video of a Demonstration
showing MPLS-TE service with SDN/OF
www.openflow.org/videos
58
Providing MPLS Services with SDN/OF
Services / Network
Applications
TE 2.0
VPNs 2.0
Routing
Optimized
FRR/ AutoBw
Discovery
MPLS-TP
Control
Label
Distribution
Multi-layer
Control
Recovery
NETWORK OPERATING SYSTEM
Simpler
Control Plane
OpenFlow
Simpler
Data Plane
PUSH
SWAP
POP
Source: Stuart Elby, Verizon
SDN in Carrier Networks
Reduce TCO – Use circuits or MPLS or
both with IP; and SDN controlarchitecture
Control – Simplicity, Extensibility, Flexible
Automated, Programmatic, and GloballyOptimized
Innovate – Faster pace of Innovation than
today. Differentiate service-offerings from
other carriers.
Software Defined Networks
Thanks!