MPLS-Based Hierarchical SDN for Hyper-Scale DC/Cloud
draft-fang-mpls-hsdn-for-hsdc-04
Luyuan Fang, Microsoft
Deepak Bansal, Microsoft
Fabio Chiussi
Chandra Ramachandran, Juniper
Ebben Aries, Facebook
Shahram Davari, Broadcom
Barak Gafni, Mellanox
Daniel Voyer, Bell Canada
Nabil Bitar, Alcatel Lucent
IETF 94, MPLS WG, November 4, 2015
Status
• HSDN architecture has been widely supported from both industry and academia
• It created the foundation architecture for scalability to tens of millions endpoints,
which was not previously achievable
• It is being adopted and applied to different forwarding technologies (L3, L2, L1, IPv4,
IPv6, MPLS, SR)
• The key concept of partition and hierarchy is being applied to the control plane
Reference: “Hierarchical SDN for the Hyper-Scale, Hyper-Elastic Data Center and Cloud,” Proc. ACM
SIGCOMM Symposium on SDN Research, June 2015, http://dl.acm.org/citation.cfm?id=2775009
• Contains the LFIB computation details with ECMP and TE, scalability analysis, and performance data
Benefit of Hierarchical SDN (HSDN)
• Partitioning is crucial for scaling to 10’s of millions endpoints
• HSDN is the architecture for partitioning the DC and DCI
• The principle applies to any forwarding: MPLS, SR, IPv4, and IPv6, L2 or even L1
• The control plane can be implemented with full SDN approach or using BGP-LU
for label distribution (draft-fang-idr-bgplu-for-hsdn-01)
• Two game-changing properties of HSDN
• All paths in the network can be pre-established in the LFIBs (with small LFIBs)
• Labels can identify paths, not just destinations
All Paths are set: support End-to-End Any-to-Any TE and ECMP concurrently
HSDN: Hierarchical Underlay Partitioning
UP0
Level 0
PL0
UPBN1
DC
1
GW
Level 1
SN
PL1
Level 2
PL2
Overlay
Level
VL
UPBN2
1,1
UPBG2 UPBN2
1,1
1,1
UPBN2
1,2
UP2,1,1
UPBN1
DC
1
GW
UPBG1,1
UP1,1
SN
UPBG2 UPBN2
1,2
1,2
DCI/WAN
UPBN1
N
SN
UPBN2
1,3
UP2,1,2
UP1,N
SN
UPBG2 UPBN2
1,3
1,3
UP2,1,M-1
UPBN2
1,M
UPBG2 UPBN2
1,M
1,M
UP2,1,M
ToR
ToR
ToR
ToR
ToR
ToR
ToR
ToR
ToR
ToR
ToR
ToR
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
VM VM
•
One path label per level of underlay partition, plus one VN label
•
Labels are “static,” globally unique within each partition
UPBG2
N,1
UP2
N,1
Example:
•
UPBG1, UPBN1
N
N
UP0 = DCI; UP1s = DCs; UP2s = Clusters  With 3 levels, easily scale to 10’s of millions of endpoints
UPBG2
N,M
UP2
N,M
HSDN Forwarding: The Life of a Packet
UP0
HSDN Label Stack
PL0 PL1 PL2
VL
4
28 37 14 IP Payload
UPBN1
DC
4
GW
PL0
Identifies dest.
UPBN1 or UPBG1
28 37 14 IP Payload
UPBN1
4
UPBG1,4
UPBN1
DC
N
GW
UP1,4
UPBN1
N
UPBG1,N
UP1,N
37 14 IP Payload
PL1
Identifies dest.
UPBN2 or UPBG2
within UP1
PL2
Identifies dest.
server within UP2
VL
UPBN2
4,28
UPBG2 UPBN2
4,28
4,28
UPBN2
4,M
UP2,4,28
ToR
ToR
UPBG2 UPBN2
4,M
4,M
UPBN2
N,1
UP2,4,M
ToR
UPBG2 UPBN2
N,1
N,1
UP2,N,1
ToR
ToR
UPBN2
N,M
UP2,N,M
ToR
ToR
14 IP Payload
VN 14
IP Payload
VM VM
VM VM
VM VM
VM VM
UPBG2 UPBN2
N,M
N,M
VM VM
VM VM
ToR
4
IP Payload
28 37 14 IP Payload
Push VL,
Push PL2,
Push PL1,
Push PL0
Identifies VN
• Route optimization
• Forward a packet from any source to any destination using the same (or less) number of hops as in a flat architecture
and without introducing any additional latency
• “Turn Around” entry to optimize label usage
Next Steps
• Issue -05 draft adding new co-authors and contributors
• Request for WG adoption