Rozšiřitelnost a vysoká dostupnost v L2 sítích Techtorial Jiří Tesař Systems Engineer CCIE #14558 jitesar@cisco.com Sponsor Logo CIscoEXPO Sponsor Sponsor Sponsor Logo Logo Logo © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 1 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 2 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 802.1ah Terminologies BEB: Backbone Edge Bridge – encapsulates customer frames for transmission across backbone. B-BEB: B type BEB – contains a B-component, supports bridging in the provider backbone based on B-MAC and B-TAG info. I-BEB: I type BEB – contains an I-component for bridging in the customer space, including customer MAC, service VLAN IDs. B-TAG: Backbone VLAN Tag – an S-TAG used in conjunction with backbone MAC addresses. I-TAG: Service Instance Tag - encapsulates customer addresses and contains the Service Instance identifier (I-SID). I-SID: Service Instance identifier - A field of the Service Instance tag which identifies the service instance of the frame. S-TAG: A field defined in the 802.1ad Q-in-Q encapsulation which identifies the Service VLAN (S-VLAN). 3 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 802.1ah Terminologies and Interconnections CE MPLS Core PE I Peer 802.1ah PBBN B BEB I IB BEB B I BEB B I B I PE/ BEB MPLS Core P I BEB I type Backbone Edge Bridge CiscoEXPO C S I C-Tagged Interface S-Tagged Interface I-Tagged Interface B BEB B type Backbone Edge Bridge S CE C CE Q C CE Q C B S PEB C C Q C B-Tagged Interface MPLS Interface PE/ BEB PB PEB Q CE IB type Backbone Edge Bridge MPLS PE and Backbone Edge Bridge Provider Bridge (S Bridge) Provider Edge Bridge (C + S) Bridge 802.1Q C Bridge Customer Equipment Cisco Public CE M IB BEB © 2010 Cisco Systems, Inc. All rights reserved. CE PB 802.1Q I BEB M Legend: S S I BEB B B I S B BEB CE PEB PB B B C S S S BCB (PB) B 802.1ad / Q-in-Q PBN B B B BEB Hierarchical B 802.1ah BEB PBBN C 802.1ah PBBN 4 IEEE 802.1ah – Provider Backbone Bridges (PBB) I-TAG: Contains 24 Bits to Identify a Service Instance B-DA B-SA Second MAC-Header B-TAG TPDNRI C-DA C-SA C-TAG B-TAG: Equals S-TAG L2 PDU FCS P802.1ah (Provider Backbone Bridges) Encapsulation Example Service Scalability Define a new “Service Instance Identifier”—24 Bits wide (taking the place of the former “VLAN”): I-SID Domain Isolation, MAC-Address Scalability Encapsulate Customer MAC-frames at the edge of the network into a “Provider MACFrame”: New MAC-Header with B-TAG “Backward Compatibility” to 802.1ad Packet header of Provider Backbone Bridges (PBB, P802.1ah) and Provider Bridges (PB, P802.1ad) look the same 802.1ah assumes existing L2 control plane mechanisms such as spanning tree; however these are not required 5 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 802.1ah I-TAG B-DA I-PCP C-SA C-TAG 4–6 Res2 Ether-Type (0x88-e7) Bits C-DA 3 Res1 1–2 B-TAG TPDNRI I-DEI NCA Octets B-SA L2 PDU 7–12 I-SID FCS 13–18 C-SA C-DA 8 7 6 5 4 3 2 1 802.1ah Frame Format Settled Priority Code Point (I-PCP) Drop Eligible Indicator (I-DEI) No Customer Addresses (NCA) Reserved 1 (Res1) Reserved 2 (Res2) Backbone Service Instance Identifier (I-SID) 6 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Ethernet Encapsulation Evolution C-DA: Customer dest addr C-SA: Customer src addr C-TAG: Customer tag S-TAG: Service tag B-DA: Backbone dest addr S-SA: Backbone src addr I-TAG: Service instance tag VID: VLAN identifier (part of C-/S-/ B-TAG) I-SID: Backbone service instance identifier (part of I-TAG) Service Instances (I-SID) 224=16,777,216 B-DA Service Instances (VID) 12 2 =4,096 B-TAG C-DA C-DA C-DA C-SA C-SA C-DA C-SA S-TAG S-TAG C-SA C-TAG C-TAG C-TAG Payload Payload Payload Payload FCS FCS FCS FCS CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public B-SA I-SID PB: Provider Bridges PBB: Provider backbone bridges I-TAG Service Instances (VID) 212=4,096 802.1Q/ad service Instances (212) 802.1ah service Instances (224) 7 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 8 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 7600 – 802.1ah Line Card Support ES+ ES+ UNI NNI Ingress IB-BEB ES+ or Any DFC UNI NNI ES+ or Any DFC BCB Egress IB-BEB 802.1ah Imposition/Disposition is done on UNI facing ES+ cards NNI Facing Line card 1. 2. CiscoEXPO Any DFC card Adds B-VID © 2010 Cisco Systems, Inc. All rights reserved. 7600 802.1ah LC Requirements Ingress LC Egress LC Native 802.1ah ES+ Any DFC card (Recommend ES+) 802.1ah + MPLS ES+ ES+, ES20, SIP600, SIP400 (Recommend ES+) Cisco Public 9 7600 VLAN Local Significance Support Interface Types ES+ ES20 SIP400 67xx EVC Dot1q Yes Yes Yes N/A EVC QinQ Yes Yes Yes N/A Sub-interface Dot1q Yes No Yes No Sub-interface QinQ Yes Yes Yes N/A VLAN Local Significance does means • VLAN is terminated in the NPU => VLAN lookup, rewrites, etc … are performed in NPU • Same VLAN tag can be used on multiple ports • VLAN tag leaving the port is different to VLAN allocated in internal Database VLAN Local Significance does NOT mean • More than 4000 VLANs are supported for Layer 3 termination 10 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Flexible Forwarding Model P2P XCONNECT EVC to L3/VRF C-BRIDGE C P2P X C ONNE MPLS L3 T CON VFI X NECT B-BRIDGE EFPs L2 B R L2 BRIDGED L2 L2 IDG E D EFPs Local Connect TRUNK Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 11 Flexible Ethernet Edge Example Access port ES Local connect Service instance or Ethernet Flow Point CiscoEXPO AS core interface, L2 trunk or L3 MPLS CS service instance 1 ethernet encapsulation dot1q 20 second-dot1q 10 rewrite ingress tag pop 1 sym bridge-domain 10 c-mac 802.1ah (PBB or .1ah over VPLS service instance 2 ethernet encapsulation dot1q 11-100 rewrite ingress tag push dot1q 101 xconnect 1.1.1.1 1000 en mpls E-LINE (VPWS) service instance 3 ethernet encapsulation dot1q 101 second-dot1q 10 rewrite ingre tag translate 2-to-1 100 bridge-domain 200 Interface vlan 200 xconnect vfi myvpls E-LAN (VPLS or Local bridging) service instance 4 ethernet encapsulation dot1q 102 rewrite ingress tag pop 1 bridge-domain 201 Interface vlan 201 ip address 2.2.2.2 255.255.255.0 ip vrf myvrf L3 termination © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 12 IEEE 802.1ah Control Plane Model IB-BEB I-Component B-Component C-MAC Lookup Function MAC Relay B-MAC Lookup Function MAC Relay EFP I-EFP B-EFP (Physical) (Virtual) (Virtual) Switch Port (Physical) CIP PIP CBP B-MAC Tagging/ I-SID Insertion B-VLAN Re-write/ I-SID Validation PBP Ingress EFP (802.1ah UNI) MAC Tunnel Egress switchport (NNI) int gig1/1 ethernet mac-tunnel virtual abc.com bridge-domain 100 int gig1/2 service instance 15 ethernet encapsulation dot1q 9 second-dot1q 8 bridge-domain 10 c-mac Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. service instance 31 ethernet encapsulation i-sid 10000 bridge-domain 10 c-mac switchport switchport mode trunk switchport allowed vlan 100 13 802.1ah on ES+ NPU Overview 802.1ah is implemented on ES+/7600 for first time on a Cisco platform 802.1ah utilizes both PFC/DFC ASIC and NPU to perform the 2 required layer2 switching decisions for dot1ah Dot1ah on ES+/7600 follows the IB Backbone Edge Bridge model PFC/DFC represents the B-component of the IB-BEB and switches the packet towards the provider backbone port or NNI 802.1ah (NPU) represents the I-component as well as the provider instance port (tunnel engine) and switches the packet towards the customer instance port or UNI PFD/DFC learns Backbone MAC addresses or B-MACs and floods on Backbone VLANs or B-VLANs 802.1ah (NPU) learns Customer MAC addresses or C-MACs and floods on Customer bridge-domains or C-BDs 14 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 15 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 7600 – PBB IB-BEB Logical Flow 7600 Ingress C-MAC1 Egress ISID-1 B-MAC1 Port, 802.1q or 802.1ad (QinQ) EFP Service instances C-MAC2 ISID-2 VLAN local EFP or switchports with the B-VLANs Significance per Port C-MAC3 ISID-3 VLAN tag translation and manipulation C-MAC4 B-MAC2 ISID-4 802.1q/qinq/ 802.1ad PBN PBBN AS Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 16 7600 – PBB IB-BEB Configuration ELAN Service Implementation Step 1 Ingress EFP configuration (UNI) interface TenGigabitEthernet3/1 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP - ELAN Service encapsulation dot1q 100 second-dot1q 1-4094 rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-H-QoS-parent l2protocol forward bridge-domain 100 c-mac interface TenGigabitEthernet3/2 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP - ELAN Service encapsulation dot1q 100 second-dot1q 1-4094 rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-H-QoS-parent l2protocol forward bridge-domain 100 c-mac Step 2 Mac-in-Mac tunnel configuration ethernet mac-tunnel virtual 1 description ** IB-BEB - Mac Tunnel 1 bridge-domain 1000 service instance 1 ethernet description ** ELAN Service - ISID encapsulation dot1ah isid 10000 bridge-domain 100 c-mac Step 3 Egress EFP configuration (NNI) interface TenGigabitEthernet3/3 dot1q tunneling ethertype 0x88A8 service instance 1 ethernet description ** B-VLAN - MAC Tunnel 1 encapsulation dot1q 1000 rewrite ingress tag pop 1 symmetric service-policy output vz-core-queuing bridge-domain 1000 or Egress switchport configuration (NNI) interface TenGigabitEthernet3/3 switchport switchport mode trunk switchport allowed vlan 1000 Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 17 7600 – MPLS + 802.1ah IB-BEB Logical Flow Ingress C-MAC1 Egress (ES +40) ISID-1 VPWS (P2P) Pseudowire B-MAC1 C-MAC2 Port, 802.1q or 802.1ad (QinQ) EFP Service instances ISID-2 MPLS Interface/ Sub-interfaces C-MAC3 ISID-3 VLAN tag translation and manipulation C-MAC4 MPLS Transport Network (H)-VPLS Pseudowire(s) B-MAC2 ISID-4 VFI VPWS (P2P) Pseudowire 802.1q/qinq/ 802.1ad PBN Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. AS 18 7600 – MPLS + PBB IB-BEB Configuration VPWS + 802.1ah Service Implementation Step 1 Ingress EFP configuration (UNI) interface TenGigabitEthernet3/1 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP – VPWS Service encapsulation dot1q 100 second-dot1q 1-4094 rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-H-QoS-parent l2protocol forward bridge-domain 100 c-mac Step 2 Mac-in-Mac tunnel configuration ethernet mac-tunnel virtual 1 description ** IB-BEB - Mac Tunnel 1 bridge-domain 1000 service instance 1 ethernet description ** VPWS Service - ISID encapsulation dot1ah isid 10000 bridge-domain 100 c-mac Step 3 VPWS configuration interface Vlan1000 description ** IB-BEB – VPWS Service xconnect 2.2.2.2 3000 encapsulation mpls Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 19 7600 – MPLS + PBB IB-BEB Configuration VPLS + 802.1ah Service Implementation Step 1 Ingress EFP configuration (UNI) interface TenGigabitEthernet3/1 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP – VPLS Service encapsulation dot1q 100 second-dot1q 1-4094 rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-H-QoS-parent l2protocol forward bridge-domain 100 c-mac interface TenGigabitEthernet3/2 dot1q tunneling ethertype 0x88A8 service instance 100 ethernet description ** UNI EFP - VPLS Service encapsulation dot1q 100 second-dot1q 1-4094 rewrite ingress tag pop 1 symmetric service-policy input vz-ingress-policer service-policy output vz-H-QoS-parent l2protocol forward bridge-domain 100 c-mac Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. Step 2 Mac-in-Mac tunnel configuration ethernet mac-tunnel virtual 1 description ** IB-BEB - Mac Tunnel 1 bridge-domain 1000 service instance 1 ethernet description ** VPWS Service - ISID encapsulation dot1ah isid 10000 bridge-domain 100 c-mac Step 3 VPLS configuration l2 vfi Vz-MAC-Tunnel-1 manual vpn id 3000 neighbor 2.2.2.2 encapsulation mpls neighbor 3.3.3.3 encapsulation mpls interface Vlan1000 description ** IB-BEB – VPLS Service xconnect vfi Vz-MAC-Tunnel-1 manual 20 Flexible Ethernet Edge for .1ah The Cisco implementation will provide for the services mandated by 802.1ah, and will extend them to support all the following offerings: S-Tagged Service Multiplexed: Each S-VID maps to an I-SID. It is possible to retain or pop the STAG. (Retention of S-TAG is an extension of 802.1ah) Bundled (same as 802.lah): Multiple S-VIDs map to an I-SID. The S-TAG must be retained C-Tagged Service (extension of 802.1ah) Multiplexed: Each C-VID maps to an I-SID. It is possible to retain or pop the CTAG. Bundled: Multiple C-VIDs map to an I-SID. The C-TAG must be retained. S/C-Tagged Service (extension of 802.1ah) Multiplexed: Each S-VID/C-VID pair maps to an I-SID. It is possible to retain or pop the S-TAG only or both S-TAG/C-TAG pair. Bundled: Multiple S-VID/C-VID pairs maps to an I-SID. The S-TAG/C-TAG pair must be retained. Port Based Service (same as 802.1ah): All frames are mapped to the same I-SID. All tags, if any, are retained. 21 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public PBB IB-BEB Packet Flow CE Side PBBN Side S-tagged Service - Multiplexed C-DA C-SA S-TAG C-TAG 0x800 Data or B-DA B-SA B-TAG I-TAG C-DA B-DA B-SA B-TAG I-TAG B-DA B-SA B-TAG I-TAG C-DA C-DA C-SA S-TAG C-TAG 0x800 C-SA C-TAG 0x800 Data Data S-tagged Service - Bundled C-DA C-SA S-TAG C-TAG 0x800 Data C-SA S-TAG C-TAG 0x800 Data C-tagged Service - Multiplexed C-DA C-SA C-TAG 0x800 Data or B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 B-DA B-SA B-TAG I-TAG C-DA C-SA B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 0x800 Data Data C-tagged Service - Bundled C-DA C-SA C-TAG 0x800 Data Data S/C-tagged Service - Multiplexed C-DA C-SA S-TAG C-TAG 0x800 Data or or B-DA B-SA B-TAG I-TAG C-DA C-SA S-TAG C-TAG 0x800 B-DA B-SA B-TAG I-TAG C-DA C-SA C-TAG 0x800 B-DA B-SA B-TAG I-TAG C-DA C-SA B-DA B-SA B-TAG I-TAG C-DA B-DA B-SA B-TAG I-TAG 0x800 Data Data Data S/C-tagged Service - Bundled C-DA C-SA S-TAG C-TAG 0x800 Data C-SA S-TAG C-TAG 0x800 Data Port Based Service C-DA Presentation_ID C-SA … © 2006 Cisco Systems, Inc. All rights reserved. C-DA C-SA … 22 MAC Address Scalability in H-VPLS H-VPLS H-VPLS with PBB IP/MPLS IP/MPLS IP/MPLS c-mac c-mac : : : : : : : : : : : : : : : : : : : : : : : c-mac Presentation_ID IP/MPLS c-mac c-mac : : : : : : : : : : : : : : : : : : : : : : : c-mac © 2006 Cisco Systems, Inc. All rights reserved. c-mac c-mac : : : : : c-mac b-mac b-mac : b-mac b-mac b-mac : b-mac c-mac c-mac : : : : : c-mac No customer MAC addresses on N-PE nodes N-PEs only learn backbone MAC addresses imposed by U-PEs 23 Deployment Scenario: H-VPLS extension + 802.1ah 802.1Q CE VPWS/ H- VPLS w/ 802.1ah C-VLAN uPE/ IB-BEB CE VPWS VPWS/ H- VPLS w/ 802.1ah 802.1ad/Q-in-Q VPWS/VPLS uPE/ IB-BEB nPE nPE VPWS VPWS VSI VPWS/VPLS IP/MPLS Core CE BEB I-SID nPE VSI nPE uPE/ IB-BEB I-SID E-Line Service uPE/ IB-BEB E-LAN Service S-VLAN CE 802.1ad/Q-in-Q VSI CE S-VLAN uPE/ IB-BEB I-SID CE CE VPWS CE 802.1ad/ Q-in-Q S-VLAN CE CE S-VLAN MPLS Access Presentation_ID Aggregation © 2006 Cisco Systems, Inc. All rights reserved. Core Transport Aggregation Access 24 Scalability Scalability Factor Scalability Number Total number of EVCs in the system 32000 Total number of EVCs per linecard 16000 Total number of ISIDs in the system Total C-MAC addresses per LC 16M 128000 (32000 per NPU) Total number of EVCs per ISID per NPU 110 Total number of EVCs per ISID for a two port Excalibur 220 Total number of EVCs per ISID for a four port Excalibur 440 Total B-bridge-domains per chassis 4094 Total I-SIDs or MAC-Tunnels 16000 Total entries in a C-MAC table 32000 25 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 26 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Test: Verify PCP Propagation and Queuing Behavior for ELAN Service Using UNI Ingress 2R3C Policer Marking IXIA Sniffer Ten3/3 L2 Link Ten3/1 PE1Ten3/2 IB BEB 2R3C Queuing, etc.. Policer ES2 IXIA C-DA C-SA S-Tag, CoS=1 C-Tag, CoS =2 C payload FCS 1. 2. Ten3/4 MPLS Link On PE1-IB_BEB Ten1/0/1 IXIA Ten1/0/0 ES1 ES3 IXIA Ten3/3 L2 Link Ten3/1 P- PE2- BCB IB-BEB Ten3/2 Ten3/4 MPLS Link B-DA ES4 B-SA IXIA B-Tag, CoS=3,4,5 I-Tag, CoS=3,4,5 C-DA C-SA C-Tag, CoS =2 C-DA C-SA S-Tag, CoS=3,4,5 C-Tag, CoS =2 C payload FCS C payload FCS Ingress card frames are remarked to CoS 5,4,3 by 2R3C Policer Egress card frames are sent to separate queues based on Policer marking for Queuing, etc …. Show end-2-end remarked CoS mapping in PBB frame and Egress PE2-IB_BEB Presentation_ID Sniffer capture to show COS is mapped from S-TagI-TagB-Tag and back to egress S-Tag © 2006 Cisco Systems, Inc. All rights reserved. 27 References IEEE 802.1ah - Provider Backbone Bridges, Draft 4.2, April 2008. “VPLS Interoperability with Provider Backbone Bridges”, draftsajassi-l2vpn-vpls-pbb-interop-04-txt, March 2009. “Extensions to VPLS PE model for Provider Backbone”, Bridging draft-balus-sajassi-l2vpn-pbb-vpls-00.txt, March 2009. “Provider Backbone Bridging and MPLS: Complementary Technologies for Next-Generation Carrier Ethernet Transport”, S. Salam and A. Sajassi, IEEE Communications Magazine, Vol. 46, No. 3, March 2008. “The Evolution of Carrier Ethernet Services – Requirements and Deployment Case Studies’, L. Fang, N. Bitar, R. Zhang, and M. Taylor, IEEE Communications Magazine, Vol. 46, No. 3, March 2008. Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 28 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 29 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public L2 Convergence EVC L2 Convergence - SRD and before MST (BPDU relay, switchport, EVC) REP (switchport) PW Redundancy (MPLS aggregation) Flex-Link Etherchannel/LACP (single-homed devices) Interface-Backup (single-homed devices) New features in SRE MST AG – simplify MST based deployments REP – support for EVC added mLACP – dual homed LACP 30 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 30 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 31 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Why MST Access Gateway? Avoid running full Spanning Tree protocol on NPE – STP is challenging to troubleshoot Terminate multiple Ethernet access rings running MST on NPE without running full STP Each ring can have its own independent topology Isolate topology changes/MAC flushes localized to each ring Improve scalability No full STP processes on NPE routers Maintain existing STP topologies on the access networks Access nodes just speak regular MSTP/RSTP Platform Support ASR9K – since FCS 7600 – 12.2SRE Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 32 MST AG operation SRE MST AG ports send preconfigured BPDU’s with root or zero cost to root information towards access network. Access network sees a loop because of root reachability from both NPE’s. Both NPE’s can send the same information or arbitrarily can be set as best and second best bridge via priority or cost setting for load balancing purposes Root bridge can be one of the NPE’s or arbitrarily set non-existent bridge address MST AG ports are always in Designated state and are forwarding L2 domain runs regular MST protocol. All convergence operations and port state transitioning happen in the access network. Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 33 MST AG TCN Propagation NPE’s snoop and relay TCN from BPDU received from access network NPE’s trigger MAC withdrawal to neighbors TCN is forwarded only to the port within the same MST AG group thus providing L2 domains isolation Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 34 Access Network Failure Scenarios Failure Scenarios 1,2 and 3 cause a primary data path disruption UPE-2 BPDU on Atlernate Port is has now the best BPDU – port transitions to root port role and forwarding state and data path is restored TCN propagates across L2 domain and is relayed between NPE’s NPE’s trigger MAC withdrawal ‘ Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 35 Root Port Recovery When root bridge recovers it starts sending best BPDU towards the access network and convergence to the original path occurs Sending of the best BPDU has to be delayed to allow core convergence; e.g. if the router was reloaded 7600 router runs STP state machine on MST AG when the port is coming from down to up state. The ports is going through LST-LRN-FWD states. To disable this behavior spanning tree port fast has to be configured on MST AG ports. BPDU’s are sent immediately upon port recovery which can cause traffic black-hole if core has not converged. EEM can be used to delay port-up event under certain scenarios. ASR 9K does not run spanning tree state machine and defines a dedicated timer to delay the best BPDU generation. Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 36 NPE isolation failure scenario N-PE isolation occurs if all core facing interfaces are not available resulting in VPLS, Psedowires or L3 connectivity failure N-PE isolation failure is not propagated into access interfaces therefore STP topology remains unchanged, this results in traffic blackhole as access network continues forwarding towards isolated PE An uplink tracking feature is under consideration for future releases Current solution is based on EEM – when router isolation is discovered the access interfaces from redundant networks can be shut down which triggers MST convergence. Upon recovery, timer can be set to delay access links recovery and avoid immediate BPDU sending to the access network. Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 37 Using EEM for Uplink Tracking Backbone uplink on NPE-1 is going down event manager applet Backbone-DN event syslog pattern "%LINK-3-UPDOWN: Interface GigabitEthernet1/40, changed s" action 1.0 cli command "enable" action 1.1 cli command "conf t" action 1.2 cli command "int g1/31" action 1.3 cli command "sh” Backbone uplink on NPE-1 is going up event manager applet Backbone-UP event syslog pattern "Interface GigabitEthernet1/40, changed state to up" action 1.0 cli command "enable" action 1.1 cli command "conf t" action 1.2 cli command "int g1/31" action 1.3 cli command "no sh” Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 38 Special PW failure scenario Special PW failure can be result of PE isolation or a miss-configuration Unlike MST, R-L2GP special PW failure does not cause a loop and therefore does not cause permanent traffic loss because BPDU forwarding topology remains unchanged and is not affected by this failure TCN will not be relayed between the two NPE’s – MAC flush may not happen in a part of L2 domain which may cause temporary traffic loss until MAC aging occurs. Bidirectional traffic will be restored immediately. MAC withdrawal will still be generated by the NPE receiving TCN Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 39 EVC STP Modes Comparison STP mode EVC Support RPVST/ PVST NO BPDU relay only VLAN-STP mapping Per VLAN Dynamic Port State N/A Dynamic BPDU always FWD N/A MST MST AG SRD SRE Single MST Region Single MST Region VLAN to Instance mapping applies to all ports VLAN to Instance mapping applies to all ports YES YES NO Designated ports, always forwarding NO preconfigured BPDU’s YES TCN isolation YES NO TCN forwarded between ports within the same L2GP group MAC Withdrawal NO YES YES Complexity Medium High Low Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. Prone to miss-configuration 40 MST AG configuration steps 1. Configure MST parameters MST AG reuses global MST configuration template to construct BPDU’s. To insure proper MST function, parameters like name, revision and timers should match on other bridges. Note: due to single domain support the same MST parameters will be used on all MST AG groups. In particular IST to VLAN mapping. spanning-tree mode mst spanning-tree mst configuration name c7600 revision 1 instance 1 vlan 3500-3599 spanning-tree mst hello-time 1 spanning-tree mst forward-time 4 spanning-tree mst max-age 6 Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 41 MST AG configuration steps 2. Configure MST AG Pseudo-Information (NPE-1) spanning-tree pseudo-information transmit 1 remote-id 2 ! use the number of pseudo-information of the peer router mst 0-1 root 24576 001e.f7f6.6040 ! root bridge and priority that will be send in BPDU on MST AG ports 3. Assing MST AG Pseudo-Information to a port interface GigabitEthernet1/32 no ip address spanning-tree portfast trunk spanning-tree pseudo-information transmit 1 ! the port will send preconf. BPDU’s as per MST global and pseudo-inf. gr. 1 conf. service instance 3500 ethernet encapsulation dot1q 3500 rewrite ingress tag pop 1 symmetric bridge-domain 3500 Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 42 MST AG configuration steps Configure Special PW for TCN relay This is the same configuration step as for MST BPDU relay interface Vlan1 no ip address xconnect vfi BPDU end NPE-1#sh run | sec BPDU l2 vfi BPDU manual vpn id 1 forward permit l2protocol all neighbor 10.1.1.6 encapsulation mpls Configure Service Instances and Bridge Domains Configure all Egde Ports explicitly with “portfast” feature avoid LRN/LSTN states when bridge is converging Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 43 MST AG configuration validation NPE-1# sh spanning-tree mst 1 ##### MST1 Bridge Root vlans mapped: 3500-3599 address 001e.f7f6.6040 priority this switch for MST1 Interface ---------------Gi1/1 Gi1/32 PW 10.1.1.6:1 Role ---Desg Desg Desg Sts --FWD FWD FWD Cost --------20000 20000 200 32769 (32768 sysid 1) Prio.Nbr -------128.1 128.32 128.55 Type -------------Edge P2p P2p R-L2GP P2p R-L2GP NPE-1#sh spanning-tree pseudo-information 1 configuration Pseudo id 1, type transmit: remote_id 2 mst_region_id 0, port_count 1, update_flag 0x0 mrecord 0x1A6BE02C, mrec_count 2: msti 0: root_id 24576.001e.f7f6.6040, root_cost 0, update_flag 0x0 msti 1: root_id 24577.001e.f7f6.6040, root_cost 0, update_flag 0x0 NPE-1# sh spanning-tree pseudo-information 1 interfaces Pseudo id 1: GigabitEthernet1/32 Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 44 Other useful commands on RP sh spanning-tree mst configuration! sh spanning-tree details! sh vlan id 3500! on SP deb spanning-tree pseudo-information! debug spanning-tree bpdu! deb spanning-tree mstp tc! deb spanning-tree mstp flush !! Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 45 MST AG Restrictions Supported on ES+ and ES20 Applicable to EVC with Bridge Domain only No xconnect, connect or subinterface support No EVC untagged, priority tagged or default encapsulation support Native VLAN is used for BPDU forwarding Single MST region support All MST AG groups share MST Instance - VLAN mapping, name and revision No MST boundary function (for RPVST/PVST/RSTP interoperability) Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 46 7600 MST AG Scale Feature Scale Comment STP Regions 1 All R-L2GP groups have to use common MST configuration; name, version, timers, IST-VLAN mapping MST instances 64 As above R-L2GP groups 256 Ports in No limit R-L2GP group Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. All ports in a chassis can be assigned to a single R-L2GP group 50,000 vport limit per chassis 47 ARS9K MST Access Gateway Interface gig 0/0/0/10.1 l2 encap untagg spanning-tree ring-termination ring1 preempt delay { until <hh:mm:ss> | for <n> { hours | minutes | seconds } } interface GigabitEthernet0/0/0/10.1 name cisco revision 1 bridge-id 0000.0000.0001 instance 0 I’m the root root-id 0000.0000.0001 priority 4096 root-priority 4096 ! instance 1 vlan-ids 101,103,105,107 root-id 0000.0000.0002 priority 8192 root-priority 4096 ! instance 2 I’m the root vlan-ids 102,104,106,108 root-id 0000.0000.0001 priority 4096 root-priority 4096 Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. Access switch configuration interface GigabitEthernet1/1/1 switchport mode trunk spanning-tree mst 0,2 cost 100000 MST root for instance 0,2 VFI VFI MST root for instance 1 Access switch configuration interface GigabitEthernet1/1/1 switchport mode trunk spanning-tree mst 1 cost 100000 Interface gig 0/0/0/10.1 l2 encap untagg spanning-tree ring-termination ring1 preempt delay { until <hh:mm:ss> | for <n> { hours | minutes | seconds } } interface GigabitEthernet0/0/0/10.1 name cisco revision 1 VFI bridge-id 0000.0000.0002 instance 0 root-id 0000.0000.0001 priority 8192 VFI root-priority 4096 ! instance 1 I’m the root vlan-ids 101,103,105,107 root-id 0000.0000.0002 priority 4096 root-priority 4096 ! instance 2 vlan-ids 102,104,106,108 root-id 0000.0000.0001 priority 8192 root-priority 4096 48 Conclusions MST AG provides an appealing option to operate STP networks to service providers: Maintaining access networks without modification Lower maintenance complexity on N-PE’s – no full spanning tree support Lower troubleshooting complexity on the network STP isolation for L2 aggregation domains separated by VPLS core Deterministic root location Improvements from MST/EVC: TCN isolation between access domains More robust implementation, special PW failure does not cause traffic black-hole Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 49 Agenda IEEE 802.1ah Technology and Benefits 7600 Platform 802.1ah Architecture Implementing 802.1ah + VPWS/VPLS Services on 7600 Implementing 802.1ah QoS on 7600 L2 Convergence Overview and Evolution MST Access Gateway Concept mLACP Conclusions 50 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public mLACP SRE Standby POA DHD Virtual LACP Peer Inter-chassis Communication EtherChannel with mLACP Active POA mLACP provides a good mechanism for multi-chassis resiliency DHD is attached to a group of Points of Attachments which look like a single node mLACP appears to DHD as a single 802.3ad LACP POA work in active/standby mode ICC exchanges redundancy information between chassis Links to standby PoA are in hot-standby state Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 51 ICCP Overview ICCP is implemented according to the standard draft-martini-pwe3iccp-00.txt ICCP is an extensible Protocol to synchronize event/states between multiple chassis which are part of the redundant group. ICCP is a reliable protocol which runs over TCP ICCP PDUs are exchanged between Peers to keep the application state consistent across Routers. Control Messages to setup, notify and exchange heartbeats. Data Messages to exchange the application state consistent across the chassis. Ex: LACP Parameters ICCP failure detection ICC Heartbeat Slow (~ 30 sec) /32 Next-hop Tracking Depends on IGP timers BFD ~50 – 150 msec Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 52 Pseudowire Redundancy in SRE VCCV over Primary and Backup PW Preferential Forwarding status bit according to draft-ietf-pwe3-redundancy-bit Upon Receipt of PW switchover status request, The receiver should clear the preferential status forwarding bit and activate the PW. Back up Pseudowire will be preprovisioned in the data plane.But forwarding is disabled. Supported with Scale EoMPLS configuration only. VPLS Redundancy is supported only with mLACP configuration. Supported on ES40,ES20 only. Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 53 mLACP with two sided VPWS/VPLS redundancy MPLS L2 Standby PE1 Standby PW 2 Active PW 1 Standby L1 Active Active PE3 E Active L3 PE2 Active Active PW 3 DHD1 SRE Standby PW 4 L4 PE4 Standby DHD2 Standby VPWS Two PEs form one virtual group on each site, one PE is primary the other is backup PE’s send primary/backup information during PW signaling PW with both sides status <active> are established, others are hot standby MPLS uplinks, attachment circuits and PW status tracking Message exchange within virtual group (for mLACP it is ICC) with redundancy status VPLS PW will be active between PE’s with active access circuits only Single active path through VPLS domain between PE virtual group Similar model applies to REP access Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 54 Pseudowire Redundancy Two-way PE1 IP/MPLS Prim. Primary Pseudowire PE2a Prim. CE1 CE2 ICCP ICCP LACP Back. IP or MPLS PE2b Redundant Pseudowires ICCP = Inter-Chassis Control Protocol LACP Back. LACP = Link Aggregation Control Protocol Failures within MPLS network are protected by MPLS FRR Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy (Note: both sides of the PW are protected) Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for LACP and PW redundancy Synchronization of state (active/standby) between the ACs and PWs Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 55 Pseudowire Redundancy Two-way PE1 IP/MPLS Prim. Primary Pseudowire PE2a Prim. CE1 CE2 ICCP ICCP LACP Back. IP or MPLS PE2b Redundant Pseudowires ICCP = Inter-Chassis Control Protocol LACP Back. LACP = Link Aggregation Control Protocol Failures within MPLS network are protected by MPLS FRR Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy (Note: both sides of the PW are protected) Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for LACP and PW redundancy Synchronization of state (active/standby) between the ACs and PWs Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 56 Pseudowire Redundancy Two-way PE1 IP/MPLS Prim. Primary Pseudowire PE2a Prim. CE1 CE2 ICCP ICCP LACP Back. IP or MPLS PE2b Redundant Pseudowires ICCP = Inter-Chassis Control Protocol LACP Back. LACP = Link Aggregation Control Protocol Failures within MPLS network are protected by MPLS FRR Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy (Note: both sides of the PW are protected) Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for LACP and PW redundancy Synchronization of state (active/standby) between the ACs and PWs Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 57 mLACP CLI Interchassis Redundancy Group redundancy interchassis group 1 member ip 10.12.1.33 ! this is IP-address of opposite end of direct link between NPE-1/2 backbone interface GigabitEthernet1/40 mlacp system-priority 100 mlacp node-id 0 ! monitor peer [bfd | route-watch] Pseudowire Class to reflect or decouple AC and PW status pseudowire-class HS-PW encapsulation mpls status peer topology dual-homed !This command reflects AC circuit status on all PWs Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 58 mLACP CLI (cont) Port-Channel Definition interface Port-channel1 description mLACP no ip address lacp fast-switchover lacp max-bundle 1 mlacp lag-priority 100 mlacp interchassis group 1 service instance 3701 ethernet encapsulation dot1q 3701 xconnect 10.1.1.1 3701 pw-class HS-PW backup peer 10.1.1.4 3701 pw-class HS-PW Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 59 Platform Specifics for mLACP mLACP is SSO Aware mLACP is only supported with EVC configuration. No support for Subinterfaces, Access subinterfaces, Switchport configurations. mLACP is only supported with ES20,ES40 mLACP is not supported with EVC Routed Pseudowire configuration Exception is inter-POA Routed PW use case for VRRP/HSRP Number of chassis part of redundancy group is 2 802.1ah supports only 1 member link on the Port-Channel. mLACP configuration should be active-standby with 1 member link. ASR9K target for mLACP is in release 4.0 Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 60 Attachment Circuit Redundancy Options summary STP based solution can’t provide sub second convergence time and is difficult to support REP is simple, spanning tree free protocol and can coexist with spanning tree topologies. REP integration with EVC in SRE. Etherchannel / LACP provides a good link redundancy scheme for single homed devices, supported with EVC starting from SRC mLACP will provide good redundancy scheme for dual homed devices. Hot-standby PW synchronization with mLACP and REP will be required Presentation_ID © 2006 Cisco Systems, Inc. All rights reserved. 61 Reference 1. LDP Specification - RFC3036 2. Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP) RFC4447 3. PWE3 - RFC 3985 4. Inter-Chassis Communication Protocol (ICCP) to synchronize multi-chassis LACP and PW redundancy state - draft: pwe3-iccp 5. Pseudowire Virtual Circuit Connectivity Verification (VCCV) - RFC5085 6. Bidirectional Forwarding Detection (BFD) for the Pseudowire Virtual Circuit Connectivity Verification (VCCV) draft-ietf-pwe3-vccv-bfd-03 7. Pseudo Wire (PW) OAM Message Mapping draft-ietf-pwe3-oam-msg-map-09 62 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Questions & Answers 63 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 64 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Backup Slides CIscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 65 Pseudowire Operation - Creation IGP = transport infrastructure Targeted LDP for L2VPN PW creation: PW Label Withdrawal It will result in the Label Mapping Message being advertised only if the attachment circuit is active PW Status TLV Mapping for primary and and backup, but using TLV Status for detection 66 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public PW Status 1. Until the SRC: when the AC associated with a PW is down (or being held down for PW redundancy) labels advertised to peers are withdrawn. 2. RFC4447 specifies extensions for LDP which allow PW status to be carried in notification messages to peers. This diverges LDP label mappings from the AC status notification and allows labels to be retained through AC status changes: - as soon as the xconnect is provisioned, - and until the xconnect is unprovisioned or AC interface shutdown. 3. The router can send pseudowire status to a peer router, even when the attachment circuit is down interface Loopback0 ! ip address 10.1.1.1 255.255.255.255 ! ! ! Router# show mpls l2transport vc detail pseudowire-class atomstatus ! … encapsulation mpls ! Last remote LDP TLV status rcvd: AC DOWN(rx,tx faults) status ! ! ! interface GigabitEthernet10/5 ! xconnect 10.1.1.2 123 pw-class atomstatus ! CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public 67 Virtual Circuit Connectivity Verification Pseudowire VCCV Control channel between a pseudowire's ingress and egress points over which connectivity verification messages can be sent Encapsulated using PWE3, follows data paths Control Channel (CC) Types in-band, out-of-band, … Connectivity Verification (CV) Types LSP Ping [RFC4379], ICMP Ping [RFC0792], BFD Can additionally carry fault detection status between the endpoints of the PW Translated into the native OAM status codes used by the native access technologies 68 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public Pseudowire - Detection, Notification 1. VCCV-BFD Connectivity Verification fault detection only 1. LDP status TLV mechanism for AC and PW status and defect notification 1. PW OAM Message Mapping specifies the mapping and notification of defect states between a Pseudo Wire and the Attachment Circuits (AC) of the end-to-end emulated service 69 CiscoEXPO © 2010 Cisco Systems, Inc. All rights reserved. Cisco Public