LTRDCT-3077 Deploying a Cisco Data Center Jeff Lester – Solutions Integration Architect Matt Tarkington – Consulting Engineer Cisco Spark Questions? Use Cisco Spark to communicate with the speaker after the session How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Lab Proctors Matt Tarkington Jeff Lester Consulting Engineer Services Solution Validation Services mtarking@cisco.com Solutions Integration Architect Advanced Services jlester@cisco.com LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Agenda • Introduction • Pod Overview • Hands on Lab 1. 2. 3. 4. 5. 6. 7. Configure VXLAN EVPN Configure UCSM LAN Configure UCSM SAN Configure UCSM Server Configure VXLAN Multi-Pod using NX-API Verify UCS & VXLAN EVPN (Optional) Configure OTV About the Lab About the Lab • Cisco AnyConnect is used to connect to the lab environment • Connect to 64.102.250.58 • Username is ciscolive • Password is C1sc0L1v32017 • Each student has a specific POD for the lab • PODXY • • • RDP X = Pod Number (1-8) Y = Student ID (A or B) RDP is used to access the lab at Cisco Live • IP Address is 10.99.1.1X • Username is podXa or podXb • Password is cisco.123 LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Nexus 7000 Nexus 7000 PSU FAN SU P IOM PSU N7K-Spine-A FAN STATUS FAN SU P IOM N7K-Spine-B FAN STATUS ID ID Cisco Nexus 7004 Pod Topology CONSOLE N7K-SU P2E 1 K LIN 2 RE EJ QU EC ES T T CT EST EJEEQU R 2 CONSOLE MGMT LOG F LASH 1 RE EJ Q EC UES T T T CT ES EJEEQU R 2 MGMT LOG F LASH Cisco Nexus 7004 PORT GROUP 1 3 1 3 4 5 PORT GRO UP 4 6 7 PORT GROUP 6 8 9 10 11 PORT GRO UP 1 3 12 1 STATUS PORT GROUP 1 PORT GROUP 2 PORT GRO UP 3 PORT GRO UP 4 SLOT0 LOG FLASH SLOT0 2 3 4 5 6 7 8 9 10 11 N7K-AC-3KW IN OUT FAU LT ID O 16 A 12 13 14 15 16 17 18 19 20 21 22 23 24 100 -240 V ~ 25 16 A 26 27 28 29 PORT GRO UP 3 2 3 4 5 31 32 33 34 35 7 37 PORT G ROUP 1 38 39 40 41 42 43 44 45 46 47 48 1 16 A 50 /60 Hz N 7K-AC-3KW IN OUT FAULT ID O 2 3 4 5 6 PORT G ROUP 2 7 8 9 10 11 100 -240 V ~ 16 A N7K-AC-3KW IN OUT FAULT ID O I I 50 /60 Hz 100 -240 V ~ 12 13 PORT GROUP 3 14 15 16 17 18 19 20 21 23 16 A 50 /60 Hz I 100 -240 V ~ 1/52 24 F3 25 16 A 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 31 32 33 34 35 36 37 38 39 N 7K-AC-3KW IN 16 A 50 /60 Hz 40 41 42 43 44 45 46 47 48 OUT FAULT ID O 100 -240 V ~ 16 A 50 /60 Hz 1/52 1/9-16 vPC X2 CISCO UCS 6248UP UCS6200-A 1 2 3 4 5 6 7 8 9 10 11 12 13 vPC X4 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 UCS6200-B ID STAT UCS 6200 UCS 6200 1/17-18 1/1-8 1/1-8 PoX6 UCS 5108 4 x B240 M2 Nexus 5500 1 PORT GRO UP 5 30 Nexus 9300 32 1/17-18 CISCO NEXUS 5548 NUP 29 I 100 -240 V ~ 50 /60 Hz 28 N9K-Leaf-B STAT PoX5 27 1/49-50 PoX0 vPC X3 30 31 ID 26 OUT FAULT ID O 1/51 1/49-50 vPC X1 6 12 3/ {2,6,10} 1/9-16 5 11 PORT GRO UP 6 N 7K-AC-3KW IN N9K-Leaf-A 4 10 PORT GROUP 4 22 OUT FAU LT ID O 3/ {1,5,9} 3/ {2,6,10} Nexus 9300 3 9 PORT G ROUP 5 PORT G ROUP 6 100 -240 V ~ 50 /60 Hz 36 I 1/51 2 PO RT GROUP 6 8 PO RT GROUP 2 N7K-AC-3KW IN 3/ {1,5,9} 1 T SE RE PORT GROUP 4 6 ID PORT GROUP 5 30 OUT FAU LT ID O I 50 /60 Hz F3 N7K-AC-3KW IN OUT FAU LT ID O I 100 -240 V ~ CISCO UCS 6248UP T SE RE RE EJ Q EC UES T T 4 1 I RE EJ QU EC ES T T LOG FLASH T CT ES EJEEQU R 2 ET H PORT GROUP 5 4 N7K-AC-3KW IN 1 HOST PO RTS MG MT N7K-F312FQ-25 ID PO RT GROUP 2 T AC T GM E EM IV RM ST CT SY A PW US AT ST CT EST EJEEQU R ET H K LIN SERIAL PORT PORT GROUP 3 2 1 2 MG MT CONSOLE ID ESD T AC HOST PO RTS SERIAL PORT ET H N7K-F312FQ-25 STATUS US AT ST 2 T SE RE SL OT0 K LIN MT E EM MG R ST TIV SY AC PW ID N7K-SU P2E T AC H OST PO RTS SERIAL PORT 1 T SE RE SL OT0 ET H K LIN T GM E EM IV RM ST CT SY A PW ID S U AT ST ESD 1 H OST PO RTS SERIAL PORT CONSOLE N7K-SU P2E T AC MT E EM MG R ST TIV SY AC PW ID S U AT ST N7K-SU P2E Nexus 5500 32 CISCO NEXUS 5548 NUP 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 UCS B440 M1 /M2 N5K-SAN-A 3 ID STAT 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 U C S 5108 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! SLOT SLOT SLOT 1 1 2 2 ! C o nso le R ese t C o nso le R ese t 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ID STAT 1/19 UCS B440 M1 /M2 1/19 N5K-SAN-B 3 ! SL OT ! SL OT 3 SLOT SLOT 1 2 ! SLOT 4 UCS B440 M1 /M2 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! ! SL OT SLOT 5 1 SLOT 2 ! C o nso le SLO T 6 R ese t UCS B440 M1 /M2 ! SL OT 7 SLOT 1 ! C o nso le SLOT SLOT 2 8 R ese t DS-X9708 -K9 DS-X9708 - K9 STATU S OK 10 -G b p s8 -po rt F Co E Mo d u l e 1 2 3 4 LINK 5 6 7 FAIL OK FAIL OK FAIL OK FAIL 8 STAT US DS-X9708 -K9 10 -Gb p s8 -p o rt FCo E Mo d u l e 1 2 3 4 LINK 5 6 7 8 DS-X9708 -K9 STATU S VSAN 901 10 -G b p s8 -p o rt F Co E Mo d u l e 1 2 3 4 LINK 5 6 7 8 ST ATU S D S-X9 2 2 4-9 6 K9 1 /2 / 4/ 8 G b ps 24 -p ort F C M o d ul e 10 -G b p s8 -po rt F Co E Mo du l e D S-X9 2 2 4-9 6 K9 1 1 /2 / 4 /8 G b p s 24 -p ort 2 3 4 LINK 5 6 7 15 16 8 F C M o d ul e ST ATUS LINK 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 STATU S LINK 1 2 3 4 5 6 7 8 9 10 11 12 13 14 17 18 19 20 21 22 23 D S - X9530 -SF 2 - K9 T M M G S E E T IV TU RM A YS CT W ST S A P ET S RE MG MT CO N SO LE 10 /100 LINK COM 1 24 VSAN 902 D S - X9530 -SF 2- K9 USB 1 USB 2 AC T SU PER VI SO R2 MT S E G EM TU IV ST T RM TA SY AC S PW T SE RE MT S M G E TU TE IV T RM TA YS C W S S A P ET ES R MG MT CO N SO L E 10 /100 LINK MG MT C ON SO L E 10/100 LINK COM 1 U SB 1 U SB 2 AC T D S - X9530 -SF 2- K9 SU PER VI SO R2 GMT S M E E TU IV ST T RM A T S SY AC PW T SE RE MG MT CO N SO L E 10 /100 LINK COM 1 AC T D S- X 9530 - SF 2- K9 U SB 1 U SB 2 SU PER VI SO R2 AC T SU PER Core 9506 COM 1 U SB 1 U SB 2 VI SO R2 Core 9506 LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 About the Lab • Web based lab guide • Make sure you select the right POD from the menu • Stick to using Google Chrome • Table Toppers have POD and access information http://dc.ciscolive.com LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 9 About the Lab • Web based lab guide • Make sure you select the right POD from the menu • Stick to using Google Chrome • Table Toppers have POD and access information http://dc.ciscolive.com Pod XY Topology Connection Info LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Lab 1 VXLAN EVPN VXLAN Overview Classic Ethernet Frame 802.1Q Limitations in Layer-2 Data Centers • DMAC SMAC Payload CRC Traditionally VLAN is expressed over 12 bits (802.1Q tag) • Limits the maximum number of segments in a Data Center to 4096 VLANs Destination MAC (DMAC) Source MAC (SMAC) 4 bytes • Etype Spanning-Tree for loop prevention 802.1Q TPID 0x8100 (16 bits) TCI PCP (3 bits) CFI (1 bits) VID (12 bits) Ether Type (Etype) • Inefficient use of links • Slow convergence Data (Payload) CRC/FCS VLAN ID 12 bits TPID = Tag Protocol Identifier, TCI = Tag Control Information, PCP = Priority Code Point, CFI = Canonical Format Indicator, VID = VLAN Identifier LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 VXLAN Overview Enter VXLAN... • Uses Layer 3 Data Center Fabric • No need for Spanning-Tree Protocol • Equal-Cost Multi-Path (ECMP) • Uses IP/UDP header (MAC in IP/UDP) • Divided into Underlay and Overlay(s) • VXLAN leverages the VXLAN Network Identifier (VNI/VNID) field • Total address space of 24 bits • Support of ~16M segments LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 VXLAN Overview Enter VXLAN... • Uses Layer 3 Data Center Fabric • No need for Spanning-Tree Protocol • Equal-Cost Multi-Path (ECMP) • Uses IP/UDP header (MAC in IP/UDP) • Divided into Underlay and Overlay(s) • VXLAN leverages the VXLAN Network Identifier (VNI/VNID) field • Total address space of 24 bits • Support of ~16M segments LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 VXLAN Overview • Enter VXLAN... • Uses Layer 3 Data Center Fabric VXLAN leverages the VXLAN Network Identifier (VNI/VNID) field • Total address space of 24 bits • Support of ~16M segments • No need for Spanning-Tree Protocol • Equal-Cost Multi-Path (ECMP) • Uses IP/UDP header (MAC in IP/UDP) • Divided into Underlay and Overlay(s) 802.1Q header removed 802.1Q 802.1Q DMAC SMAC Etype Payload CRC Classic Ethernet Frame Original Layer 2 Frame VXLAN Frame Outer MAC Outer IP UDP VXLAN DMAC SMAC Etype 14B 20B 8B 8B 14B Payload CRC (new) 4B 20B + 8B +8B + 14B* = 50 Bytes of total overhead LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 Next-Hop MAC Address VXLAN Frame Format Underlay Outer IP Header UDP Header 48 Src. MAC Address 48 VLAN Type 0x8100 16 VLAN ID Tag 16 Ether Type 0x0800 16 Source Port 16 VXLAN Port 16 14 Bytes (4 Bytes Optional) UDP Length 16 Checksum 0x0000 16 VXLAN Header Original Layer-2 Frame IP Header Misc. Data 72 Protocol 0x11 (UDP) 8 Header Checksum 16 Source IP 32 Dest. IP 32 UDP 4789 VXLAN Flags RRRRIRRR 8 Reserved 24 VNI 24 Reserved 8 8 Bytes LTRDCN-3077 20 Bytes Src and Dst addresses of the VTEPs Hash of the inner L2/L3/L4 headers of the original frame. Enables entropy for ECMP Load balancing in the Network. 8 Bytes Overlay 50 (54) Bytes of Overhead Outer MAC Header Src VTEP MAC Address Dest. MAC Address Allows for 16M possible Segments © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 VXLAN EVPN Overview • Workload MAC and IP Addresses learnt by VXLAN Edge Devices (NVEs) • Multi-Protocol BGP (MP-BGP) based Control-Plane using EVPN NLRI (Network Layer Reachability Information) • • Forward based on MAC or IP address learnt via Control-Plane (MP-BGP EVPN) Make Forwarding decisions at VTEPs for Layer-2 (MAC) and Layer-3 (IP); Integrated Route/Bridge (IRB) Bridge – L2 VNI • Route – L3VNI • • Reduce impact of ARP on the Network using ARP Suppression • Leverage Distributed Anycast Gateway capabilities • Host/IP Mobility – Location (VTEP), Identifier (MAC, IP of End-Host) LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 VXLAN EVPN Overview – Control Plane • Cisco’s VXLAN/EVPN does provide automated Route Distinguisher (RD) • Automatic uses Type 1 format • 4-byte IP Address (Router ID) • 4-byte Value (VRF ID) vrf context VRF-A vni 50000 rd auto address-family ipv4 route-target both route-target both address-family ipv6 route-target both route-target both VRF Info Name: VRF-A RD: 15:10.0.0.2 (auto) Name: VRF-A Imp Route-Target RD: 3:10.0.0.1 (auto) RR RR 65500:50000 (auto) Exp Route-Target 65500:50000 (auto) Imp Route-Target 65500:50000 VRF Info (auto) Exp Route-Target 65500:50000 (auto)VRF-A Name: VRF Info RD: 62:10.0.0.3 (auto) Imp Route-Target 65500:50000 (auto) Exp Route-Target 65500:50000 (auto) V2 V1 unicast auto auto evpn unicast auto auto evpn RR V3 LTRDCN-3077 BGP Route-Reflector iBGP Peering © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17 VXLAN EVPN Overview – Control Plane • BGP Advertisement VPN-EVPN: RD:[MAC_A][IP_A] BGP Next-Hop: V1 Route Target: 65500:50000 Label (L3VNI): 50000 Cisco’s VXLAN/EVPN does provide automated Route Target (RT) • 8-byte Route Target (2 x 4-byte) • ASN : VNI vrf context VRF-A vni 50000 rd auto address-family ipv4 route-target both route-target both address-family ipv6 route-target both route-target both RR RR MAC_A / IP_A >> LOCAL Route-Type2 V2 V1 unicast auto auto evpn unicast auto auto evpn MAC_A / IP_A >> V1 Route-Type2 RR V3 BGP Route-Reflector iBGP Peering Host A MAC_A / IP_A LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 VXLAN EVPN – ARP Handling on Lookup “Miss” (1) RR MAC, IP VNI NH MAC_C, IP_C 30001 IP_V3 MAC_Y, IP_Y 30002 IP_V3 RR VNI NH MAC_A, IP_A 30001 IP_V1 MAC_C, IP_C 30001 IP_V3 MAC_Y, IP_Y 30002 IP_V3 V2 V1 Host A MAC_A / IP_A MAC, IP V3 MAC, IP VNI NH MAC_A, IP_A 30001 IP_V1 Host B MAC_B / IP_B Virtual Switch Host C MAC_C / IP_C LTRDCN-3077 Host Y MAC_Y / IP_Y © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 VXLAN EVPN – ARP Handling on Lookup “Miss” (1) RR MAC, IP Missing “B”VNI NH MAC_C, IP_C 30001 IP_V3 MAC_Y, IP_Y 30002 IP_V3 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 2 Host A MAC_A / IP_A ARP Request sent for IP_B sent from Host A MAC, IP VNI NH MAC_A, IP_A 30001 IP_V1 MAC_C, IP_C 30001 IP_V3 MAC_Y, IP_Y 30002 IP_V3 V2 V1 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF 2 1 1 RR V3 MAC, IP VNI NH MAC_A, IP_A 30001 IP_V1 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF Host B MAC_B / IP_B Virtual Switch 2 Miss of IP_B. Forward ARP Request to all Ports except source-port (ARP snooping) Host C MAC_C / IP_C LTRDCN-3077 Host Y MAC_Y / IP_Y © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 VXLAN EVPN – ARP Handling on Lookup “Miss” (2) RR RR MAC, IP VNI NH MAC, IP VNI NH MAC_A, IP_A 30001 IP_V1 MAC_C, IP_C 30000 V3 MAC_C, IP_C 30001 IP_V3 MAC_Y, IP_Y 30001 V3 MAC_Y, IP_Y 30002 IP_V3 V2 V1 ARP Response from IP_B Src MAC: MAC_B Dst MAC: MAC_A 3 Host A MAC_A / IP_A 3 MAC, IP VNI NH MAC_A, IP_A 30000 V1 V3 Host B MAC_B / IP_B ARP Response is sent to V2 Virtual Switch Host C MAC_C / IP_C LTRDCN-3077 Host Y MAC_Y / IP_Y © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 VXLAN EVPN – ARP Handling on Lookup “Miss” (2) RR ARP Response for IP_B Src MAC: MAC_B Dst MAC: MAC_A MAC, IP VNI NH MAC_C, IP_C 30000 30001 V3 IP_V3 MAC_Y, IP_Y 30001 30002 V3 IP_V3 MAC_B, IP_B 30001 IP_V2 V1 RR 4 MAC, IP VNI NH MAC_A, IP_A 30001 IP_V1 MAC_C, IP_C 30001 IP_V3 MAC_Y, IP_Y 30002 IP_V3 V2 ARP Response from IP_B Src MAC: MAC_B Dst MAC: MAC_A 4 3 Host A MAC_A / IP_A 3 V3 MAC, IP VNI NH MAC_A, IP_A 30000 30001 V1 IP_V1 MAC_B, IP_B 30001 IP_V2 Host B MAC_B / IP_B ARP Response is sent to V2 Virtual Switch 4 V2 will populate this information in the control-plane (learn) and forward it subsequently Host C MAC_C / IP_C LTRDCN-3077 Host Y MAC_Y / IP_Y © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 VXLAN EVPN – Packet Forwarding (Bridging) RR RR MAC, IP VNI NH MAC, IP VNI NH MAC_A, IP_A 30001 Local MAC_B, IP_B 30001 Local MAC_B, IP_B 30001 IP_V2 MAC_A, IP_A 30001 IP_V1 V2 V1 Host A MAC_A / IP_A V3 Host B MAC_B / IP_B LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 VXLAN EVPN – Packet Forwarding (Bridging) RR RR MAC, IP VNI NH MAC, IP VNI NH MAC_A, IP_A 30001 Local MAC_B, IP_B 30001 Local MAC_B, IP_B 30001 IP_V2 MAC_A, IP_A 30001 IP_V1 SMAC: MAC_A DMAC: MAC_B V2 V1 SIP: IP_A DIP: IP_B SMAC: MAC_A DMAC: MAC_B SIP: IP_A DIP: IP_B 1 2 Host A MAC_A / IP_A VXLAN VNID: 30001 SIP: IP_A DIP: IP_B Underlay Host B MAC_B / IP_B UDP VXLAN VNID: 30001 Overlay SMAC: MAC_A DMAC: MAC_B SIP: IP_V1 DIP: IP_V2 V3 UDP 4 SMAC: MAC_A DMAC: MAC_B Overlay SIP: IP_V1 DIP: IP_V2 3 SMAC: hop-by-hop DMAC: MAC_V2 Underlay SMAC: MAC_V1 DMAC: hop-by-hop SIP: IP_A DIP: IP_B LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 VXLAN EVPN – Packet Forwarding (Routing) RR RR MAC, IP VNI NH VRF MAC, IP VNI NH VRF MAC_A, IP_A 30001 Local 50001 MAC_A, IP_A 30001 Local 50001 MAC_F, IP_F 30005 IP_V2 50001 MAC_F, IP_F 30005 E1/4 50001 V2 V1 Host A MAC_A / IP_A V3 Host F MAC_F, IP_F LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 VXLAN EVPN – Packet Forwarding (Routing) RR RR MAC, IP VNI NH VRF MAC, IP VNI NH VRF MAC_A, IP_A 30001 Local 50001 MAC_A, IP_A 30001 Local 50001 MAC_F, IP_F 30005 IP_V2 50001 MAC_F, IP_F 30005 E1/4 50001 SMAC: MAC_A DMAC: MAC_GW V2 V1 SIP: IP_A DIP: IP_F SMAC: MAC_GW DMAC: MAC_F SIP: IP_A DIP: IP_F 1 2 Host A MAC_A / IP_A VXLAN VNID: 50001 SIP: IP_A DIP: IP_F Underlay Host F MAC_F, IP_F UDP VXLAN VNID: 50001 Overlay SMAC: MAC_A DMAC: MAC_GW SIP: IP_V1 DIP: IP_V2 V3 UDP 4 SMAC: MAC_GW DMAC: MAC_F Overlay SIP: IP_V1 DIP: IP_V2 3 SMAC: hop-by-hop DMAC: MAC_V2 Underlay SMAC: MAC_V1 DMAC: hop-by-hop SIP: IP_A DIP: IP_F LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 Lab 1 Modules 1. Configure MP-BGP EVPN on the Spines 2. Configure MP-iBGP EVPN on the VTEPs 3. Configure VLAN to VXLAN VNI Mapping on the VTEPs 4. Configure NVE Tunnel Endpoint Interface on the VTEPs 5. Configure VXLAN Bridging on the VTEPs 6. Configure VXLAN Routing on the VTEPs 7. Verify VXLAN Configuration LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 Lab 2 UCSM LAN UCSM LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 UCSM Service Profile Flow LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 UCSM LAN N9K-Leaf-A N9K-Leaf-B Nexus 9300 Nexus 9300 vPC X1 CISCO UCS 6248UP Fabric Ports vPC ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 vPC X2 vPC X3 30 31 32 CISCO UCS 6248UP UCS6200-A STAT B 1/9-12 1/9-12 2 3 4 5 6 7 8 9 10 11 12 13 vPC X4 14 15 16 17 18 19 20 UCS 5108 4 x B240 M2 X2 UCS B440 M1 /M2 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 SL OT 1 1 24 25 26 27 28 29 30 31 32 Fabric Ports vPC A 1/13-16 X3 B 1/13-16 X4 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! SLOT 22 23 UCS 6200 X1 U CS 5108 21 UCS6200-B STAT UCS 6200 A 1 ID ! C on so le R e se t C on so le R e se t ! SLOT SL OT 2 2 UCS B440 M1 /M2 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! SLOT 3 SL OT SLOT 1 2 ! SL OT 4 UCS B440 M1 /M2 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! SLOT 5 SL OT SLOT 1 2 ! C on so le R e se t C on so le R e se t SL OT 6 UCS B440 M1 /M2 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! SLOT 7 SL OT 1 ! OK FAIL OK FAIL OK FAIL OK SLOT SL OT 2 8 FAIL VLANs 11-12,21-22 LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 UCSM LAN N9K-Leaf-A N9K-Leaf-B MAC Pool Nexus 9300 First MAC Address MAC Nexus Pool9300 First MAC Address PODXA-A-MAC-POOL 00:25:B5:0X:AA:00 PODXB-A-MAC-POOL 00:25:B5:0X:BA:00 PODXA-B-MAC-POOL 00:25:B5:0X:AB:00 PODXB-B-MAC-POOL 00:25:B5:0X:BB:00 vPC X1 CISCO UCS 6248UP ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 vPC X2 vPC X3 30 31 32 CISCO UCS 6248UP UCS6200-A 1 2 3 4 5 6 7 8 9 10 11 12 13 vPC X4 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 UCS6200-B ID STAT STAT UCS 6200 UCS 6200 UCS 5108 4 x B240 M2 UCS B440 M1 /M2 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 U CS 5108 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! SLOT SL OT 1 1 ! C on so le ! SLOT SL OT 2 2 R e se t UCS B440 M1 /M2 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! SLOT SL OT SLOT 3 1 SLOT SL OT SLOT 1 2 2 ! C on so le SL OT 4 R e se t UCS B440 M1 /M2 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 5 ! C on so le R e se t C on so le R e se t SL OT 6 UCS B440 M1 /M2 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! SLOT 7 SL OT 1 ! OK FAIL OK FAIL OK FAIL OK SLOT SL OT 2 8 FAIL LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Lab 2 Modules 1. Configure Named VLANs 2. Configure Ethernet Uplink Port-Channels 3. Create MAC Address Pools 4. Create vNIC Templates LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 Lab 3 UCSM SAN UCSM Service Profile Flow LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31 SAN Concepts Legend VNP_PORT VF_PORT VE_PORT FCoE FC zone name PODXA-SP-1 vsan 901 member pwwn 50:0a:09:81:8a:72:87:8D member pwwn 20:0X:00:25:B5:0A:0A:00 zone name PODXA-SP-2 vsan 901 member pwwn 50:0a:09:81:8a:72:87:8D member pwwn 20:0X:00:25:B5:0A:0A:01 zone name PODXA-SP-1 vsan 902 member pwwn 50:0a:09:82:8a:72:87:8D member pwwn 20:0X:00:25:B5:0B:0A:00 zone name PODXA-SP-2 vsan 902 member pwwn 50:0a:09:82:8a:72:87:8D member pwwn 20:0X:00:25:B5:0B:0A:01 No FC Domain ID zoneset name PODX-A vsan 901 member PODXA-A-SP-1 member PODXA-A-SP-2 CISCO UCS 6248UP 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 NPV ID 23 24 25 26 27 28 29 30 31 32 CISCO UCS 6248UP UCS6200-A 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 NPV 28 29 30 31 zoneset name PODX-B vsan 902 member PODXA-B-SP-1 member PODXA-B-SP-2 32 UCS6200-B STAT zoneset activate name PODX-B vsan 902 UCS 5108 4 x B240 M2 Nexus 5500 2 3 UCS 6200 zoneset activate name PODX-A vsan 901 1 2 ID STAT UCS 6200 CISC O NEXU5548UP SN 1 21 22 23 24 25 26 27 28 29 30 31 Nexus 5500 32 CISC O NEXU5548 S NUP 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 U CS B440 M1/M2 ID STAT NPIV N5K-SAN-A 3 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 UC S 5108 SLOT SLOT 1 1 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 WWPN-B WWPN-A ! ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 WWNN ! ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 C o nso le ! SL OT SLOT 2 2 ID STAT NPIV N5K-SAN-B 3 R e se t U CS B440 M1/M2 ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 ! SLOT 3 SLOT SL OT 1 2 WWPN-A SLOT SLOT 5 1 ! WWPN-B ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 ! C o nso le SLOT 4 R e se t U CS B440 M1/M2 ! 2 A G 0 S 03 A 0- S B 30 k G 0 A 01 03 WWNN ! SL OT 2 ! C o nso le R e se t C o nso le R e se t SLO T 6 U CS B440 M1/M2 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 SLOT 7 Eth1/28 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 1 OK 1 2 3 4 LINK 5 6 7 8 DS-X 9708 -K9 STA TUS 10 -Gb p s 8-p o rt FCo E Mod u l e D S-X9 22 4 -9 6 K9 LINK D S- X 9530 - SF 2- K 9 1 2 3 4 LINK 5 6 7 8 NPIV 1 / 2/ 4 / 8 G b p s 2 4 -p o rt F C M o d u le STAT U S 1 2 3 4 MT S M E U T TE TIV RMG TA YS C W S S A P T E ES R T M M E US TE MG IV AT YS R CT W ST S A P ET ES R 5 6 MG MT C ON SO L E 10 /100 LINK MG MT C ON SO L E 10 /100 LINK 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 COM 1 23 24 COM 1 SU PER VI SO ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 WWPN-B WWNN OK FAIL ! SL OT SLOT 2 8 Eth1/28 DS-X9708 -K9 OK FAIL OK FAIL ST ATUS 10-G b ps 8-p o rt FC oE Mo d ule U SB 2 2 R Core 9506 WWPN-B 22 4 -9 6 K9 D S- X 9530 - SF 2- K 9 1 VI SO 3 2 4 3 LINK 4 5 LINK 5 6 6 7 7 8 8 NPIV 1 / 2/ 4 / 8 G bp s 2 4-p o rt F C M o d u le 1 2 3 4 T M E EM MG US V T ST TI R A W ST SY AC P T SE RE T GM E EM US V T ST TI RM TA SY AC S PW T SE E R 5 6 MG MT C ON SO L E 10/100 LINK MG MT C O NSO L E 10/100 LINK 7 8 COM 1 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 VSAN 902 U SB 1 U SB 2 AC T SU PER 2 R D S- X 9530 - SF 2- K 9 ACT SU PER COM 1 USB 1 USB 2 VI SO R2 Core 9506 WWNN FC Domain ID FC Domain ID World Wide Port Name 2 ST ATUS D S-X9 LINK U SB 1 1 DS-X9708 -K9 10-Gb ps 8-p o rt FCo E Mo d u le STATU S VI SO R2 D S- X 9530 - SF 2- K 9 ACT FAIL WWPN-A U SB 1 U SB 2 AC T SU PER 22 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! ST ATUS 10-Gb p s 8-p o rt FCo E Mod u l e VSAN 901 ! WWPN-A SLOT DS-X9708 -K9 World Wide Node Name LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 UCSM SAN Fabric Ports vPC A 1/17-18 X7 CISCO UCS 6248UP 1 2 3 4 5 6 7 8 9 ID 10 11 12 13 14 15 VSAN VSAN 901 902 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 CISCO UCS 6248UP UCS6200-A 2 3 4 5 6 7 8 9 10 11 12 13 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 PO B 1/17-18 X8 32 UCS6200-B UCS 6200 UCS 5108 4 x B240 M2 Nexus 5500 1 2 Ports STAT UCS 6200 CISCO NEXUS 5548 NUP 1 ID STAT Fabric 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Nexus 5500 32 CISCO NEXUS 5548 NUP 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 UCS B440 M1 /M2 N5K-SAN-A 3 ID STAT 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 UC S 5108 SL OT SLOT 1 1 ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 ! WWPN-A ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 ! 2 A G 00 S 3 A B 0- S G 30 k 0 A 01 03 WWPN-B 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 SL OT 3 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 WWNN ! ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 C o nso le R ese t C o nso le R ese t ! SLOT SLOT 2 2 ID STAT UCS B440 M1 /M2 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! SLOT SLOT 1 2 WWPN-A N5K-SAN-B 3 ! ! WWPN-B ! SLOT 4 UCS B440 M1 /M2 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 SL OT ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 5 1 SLOT ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 ! 2 A G 0 S 03 A 0- S B 30 k0 G 00 A 1 3 WWNN SLOT SL OT 7 ! ! SLOT 2 ! C o nso le SLO T 6 R ese t UCS B440 M1 /M2 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 Eth1/28 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 STATU S OK 1 2 3 4 LINK 5 6 7 8 DS-X9708 -K9 STATU S VSAN 901 10 -G b p s8 -p o rt F Co E Mo d u l e D S-X9 2 2 4-9 6 K9 1 1 /2 / 4/ 8 G b ps 24 -p ort 2 3 4 LINK 5 6 7 8 F C M o d ul e ST ATUS LINK 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 ! 2 A G 00 S 30 A B - S G 30 k0 00 A 1 3 WWPN-B ! ! DS-X9708 -K9 10 -G b p s8 -po rt F Co E Mo d u l e ! WWPN-A 1 FAIL WWNN OK FAIL C o nso le SLOT SLOT 2 8 R ese t Eth1/28 DS-X9708 - K9 OK FAIL OK FAIL STAT US 10 -Gb p s8 -p o rt FCo E Mo d u l e 1 2 3 4 LINK 5 6 7 8 DS-X9708 -K9 ST ATU S 10 -G b p s8 -po rt F Co E Mo du l e D S-X9 2 2 4-9 6 K9 1 1 /2 / 4 /8 G b p s 24 -p ort 2 3 4 LINK 5 6 7 15 16 8 F C M o d ul e STATU S LINK 1 2 3 4 5 6 7 8 9 10 11 12 13 14 17 18 19 20 21 22 23 D S - X9530 -SF 2 - K9 T GM S E EM TU IV ST T RM TA SY AC S PW T SE RE MG MT CO N SO LE 10 /100 LINK COM 1 AC T USB 1 USB 2 SU PER VI SO R2 D S - X9530 -SF 2- K9 MT S M G E E TU IV ST T RM TA SY AC S PW T SE E R MG MT CO N SO L E 10 /100 LINK COM 1 AC T SU PER VI SO R2 U SB 1 U SB 2 WWPN-A 24 VSAN 902 D S - X9530 -SF 2- K9 WWPN-B MT M G E US E T IV T AT RM YS T W S S AC P ET S RE MT M G S E TE IV TU RM A YS CT W ST S A P ET ES R MG MT CO N SO L E 10 /100 LINK MG MT C ON SO L E 10/100 LINK COM 1 U SB 1 U SB 2 AC T SU PER VI SO R2 D S- X 9530 - SF 2- K9 AC T SU PER Core 9506 COM 1 U SB 1 U SB 2 VI SO R2 Core 9506 WWNN LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 UCSM SAN Student Name: Assignment Order First WWNN: Size: A PODXA-WWNN-POOL Default 20:0X:00:25:B5:00:0A:00 4 Student Name: Assignment Order First WWNN: Size: B PODXB-WWNN-POOL Default 20:0X:00:25:B5:00:0B:00 4 Student Name: Assignment Order First WWNN: Size: A PODXA-A-WWPN-POOL Default 20:0X:00:25:B5:0A:0A:00 4 A PODXA-B-WWPN-POOL Default 20:0X:00:25:B5:0B:0A:00 4 Student Name: Assignment Order First WWNN: Size: B PODXB-A-WWPN-POOL Default 20:0X:00:25:B5:0A:0B:00 4 B PODXB-B-WWPN-POOL Default 20:0X:00:25:B5:0B:0B:00 4 WWNN Pools WWPN Pools LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 UCSM SAN LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 Lab 3 Modules 1. Configure UCS 6200 Fabric Interconnect VSANs 2. Configure FCoE Port-Channels 3. Create SAN Pools 4. Create vHBA Templates LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 Lab 4 UCSM Server UCSM Service Profile Flow LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 UCSM Service Profiles IPv4 Address block for KVM/CIMC LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 UCSM Service Profiles SAN Boot Policy LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 UCSM Service Profiles UUID Pool Policy LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 UCSM Service Profiles Server Pool Policy LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42 UCSM Service Profiles Service Profile Template LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 UCSM Service Profiles Service Profiles from Template LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 Lab 4 Modules 1. Configure an IP Pool for External Blade Management 2. Configure Primary and Secondary Server SAN Boot Policy 3. Create a UUID Suffix Pool 4. Create a Server Pool 5. Create a Service Profile Template 6. Create Service Profiles from a Service Profile Template LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 Lab 5 VXLAN Multi-Pod using NX-API VXLAN Multi-Pod Overview LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 VXLAN Multi-Pod using NX-API LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 Lab 5 Modules 1. Configure VXLAN Multi-Pod using NX-API 2. Verify VXLAN Multi-Pod LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 Lab 6 Verify UCS and VXLAN EVPN Verify Blade Servers LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 Verify VXLAN Bridging & Routing LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Summary Summary • A CLOS Fabric (Spine/Leaf Architecture) was configured for VXLAN EVPN to extend Layer 2 domains • VLANs were mapped to VXLAN VNIs • VXLAN Bridging and Routing were configured • All the logical components were built for compute services using the UCS BSeries; UCS 5108, UCS 6200 Fabric Interconnects, and Blade Servers • Servers were built with Service Profiles created within the UCSM GUI • Server Blades SAN booted over multi-hop FCoE LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54 Cisco Spark Questions? Use Cisco Spark to communicate with the speaker after the session How 1. Find this session in the Cisco Live Mobile App 2. Click “Join the Discussion” 3. Install Spark or go directly to the space 4. Enter messages/questions in the space cs.co/ciscolivebot#LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public • Please complete your Online Session Evaluations after each session • Complete 4 Session Evaluations & the Overall Conference Evaluation (available from Thursday) to receive your Cisco Live T-shirt • All surveys can be completed via the Cisco Live Mobile App or the Communication Stations Complete Your Online Session Evaluation Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at www.ciscolive.com/global/on-demand-library/. © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public Continue Your Education • Demos in the Cisco campus • Walk-in Self-Paced Labs • Tech Circle • Meet the Engineer 1:1 meetings • Related sessions LTRDCN-3077 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 Thank you