vPC Best Practices and Design
on NX-OS
Nemanja Kamenica (nkamenic@cisco.com)
Engineer, Technical Marketing
BRKDCT-2378
Session Goal
•
To provide a thorough understanding of the Virtual Port Channel, design and
best practices for configure of vPC.
•
This session will examine best practice of use of vPC in different environments,
with firewall, with Nexus 2000, in routed environment, and in FCoE, FabricPath,
VxLAN, and ACI environment.
•
This session will not examine in depth FCoE, Fabric Path, VxLAN, ACI, Nexus
2000, and firewalls.
3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
3
Agenda
•
Introduction to vPC
•
Feature Overview
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
Virtual Port Channel - vPC
Benefits
•
MC-LAG on Cisco Nexus Devices
•
Provides device level redundancy with faster
convergence
•
Eliminates Spanning Tree blocked ports by
providing loop-free topology
•
Better bandwidth utilization
•
Deployed by almost 95% of Nexus
customers
Unified Fabric
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
5
vPC Feature Overview
vPC Concept & Benefits
S1
S2
S1
S3
STP
•
•
S2
S2
S1
S3
vPC Physical Topology
S3
vPC Logical Topology
No Blocked Ports, More Usable Bandwidth, Load Sharing
Fast Convergence
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
6
Data Center Technology Evolution
MPLS, OTV,
LISP
MPLS, OTV,
LISP
ACI
VXLAN
FabricPath with vPC+
FEX with vPC
VPC
2014-2015
STP
2013-2014
2010
2010
2009
2008
Used to redundantly connect network entities at the edge of the Fabric
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
7
Agenda
•
Introduction to vPC
•
Feature Overview
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
Feature Overview
L3
vPC Terminology
Layer
3 Cloud
P
vPC
Peer
Orphan
Port
vPC Peer
Keepalive Link
S
Peer-Link
vPC Domain
S1
S2
CFS
vPC Member
Port
vPC
Orphan
Device
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
9
vPC Peer-keepalive link
L3
•
L3 link, connects vPC peers
•
Carries period hart beet between vPC peers
•
Uses UDP port 3200
•
Sends Keepalive heart beets every 1 s
vPC Domain
S1
S2
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
10
vPC Peer-link
L3
•
vPC peer link is a port channel that carries:
•
vPC VLANs
• CFS messages
• Flooded traffic from the other peer device
• STP BPDUS, HSRP hello messages and IGMP
updates
vPC Domain
S1
S2
•
vPC imposes the rule that peer-link should
never be blocking
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
11
vPC
L3
•
Consists of port-channel member of vPC
•
L2 port channel
•
Ports in vPC can be in access or trunk mode
•
VLANs allowed on vPC need to be allowed on
peer-link
•
vPC Domain
S1
S2
LACP and Static port channel configuration
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
12
Cisco Fabric Services Protocol
L3
•
Synchronization and consistency checking
mechanism
•
Runs on VPC Peer-link
•
CFS protocols mechanism:
•
•
•
•
•
•
vPC Domain
Validation and comparison for consistency check
S1
Synchronization of MAC addresses for member ports
Status of member ports advertisement
STP management
Synchronization of HSRP and IGMP snooping
S2
CFS
S3
Enabled by default
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
13
vPC Consistency check
•
System configuration must be in sync
•
Type 1 Consistency Check
•
Graceful Consistency check – suspends:
•
•
•
•
Per-interface inconsistent parameters – vPC member ports on secondary peer set to down state
Globally inconsistent parameters – misconfigured member ports on secondary peer suspended
Parameters: STP mode, STP VLAN state, STP global settings, LACP mode, MTU…
Type 2 Consistency Check
•
Forwards traffic in case of inconsistency
• Possible undesirable traffic forwarding behavior
• Parameters: VLAN interface (SVI), ACL, QOS, IGMP snooping, HSRP…
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
14
Agenda
•
Introduction to vPC
•
Feature Overview
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
vPC Configuration Best Practices
L3
Building a vPC domain – Configuration Steps
1. Define domains
2. Establish Peer Keepalive connectivity
3. Create a Peer link
S2
S1
CFS
4. Create vPCs
5. Make Sure Configurations are Consistent
(Order does Matter!)
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
16
vPC Configuration Best Practices
vPC Domain-ID
•
•
The vPC peer devices use the vPC domain ID to
automatically assign a unique vPC system MAC
address
vPC Domain 10
S1
S2
You MUST use unique Domain id’s for all vPC
pairs defined in a contiguous layer 2 domain
vPC Domain 20
S3
S4
! Configure the vPC Domain ID – It should be unique within the layer 2
domain
NX-1(config)# vpc domain 20
! Check the vPC system MAC address
NX-1# show vpc role
<snip>
vPC system-mac
: 00:23:04:ee:be:14
S5
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
17
vPC Configuration Best Practices
vPC Peer-Keepalive link
Preference
Recommendations
(in order of
preference):
Nexus 7X00 /
9500 series
Nexus 9300 /6000 /
5X00 / 3X00 series
1
Dedicated link(s)
(1GE/10GE LC)
mgmt0 interface
2
mgmt0 interface
Dedicated link(s)
(1GE/10GE LC)
3
L3 infrastructure
L3 infrastructure
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
18
vPC Configuration Best Practices
vPC Peer-Keepalive link – Dual Supervisors
Management Switch
•
When using dual supervisors and mgmt0 interfaces
to carry the vPC peer-keepalive, DO NOT connect
them back to back between the two switches
•
Only one management port will be active a given point
in time and a supervisor switchover may break keepalive connectivity
•
Use the management interface when you have an outof-band management network (management switch in
between)
BRKDCT-2378
Management
Network
vPC_PKL
vPC_PKL
vPC_PL
vPC1
vPC2
Standby Management Interface
Active Management Interface
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
19
vPC Configuration Best Practices
vPC Peer-Link
S1
S2
S2
S1
S3
•
•
•
•
S3
vPC Peer-link should be a point-to-point connection
Peer-Link member ports can be 10/40/100GE interfaces
Peer-Link bandwidth should be designed as per the vPC
vPC imposes the rule that peer-link should never be blocking
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
20
Design Best Practices
Mixed Hardware across vPC Peers : Line Cards
•
Always use identical line cards on either sides of the peer link and vPC member ports !
Examples:
vPC Primary
vPC Secondary
vPC Primary
vPC Peer-link
S1
N7700
F2E
vPC Peer-link
S2
N7700
F2E
F3
vPC Secondary
S1
M1
F3
F3
vPC
S2
M2
F3
vPC
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
21
Design Best Practices
Mixed Hardware across vPC Peers : Nexus 9500
X
vPC Primary
vPC Secondary
Y
N9K-X9636PQ
N9K-X9432PQ
N9K-X9564PX
N9K-X9464PX
N9K-X9564TX
N9K-X9464TX
N9K-X9536PQ
N9K-X9736PQ
vPC
vPC Peer-link
S1
N9500
X
S2
N9500
Y
X
Y
vPC
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
22
Design Best Practices
Mixed Hardware across vPC Peers : Chassis & Supervisors
•
•
•
•
N7000 and N7700 in same vPC Construct -Supported
VDC type should match on both peer device
vPC peers can have mixed SUP version* (SUP1, SUP2, SUP2E)
N5500 and N5600 in same vPC Construct –Not Supported
vPC Primary
vPC Secondary
vPC Primary
vPC Peer-link
S2
N7700
S1
N7000
S1
N5500
vPC
vPC Secondary
vPC Peer-link
S2
N5600
vPC
*Recommended only for short period such as migration
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
23
vPC Configuration Best Practices
vPC Loop Avoidance
•
Data plane loop control
•
vPC peer forwards traffic locally when possible
•
Traffic coming from vPC member port, crossing
Peer-link is NOT allows to egress any vPC
member port
•
Exception of the rule, when member port goes
down
S3
S2
vPC 1
S1
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
24
vPC Configuration Best Practices
Spanning Tree (STP)
STP is running to manage
loops outside of vPC domain,
or before initial vPC
configuration !
S1
S2
S4
S3
S5
•
•
All switches in Layer 2 domain should run either Rapid-PVST+ or MST
Do not disable spanning-tree protocol for any VLAN
•
Always define the vPC domain as STP root for all VLANs in that domain
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
25
vPC Configuration Best Practices
vPC Peer-switch
P
Without Peer-switch:
• STP for vPCs controlled by vPC primary
• vPC primary send BPDU’s on STP designated ports
• vPC secondary device proxies BPDU’s to primary
Nexus(config-vpc-domain)# peer-switch
S
BPDUs
P
S
With Peer-switch:
• Peer-Switch makes the vPC peer devices to appear as
single STP root
• BPDUs processed by the logical STP root formed by the
2 vPC peer devices
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
26
Hybrid topology (vPC and non-vPC)
STP Root
VLAN 1
VLAN 2
STP Root
VLAN 1
Bridge Priority
VLAN 1  4K
VLAN 2  8K
vPC Primary
STP Root
VLAN 2
vPC Secondary
S1
S2
peer-switch
Bridge Priority
VLAN 1  8K
VLAN 2  4K
VLAN 1
(blocked)
vPC1
S3
S4
VLAN 2
(blocked)
•
Hybrid topology where vPC and non-vPC devices coexist in a vPC domain
•
Need additional configuration parameters : spanning-tree pseudo-information
•
STP pseudo configuration takes precedence over global STP configuration
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
27
vPC Configuration Best Practices
vPC Peer-Gateway
• Allows a vPC switch to act as the active
gateway for packets addressed to the peer
router MAC
S1
S2
• Keeps forwarding of traffic local to the vPC node
and avoids use of the peer-link
• Allows Interoperability with features of some NAS
or load-balancer devices
S3
S4
Nexus(config-vpc-domain)# peer-gateway
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
28
vPC Configuration Best Practices
PVLAN on vPC
•
PVLAN configuration across both vPC switches
should be identical
•
PVLAN configuration not supported on Peer-Link
•
Type-1 Consistency Check
• Port mode is a type-1 check
• vPC member port brought down if PVLAN port
mode differs between vPC peers
•
vPC Primary
S1
vPC Secondary
P
P
PVLANPROMISC
(3500, 3501)
S2
PVLANPROMISC
(3500, 3501)
Type-2 Consistency Check
• PVLAN will bring down mismatched couples
C
Community
VLAN
Note : This feature is currently not supported on N9X00
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
29
vPC Configuration Best Practices
PVLAN vPC Type 1 Consistency Check
vPC Primary
S1
vPC Secondary
P
PVLAN
Promiscuous
Trunk
vPC Primary
S2
P
S1
PVLAN
Isolated
Trunk
S3
vPC Primary
S1
vPC Secondary
I
I
S2
S3
vPC Secondary
I
T
S2
Type 1 Consistency
Failure
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
30
vPC Configuration Best Practices
PVLAN vPC Type 2 Consistency Check
vPC Primary
S1
vPC Secondary
P
S2
P
PVLANPROMISC
(10, 201)
vPC Primary
S1
Secondary
Trunk (2,31)
(3,30), (4,100)
PVLANPROMISC
(10, 201)
S2
Secondary
Trunk (2,31)
(3,30), (4,100)
S3
vPC Primary
S1
I
I
S3
Type 2 Consistency
Failure
vPC Secondary
vPC Secondary
I
I
Secondary
Trunk (3,31)
(2,30), (4,100)
S2
Secondary
Trunk (2,31)
(3,30), (4,100)
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
31
Failure Scenarios
vPC Failure Scenario
vPC Peer-Keepalive up, vPC Peer-Link vPC, vPC member port down
 On of the vPC member ports fails (optics failure
or cable failure)
• vPC primary and secondary peer remain
primary and secondary, no change in roles.
• Result in change in path, and traffic that is
designated to the other peer, will cross peerlink to get to destination
• This is not desirable behavior, and peer-link
can be oversubscribed.
P
S1
S2
vPC_PLink
vPC1
vPC2
SW3
SW4
P Primary vPC
S
BRKDCT-2378
S
vPC Peer-keepalive
Secondary vPC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
33
vPC Failure Scenario
vPC Peer-Keepalive Link down
vPC peer-keepalive Link failure (link loss):
P
vPC Peer-keepalive
S
• vPC peer-link up
• Status of other vPC peer known
S1
S2
• Both peers Active
vPC_PLink
• No down time in the network
vPC1
vPC2
SW4
SW3
Keepalive Heartbeat
BRKDCT-2378
P
Primary vPC
S
Secondary vPC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
34
vPC Failure Scenario
vPC Peer-Keepalive Link up & vPC Peer-Link down
vPC peer-link failure (link loss):
P
vPC Peer-keepalive
S
• vPC peer-keepalive up
• Status of other vPC peer known
S1
S2
vPC_PLink
• Both peers Active
Suspend secondary
vPC Member Ports
• Secondary vPC peer disables all vPC’s
vPC1
• Traffic from vPC primary.
• Traffic from orphan devices connected to
secondary peer will be blackholed
vPC2
SW4
SW3
Keepalive Heartbeat
BRKDCT-2378
P
Primary vPC
S
Secondary vPC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
35
vPC Failure Scenario – Dual Active
vPC Peer-Keepalive down followed by vPC Peer-Link down
1. vPC peer-keepalive DOWN
2. vPC peer-link DOWN
3. DUAL-ACTIVE or SPLIT BRAIN
S1
• vPC primary peer remains primary and
secondary peer becomes operational primary
role
• Result in traffic loss / uncertain traffic behavior
• When links are restored, the operational
primary (former secondary) keeps the primary
role & former primary becomes operational
secondary
BRKDCT-2378
P
P
vPC Peer-keepalive
S2
vPC_PLink
Traffic Loss / Uncertain Traffic
Behavior
vPC1
vPC2
SW3
SW4
P Primary vPC
S
Secondary vPC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
36
Additional Features
vPC Configuration Best Practices
vPC Orphan ports suspend
S
P
•
Single attached devices to vPC
domain, will blackhole traffic if
peer-link fails
•
With Orphan Port Suspend
feature, will suspend orphan ports
on vPC secondary peer
•
Active or
Standby
S1
When peer-link is restored, vPC
secondary restores orphan ports
S2
Active or
Standby
S3
Nexus(config-if)# vpc orphan-ports suspend
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
38
vPC Configuration Best Practices
vPC ARP sync
•
•
When peer device goes down or
peer link goes down, SVIs are
suspended
After restore of the peer device, or
peer link, ARP table is empty traffic blackholed
•
Before bringing up SVI, peer
devices synchronize ARP table
over CFS
•
Reduces convergence time
ARP TABLE
ARP TABLE
IP1
MAC1
VLAN 100
IP1
MAC1
-
VLAN- 100
IP2
MAC2
VLAN 200
IP2
MAC2
-
VLAN- 200
L3
L2
SVI
100
SVI
200
CFS
SVI
100
SVI
200
Nexus(config-vpc-domain)# ip arp synchronize
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
39
vPC Configuration Best Practices
vPC Delay Restore
•
After vPC peer reload, traffic might
be blackholed, before L3
connectivity is reestablished
•
vPC link bring up can be delayed
to allow L3 routing protocol
convergence
•
Default time 30 seconds
OSPF
L3
L2
Nexus(config-vpc-domain)# delay restore <1-3600 sec>
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
40
vPC Configuration Best Practices
Operational
Primary
vPC auto-recovery
P
S
S1
P
S2
S
S1
S3
P
S2
S1
S2
S3
S3
1. vPC peer-link down : S2 - secondary shuts all its vPC member ports
2. S1 down : vPC peer-keepalive link down : S2 receives no keepalives
3. After 3 keepalive timeouts, S2 changes role and brings up its vPC
Nexus(config-vpc-domain)# auto-recovery
BRKDCT-2378
P
Primary vPC
S
Secondary vPC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
41
vPC Configuration Best Practices
vPC auto-recovery reload delay
•
Until peer adjacency is reestablished
between vPC devices, vPC member ports
are suspended
•
vPC auto-recovery reload delay allows
“alive” vPC peer to assume primary role
after delay time is expired
•
Delay timer can be tuned
S1
S2
S3
Nexus(config-vpc-domain)# auto-recovery reload-delay <240-3600 seconds>
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
42
vPC Configuration Best Practices
vPC auto-recovery
Auto-recovery addresses two cases of single switch behavior
•Peer-link fails and after a while primary switch (or keepalive link) fails
•Both VPC peers are reloaded and only one comes back up
How it works
•If Peer-link is down on secondary switch, 3 consecutive missing peer-keepalives will
trigger auto-recovery
•After reload (role is ‘none established’) auto-recovery timer (240 sec) expires while
peer-link and peer-keepalive still down, autorecovery kicks in
•Switch assumes primary role
•VPCs are brought up bypassing consistency checks
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
43
vPC Configuration Best Practices
vPC Self-Isolation
P
Error
Triggered
Operational
Primary
S
P
S
ISOLATED
Self- Isolate
S1
S2
S3
S1
S2
P
S1
S2
S3
S3
1. Error Triggered : All Line cards Fail or All Vlans’s down on peer-link
2. S1 sends “self-isolation” message through the peer-keepalive
3. S2 takes over as operational Primary and S1 is isolated from the vPC domain
BRKDCT-2378
P
Primary vPC
S
Secondary vPC
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
44
vPC Configuration Best Practices
Example Configuration and Verification on Nexus 7x00
vPC domain 100
peer-keepalive destination
10.126.216.44
peer-gateway
self-isolation
vPC domain 100
peer-keepalive destination
10.126.216.41
peer-gateway
self-isolation
Switch# show vPC brief
<snip>
vPC domain id
: 100
<snip>
vPC role
: primary
<snip>
Self-isolation
: Enabled
Switch# show vPC brief
<snip>
vPC domain id
: 100
<snip>
vPC role
: secondary
<snip>
Self-isolation
: Enabled
2015 Sep 29 22:33:03 S1 %$ VDC-1 %$
%vPC-2-ENTER_SELF_ISOLATION: Local
switch goes into self isolation
state due to all linecards failure.
Please resume failed linecards and
do shut/no shut on peer-link to exit
self-isolation state
2015 Sep 30 10:33:14 S2 %$ VDC-1 %$
%vPC-2-ENTER_SELF_ISOLATION: Remote
switch goes into self isolation
state due to all linecards failure.
Please resume failed linecards and
do shut/no shut on peer-link to exit
self-isolation state
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
45
vPC Configuration Best Practices
vPC Self-Isolation
• vPC self-isolation is turned OFF by default
• No Impact on vPC operation if sellf-isolation enabled
• Functional only when enabled on both vPC peers.
• Not part of vPC type-1 and type-2 consistency checks
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
46
vPC Configuration Best Practices
Why Object-Tracking ?
•
S5
S4
Modules hosting peer-link and uplink fail on
the vPC primary
Primary
•
Peer-Link is down and vPC Secondary
shut all its vPC
•
Auto-Recovery does not kick in as peerkeepalive link is active
•
Traffic is black holed
Secondary
L3
L2
S1
S2
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
47
vPC Configuration Best Practices
Object-tracking
•
•
vPC object tracking, tracks both peer-link and
uplinks in a list of Boolean OR
Object Tracking triggered when the track object
goes down
•
Suspends the vPCs on the impaired device
•
Traffic forwarded over the remaining vPC peer
! Track
track 1
! Track
track 2
track 3
the vpc peer link
interface port-channel11 line-protocol
the uplinks
interface Ethernet1/1 line-protocol
interface Ethernet1/2 line-protocol
S4
S5
L3
L2
S1
S2
! Combine all tracked objects into one.
! “OR” means if ALL objects are down, this object will go down
track 10 list boolean OR
object 1
object 2
object 3
! If object 10 goes down on the primary vPC peer,
! system will switch over to other vPC peer and disable all local vPCs
vpc domain 1
track 10
S3
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
48
vPC Configuration Best Practices
Spanning Tree Bridge Assurance
Stopped receiving
BPDUS!
Root
Malfunctioning
switch
BPDUs
Network
Network
BA Inconsistent
Network
Network
BPDUs
BPDUs
Network
Network
BA Inconsistent
Blocked
Stopped receiving
BPDUS!
Edge
Edge
%STP-2-BRIDGE_ASSURANCE_BLOCK: Bridge Assurance blocking port Ethernet2/48
VLAN0700
switch# show spanning vl 700 | in -i bkn
Eth2/48
Altn BKN*4
128.304 Network P2p *BA_Inc
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
49
Spanning Tree Bridge Assurance
Almost like a routing protocol…
•
Turns STP into a bidirectional protocol
•
Ensures spanning tree fails “closed” rather than “open”
•
All ports with “network” port type send BPDUs regardless of state
•
If network port stops receiving BPDUs, port is placed in BA-Inconsistent state
(blocked)
%STP-2-BRIDGE_ASSURANCE_BLOCK: Bridge Assurance blocking port
Ethernet2/48 VLAN0700.
switch# sh spanning vl 700 | in -i bkn
Eth2/48
Desg BKN*4
128.304 Network P2p *BA_Inc
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
50
vPC Configuration Best Practices
vPC & Bridge Assurance (BA)
• STP Bridge Assurance is enabled by default on vPC Peer-Link
• DON’T disable Bridge Assurance on vPC Peer-link
• NO Bridge Assurance on vPC member ports (even with peer-switch)
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
51
vPC Configuration Best Practices
Unidirectional Link Detection (UDLD)
•
Light-weight Layer 2 failure detection protocol
•
Designed for detecting:
•
One-way connections due to physical or soft failure
• Mis-wiring detection (loopback or triangle)
•
Cisco proprietary, but listed in informational RFC 5171
•
Runs on any single Ethernet link, even inside bundle
•
Centralized implementation in switching platforms
•
Message interval: 7 - 90 sec (default: 15 seconds)
•
Detection: 2.5 x interval + timeout value (4 sec)  ~ 41 sec
BRKDCT-2378
Rx
Tx
Rx
Tx
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
52
vPC Configuration Best Practices
UDLD with vPC
• UDLD NOT recommended on vPC peer-link
• UDLD NOT recommended on vPC member ports if LACP is used
• UDLD only in normal mode on vPC member ports if required
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
53
Agenda
•
Introduction to vPC
•
Feature Overview
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
Design Best Practices
FHRP with vPC
FHRP
“Active”:
Active for
shared L3 MAC
FHRP
“Standby”:
Active for
shared L3 MAC
L3
L2
S1
S3
•
•
•
S2
S4
FHRP in Active/Active mode with vPC
No requirement for aggressive FHRP timers
Best Practice : Use default FHRP timers
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
55
Design Best Practices
ASA Cluster
Cluster
Control Link
Cluster
Data Link
ASA Cluster Mode
•
Use unique vPC for ASA Cluster Data Links to vPC domain
•
Use vPC per ASA device for Cluster Control Link (CCL) to vPC domain
•
Leverage peer-switch configuration
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
56
Nexus 2000 (FEX) Straight-Through Deployment
with vPC
• Port-channel connectivity from the server
• Two Nexus switches bundled into a vPC
pair
S2
S1
• Suited for servers with Dual NIC and
capable of running Port-Channel
Fabric Links
FEX
101
FEX
102
HIF
HIF
vPC
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
57
Nexus 2000 (FEX) Active-Active Deployment with
vPC
S1
•
•
•
Fabric Extender connected to two Nexus
5X00 / 6000 /7x00
Suited for servers with Single NIC or
Dual NIC not having port-channel
capability.
Scale implications of less FEX per
system and less vPC
Note :
• This design is currently not supported on Nexus 9X00
• Nexus 7X00 will support this from release 7.2
S2
Fabric Links
Fex 102
Fex 101
HIF
BRKDCT-2378
HIF
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
58
Nexus 2000 (FEX) Active-Active Scale & Limitations
(N7X00)
•
N7X00 can support up to 64 FEXs
•
N7X00 supports only 15 Active-Active FEX in 7.2(0)D1(1)
•
N7X00 supports only 32 Active-Active FEX in 7.3(0)D1(1)
•
Straight-Through FEX and Active-Active FEX cannot exist on the same ASIC
instance
•
Layer 3 HIF ports are not supported with Active-Active FEX
•
Active-Active FEX is not supported with vPC+
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
59
Nexus 2000 (FEX) - Enhanced vPC
•
•
•
Port-channel connectivity to dual-homed
FEXs
From the server perspective a single access
switch with port-channel support – each line
card supported by redundant supervisors
Ideal design for a combination of single
NIC and Dual NIC servers with portchannel capability
Note :
This design is currently not supported on N7000 / N7700 and
N9X00
S2
S1
Fabric Links
Fex 100
Fex 101
HIF
BRKDCT-2378
HIF
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
60
Physical Port vPC – Nexus 7x00
vPC domain
vPC domain
FEX101
e101/1/1
Port-channel vPC
FEX101
FEX102
Po1
VPC1
VPC1
Po1
e101/1/1
e102/1/1
FEX102
VPC1
interface e101/1/1
switchport
vpc 1
lacp mode active
VPC1
e102/1/1
Physical port vPC
• vPC configuration on a physical Layer 2 port as opposed to a port-channel
• Front panel ports and FEX ports connected to F2/F2e/F3 only
• Improves scaling as separate port-channel interface not created for single-link vPC member port
• Key benefit: more than 1000 host facing vPCs with FEX
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
61
Data Center Interconnect - vPC
Data Center Interconnect - DCI
•
DCI provides connection of distant date centers
•
Extend VLANs between data centers
•
Technologies for DCI:
Overlay Transportation Virtualization – OTV ( Multiple DC Interconnect)
• Virtual Port Channel – vPC ( Two DC Interconnect)
•
•
vPC DCI:
•
STP Isolation Between DC
• Easy to Configure
• Resilient Solution
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
63
vPC - Data Center Interconnect(DCI)
DC 1
DC 2
E
-
F
vPC domain 21
Long Distance
Dark Fiber
F
E
CORE
CORE
vPC domain 11
-
N
N
N
N
N
Network port
E
Edge or portfast
-
Normal port type
B
BPDUguard
F
BPDUfilter
R
Rootguard
802.1AE (Optional)
- R
F
F E
R
R
-
-
N
N
-
N
R
R
N
-
vPC domain 10
vPC domain 20
R
R
-
-
E
E
B
B
Server Cluster
ACCESS
ACCESS
E
AGGR
AGGR
-
R
Server Cluster
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
64
Design Best Practices
vPC -Data Center Interconnect (DCI)
•
vPC Domain id for vPC layers should be UNIQUE
•
BPDU Filter on the edge devices to avoid BPDU propagation
•
STP Edge Mode to provide fast Failover times
•
No Loop must exist outside the vPC domain
•
No L3 peering between Nexus devices (i.e. pure layer 2)
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
65
Layer 3 over vPC
Dynamic Routing over vPC
Problem?
1)
Packet arrives at R1
2)
R1 does lookup in routing table and sees 2 equal paths
going north (to S1 & S2)
3)
Assume it chooses S1 (ECMP decision)
4)
R1 now has rewrite information to which router it needs
to go (router MAC S1 or S2)
5)
L2 lookup happens and outgoing
interface is port-channel 1
6)
Hashing determines which port-channel member is
chosen (say to S2)
7)
Packet is sent to S2
8)
S2 sees that it needs to send it over the peer-link to S1
based on MAC address
S3
Po2
S1
S2
Po1
R1
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
67
Dynamic Routing over vPC
Problem?
S3
9)
S1 performs lookup and sees that it needs to send
to S3
10)
S1 performs check if the frame came over peer link
& is going out on a vPC.
11)
Frame will ONLY be forwarded if:
• Outgoing interface is NOT a vPC or
• Outgoing vPC doesn’t have active interface on
other vPC peer (in the example S2)
Po2
S1
S2
Note:
Po1
•
Use of Peer-Gateway allows routing/forwarding traffic for the peer-router MAC
locally, but does NOT Enable Dynamic Routing on vPC VLANs
•
Even with Peer-Gateway Routing protocols (e.g. OSPF) TTL expiry when
traversing in transit the peer vPC Router device.
R1
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
68
Dynamic Routing over vPC
Devices without L3 over vPC support
•
Not recommended to attach L3 devics to vPC domain via L2 port-channel
•
Common workarounds:
•
Individual L3 links for routed traffic
• Static route to FHRP VIP
A
SVI 1
IP Z
VIP A
SVI 1
IP Y
VIP A
S2
S1
SVI 2
IP X
B
SVI 1
IP Z
VIP A
SVI 1
IP Y
VIP A
S1
L3 ECMP
SVI 2
IP X
S2
Router
Router
SVI 1
IP Z
VIP A
SVI 1
IP Y
VIP A
S1
S2
SVI 2
IP X
Router
Static Route to VIP A
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
69
Design Best Practices
Backup Routing Path
•
•
•
•
Point-to-point dynamic routing protocol
adjacency between the vPC peers to
establish a L3 backup path to the core
through PL in case of uplinks failure
Define SVIs associated with FHRP as
routing passive-interfaces in order to avoid
routing adjacencies over vPC peer-link
A single point-to-point VLAN/SVI (aka
transit vlan) will suffice to establish a L3
neighbor
Use one transit vlan to establish L3 routing
backup path over the vPC peerlink in case L3
uplinks were to fail, all other SVIs can use
passive-interfaces
S3
S4
P
P
OSPF/EIGRP
P
P
VLAN 99
L3
L2
OSPF/EIGRP
S1
Alternatively, use an L3 point-to-point link
between the vPC peers to establish a L3
backup path
Primary
vPC
Secondary
vPC
S2
S5
P
BRKDCT-2378
Routing Protocol Peer
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
70
Dynamic Routing over vPC
Dynamic routing over vPC
Configuration
Nexus(config-vpc-domain)# layer3 peer-router
•
Dynamic peering between Layer 3 device and vPC peers over vPC VLAN
•
Traffic does not get decremented TTL if travers peer-link
•
“Peer-Gateway” should be enabled.
•
NOT supported on Nexus 3000 and Nexus 9000
S1
vpc domain 200
peer-keepalive destination
10.10.12.42 source 10.10.12.52
peer-gateway
layer3 peer-router
S1
S2
P
P
S2
vpc domain 200
peer-keepalive destination
10.10.12.52 source 10.10.12.42
peer-gateway
layer3 peer-router
P
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
72
Dynamic routing over vPC
Example Configuration and Verification on Nexus 7x00
S1
vpc domain 200
peer-keepalive
destination 10.10.12.42
source 10.10.12.52
peer-gateway
layer3 peer-router
Switch# show vPC brief
<snip>
vPC domain id
: 100
<snip>
Peer Gateway
: Enabled
<snip>
Operational Layer3 Peer : Enabled
P
P
S2
vpc domain 200
peer-keepalive
destination 10.10.12.52
source 10.10.12.42
peer-gateway
layer3 peer-router
Switch# show vPC brief
<snip>
vPC domain id
: 100
<snip>
Peer Gateway
: Enabled
<snip>
Operational Layer3 Peer : Enabled
P
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
73
Benefits of Dynamic Routing over vPC
• No Static routes
• No Parallel links
• No design changes
• Route peering across vPC’s over existing infrastructure
• Routing between vPC DCI
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
74
Dynamic routing over vPC
Use Case 1 : Firewall at Aggregation layer
L3 Cloud
•
Peering Firewalls in routed mode over vPC
•
Firewalls may be in active-standby mode
•
Static routing / L3 P2P links NOT required
•
External and internal traffic traverse same
port channel to firewall.
S1
S2
FW-A
FW-B
Dynamic Peering Relationship
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
75
Dynamic routing over vPC
Use Case 2 : Remote Orphan Site Peering in DCI Deployment
• vPC as Data Center Interconnect (DCI)
• Each Switch has routing adjacency with both
vPC device in other DC
Remote Site 1
S1
Remote Site 2
S2
• Each DC connected to a remote site by
orphan port
• Remote sites forms routing adjacency with
both peers of its directly connected DC
S3
BRKDCT-2378
S4
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
76
Dynamic Routing over vPC
New Supported Designs
Dynamic routing over vPC
Supported Designs
Layer 3 over DCI - vPC
Layer 3 services devices with vPC
P
P
P
P
P
P
P
Note : Supported only in Nexus 7X00 on F3 and F2E Line Cards, and Nexus 5x00
Supported on Nexus 9X00 in ACI mode
Currently not supported on Nexus 3X00, Nexus 9X00 (standalone mode), Nexus 7000 M-series Line card
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
78
Dynamic routing over vPC
Supported Designs
STP inter-connection using a vPC VLAN
P
P
Orphan device with vPC peers over vPC VLAN
P
P
P
P
Note : Supported only in Nexus 7X00 on F3 and F2E Line Cards, and Nexus 5x00
Supported on Nexus 9X00 in ACI mode
Currently not supported on Nexus 3X00, Nexus 9X00 (standalone mode), Nexus 7000 M-series Line card
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
79
Dynamic routing over vPC
Supported Designs
Peering with vPC peers over FEX vPC host interfaces
P
P
P
Note : Supported only in Nexus 7X00 on F3 and F2E Line Cards starting from release 7.2.
Supported on Nexus 9X00 in ACI mode
Currently not supported on Nexus 5X00, Nexus 3X00, Nexus 9X00 (standalone mode), Nexus 7000 M-series Line card
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
80
Dynamic Routing over vPC
Unsupported Designs
Dynamic routing over vPC
Unsupported Design
B
Peering across vPC interfaces with unequal L3
metrics
•
•
SVI
Router2
Int VLAN 20
The routing metric on S1 is less than the routing
metric on S2 (preferred path using S1).
Traffic from A to B may hash to S2. This traffic
will need to traverse to peer-link to get to B
through S1.
• Due to the vPC loop avoidance rule S1 will not
allow traffic to flow to B.
Po2
Int VLAN 20
S2
Int VLAN 20
S1
Po100
Int VLAN 10
Metric 10
Int VLAN 10
Metric 20
Po1
Router1
Int VLAN 10
SVI
A
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
82
Design Best Practices
vPC and Multicast
• vPC supports PIM-SM ( on all platforms)
Source
• vPC supports PIM-SSM (on N9000 and N5600)
• vPC uses CFS to sync IGMP state
S1
Source
S2
Receivers
• Sources in vPC domain
− both vPC peers are forwarders
− Duplicates avoided via vPC loop-avoidance logic
• Sources in Layer 3 cloud
− Active forwarder elected on unicast metric
− vPC Primary elected active forwarder in case metric
are equal
− Active forwarder concept is per multicast group/source
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
83
Agenda
•
Feature Overview
•
vPC Terminology and Roles
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
vPC Configuration Best Practices
vPC Shutdown
•
Isolates a switch from the vPC complex to:
•
Debug
•
Troubleshoot
•
Physically isolate
Primary
Secondary
vPC
S2
S1
•
Minimal disruption of traffic flows
•
“no shutdown” brings switch up
•
Part of configuration, persistent after reload
•
Recommended to have “peer-switch” enabled
S3
switch# configure terminal
switch(config)# vpc domain 100
switch(config-vpc)# shutdown
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
85
Graceful Insertion and Removal - GIR
Change window begins
vPC
vPC
system mode maintenance
One command!
Pre-change System Snapshot
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
86
Graceful Insertion and Removal - GIR
Change window complete
vPC
vPC
system mode normal
One command!
Pre/Post-change Snapshot Comparison
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
87
Graceful Insertion and Removal
•
Flexible framework providing a comprehensive, systemic method to isolate a
node.
•
Configuration profile foundation in NX-OS
•
Initial support for:
•
•
•
•
•
•
•
vPC/vPC+
ISIS
OSPF
EIGRP
BGP
Interface
Per VDC on Nexus 7x00
Platform
Release
Nexus 5x00/6000
NX-OS 7.1
Nexus 7x00
NX-OS 7.2
Nexus 9000
NX-OS 7.X
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
88
ISSU with vPC
•
ISSU (In Service Software Upgrade) recommended
way system upgrade in a vPC environment
•
vPC system can be independently upgraded
•
Upgrade must be run one peer at a time
•
Start with vPC primary switch
•
Configuration is locked on “other” vPC peer during
ISSU
•
vPC run seamlessly with two different version of
software
•
Aggressive timers not supported
BRKDCT-2378
5.2(x) / 6.2(x)
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
89
Agenda
•
Feature Overview
•
vPC Terminology and Roles
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
vPC with FCoE
Fiber Channel over Ethernet - FCoE
•
FC Payload
FCS
Ethernet FCoE
FC
Header Header Header
EOF
Fiber Channel traffic over Ethernet
CRC
•
Ethernet standards to support FCoE:
Priority Flow Contol – PFC
• Enhanced Transmission Selection – ETS
• Data Center Bridging Exchange – DCBX
•
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
92
vPC with FCoE
Unified Fabric Design
•
vPC with FCoE is supported between
hosts Nexus 7X00, Nexus 5X00 and
N5X00 & N2X00 pairs.
•
vPC and FCoE only on the first hop
•
Each vPC peer must be part of separate
fabric.
•
Best Practice: Use static port channel
rather than LACP with vPC and boot from
SAN.
[If NX-OS is prior to 5.1(3)N1(1)]
LAN Fabric
Fabric A
Fabric B
VLAN 10 ONLY HERE!
Nexus 5000
FCF-A
Nexus 5000
FCF-B
VLAN 10,20
STP Edge Trunk
VLAN 10,30
vPC contains only 2 X 10GE
links – one to each Nexus 5X00
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
93
vPC with FabricPath
FabricPath: an Ethernet Fabric
Shipping on Nexus 7x00, Nexus 600x and Nexus 5x00
FabricPath
•
•
•
•
•
•
•
Spanning Three Protocol independence
High MAC address scalability with conversation learning on Edge ports
Unique Switch ID (SID) identifies switches in FabricPath fabric
IS-IS for control plane information exchange
Multidestination Trees for BUM traffic
Loop mitigation with TTL
Simple CLI configuration Switch(config-if)# switchport mode fabricpath
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
95
vPC vs vPC+
Architecture of vPC and FabricPath with vPC+
CE
FP
CE Port
FP Port
CE VLAN’s
FP VLAN’s
vPC+
vPC
• Physical architecture of vPC and vPC+ is the same from the access edge
• Functionality/Concepts of vPC and vPC+ are the same
• Key differences are addition of Virtual Switch ID and Peer Link is a FP Core Port
• vPC+ is not supported on Nexus 9X00 & Nexus 3X00 Series
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
96
Dynamic Routing over vPC+
• Layer 3 devices can form routing adjacencies with
both the vPC+ peers over vPC
• The peer link ports and VLAN are configured in
FabricPath mode.
FabricPath
• PIM-SSM multicast
P
P
• L3 peering with vPC+ plus devices is not
supported on N7X00
vPC
N55xx, N56xx,
N6000
Router/ Firewall
P
Fabricpath Link
Dynamic Peering Relationship
Routing Protocol Peer
BRKDCT-2378
P
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
97
vPC with VXLAN
Virtual Extensible LAN - VXLAN
Benefits
•
VXLAN is a new network overlay
technology
•
VXLAN builds Layer-2 & Layer-3
overlay network on top of an IP
routed network
•
VXLAN uses MAC in IP-UDP
encapsulation (UDP dest. port
4789)
•
MAN/WAN
VLAN scale – VXLAN extends the
L2 segment ID field to 24-bits
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
99
VXLAN Packet Format
16
16
Reserved
16
VNI
16
Reserved
32
8 Bytes
VXLAN
RRRR1RRR
32
Checksum
0x0000
UDP
Src. Port
16
VXLAN Port
Outer
Dst. IP
8
Outer
Src. IP
72
FCS
8 Bytes
Header
Checksum
Protocol
0x11
16
Original
FCS L2 Frame
20 Bytes
IP Header
Misc Data
16
Ether Type
0x0800
16
VLAN ID
Tag
VLAN Type
48
0x8100
Src.
MAC Addr.
Dst.
MAC Addr.
14 Bytes
(4 bytes optional)
48
VXLAN
Header
UDP Header
UDP Length
Outer
IP Header
Outer
Mac Header
8
24
24
8
•
VXLAN is a Layer 2 overlay scheme over a Layer 3 network.
•
VXLAN uses Ethernet in UDP encapsulation
•
VXLAN uses a 24-bit VXLAN Segment ID (VNI) to identify Layer-2 segments
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
100
VXLAN Terminology
VTEP – Virtual Tunnel End Point
Transport IP Network
VTEP
VTEP
•
•
IP Interface
IP Interface
Local LAN Segment
Local LAN Segment
End System End System
End System End System
VXLAN terminates its tunnels on VTEPs (Virtual Tunnel End Point).
VTEP has two interfaces :
1. Bridging functionality for local hosts
2. IP identification in the core network for VXLAN encapsulation / de-encapsulation.
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
101
vPC and VXLAN
vPC VTEP
•
When vPC is enabled an ‘anycast’ VTEP
address is programmed on both vPC
peers
•
Multicast topology prevents BUM traffic
being sent to the same IP address across
the L3 network (prevents duplication of
flooded packets)
•
vPC peer-gateway feature must be
enabled on both peers
•
Backup SVI, configured with PIM
•
VXLAN header is ‘not’ carried on the vPC
Peer link
VXLAN
vPC VTEP
vPC VTEP
VLAN
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
102
VTEP1
vlan 10
vn-segment 10000
VXLAN & vPC
interface loopback 0
ip address <VTEP individual IP – orphan)
ip address <VTEP anycast IP – per VPC domain> secondary
!
interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
!
Vlan 99
!
Interface vlan 99
ip address 99.1.1.1/24
ip ospf cost 10
ip router ospf 1 area 0.0.0.0
ip pim sparse-mode
!
vpc nve peer-link-vlan 99
vPC Configuration
Map VNI to VLAN
Source Interface
individual IP is used for single attached Hosts
anycast IP is used for VPC attached Hosts
VXLAN Tunnel Interface
Vlan for VXLAN vPC peer-link
vtep
SVI for the VXLAN vPC peer-link
1
Enable the VLAN on the VXLAN
vPC peer-link
H1
10.10.10.10
VLAN 10
(vpc)
vtep
2
VTEP2
vtep
vlan
3
vtep
10
4
vn-segment 10000
interface loopback 0
ip address <VTEP individual IP - orphan>
ip address <VTEP anycast IP – per VPC domain> secondary
!
interface nve1
source-interface loopback0
member vni 10000 mcast-group 235.1.1.1
!
Vlan 99
!
Interface vlan 99
ip H2
address 99.1.1.2/24
ip ospf cost 10
10.10.10.20
VLAN
ip 10
router ospf 1 area 0.0.0.0
(vpc)
ip pim sparse-mode
!
vpc nve peer-link-vlan 99
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
103
VXLAN & vPC
Dual attached Host to dual attached Host (Layer-2)
•
Host 1 (H1) and Host 2 (H2) are dual
connected to a vPC domain
•
As H1 is behind a VPC interface, the
anycast VTEP IP is the source for
the VXLAN encapsulation
•
vtep
1
vtep
2
vtep
20
vtep
3
vtep
4
vtep
30
As H2 is behind a VPC interface, the
anycast VTEP IP is the target
H1
10.10.10.10
VLAN 10
(vpc)
BRKDCT-2378
H2
10.10.10.20
VLAN 10
(vpc)
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
104
vPC with ACI
Nexus 9000 + APIC = ACI
APIC
APIC
APIC
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
106
ACI uses a
policy based approach
on
that focuses
the application.
QoS
QoS
QoS
Filter
Service
Filter
Web
App
DB
External
Network
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
107
vPC and ACI
ACI fabric utilised for control-plane
• No dedicated peer-link between vPC peers:
•
vPC
peers
vPC
Domains
Fabric itself serves as the peer-link
ACI
fabric
• No out-of-band mechanism to detect peer
liveliness:
•
•
•
Due to rich fabric-connectivity (leaf-spine), it is
very unlikely that peers will have no active path
between them
CFS (Cisco Fabric Services) is replaced by
Zero Message Queue (ZMQ)
vtep
1
vtep
2
vtep
3
vPC
vPC
As ACI fabric is VXLAN-based, an anycast
VTEP is shared by both leaf switches in a
vPC domain
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
108
Agenda
•
Feature Overview
•
vPC Terminology and Roles
•
Configuration Best Practices
•
Design Best Practices
•
vPC Operations and Upgrade
•
vPC with Fabric Technologies
•
Key Takeaways
Key Takeaways
vPC in 2016
VXLAN, ACI, Fabricpath
VXLAN
• L2 segment scalability
• VTEP redundancy with
vPC
vPC Benefits
ACI
• No Blocked Ports
• High availability
• Fast Convergence
• Policy Based
• Fabric for vPC control
plane
Fabricpath
FCoE
•
•
•
• Unified Fabric for LAN &
SAN
Eliminates Spanning-Tree *
High resiliency
vPC+ for legacy switches,
servers, hosts
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
110
Related Sessions
Session Id
Session Name
BRKDCT-2404
VXLAN deployment models - A practical perspective
BRKDCT-3313
Fabricpath Operations and Troubleshooting
BRKDCT-2458
Nexus 9000/7000/6000/5000 Operations and
Maintenance Best Practices
BRKACI-2008
A Technical Introduction into ACI
BRKDCT-2333
Data Centre Network Failure Detection
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
111
Reference Material
•
vPC Best Practices Design Guide:
http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guid
e.pdf
•
vPC design guides:
http://www.cisco.com/en/US/partner/products/ps9670/products_implementation_design_guides_list.html
•
vPC and VSS Interoperability white Paper:
http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/white_paper_c11_589890.html
•
VXLAN Overview :
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-729383.html
•
Fabrcipath whitepaper :
http://www.cisco.com/c/en/us/products/collateral/switches/nexus-7000-series-switches/white_paper_c11-687554.html

ACI Overview
http://www.cisco.com/c/en/us/products/collateral/cloud-systems-management/aci-fabric-controller/white-paper-c11-729587.html
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
112
Complete Your Online Session Evaluation
•
Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 Amazon gift card.
•
Complete your session surveys
through the Cisco Live mobile
app or from the Session Catalog
on CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
CiscoLive.com/Online
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
113
Continue Your Education
•
Demos in the Cisco campus
•
Walk-in Self-Paced Labs
•
Lunch & Learn
•
Meet the Engineer 1:1 meetings
•
Related sessions
BRKDCT-2378
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
114
Please join us for the Service Provider Innovation Talk featuring:
Yvette Kanouff | Senior Vice President and General Manager, SP Business
Joe Cozzolino | Senior Vice President, Cisco Services
Thursday, July 14th, 2016
11:30 am - 12:30pm, In the Oceanside A room
What to expect from this innovation talk
• Insights on market trends and forecasts
• Preview of key technologies and capabilities
• Innovative demonstrations of the latest and greatest products
• Better understanding of how Cisco can help you succeed
Register to attend the session live now or
watch the broadcast on cisco.com
Thank you