Nexus 7000
virtual Port-Channel
Best Practices & Design Guidelines
Roberto Mari
Technical Marketing Engineer
Data Center Business Unit
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
November 2009
version 1.1
1
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
2
Feature Overview & Terminology
vPC Definition
 Allow a single device to use a
port channel across two upstream
switches
 Eliminate STP blocked ports
 Uses all available uplink
bandwidth
Logical Topology without vPC
 Dual-homed server operate in
active-active mode
 Provide fast convergence upon
link/device failure
 Reduce CAPEX and OPEX
 Available on current and future
hardware for M1 and D1
generation cards.
Logical Topology with vPC
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
4
Feature Overview & Terminology
vPC Terminology
vPC peer-keepalive
link
 vPC peer – a vPC switch, one of a pair
 vPC member port – one of a set of ports
(port channels) that form a vPC
vPC peer-link
CFS protocol
vPC peer
vPC
vPC
vPC
member
member
port
port
vPC
non-vPC
device
 vPC – the combined port channel between
the vPC peers and the downstream device
 vPC peer-link – Link used to synchronize
state between vPC peer devices, must be
10GbE
 vPC peer-keepalive link – the keepalive
link between vPC peer devices, i.e., backup
to the vPC peer-link
 vPC VLAN – one of the VLANs carried
over the peer-link and used to
communicate via vPC with a peer device.
 non-vPC VLAN – One of the STP VLANs
not carried over the peer-link
 CFS – Cisco Fabric Services protocol, used
for state synchronization and configuration
validation between vPC peer devices
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
5
Building a vPC Domain
Configuration Steps
Following steps are needed to build a vPC (Order does Matter!)
1. Configure globally a vPC domain on both vPC devices
2. Configure a Peer-keepalive link on both vPC peer switches (make sure is operational)
NOTE: When a vPC domain is configured the keepalive must be operational to allow a
vPC domain to successfully form.
3. Configure (or reuse) an interconnecting port-channel between the vPC peer switches
4. Configure the inter-switch channel as Peer-link on both vPC devices (make sure is
operational)
5. Configure (or reuse) Port-channels to dual-attached devices
6. Configure a unique logical vPC and join port-channels across different vPC peers
vPC peerkeepalive link
vPC peer-link
vPC peer
Standalone
Port-channel
© 2009 Cisco Systems, Inc. All rights reserved.
vPC
Cisco Confidential
vPC member port
6
Building a vPC Domain
Peer Link
 Definition:
Standard 802.1Q Trunk
vPC peer-link
Can Carry vPC and non vPC VLANs*
Carries Cisco Fabric Services messages (tagged as CoS=4
for reliable communication)
Carries flooded traffic from a vPC peer
Carries STP BPDUs, HSRP Hellos, IGMP updates, etc.
 Requirements:
Member ports must be 10GE interfaces one of the N7KM132XP-12 modules
Peer-link are point-to-point. No other device should be inserted
between the vPC peers.
 Recommendations (strong ones!)
Minimum 2x 10GbE ports on separate cards for best
resiliency.
Dedicated 10GbE ports (not shared mode ports)
*It is Best Practice to split vPC and non-vPC
VLANs on different Inter-switch Port-Channels.
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
7
Building a vPC Domain
Peer Link with Single 10G Module
 Common Nexus 7000 configuration:
1x 10G, 7x 1G cards
 vPC recommendation is 2 10G cards
 Potential problem occurs if Nexus 7000 is L3 boundary with
single 10G card
 Use Object Tracking Feature available in 4.2
 More information from CCO:
http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nxos/interfaces/configuration/guide/if_vPC.html#wp1529488
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
8
Building a vPC Domain
Peer Link with Single 10G Module – Object Tracking
Scenario:
 vPC deployments with a single N7KM132XP-12 card, where core and peerlink interfaces are localized on the same
card.
 This scenario is vulnerable to accesslayer isolation if the 10GE card fails on
the primary vPC.
vPC Object Tracking Solution:
e1/… e1/…
e1/… e1/…
e1/…
L3
L2
vPC PL
e1/…
e1/…
e1/…
vPC PKL
vPC
Primary
e2/…
e2/…
vPC
Secondary
 Leverages object tracking capability in
vPC (new CLI commands are added).
 Peer-link and Core interfaces are
tracked as a list of boolean objects.
 vPC object tracking suspends vPCs on
the impaired device, so traffic can get
diverted over the remaining vPC peer.
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
rhs-7k-1(config-vpc-domain)# track <object>
9
Building a vPC Domain
Peer-Keepalive (1 of 2)
 Definition:
 Heartbeat between vPC peers
 Active/Active (no Peer-Link) detection
vPC peerkeepalive link
 Messages sent on 2 second interval
 3 second hold timeout on peer-link loss
 Fault Tolerant terminology is specific to VSS and deprecated in
vPC.
 Packet Structure:
 UDP message on port 3200, 96 bytes long (32 byte payload),
includes version, time stamp, local and remote IPs, and domain ID.
 Keepalive messages can be captured and displayed using the
onboard Wireshark Toolkit.
 Recommendations:
 Should be a dedicated link (1Gb is adequate)
 Should NOT be routed over the Peer-Link
 Can optionally use the mgmt0 interface (along with management
traffic)
 As last resort, can be routed over L3 infrastructure
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
11
Building a vPC Domain
Peer-Keepalive (2 of 2)
Cautions/Additional Recommendations:
 When using supervisor management interfaces to carry the vPC peerkeepalive, do not connect them back to back between the two switches.
 Only one management port will be active a given point in time and a
supervisor switchover may break keep-alive connectivity
 Use the management interface only if you have an out-of-band
management network (management switch in between).
Management
Switch
vPC_PK
Management
Network
vPC_PK
Standby Management
Interface
Active Management
Interface
vPC_PL
vPC1
© 2009 Cisco Systems, Inc. All rights reserved.
vPC2
Cisco Confidential
12
Building a vPC Domain
vPC Member Port
 Definition:
Port-channel member of a vPC peer.
 Requirements:
Configuration needs to match other vPC
peer’s member port config.
In case of inconsistency a VLAN or the
entire port-channel may suspend (i.e.
MTU mismatch).
Number of member ports on both vPC
peers is not required to match.
Up to 8 active ports between both vPC
peers (16-way port-channel can be build
with multi-layer vPC)
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
vPC
member
port
13
Building a vPC Domain
VDC Interaction
 vPC works seamlessly in any VDC based environment.
 One vPC domain per VDC is supported, up to the maximum number of
VDCs supported in the system.
 It is still necessary to have a separate vPC peer-link and vPC PeerKeepalive Link infrastructure for each VDC deployed.
Can vPC run between VDCs on the same switch?
 This scenario should technically work, but it is NOT officially supported
and has not been extensively tested by our QA team.
 Could be useful for Demo or hands on, but It is NOT recommended for
production environments. Will consolidate redundant points on the same
box with VDCs (e.g. whole aggregation layer on a box) and introduce a
single point of failure.
 ISSU will NOT work in this configuration, because the vPC devices can
NOT be independently upgraded.
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
14
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
15
Attaching to a vPC domain
The One and Only Rule…
ALWAYS
dual attach
devices to a vPC
Domain!!!
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
16
Attaching to a vPC Domain
IEEE 802.3ad and LACP
 Definition:
Port-channel for devices for devices dual-attached to
the vPC pair.
Provides local load balancing for port-channel
members
STANDARD 802.3ad port channel
 Access Device Requirements
STANDARD 802.3ad capability
LACP Optional
vPC
 Recommendations:
 Use LACP when available for better failover and misconfiguration protection
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
vPC
Regular
member
Portport
channel
port
17
Attaching to a vPC Domain
”My device can’t be dual attached!”
Recommendations (in order of preference):
1.
ALWAYS try to dual attach devices using vPC (not applicable for routed links).
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dualactive scenarios. Ensures full redundant active/active paths through vPC.
CONS: None
2. If (1) is not an option – connect the device via a vPC attached access switch (could use VDC to create a
“virtual access switch”).
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dualactive scenarios. Availability limited by the access switch failure.
CONS: Need for an additional access switch or need to use one of the available VDCs. Additional
administrative burden to configure/manage the physical/Virtual Device
3. If (2) is not an option – connect device directly to (primary) vPC peer in a non-vPC VLAN* and provide
for a separate interconnecting port-channel between the two vPC peers.
PROS: Traffic diverted on a secondary path in case of peer-link failover
CONS: Need to configure and manage additional ports (i.e. port-channel) between the Nexus 7000
devices.
4. If (3) is not an option – connect device directly to (primary) vPC peer in a vPC VLAN
PROS: Easy deployment
CONS: VERY BAD. Bound to vPC roles (no role preemption in vPC) , Full Isolation on peer-link failure
when attached vPC toggles to a secondary vPC role.
* VLAN that is NOT part of any vPC and not present on vPC peer-link
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
18
Attaching to a vPC Domain
vPC and non-vPC VLANs (i.e. single attached .. )
S
P
P
2. Attached via VDC/Secondary Switch
1. Dual Attached
P
S
Orphan
Ports
S
P
P
Primary vPC
S
Secondary vPC
3. Secondary ISL Port-Channel
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
S
4. Single Attached to vPC Device
19
Attaching to a vPC Domain
”My device only does STP!”
Recommendations (in order of preference):
1. ALWAYS try dual attach devices using vPC
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with
vPC dual-active scenarios. Ensures full redundant active/active paths through vPC.
CONS: None
2. If (1) is not an option – connect the device via two independent links using STP. Use nonvPC VLANs ONLY on the STP switch.*
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with
vPC dual-active scenarios. Ensures full redundant Active/Active paths on vPC VLANs.
CONS: Requires an additional STP port-channel between the vPC devices. Operational
burden in provisioning and configuring separate STP and vPC VLAN domains. Only
Active/Standby paths on STP VLANs.
3. If (2) is not an option – connect the device via two independent links using STP. (Use vPC
VLANs on this switch)
PROS: Simplify VLAN provisioning and does not require allocation of an additional 10GE
port-channel.
CONS: STP and vPC devices may not be able to communicate each other in certain failure
scenarios (i.e. when STP Root and vPC primary device do not overlap). All VLANs carried
over the peer-link may suspend until the two adjacency forms and vPC is fully
synchronized".
* Run the same STP mode as the vPC domain. Enable portfast/port type edge on host facing ports
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
20
vPC Design principles
Attaching to a vPC Domain - vPC and non-vPC VLANs (STP/vPC Hybrid)
Non vPC port-channel
P
SR
S
PR
S
P
1. All devices Dual Attached via vPC
SR
P
2. Separate vPC and STP VLANs
PR
S
P
Primary vPC
S
Secondary vPC
PR
Primary STP Root
SR
Secondary STP Root
3. Overlapping vPC and STP VLANs
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
21
Attaching to a vPC Domain
16-way Port-Channel (1 of 2)
 Multi-Layer vPC can join 8 active
ports port-channels in a unique 16way port-channel*
 vPC peer side load-balancing is
LOCAL to the peer
 Each vPC peer has only 8 active
links, but the pair has 16 active load
balanced links
Nexus
7000
16-way port
channel
Nexus
5000
* Possible with any device supporting
vPC/MCEC and 8-way active port-channels
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
22
Attaching to a vPC Domain
16-way Port-Channel (2 of 2)
 16 active ports between 8
active port-channel devices
and 16 active port-channel
devices?
 vPC peer side load-balancing
is LOCAL to the peer
 Each vPC peer has only 8
active links, but the pair has 16
active load balanced links to
the downstream device
supporting 16 active ports
 D-series N7000 line cards will
also support 16 way active
port-channel load balancing,
providing for a potential 32
way vPC port channel!
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Nexus
7000
16-port port-channel
Nexus
5000
Nexus 5000 16-port port-channel
support introduced in 4.1(3)N1(1a)
release
23
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
24
Layer 3 and vPC
Recommendations
 Use separate L3 links to hook up routers to a vPC domain is still standing.
 Don’t use L2 port channel to attach routers to a vPC domain unless you can
statically route to HSRP address
 If both, routed and bridged traffic is required, use individual L3 links for routed
traffic and L2 port-channel for bridged traffic
Switch
Switch
Po2
7k1
Po2
7k2
L3 ECMP
Po1
Router
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Router
25
Layer 3 and vPC
What can happen… (1 of 3)
vPC view
Layer 2 topology
Layer 3 topology
7k vPC
7k1
7k1
7k2
7k2
R
R
R
R could be any router,
L3 switch or VSS
building a port-channel
© 2009 Cisco Systems, Inc. All rights reserved.
Port-channel looks like
a single L2 pipe.
Hashing will decide
which link to chose
Cisco Confidential
Layer 3 will use ECMP
for northbound traffic
26
Layer 3 and vPC
What can happen… (2 of 3)
1) Packet arrives at R
S
2) R does lookup in routing table and sees 2
equal paths going north (to 7k1 & 7k2)
Po2
3) Assume it chooses 7k1 (ECMP decision)
4) R now has rewrite information to which
router it needs to go (router MAC 7k1 or
7k2)
5) L2 lookup happens and outgoing
interface is port-channel 1
7k1
7k2
Po1
6) Hashing determines which port-channel
member is chosen (say to 7k2)
7) Packet is sent to 7k2
8) 7k2 sees that it needs to send it over the
peer-link to 7k1 based on MAC address
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
R
27
Layer 3 and vPC
What can happen… (3 of 3)
9) 7k1 performs lookup and sees that it
needs to send to S
S
Po2
10) 7k1 performs check if the frame came
over peer link & is going out on a vPC.
11) Frame will only be forwarded if outgoing
interface is NOT a vPC or if outgoing
vPC doesn’t have active interface on
other vPC peer (in our example 7k2)
7k1
7k2
Po1
R
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
28
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
29
Spanning Tree Recommendations
Overview – STP Interoperability
 STP Uses:
• Loop detection (failsafe to vPC)
• Non-vPC attached device
• Loop management on vPC addition/removal
 Requirements:
• Needs to remain enabled, but doesn’t dictate vPC member
port state
• Logical ports still count, need to be aware of number of
VLANs/port-channels deployed!
 Best Practices:
• Not recommended to enable Bridge Assurance feature on
vPC channels (i.e. no STP “network” port type). Tracked by
CSCsz76892.
vPC
vPC
STP
is running to manage
• Make sure all switches in you layer 2 domain are running
loops outside of vPC’s
with Rapid-PVST or MST (IOS default is non-rapid PVST+),
direct domain, or before
to avoid slow STP convergence (30+ secs)
initial vPC configuration
• Remember to configure portfast (edge port-type) on host
facing interfaces to avoid slow STP convergence (30+
secs)
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
30
Spanning Tree Recommendations
Port Configuration Overview
Data Center Core
Primary
vPC
vPC
Domain
Primary
Root
R
Network port
E
Edge or portfast port type
-
Normal port type
B
BPDUguard
R
Rootguard
L
Loopguard
Secondary
vPC
HSRP
ACTIVE
Aggregation
N
R
-
R
HSRP
STANDBY
N
N
-
-
R
R
Layer 3
Secondary
Root
R
R
Layer 2 (STP + Rootguard)
-
R
-
Access
-
L
-
E
E
E
E
E
B
B
B
B
B
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Layer 2 (STP + BPDUguard)
31
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
32
Data Center Interconnect
Multi-layer vPC for Agg and DCI
DC 1
-
-
R
-
-
Normal port type
B
BPDUguard
F
BPDUfilter
R
Rootguard
DC 2
N
N
N
F
Edge or portfast port type
F
-
R
R
- R
-
-
N
N
-
N
R
R
vPC domain 10
vPC domain 20
AGGR
AGGR
-
F
N
-
-
E
vPC domain 21
Long Distance
F
Network port
CORE
CORE
vPC domain 11
N
N
-
R
R
E
B
 vPC Domain id for facing vPC layers should be different
 No Bridge Assurance on interconnecting vPCs
 BPDU Filter on the edge devices to avoid BPDU propagation
 No L3 peering between DCs (i.e. L3 over vPC)
Server Cluster
© 2009 Cisco Systems, Inc. All rights reserved.
-
E
ACCESS
ACCESS
Key Recommendations
B
Server Cluster
Cisco Confidential
33
Data Center Interconnect
Encrypted Interconnect
DC-2
DC-1
Nexus 7010
Nexus 7010
vPC
vPC
CTS Manual Mode
(802.1AE 10GE line-rate
encryption)
No ACS is required
Nexus 7010
© 2009 Cisco Systems, Inc. All rights reserved.
Nexus 7010
Cisco Confidential
34
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
35
HSRP with vPC
FHRP Active/Active
 Support for all FHRP protocols
in Active/Active mode with vPC
 No additional configuration
required
HSRP/VRRP
“Active”:
Active for
shared L3 MAC
 Standby device communicates
with vPC manager produces to
determine if vPC peer is
“Active” HSRP/VRRP peer
HSRP/VRRP
“Standby”:
Active for
shared L3 MAC
L3
L2
 General HSRP best practices
still applies.
 When running active/active
aggressive timers can be
relaxed (i.e. 2-router vPC
case)
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
36
HSRP with vPC
Do NOT use Object Tracking
Cautions:
 Not recommended using HSRP link tracking in a vPC configuration
 Reason: vPC will not forward a packet back on a vPC once it has
crossed the peer-link, except in the case of a remote member port
failure
L3 CORE
ACTIVE HSRP
STANDBY HSRP
GW
GW
VLAN 100, 200
VLAN 100
© 2009 Cisco Systems, Inc. All rights reserved.
GW
L2/L3
Aggregation
VLAN 200
Cisco Confidential
37
HSRP with vPC
L3 Backup Routing
 Use an OSPF point-to-point adjacency (or equivalent L3 protocol)
between the vPC peers to establish a L3 backup path to the Core
through in case of uplinks failure
 A single point-to-point VLAN/SVI will suffice to establish a L3
neighborship.
OSPF
OSPF
VLAN 99
L3
L2
OSPF
Primary
vPC
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Secondary
vPC
38
HSRP with vPC
Dual L2/L3 Pod Interconnect
Scenario:
 Provide L2/L3 interconnect between
L2 Pods, or between L2 attached
Datacenters (i.e. sharing the same
HSRP group).
 A vPC domain without an active
HSRP instance in a group would not
able to forward traffic.
Active
Standby
Listen
Listen
Multi-layer vPC with single HSRP:
 L3 on the N7K supports
Active/Active on one pair, and still
allows normal HSRP behavior on
other pair (all in one HSRP group)
 L3 traffic will run across Intra-pod
link for non Active/Active L3 pair
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
39
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
40
vPC and Services
Catalyst 6500 Services Chassis w. Services VDC Sandwich
Two Nexus 7000 Virtual Device Contexts used to “sandwich”
services between virtual switching layers
• Layer-2 switching in Services Chassis with transparent
services
• Services Chassis provides Etherchannel capabilities for
interaction with vPC
• vPC running in both VDC pairs to provide Etherchannel for
both inside and outside interfaces to Services Chassis
Design considerations:
• Access switches requiring services are connected to subaggregation VDC
• Access switches not requiring services may be connected to
aggregation VDC
• May be extended to support multiple virtualized service
contexts by using multiple VRF instances in the subaggregation VDC
Design Cautions:
• Be aware of the Layer 3 over vPC design caveat. If Peering at
Layer 3 is required across the two vPC layers an alternative
solution should be explored (i.e. using STP rather than vPC to
attach service chassis)
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
41
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
42
vPC Latest Enhancements
Summary
Several enhancements to vPC:
 vPC Object Tracking
 vPC Peer-Gateway
 vPC Delay Restore
 Multi-layer vPC with single HSRP group
 vPC unicast ARP handling
 vPC Exclude Interface-VLAN
 vPC single attached device Listing
 vPC Convergence and Scalability
For more details:
 4.2 Release Notes
http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/release/notes/42_nxos_release_note.html#wp218085
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
43
vPC Latest Enhancements
vPC Peer-Gateway for NAS interoperability
Local Routing for peer
router –mac Traffic
Scenario:
 Interoperability with non RFC
compliant features of some NAS devices
(i.e. NETAPP Fast-Path or EMC IPReflect)
 NAS device may reply to traffic using
the MAC address of the sender device
rather than the HSRP gateway.
vPC PL
vPC PKL
L3
L2
 Packet reaching vPC for the non local
Router MAC address are sent across the
peer-link and can be dropped if the final
destination is behind another vPC.
vPC Peer-Gateway Solution:
 Allows a vPC switch to act as the
active gateway for packets addressed
to the peer router MAC (CLI command
added in the vPC global config)
N7k(config-vpc-domain)# peer-gateway
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
44
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
45
In-Service Software Upgrade (ISSU)
vPC System Upgrade/Downgrade
 ISSU is still the recommended system
upgrade in a multi-device vPC environment
 vPC system can be independently upgraded
with no disruption to traffic.
 Upgrade is serialized and must be run one at
the time (i.e. config lock will prevent
synchronous upgrades)
 Configuration is locked on “other” vPC peer
during ISSU.
4.1(3)
4.2(1)
Begin
End
Caveats
4.1(x)
4.2(x)
None
4.2(x)
4.1(x)
None
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
4.1(3)
4.2(1)
4.1(3)
4.2(1)
46
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
47
4.2(1) vPC Enhancements
Convergence Topology
20 flows @1000 pps
L3 Core
Nexus 7000
OSPF
N7K-1
N7K-2
OSPF
Po10
16-way port-channel
L2/L3
Aggregation
Nexus 7000 vPC
4-way port-channel
Po160
Po20
L2 Access
Nexus 5000
vPC Peer Link LACP
Channel (2x10 GigE)
vPC Peer-Keepalive (GigE)
© 2009 Cisco Systems, Inc. All rights reserved.
20 flows @1000 pps
Cisco Confidential
20 flows @1000 pps
48
vPC on Nexus 7000
Convergence Numbers
Failover case
Failure Topology
Convergence Time
Failure
Failure of
secondary vPC
peer*
Failure of a
primary vPC peer*
P
P
S
S
Failover of the
vPC Peer Link
P
S
Restoration
4.1(4)
4.1(4)
North-Bound: ~700 ms
South-Bound: ~2.5 sec
North-Bound: ~3 sec
South-Bound: ~3.4 sec
4.2(1)
4.2(1)
North-Bound: ~50 ms.
South-Bound: ~100 ms
North-Bound: 100 – 900 ms
South-Bound: 1.2 -2 s
4.1(4)
4.1(4)
North-Bound: ~150 ms
South-Bound: ~3 sec
North-Bound:~4.5 secs
South-Bound: ~5 secs
4.2(1)
4.2(1)
North-Bound: ~50 ms
South-Bound: ~100 ms
North-Bound: ~400 ms-1.5 s
South-Bound: ~1.5 s
4.1(4)
4.1(4)
North-Bound: ~1.3 s
South-Bound: ~1.8 s
North-Bound: ~900 ms
South-Bound: up to 10+ s (CSCsz88998)
4.2(1)
4.2(1)
North-Bound: 100-300 ms
South-Bound: 50-500 ms
North-Bound: 150 - 900 ms
South-Bound: ~ 900 ms–1.5 s
NOTE: Convergence numbers may vary depending on the specific configuration (i.e. scaled
number of VLANs/SVIs or HSRP groups) and traffic patterns (i.e. L2 vs L3 flows).
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
49
vPC on Nexus 7000
Scalability Number Improvements
Release
Supported Scalability
4.1(5)
192 vPC’s (2-port) with the following,
200 VLANs
200 HSRP Groups
40K MACs & 40K ARPs
10K (S,G) w. 66 OIFs (L3 sources)
3K (S,G) w. 34 OIFs (L2 sources)
Latest
Ankara
4.2(1)
256 vPC’s (4-port) with the following,
260 VLANs
200 SVI/HSRP Groups
40k MACs & 40K ARPs
10K (S,G) w. 66 OIFs (L3 sources)
3K (S,G) w. 64 OIFs (L2 sources)
NOTE: Supported numbers of VLANs/vPCs are NOT related to an hardware or software limit but reflect what
has been currently validated by our QA. The N7k BU is planning to continuously increase these numbers as
soon as new data-points become available.
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
50
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
51
vPC Hands-on Lab Information
On Demand vPC Lab Overview
N7K-Aggr
N7K-Aggr
Pod 2
Pod 1
 Instructor-led hands-on lab
introducing the vPC (virtual Portchannel) feature for the Nexus 7000.
 Participants exposed to the
configuration of vPC with NX-OS.
 Lab needs to be manually booked
through Nexus 7000 TMEs.
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
N7K-1
POD 1-2 VPC
N7K-2
POD 1-2 VPC
Pod 1
Pod 2
N7K-3
POD 3-4 VPC
N7K-4
POD 3-4 VPC
Pod 3
Pod 4
N7K-7
POD 5-6 VPC
N7K-8
POD 5-6 VPC
Pod 5
Pod 6
52
vPC Hands-on Lab Information
vPC Lab Logistics and Timing
 The vPC Laboratory consists of 6 independent PODs.
 A group of 2 students is assigned to each Pod.
 Each student will configure a vPC peer device.
 PODs are logically independent. Two adjacent PODs are physically
bound to the same Nexus. Virtual Device Contexts (VDCs) are used to
define logically independent devices on the same Nexus 7010 box.
 The vPC Lab session is expected to be completed in around two hours.
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
53
Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU
 Convergence and Scalability
 vPC Hands-on Lab Information
 Reference Material
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
54
Reference Material
vPC/VSS Interop Test Details
Physical
Logical
L3 Core
N7K-1
N7K-2
L2/L3 Aggregation
Nexus 7000 vPC
Po10
E1/26
E1/25
Po100
Te1/2/1
6K-1
Po100
Te2/2/1
6K-2
L2 Access
6500 VSS
vPC Peer Link LACP
Channel (2x10 GigE)
vPC PeerKeepalive (GigE)
VSS VSL Channel
(2x10 GigE)
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
55
Reference Material
vPC/VSS Interop Test Details
 The following scenarios were tested:
• VSS and vPC member failover and convergence
• Dual active scenarios and behavior
• Best practice guidelines for STP, L3 (NSF), Multicast
 Catalyst 6500/Nexus 7000 interoperability:
• Multiple ports per chassis act as one larger ether-channel
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
56
Reference Material
Other Solution Tests and Recent vPC Documentation
 Enterprise Solutions Engineering:
http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html
 Implementing Nexus 7000 in the Data Center Aggregation
Layer with Services:
https://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/nx_7000_dc.html
 Configuration Guide for Object Tracking Feature:
http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/4_2/nxos/interfaces/configuration/guide/if_vPC.html#wp1530133
 vPC white Paper:
http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9402/white_paper_c11516396.html
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
57