Unified Fabric
aka FCOE
Dave Gibson
Senior Systems Engineer
Cisco Systems
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential (please do not distribute)
1
Legal Disclaimer
Many of the products and features described
herein remain in varying stages of development
and will be offered on a when-and-if-available
basis. This roadmap is subject to change at the
sole discretion of Cisco, and Cisco will have no
liability for delay in the delivery or failure to
deliver any of the products or features set forth
in this document.
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
2
Agenda
 The Evolution of the Data Center
 Introduction to FCoE
 Standards Defined
 Nexus and the Unified Fabric
 Nexus 5000
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
3
The Evolution of the
Data Center
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential (please do not distribute)
4
Data Center Access Layer Trends
Multi-Core CPU architectures allowing
bigger and multiple workloads on the same
machine
Server virtualization driving the need for
more I/O bandwidth per server
Growing need for network storage driving
the demand for higher network bandwidth
to the server
Increasing adoption of Blades in data
centers.
10G LOM on server Motherboard
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
5
Next-Gen Switch Design Goals
•Consolidate LAN & SAN
infrasctucture
•Standards based solution
•Reduce total cost of
ownership
•End-to-end data center
architecture
•Operational consistency
across platforms
Unified
I/O
Nexus
Family
•Enable Virtualization
•Address increase in
server processing
power
C97-485980-00
10G to
Server
© 2008 Cisco Systems, Inc. All rights reserved.
Low
Latency
Nexus
5000
Cisco Confidential
•Build with superior
performance in mind
•Support low latency
applications (e.g. HPC,
clustered app’s)
Scalable
Infrastructure
•Scale to 40G and
100G in future
•Increase feature
velocity
6
Cisco Nexus Family
 Complete data center class switching portfolio
 Consistent data center operating system across all platforms
 Infrastructure scalability, transport flexibility and operational
manageability
Nexus 7000
(Modular Switch
Platform)
Nexus 1000V
(Virtual Switch)
Nexus 4000
Nexus 2000
2008
1K
(Blade Switch)
(Fabric
Extender)
Nexus 5000
(Fixed Config
Switch)
Cisco Nexus 1000V
x86
NX-OS Data Center Operating System
Data Center Network Manager
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
7
Before I/O Consolidation
LAN
SAN B
SAN A
 Parallel LAN/SAN Infrastructure
 Inefficient use of Network Infrastructure
 5+ connections per server – higher
adapter and cabling costs
• Adds downstream port costs;
cap-ex and op-ex
• Each connection adds additional points of
failure in the fabric
 Multiple switching modules in Blade
Chassis
 Longer lead time for server provisioning
Server with
NICs and HBAs
Blade Chassis
with I/O Modules
Ethernet
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
 Multiple fault domains – complex
diagnostics
 Management complexity
FC
Cisco Confidential
9
I/O Consolidation
 Reduction of server adapters
LAN
SAN A
SAN B
 Simplification of access layer and
cabling
 Gateway free implementation – fits in
installed base of existing LAN and
SAN
Nexus 5000
Nexus 5000
 Lower Total Cost of Ownership
 Fewer Cables
 Investment Protection (LANs and
SANs)
Server with
CNAs
Data Center Bridging
and FCoE
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Blade Chassis
with Nexus
4000
Ethernet
Cisco Confidential
 Consistent Operational Model
Fibre Channel (FC)
10
Adapter Evolution:
Consolidation Network Adapter
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
11
Operating System View
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
12
Evolution of 10G Ethernet Physical Media
Role of Transport in Enabling these Technologies!
Mid 1980’s
10Mb
UTP Cat 3
Mid 1990’s
Early 2000’s
100Mb
Late 2000’s
10Gb
1Gb
UTP Cat 5
UTP Cat 5
SFP Fiber
X2
SFP+ Cu (BER better than 10 -18)
SFP+ Fiber
Cat 6/7
Technology
Cable
Distance
Power
(each side)
Transceiver
Latency (link)
SFP+ CU
Copper
Twinax
7m
~0.1W
~0.1μs
SFP+ USR
Ultra short reach
MM OM2
MM OM3
10m
100m
1W
~0.1μs
SFP+ SR
Short reach
MM 62.5 μm
MM 50 μm
26-33m
66-300m
1W
~0.1μs
SFP+ LR
Long range
SMF G.652
10km
0.5W
10GBASE-T
Cat6
Cat6a/7
Cat6a/7
55m
100m
30m
~8W
~8W
~4W
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
2.5μs
2.5μs
1.5μs
13
Introduction to FCoE
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential (please do not distribute)
14
What is Fibre Channel over Ethernet?
 From a Fibre Channel standpoint it’s
FC connectivity over a new type of cable called… an Ethernet cloud
 From an Ethernet standpoints it’s
Yet another ULP (Upper Layer Protocol) to be transported
FCoE is an extension of Fibre Channel
onto a Lossless Ethernet fabric
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
15
Unified Fabric Overview
Fibre Channel over Ethernet (FCoE)
FCoE
Benefits
• Mapping of FC Frames over
Ethernet
• Enables FC to Run
on a Lossless
Ethernet Network
Ethernet
Fibre
Channel
Traffic
C97-485980-00
3/23/2016
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
• Fewer Cables
• Both block I/O & Ethernet
traffic co-exist on same
cable
• Fewer adapters needed
• Overall less power
• Interoperates with
existing SAN’s
• Management SAN’s
remains constant
• No Gateway
16
16
FCoE Enablers
 10Gbps Ethernet
 Lossless Ethernet
Matches the lossless behavior guaranteed in FC by B2B credits
FC Payload
CRC
EOF
FCS
Normal ethernet frame, ethertype = FCoE
Same as a physical FC frame
FC
Header
FCoE
Header
Ethernet
Header
 Ethernet jumbo frames
Control information: version, ordered sets (SOF,
EOF)
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
17
Unified I/O
Fibre Channel over Ethernet (FCoE)
FCoE is managed like FC at initiator, target, and switch level
Completely based on the
FC model
Easy to
Understand
Same
Operational Model
FCoE is
FibreTechniques
Channelof
Same
C97-485980-00
Same host-to-switch and switch-toswitch behavior as FC
Traffic Management
e.g. in order delivery,
FSPF load balancing
Same Management
and Security Models
WWNs, FC-IDs, hard/soft
zoning, DNS, RSCN
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
18
Network Stack Comparison
SCSI
SCSI
SCSI
SCSI
SCSI
iSCSI
FCP
FCP
FCP
FC
FC
FC
Less Overhead
than FCIP, iSCSI
FCIP
TCP
TCP
IP
IP
FCoE
Ethernet
Ethernet
Ethernet
PHYSICAL WIRE
SCSI
C97-485980-00
iSCSI
© 2008 Cisco Systems, Inc. All rights reserved.
FCIP
Cisco Confidential
FCoE
FC
19
FCoE Frame Format
Destination MAC Address
Source MAC Address
(IEEE 802.1Q Tag)
ET = FCoE
Ver
Reserved
Reserved
Reserved
Reserved
SOF
Encapsulated FC Frame (with CRC)
EOF
Reserved
FCS
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
20
FCoE Standards
Defined
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential (please do not distribute)
22
A larger picture
 IEEE 802
• Evolution of Ethernet (10 GE, 40 GE, 100 GE, copper and fiber)
• Evolution of switching (Priority Flow Control, Enhanced
Transmission, Congestion Management, Data Center Bridging
eXchange)
 INCITS/T11
• Evolution of Fibre Channel (FC-BB-5)
• FCoE (Fibre Channel over Ethernet)
 IETF
• Layer 2 Multi-Path
•TRILL (Transparent Interconnection of Lots of Links)
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
23
DCE versus DCB
 DCE is an old Cisco marketing term
 Cisco is now using the term DCB
The term IEEE uses
 Cisco supports the DCB standard activity
By implementing products that are DCB compliant
 CIN-DCBX – Cisco, Intel, Nuova Data Center Bridging Exchange protocol,
pre-standard
 CEE-DCBX – Converged Enhanced Ethernet Data Center Bridging
Exchange protocol, which is standards base
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
24
What’s FC-BB-5
 FC-BB-5 covers the majority of the FC features, using
Ethernet
 From an Ethernet perspective, FC-BB-5 is
Ethernet control plane referred to as FIP (Fibre Channel over
Ethernet Initiation Protocol)
discover and build virtual paths between end points
Ethernet data plane providing FCoE forwarding
including both FC control plane and FC data plane (FCF)
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
25
FC-BB-6
 It is an active working group of T11 that will discuss the
future of FCoE or FCoE v2.0
 It is just started, 18 months to have a standard
Approximate target spring 2011
 You can track it on
http://www.fcoe.com
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
26
Protocol Organization
FCoE itself …
 Is the data plane protocol
FIP (FCoE initiation
protocol)
 It is used to carry most of the
 It is the control plane protocol
FC frames and all the SCSI traffic
 It is used to discover the FC entities
connected to an Ethernet cloud
 It is used to login to and logout from the
FC fabric
The two protocols have:
• Two different Ethertypes
• Two different frame formats
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
27
What’s NOT FC-BB-5
 FC-BB-5 doesn’t deal with how lossless is realized in
Ethernet
no Priority Flow Control, Bandwidth Management, etc.
 FC-BB-5 doesn’t deal with management functions
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
28
IEEE DCB standards status
DCB technologies allow Ethernet to be lossless and to
manage bandwidth allocation of SAN and LAN flows
Feature / Standard
Standards Status
IEEE 802.1Qbb
Priority Flow Control (PFC)
Enable multiple traffic types to share a common
Ethernet link without interfering with each other
PAR approved
1.0 published
IEEE 802.1Qaz
Bandwidth Management (ETS)
Enable consistent management of QoS at the
network level by providing consistent scheduling
Data Center Bridging Exchange
Protocol (DCBX)
PAR approved
1.0 published
This is part of IEEE 802.1Qaz
Management protocol for enhanced Ethernet
capabilities
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
29
Data Center Ethernet:
PFC & Bandwidth Management
CoS based
Bandwidth Management
Priority Flow Control
Transmit Queues
Ethernet Link
Receive Buffers
Offered Traffic
Zero
Zero
One
One
Two
Two
Three
STOP
PAUSE
Three
Four
Four
Five
Five
Six
Six
Seven
Seven
Eight
Virtual
Lanes
• Enables lossless behavior
for each class of service
• PAUSE sent per virtual lane
when buffers limit exceeded
3/23/2016
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
3G/s
3G/s
2G/s
3G/s
3G/s
3G/s
3G/s
4G/s
6G/s
t1
t2
t3
3G/s
HPC Traffic
3G/s
2G/s
3G/s
Storage Traffic
3G/s
3G/s
3G/s
LAN Traffic
4G/s
5G/s
t1
t2
t3
• Enables Intelligent sharing of
bandwidth between traffic classes
control of bandwidth
• 802.1Qaz Enhanced Transmission
Nuova Systems Inc.
Cisco Confidential
10 GE Realized Traffic Utilization
30
30
DCBX Overview
Auto-negotiation of capability and configuration
Priority Flow Control capability and associated CoS values
Allows one link peer to push config to other link peer
Link partners can choose supported features and willingness to accept
Discovers FCoE Capabilities
Responsible for Logical Link Up/Down signaling of
Ethernet and FC
DCBX negotiation failures will result in:
vfc not coming up
Per-priority-pause not enabled on CoS values with PFC configuration
http://download.intel.com/technology/eedc/dcb_cep_spec.pdf
/
http://www.ieee802.org/1/files/public/docs2008
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
31
FCoE control plane
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
32
FIP: FCoE Initialization Protocol
 FCoE VLAN discovery
Automatic discovery of FCoE VLANs
 Device discovery
ENodes discover VF_Port capable FCF-MACs for VN_Port to VF_Port Virtual
Links
VE_Port capable FCF-MACs discover other VE_Port capable FCF-MACs for
VE_Port to VE_Port Virtual Links
The protocol verifies the Lossless Ethernet network supports the required Max
FCoE Size
 Virtual Link instantiation
Builds on the existing Fibre Channel Login process, adding the Negotiation of
the MAC address to use
Fabric Provided MAC Address (FPMA), or
Server Provided MAC Address (SPMA)
 Virtual Links maintenance
Timer based
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
33
Server Provided MAC
Addresses
Adapter uses burned-in or configured
MAC address:
Consistent with the Ethernet
model
FCF needs a table to map between
MAC addresses and FC_IDs
Fabric Provided MAC
Addresses
MAC address assigned for each FC_ID:
Consistent with the Fibre Channel model
Multiple FC-MAPs may be supported
One per SAN
No table needed for Encapsulation
Multiple MACs may be needed for NPIV
FC-ID
7.8.9
FC-MAP
(0E-FC-00)
24
bits
FC-MAP
(0E-FC-00)
MAC
Address
Burned in or Configured
48
bits
24
bits
FC-ID
7.8.9
48
bits
Cisco Nexus 5000 uses FPMA
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
34
Initial Login Flow ladder
ENode
FCoE Switch
VLAN
Discovery
VLAN
Discovery
FCF
Discovery
FCF
Discovery
FLOGI/FDISC
FLOGI/FDISC Accept
FC Command
FC Command
responses
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
FIP:
FCoE
Initialization
Protocol
FCOE
Protocol
35
FCoE data plane
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential (please do not distribute)
44
ENode: Simplified Model
 ENode (FCoE Node): a Fibre Channel HBA implemented within an
Ethernet NIC aka CNA (Converged Network Adapter)
 FCoE LEP : The data forwarding component that handles FC
frame encapsulation/decapsulation
 FCoE Controller is the functional entity that performs the FIP and
instantiates VN_Port/FCoE_LEP pairs.
FC Node
FCoE_Controller
FCoE_Controller
FCoE_LEP
FCoE_LEP
Enet
port
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
Enet
port
45
FCoE Switch: Simplified Model
 FCF (Fibre Channel Forwarder), the forwarding entity
inside an FCoE switch
FC
port
FCoE Switch
FCF
FC
port
FCoE_Controller
FC
port
FCoE_LEP
FC
port
Ethernet Bridge
Eth
port
C97-485980-00
Eth
port
Eth
port
© 2008 Cisco Systems, Inc. All rights reserved.
Eth
port
Cisco Confidential
Eth
port
Eth
port
Eth
port
Eth
port
46
FCoE Network
Topology
© 2009 Cisco Systems, Inc. All rights reserved.
Cisco Confidential (please do not distribute)
47
FCoE: Initial Deployment
SAN A
SAN B
10GE
Backbone
VF_Ports
Nexus 5000 (FCF)
VN_Ports
10GE
4/8 Gbps FC
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
48
FCoE: Adding Blade Servers
SAN A
SAN B
10GE
Backbone
VF_Ports
10GE
VN_Ports
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
4/8 Gbps FC
Cisco Confidential
49
FCoE: Adding Native FCoE Storage
SAN A
SAN B
10GE
Backbone
VN_Ports
VF_Ports
10GE
VN_Ports
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
4/8 Gbps FC
Cisco Confidential
50
FCoE: Adding VE_ports
SAN A
SAN B
10GE
Backbone
VE_Ports
VF_Ports
10GE
VN_Ports
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
4/8 Gbps FC
Cisco Confidential
51
Nexus Topologies
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
52
The Unified Data Center Architecture
NEXUS 7000
L3
L3
Service
Appliances
Catalyst
6500
NEXUS 5000
NEXUS 2000
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
VM VMVM
VM VM
B
Virtual Access: A virtual layer of network intelligence
offering access layer-like controls to extend
traditional visibility, flexibility and mgmt into virtual
server environments. Virtual network switches bring
access layer switching capabilities to virtual servers
without burden of topology control plane protocols.
Virtual Adapters provide granular control over virtual
and physical server IO resources
NEXUS 1000v
VM VM
VM VM
VM VM
VM VM
VM VM
POD
Rack 1 Rack 2 Rack 3
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Aggregation: Typical L3/L2 boundary. DC
aggregation point for uplink and DC services offering
key features: VPC, VDC, 10GE density and 1st point
of migration to 40GE and 100GE
Access: Classic network layer providing non-blocking
paths to servers & IP storage devices through VPC.
It leverages Distributed Access Fabric Model (DAF)
to centralize config & mgmt and ease horizontal
cabling demands related to 1G and 10GE server
environments
Unified
Compute
System
NEXUS 7000 VPC
A
vL2
Service
Modules
NEXUS 7000 VPC
L2
L2
Core: L3 boundary to the DC network. Functional
point for route summarization, the injection of default
routes and termination of segmented virtual transport
networks
Rack 1
Cisco Confidential
Rack x
53
Fitting the pieces together…
DC Core
Nexus 7000
10GbE Core
Gigabit Ethernet
WAN
10 Gigabit Ethernet
IP+MPLS WAN
Agg Router
4, 8Gb Fibre Channel
10 Gigabit FCoE/DCE
DC Aggregation
Nexus 7000
10GbE Agg
Catalyst 6500
DC Services
Catalyst 6500
10GbE VSS Agg
DC Services
SAN A/B
MDS 9500
Storage
Services
DC Access
Catalyst 6500
End-of-Row
C97-485980-00
Catalyst 49xx CBS 3100
| MDS 9100
Rack
Blade
© 2008 Cisco Systems, Inc. All rights reserved.
1GbE
Server Access
Nexus 7000
End-of-Row
Cisco Confidential
Nexus 5K|2K
Top of Rack
Nexus 1000V VN-Link
UCS blade
or
Nexus 4K
1GbE,10GbE Server Access
MDS 9500
Storage
54
Storage
Cisco Nexus 5000
Architecture
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
90
Hardware Architecture
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
91
Virtual Output Queues
Egress Port
VOQ 1
Packet
Buffer
VOQ N
Ingress Port
Scheduler
Packet
Buffer
Q1
Q1
Q8
Q8
Crossbar
Fabric
Q1
Q8
Egress
Queue
Egress
Queue
VOQ 1
Packet
Buffer
Q1
VOQ N
Q8
Q1
Q8
Egress
Queue
Egress Port
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
92
Policy Enforcement
VLAN Membership
Check
pass
fail
pass
fail
MAC and L3 Binding
(IP & Fibre Channel)
Control Plane Redirect/Snooping
Switch Port Analyzer (SPAN) and Diagnostic Sampling
Interface, VLAN, and
MAC Binding
pass
Parsed Packet
fail
Virtual Interface
Table (512)
Fibre Channel Zone
Membership Check
pass
permit
Determine
Destination
(ingress only)
deny
To Sup
Vlan State Table
(1K)
Fibre Channel
Switch Table (4K)
Multicast Vector
Table (4K)
RBACL
RBACLLabel
LabelTable
Table
(2K)
(2K)
deny
ACL Search Engine
(2K)
Role Based ACLs
(egress)
permit
Vlan
Vlan Translation
Translation
Table
Table(4K)
(4K)
Ethernet
Learning
Policy Enforcement
Binding
BindingTable
Table
(2K)
(2K)
Zoning
ZoningTable
Table
(2K)
(2K)
deny
QoS ACLs (ingress)
C97-485980-00
Station
StationTable
Table
(16K)
(16K)
VLAN ACLs (ingress)
permit
Collect Interface
Configuration and
State
fail
Port ACLs
permit
To SPAN
session
 Frames evaluated by multistage
engine searches occur in parallel
results, and are evaluated in pipeline
diagnostics, and control plane tap
pipelines.
policer
drop
Fibre Channel
Multipath Table (1K)
Multipath
Expansion
(ingress only)
PortChannelTable
(16)
Editing Instructions &
Virtual Output Queue List
Multipath Expansion
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
93
Default QoS Configuration
•
Qos is always on.
•
Four default class of
services defined when
system boots up
•
•
•
•
•
Two for control traffic.
One for FCoE traffic
and another one for
Ethernet traffic
Match CoS 3 for classfcoe.
Class-fcoe is nodrop with MTU 2240.
Match any for classdefault
Class-fcoe and
class-default get 50% of
guaranteed bandwidth
by default
C97-485980-00
switch1# sh policy-map
Type qos policy-maps
====================
policy-map type qos default-in-policy
class type qos class-fcoe
set qos-group 1
class type qos class-default
set qos-group 0
Type queuing policy-maps
========================
policy-map type queuing default-in-policy
class type queuing class-fcoe
bandwidth percent 50
class type queuing class-default
bandwidth percent 50
policy-map type queuing default-out-policy
class type queuing class-fcoe
bandwidth percent 50
class type queuing class-default
bandwidth percent 50
Type network-qos policy-maps
===============================
policy-map type network-qos default-uf-policy
class type network-qos class-fcoe
pause no-drop
mtu 2240
class type network-qos class-default
mtu 1538
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
switch2# show class-map
Type qos class-maps
===================
class-map type qos class-fcoe
match cos 3
class-map type qos class-default
match any
Type queuing class-maps
=======================
class-map type queuing class-fcoe
match qos-group 1
class-map type queuing class-default
match qos-group 0
Type network-qos class-maps
==============================
class-map type network-qos class-fcoe
match qos-group 1
class-map type network-qos class-default
match qos-group 0
96
Nexus 5000 Software Features Set
Layer 2
802.1w (Rapid Spanning Tree), 802.1s (Multiple Spanning
Tree), RPVST+, Root Guard, Uplink Guard, Bridge Assurance,
PortFast, CDP, PVLANs, UDLD, LACP, IGMP Snooping,
802.1Q trunks, Port-Channel, SVI, SPAN, Jumbo Frames, NTP,
Link State Tracking (LST)
Management/
Security
Radius, Tacacs+, AAA, CallHome, SSHv1/V2, telnet, IPv4 &
IPv6 mgmt, SNMP MiBs, Traps, EthAnalyzer (wireshark),
RBAC, DCNM, RME support via Cisco Works, syslog,
coredump, RMON, first-setup script, accounting log, checkpoint
and configuration rollback
ACL/QOS
FCOE
C97-485980-00
PACLs, VACLs, Session based ACLs, ACL based QOS,
egress Bandwidth Limiting, 802.1p priority, strict priority
scheduling, WRED, Tail Drop, Storm Control (broadcast,
multicast), Egress Shaper
FIP Snooping Bridge, DCBXP, PFC (Priority Flow Control),
8 Virtual Lanes, ETS (Enhance Transmission Selection)
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
99
Nexus 5000 and
FC Connectivity
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
100
Switch Mode
 Nexus 5000 FC module can be ISL’ed to another FC switch
(E_port)
 Zoning, DPVM, etc. are enforced on the Nexus 5000
 Domain manager, FSPF, zone server, fabric login server,
name server run on Nexus 5000
 Require a domain ID for every VSAN
 Interop mode considerations when connecting to non-Cisco
FC switches
 Note: Nexus 5000 supports direct connectivity to FC
initiator (server HBAs) and targets (storage arrays)
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
101
N-Port Virtualization (NPV) mode
 Nexus 5000 FC module can work in NPV mode
Server-facing ports are regular F ports
Uplinks toward SAN core fabric are NP ports
 Nexus 5000 switches assign FCIDs to attached devices
First byte in FCID received from core SAN switch
 One VSAN per uplink on Nexus 5000 (will change in future)
No trunking or channelling of NP ports
 Zoning, DPVM, etc. are not enforced on the Nexus 5000
 Domain manager, FSPF, zone server, fabric login server,
name server
They do not run on Nexus 5000
 No local switching
All traffic routed via the core SAN switches
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
102
N-Port Virtualization (NPV): An Overview
NPV-Core Switch (MDS or 3rd party switch with NPIV support)
FC
F-port
NP-port
Can have multiple
uplinks – one VSAN per uplink
Two uplinks can be in the same VSAN
No port channel or trunking
F-ports
N-ports
Host
Host
Host
Nexus 5000 to SAN Fabric A & B
Assign FCIDs to servers – no domain to configure!
Servers log in (FLOGI) locally
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
103
Working with
Nexus 2148
(Optional)
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
113
Nexus 2000 Fabric Extender
Virtual Chassis
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
114
Nexus 2000 Fabric Extender
1GE Connectivity
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
115
Fabric Extender
Static Pinning
Uplink Modes
 Fabric Extender associates (pins)
a server side (1GE) port with an
uplink (10GE) port
 Server ports are either individually
pinned to specific uplinks (static
pinning) or all interfaces pinned to
a single logical port channel
 Behaviour on FEX uplink failure
depends on the configuration
Server Interface
goes down
Port Channel
 Static Pinning – Server ports
pinned to the specific uplink are
brought down with the failure of
the pinned uplink
 Port Channel – Server traffic is
shifted to remaining uplinks based
on port channel hash
Server Interface
stays active
C97-485980-00
© 2008 Cisco Systems, Inc. All rights reserved.
Cisco Confidential
116