Cisco MDS/Nexus SAN Portfolio
and Architecture
Mark Allen – Manager Storage Technical Marketing
BRKDCN-1309
Agenda
•
Data Center Trends
•
Fibre Channel Storage Networking
•
Converged Storage Networking
•
Storage Deployments
•
Storage Network Management
•
Wrap Up
Purpose of Storage
One Main
Job:
“Give me back the correct bit
I asked you to hold on for me.”
Everything we do in storage (including
storage networking) is based around
completing that job safely, securely,
reliably, and without error
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
4
Data Center Trends
IT Environments Experiencing Accelerating
Transitions
Infrastructure Agility Needed to Enable Greater Speed of Business
INDUSTRY DYNAMICS
DATA CENTER TRENDS
Internet of Things
Server Virtualization
40% CAGR by 2017
85% by 2018
Cloud
69% CAGR by 2017
Software Defined
Infrastructure
Analytics
Increased Flash Usage
69% CAGR by 2014-19
EFFECT ON STORAGE
INFRASTRUCTURE
Platform
6% Integrated
CAGR by 2015-18
14%
Integrated Infrastructure
58%
Hyperconverged
CAGR by 2015-18
65% Growth by 2017
7x Growth SSD by 2018
STORAGE GROWTH
CAGR by 2015-18
10Xs
Growth in Information
Created by 2020
Higher Demand
on Multiprotocol Storage
Sources: Cisco Visual Networking Index (VNI), Cisco Cloud Index, IDC, Gartner, IDC WW Integrated Systems Forecast 2014-2018 (Nov ‘14)
IDC WW Hyperconverged Systems 2015-2019 Forecast (April ’15)
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
6
Cisco’s Family of Storage Networking Solutions
LAN/SAN
Nexus
SAN
COMPUTE
UCS
MDS
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
7
Cisco Multi-Protocol Portfolio
SAN, LAN & Compute
LAN/SAN
Cisco Nexus
7000
Cisco Nexus
5500
Cisco
Nexus 2000
SAN
Cisco Nexus
9000
Nexus
5672UP-16G
Nexus
2348UPQ
Cisco MDS
9718/9710/9706
Cisco Nexus
5600
24 x 40G
FCoE Module
Cisco
Nexus 3000
Cisco
MDS 9148S
COMPUTE
48x16G
FC Module
Cisco UCS
6300 Series FI
Cisco UCS
6200 Series FI
48x10G
FCoE Module
MDS 9700 24/10 SAN
Extension Module
Cisco
MDS 9396S
Cisco
MDS 9250i
Cisco UCS B-Series
Blade Servers
Cisco UCS C-Series
Rack Servers
Common OS, Common Management
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
8
MDS 9700 Series Multi Layer
Directors
Cisco MDS 9710 Multilayer Director
Investment Protection for the Next Decade
INDUSTRY’S HIGHEST
PERFORMANCE AND
CAPACITY
14 RU
1.5-Tbps/Slot
384 Line-Rate 16G FC Ports
INDUSTRY’S MOST
RELIABLE
•
Up to 8 Line Cards
•
Up to 6 Fabric Modules
•
Dual Supervisors
STORAGE DIRECTOR
N+1 Fabric
UNMATCHED
FLEXIBILITY
WITH MULTI-PROTOCOL
CONNECTIVITY
Launched May 2013
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
10
Cisco MDS 9706 Multilayer Director
Extending MDS 9710 Director Qualities to a Smaller Form Factor
9RU
3x THE PERFORMANCE
OF ANY COMPACT
DIRECTOR
INDUSTRY’S MOST
RELIABLE COMPACT
DIRECTOR
• AND 15X the performance of
current MDS 9506 director
• Eliminate loss of bandwidth
• Grow without forklift –
investment protection for future
• Eliminate Downtime
N+1 Fabric Redundancy
In-Service Software Upgrade
1.5 Tbps/slot
Switching Capacity
Front-Back Airflow
Scale up to 192 Line Rate Ports –
16G FC or 10G FCoE
Dual, Redundant Supervisors
• Preserve IT operations and
Knowledge – ease of migration
with NX-OS and DCNM
Redundant power supplies/fans
• Maintain Performance
Reduced Failure Domains
Launched September 2014
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
11
Cisco MDS 9718 Multilayer Director
Industry’s First Ultra-High Density Director
Same Features and Functionality as MDS 9706 and 9710
• High Availability – Redundant Supervisors, Power Supplies
and Fabric Modules
• Performance – 1.5 Tbps bandwidth per slot = 768 x 32Gbs
line rate ports
• Investment Protection – Same NX-OS, Line Cards, Power
Supplies as other MDS 9700 directors
Plus
• New Supervisor 1E (Enhanced) with 2x Compute and 4x
Memory of Supervisor 1
• Fully populated Fabric Modules – 32Gbps ready at FCS
Designed for Consolidation and High-Scale SAN Fabrics
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
12
Comparison: 9706, 9710 and 9718
Hardware Feature
MDS 9706
MDS 9710
MDS 9718
4
8
16
RU
9RU
14RU
26RU
Fabric Module slots
(default / available)
3/6
3/6
6/6
2
2
2
Back
Back
Back
Front to Back
Front to Back
Front to Back
4
8
16
Line Card slots
Sup Slots
Fabric Module location
Airflow
Power Supply slots
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
13
Cisco MDS 9700 - Designed for Scale
1
Host Ports
2
FC, FCoE Modules
3
Number of Fabric
Cards
FC Bandwidth per Slot
FCoE Bandwidth per Slot
1
256 Gbps
220 Gbps
2
512 Gbps
440 Gbps
3
768 Gbps
660 Gbps
4
1024 Gbps
880 Gbps
5
1280 Gbps
1100 Gbps
6
1536 Gbps
1320 Gbps
4
5
6
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
14
The Benefits Of Multiple Fabrics
Possible Front-Panel Bandwidth Per Slot
1536
Number Of
Fabric Cards
Front-Panel FC
Bandwidth / Slot
1
256 Gbps
2
512 Gbps
3
768 Gbps
N+1 redundancy at 8G FC
4
1024 Gbps
N+1 redundancy at 16G FC
5
1280 Gbps
6
1536 Gbps
Cisco MDS 9700
Bandwidth - Gbps
1280
1024
768
512
Future
Proof
Growth
256
0
1
2
3
4
Number Of Fabric Cards
5
6
Two key design benefits:
1.
Operational Redundancy: Uptime at full performance
2.
Investment Protection: Scale up with addition rather than replacement:
(32G FC, 40GE FCoE)
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
15
MDS 9710 Supervisor Modules
•
Half-width form factor
•
10/100/1000 Management port, RJ45 Console port, 2 x USB 2.0 ports
•
More powerful Supervisors allow greater scalability
Memory
# of Cores
Clock Speed
Instruction
MDS 9500
Sup 2A
MDS 9706/9710
Sup 1
MDS 9718
Sup 1E
2G
8G
32G
1
4
8
1.4 Ghz
2.1 GHz
2.1 GHz
32-bit
64-bit
64-bit
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
16
MDS 9700 Power Supplies
•
New form factor 3000W power supply module
•
•
•
Both AC and DC power supplies available
•
•
Can mix AC and DC power supplies in same
chassis
Grid A
Grid B
Grid Redundancy for all chassis
•
•
•
•
Autosensing voltage detection
80Plus Platinum Certified (>94% efficiency)
Four Power Supplies – MDS 9706
Six to Eight Power Supplies – MDS 9710
Twelve to Sixteen Power Supplies – MDS 9718
N+N:N+N Power Redundancy optional for MDS
9710 and 9718 chassis
BRKDCN-1309
Power Distribution
Unit-A
Power Distribution
Unit-B
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
17
MDS 9700 48-Port 16G Fibre Channel Line Card
48-port 16-Gbps Line Card
Performance
48 x 16 Gbps ports, 768 Gbps (3 Fabric Modules for line rate speed)
Port Speeds
2, 4, 8, 10 and 16 Gbps Fibre Channel
Optics (sfp+)
2/4/8G FC, 4/8/16G FC, 10G FC, 10GE (FC with 10GE clock)
Port Types
F, FL, TF, E, TE, SD, ST
Port Groups
Twelve 4-port port-groups
Intelligent Capabilities
VSAN, IVR, FC Redirect
Buffer-to-buffer credits
Up to 500 per port, 4095 with Enterprise License (510km @ 16G)
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
18
MDS 9700 24/10 SAN Extension Module
24-port FC, 8-port 1/10GE and 2-port 40GE FCIP line card
FC Performance & Features
Same as 48-port FC line card
FCIP Port Speed
8 x 1/10GE ports (FCS), 2 x 4GE port (post FCS)
FCIP Port Groups
IPS 1/10GE ports 1-4, 40GE port 9 (Storage Service Engine 1)
IPS 1/10GE ports 5-8, 40GE port 10 (Storage Service Engine 2)
FCIP Port Types
VE, VTE
Intelligent Capabilities
VSAN, IVR, IP Sec Encryption, IP Compress, Disk Write Acceleration, Tape Write
Acceleration
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
19
MDS 9700 48-port 10GE FCoE Line Card
48-Port 10GE FCoE Line Card
Performance
48 x 10 Gbps ports, 480 Gbps (3 Fabric Modules for line rate speed)
Port Speed
10 Gbps Ethernet
Port Types
VF, VE, VTE
Port Groups
Twelve 4-port port-groups
Intelligent Capabilities
VSAN, IVR
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
20
MDS 9700 24-port 40GE FCoE Line Card
24-Port 40GE FCoE Line Card
Performance
24 x 40 Gbps ports, 960 Gbps (5 Fabric Modules for line rate speed)
Port Speed
40 Gbps Ethernet
Port Types
VF, VE, VTE
Port Groups
Four 6-port port-groups
Intelligent Capabilities
VSAN, IVR
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
21
MDS 9000 Series Fabric Switches
Cisco MDS 9148S Fabric Switch
Front
Back
1 RU
48 x 16G FC Line Rate Performance
Expand from 12- to 48-ports in 12-port increments
VERSATILE
Dual Power Supplies and Fans for Enterprise-Class Availability
EASY TO USE
ENTERPRISE-CLASS
• Line-rate 16/8/4/2G FC Ports
• Automated Provisioning
• Non-disruptive software upgrades
• Industry-leading port range
• Quick Configuration Wizard
• Up to 32 Virtual SANs (VSANs)
• Same OS and Management across
Industry’s broadest SAN Portfolio
• Inter-VSAN Routing (IVR), QOS,
PortChannels, N-Port ID
Virtualization (NPIV), N-Port
Virtualization (NPV), Comprehensive
Security
12-port base
Scale up with 12-port licenses
Full 48-port option available
• Hardware-based slow-drain
detection and recovery
High-Performance, Easy to Deploy,
Enterprise-class Fabric Switch
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
23
MDS 9396S Fabric Switch
2RU 96-port 16G Line-Rate Switch
Hardware Platform
•
•
•
2RU 96-port 16G FC fixed fabric switch
2/4/8 and 16-Gbps Line-rate Fibre Channel speeds
Redundant power supplies and fans, with Port Side Exhaust or Port Side
Intake air flow
Front View
Enterprise Class
•
•
•
Up to 4095 B2B credits / port
FC TrustSec Encryption
Forward Error Correction
Fabric Switch-like Capabilities
•
•
•
On demand port licensing (48-port base with 12-port incremental licenses)
NPV Mode
Power-On-Auto-Provisioning (POAP), Quick Configuration Wizard
Rear View
Blend of Fabric Switch and Director Capabilities
Deploy as a Stand-Alone switch or as an Edge Switch for an Edge-Core Network
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
24
MDS 9396S: Performance of a Director
Based on the architecture of 48-port 16G FC line card of MDS9700 director
•
Same Port ASICs
•
Same Crossbar
ASIC
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
25
MDS 9000 Series Multiservice
Fabric Switch
MDS 9250i Multiservice Fabric Switch
Next-Gen Storage Services Platform for Unified Fabric
2 Ports 10GE FCIP/iSCSI
1+1 Redundant Fans
2+1 Redundant Power Supplies
1
40 Ports 16G FC
•
16G FC, 10GE FCoE, 10GE FCIP, FICON, iSCSI
Rich set of Storage Services for FC and FCoE
SANs:
•
•
FCIP, IO Accelerator (IOA), Data Mobility Migration
(DMM),
Integrated Management via Data Center Network
Manager (DCNM)
3
4
FCoE
5
6
7
8
8 Ports 10GE FCoE
Features
Line Rate Performance for:
2
Benefits
Single Platform for deploying Storage Services
across FC and FCoE Storage Area Networks
(SANs):
•
•
High-Bandwidth SAN Extension across MAN/WAN
Migrate Data between FC and FCoE arrays
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
27
MDS 9250i Multiservice Fabric Switch
BUSINESS CONTINUITY/
DISASTER RECOVERY
Production
DC
FC SAN SWITCH
Disaster Recovery
DC
FC
IP WAN
SAN
SAN
FC
FCoE
FC
SAN
FCoE
FC
DATA MIGRATION
FC SAN GATEWAY
Converged
Fabric
SAN
SAN
FCoE
Migrate data between
heterogenous storage
FC
FC
SAN
FCoE
One SAN Appliance, Multiple Use-Cases
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
28
MDS Architecture
Cisco MDS Architecture Overview
MDS Family - Arbitrated Crossbar Architecture
•
•
Crossbar Switch with Central Arbitration
Crossbar maybe external (MDS 9700 Directors, MDS
9396S) or integrated into ASIC (MDS 9250i and MDS
9148S)
Control and
Scheduling
Ingress
Crossbar establishes a temporary connection between
input and output port for duration of the frame
exchange
Crossbar
•
Frames are not transmitted unless there is an available
path
•
Scheduler uses arbiter grants to provide traffic fairness
and QoS
•
Virtual Output Queues used to eliminate HOL blocking
BRKDCN-1309
Egress
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
30
Cisco MDS Frame Flow
PHY
Optical to Electrical
interface:
• SFP+ (2/4/8/10/16 Gbps)
Error Checking
Access Control
Timestamp
Forwarding and Load balancing
VSAN Header
Inter VSAN Routing
Buffering and Virtual Output Queues
Redundant Arbiters
QoS
Arbitration Requests
Interface Statistics
Linecard
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
MAC
Port Type / Modes:
• F_Port
• Loop port (FL_Port)
• ISL (E_Port)
• EISL (TE_Port)
• SPAN Port (SD_Port)
Crossbar
Switch
Crossbar
Fabric
Switch
Crossbar
Fabric Fabric
Switch
Fabric Fabric
Forwarding
Transmit, loopback or Expire frame
Egress Access Control and QoS
Remove VSAN header (end device) or prepend
EISL header (EISL to Cisco switch)
Forwarding and Load balancing
Fabric
Redundant crossbar fabrics
Inter VSAN Routing
•
Frames always follow same logic – ingress to egress (consistent switching latency)
•
Newer generation line cards use fewer components compared to first generation line cards, however, the frame
forwarding logic is the same
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
31
Frame Flows
1. Frame arrives on ingress PHY/MAC
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
•
•
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Fabric
PHY
•
•
Forwarding
Fabric
Optical to electrical conversion
2/4/8/10/16-Gbps SFP
MAC
•
•
•
•
Decipher FC frames from incoming bit stream using SOF/EOF primitives
Issue R_RDYs (return buffer credits)
Check incoming frame for errors (CRC check)
Prepend switch-internal header
•
Ingress port, ingress VSAN, QoS markings, frame arrival time, etc
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
32
Frame Flows
2. Ingress Forwarding logic
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Forwarding
Fabric
Per-VSAN
Forwarding Table
Lookup
Incoming
Frame
Statistics Lookup
Update Flow,
Interface & VSAN
Statistics
Per-VSAN
Ingress ACL
Lookup
Multiple
Destinations?
Yes
Drop Frame
•
Fabric
Permit
Load-balancing
SRC/DST
require
Inter VSAN
Routing?
to Queuing
Yes
FC-NAT
H/W-based frame forwarding distributed on each line card:
VSAN-aware (forwarding based on VSAN & D_ID – “virutal fabric aware”)
• Ingress ACLs (Hard zoning)
• Load-balancing (FSPF and/or Port Channels)
• Inter VSAN Routing including FC-NAT (integrated – no loss of performance)
•
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
33
Frame Flows
3. Ingress Queuing / Request-to-send logic
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
Request Grant from
Arbiter
Incoming
Frame
Determine Destination
Virtual Output Queue
Queue Frame on
ingress-port VoQ
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Forwarding
Fabric
Signal free
Destination
Queue slot
Wait
For
Grant
Fabric
to Crossbar
Switch Fabric
Transmit Frame
Choose Crossbar
Channel
PQ
DWRR 2
Classify
DWRR 3
DWRR 4
Ingress FC Port
Transmit
Queue
 Virtual Output queuing
 Interface to centralized arbitration
 Buffering/Queuing (in case of congestion)
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
34
Frame Flows
4a. Centralized Arbiters
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Forwarding
 Responsible for scheduling
frames from ingress to egress
 Capable of scheduling more
than a billion frames/sec
 Ensures fairness when there
is congestion on output ports:
Congestion does not result in
blocking because of VoQs
Fabric
Switch without VOQ
Fabric
VOQ model
Top of queue
Frame to port 4
Frame to port 6
Frame to port 6
Frame to port 4
Frame to port 4
Frame to Port 6
Frame to port 5
Frame to port 5
Top of virtual
output queue
Frame to port 5
Frame to port 5
Top of virtual
output queue
Top of virtual
output queue
Frame to port 4
Frame to port 4
Frame to port 6
Frame to port 6
Frame to port 6
Input queue at port 1
Input queue at port 1
 Frame to port 4
Input queue at port 1
Input queue at port 1
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
35

Frame Flows
4b. Crossbar switch fabrics
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
Forwarding
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Fabric
•
Low-latency, high-throughput non-blocking, non-oversubscribed crossbar
switch fabric
•
Guaranteed bandwidth for frames being transmitted
•
No over-subscription in MDS 16G platforms
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Fabric
36
Frame Flows
5. Egress forwarding logic
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Forwarding
Fabric
Permit
Per-Port
Class-of-Service
Output Queue
IVR
FC-NAT Rewrite
Drop Frame
Per-VSAN Egress ACL
Lookup
Deny
•
Check frame integrity (again)
•
Egress ACLs
•
Egress QoS
Drop Frame
Valid
Check Frame for
Errors
Bad
BRKDCN-1309
Fabric
Incoming
frame
from Crossbar
Switch Fabric
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
37
Frame Flows
6. Egress MAC/PHY logic
Sup
Central
Arbiter
Sup
Central
Arbiter
Queuing
PHY / MAC
Forwarding
Crossbar
Crossbar
Switch
Switch
Crossbar
Crossbar
Fabric
Fabric
Switch
Switch
Crossbar
Crossbar
Fabric Fabric
Fabric Fabric
Switch
Switch
Fabric Fabric
Fabric Fabric
Fabric
•
Remove internal switch header
•
If output port is a Trunked E_Port (VSAN Trunking), prepend EISL header
•
Check frame timestamp (drop expired frames)
•
Transmit frame onto wire with appropriate encoding & primitives
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Fabric
38
VSANs – The Technical Details
EISL Header – What’s inside?
4B
SOF
•
Several fields are added to the VSAN header that
provide a variety of functions
8B
EISL Header
•
Each frame on a VSAN trunk carries up to an
extra 8 bytes of header which includes:
24B
FC Header
≤2112B
Payload
4B
CRC
EOF
•
•
•
•
•
User priority – 3 bits – used for QoS functionality to designate
priority of frame
VSAN ID – 12 bits – used to mark the frame as part of a
particular VSAN – supports up to 4096 VSANs
MPLS flag – 1 bit – used to designate whether this frame is
subject to Multi-Protocol Label Switching processing – future
use
Time-to-live (TTL) – 8 bits – used to help avoid routing loops
Other misc. fields including version, frame type, and other
reserved fields
BRKDCN-1309
4B
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
39
Designed For Fabric Resiliency
CRC Detection and 16 Gbps FC Forward Error Correction (FEC)
•
Lossy Media such as loose SFPs,
dirty cables can result in packets
getting corrupted
•
CRC Checking drops corrupted
frames from end devices or
internally corrupted frames
•
FEC corrects frames corrupted inflight to preserve frames
Ingress
CRC
checking
Forward
Error
Correction
Drop
frame
BRKDCN-1309
Internal
CRC
checking
Drop
frame
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
40
Converged Storage Networking
Industry’s Broadest Converged Ethernet Portfolio
Cisco NEXUS
Cisco MDS
Cisco UCS
Application Centric
Infrastructure (ACI)
Max FCoE / IP
Ports/Chassis
Nexus 7700/7000
Nexus 5600
Nexus 2300
Nexus 9000
MDS 9000
UCS FI
6200/6300
10GE
1536 (IP),
768 (FCoE)
384
48
2048
768 (FCoE only)
2 (IP)
96
40GE
384
96
6
512
384 Ports
32
Use 10/40G Ethernet to Converge IP and Fibre Channel Storage Traffic
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
42
Cisco Nexus 5672UP-16G
All Features of Nexus 5600 and More
•
Flexible – Traditional Ethernet plus
Storage: File, iSCSI, FCoE and FC
•
Reduced cost – Deploy once,
implement solutions as needed
24 Fixed 1/10G SFP+
Ports
24 Unified Ports provide
2/4/8/16G FC
or 10G Ethernet/FCoE
6x 40G QSFP+ Ports
Flexibility to use 4x10G or
40G
Enhanced 5672UP for SAN
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
43
Cisco Nexus 2348UPQ Fabric Extender
•
Flexible – Deploy ports as
required, LAN or SAN
•
Reduced cost – Lower cost
than Ethernet switch
•
Reduced management –
Configuration and OS done on
parent switch
Unified Ports provide 2/4/8/16G
FC or 10G Ethernet/FCoE
Up to 24 x 16G FC ports
Up to 48 x 2/4/8G FC ports
6 x 40G QSFP+ Ports
Flexibility to use
4 x 10G or 40G
First FEX Solution Designed for All Storage Connectivity
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
44
Next Generation UCS Fabric Interconnect
UCS FI 6332, UCS FI6332-16UP, IOM 2304
FI 6332
• High-density 40GbE
ports: enables 40G
end-to-end Fabric
32 x 40GbE QSFP+ ports
FI 6332-16UP
24 x 40GbE QSFP+ and 16 x UP ports
(1/10GbE or 4/8/16G FC)
• 2.6X increase in
throughput
• 3X lower latency
IOM 2304
8 x 40GbE server links and
4 x 40GbE QSFP+ uplinks
Enabling High Performance, Low-Latency and Lossless Fabric
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
45
Storage Deployments
MDS 9718 Use Case:
Switch Convergence
64 Target ports
per fabric
BENEFITS:
64 Target ports
per fabric
REDUCED
MANAGEMENT
 Fewer switches to manage
 No ISLs to manage
REDUCED
POWER
 Fewer switches
 Fewer ports deployed (no
ports used by ISLs)
REDUCED
CABLING
704 Server ports
per fabric
CORE-EDGE DESIGN
 6 x 384 port directors
 256 ISL ports
 1792 ports deployed
 Elimination of ISL cables
704 Server ports
per fabric
COLLAPSED CORE DESIGN
 2 x 768 port directors
 0 ISL ports
 1536 ports deployed
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
47
MDS 9718 Use Case:
Scaled Growth
BENEFITS:
384 Target ports
per fabric
384 Target ports
per fabric
CORE-EDGE DESIGN
….8….
….8….
….8….
2688 host ports per
fabric




26 x 384 port directors
3072 ISL ports
9216 ports deployed
Core at maximum capacity


Fewer switches to manage
Fewer ISLs to manage
REDUCED
CAPEX


Fewer switches deployed
Fewer ports deployed
(fewer ISLs)

Elimination of ISL
ports/cables
….8….
2688 host ports per
fabric
EDGE-CORE-EDGE DESIGN
REDUCED
OPEX
REDUCED
CABLING
CORE-EDGE DESIGN




4 x 768 port directors, 16 x 384 port directors
1536 ISL ports
7680 ports deployed
Core can continue to grow storage and edges
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Cisco 40G QSFP BiDi Transceivers
Utilizing Existing Duplex Fiber
MPO-12 Cable with 40G QSFPSR4
LC Cable with 40G QSFP-BiDi
LC Fiber Cable
Each Fiber Strand Carries
16G/10 Gbps
MPO-12 Fiber Cable
8 Fiber Strands Carry 10 Gbps Each;
4 Fiber Strands Unused
LC Fiber Cable
Duplex 20 Gbps (Receive and
Transmit on Two Different Wavelengths)
Duplex LC
12-Fiber MPO
Duplex LC
OM3 MMF: 300m (10G), 100m (16G)
OM4 MMF: 400m (10G), 125m (16G)
12-Fiber MPO
LC
LC
LC Cable with SFP+
OM3 MMF: 100m
OM4 MMF: 150m
OM3 MMF: 100m
OM4 MMF: 150m
No Need to Upgrade the Fiber Plant
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
49
40GE ISL Use Case:
ISL Consolidation
BENEFITS:
64 Target ports
per fabric
64 Target ports
per fabric
256
ISL Ports
704 Server ports
per fabric
REDUCED
MANAGEMENT
 Fewer ISLs to manage
REUSE
CABLING
 BiDi Optics allow use of
existing LC cabling
96
ISL Ports
704 Server ports
per fabric
16G FC ISLs
 6 x 384 port directors
 256 ISL ports
 1792 ports deployed
40GE FCoE ISLs
 6 x 384 port directors
 96 ISL ports
 1632 ports deployed
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
50
40GE ISL Use Cases:
Converged Networks to Fibre Channel SAN
40GE
FCoE
10GE
FCoE
NEXUS Fixed / Modular
40GE
FCoE
40GE
FCoE
10GE FCoE
UCS with 6333 Fabric Interconnect
NEXUS with FEX
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
51
Cisco Nexus 5672UP-16G Use Cases
Ethernet LAN
FC SAN
Ethernet LAN
Nexus Spine
40GE
8/16G FC
16G FC
40GE
5672UP-16
5672UP-16
10GE
5672UP-16
16G FC
10GE FCoE
Ethernet and FC Connectivity
Converged Access to FC and
IP Storage
BRKDCN-1309
IP Storage
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
52
Cisco Nexus 2348UPQ Use Case:
LAN/SAN Access Convergence
TRADITIONAL RACK
ARCHITECTURE
To LAN
UP FEX RACK ARCHITECTURE
To LAN/SAN
Parent Switch
To SAN
2 x TOR LAN
Switches
2 x TOR SAN
Switches
2 x LAN/SAN
2348UPQ FEX
Rack Mount
Servers
Rack Mount
Servers
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
53
Storage Networking Management
Data Center Components
Network:
Nexus and MDS Fabrics
Compute:
Cisco UCS
Network
Virtual Compute:
VMware
Compute
Storage Arrays:
FC Block Based Vendors
Storage
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
55
Nexus and MDS Fabric Management Solution
Simplified Operations of NX-OS
AUTOMATE
VISUALIZE
DIAGNOSE
MEASURE
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
PROGRAMABLE
SCALABLE
CONFIGURE
Slowdrain Analytics
Automates Troubleshooting
• Collects the whole fabric at once
Automates Collection
• From hours of collection to minutes
Reduces False Positives
• prioritizing ports highest severity counters
Shows Fluctuations in counters
• Graphs counters
• Enables user to zero in on specific counters
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
57
SAN Host Path Redundancy Analysis
On-Demand and Automated Redundancy Checks
Nexus5000-FabA
Fabric A
MDS9710-FabA
Masking
Database
VSAN 2,10,12
VSAN 10
VSAN 10
Port A
LUN 1 > A/B
HBA0
Fabric Separation
Host_Memoria
LUN 2 > A/B
HBA1
Fabric B
VSAN 11
VSAN 3, 13
11
Nexus5000-FabB
Enclosure
Port Down on
Redundant
Path
VSAN initiator
to target
Mismatch
13 Storage
VSAN 11
Array
Port B
LUN 3 > A
LUN 4 > A/B
MDS9710-FabB
VSAN
Segmentation
Both Paths On
Same Line
Card
Physical
Fabric
Separation
Port LUN
Masking
Mismatch
Root
Cause
Reduce
Mean Time to Repair
Risk to Switch and Array
Upgrades
Risk to Hardware
Maintenance Activities
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Visualize Hypervisor: VMware
N5K-access-71
M97-75
Windows
2k8-R2-151.102
•
VM and ESX Inventory
•
Monitor VM and ESX
CPU/Memory
•
Monitor storage and
network traffic & Events
•
Max Datastore Latency
172.20.151.101
Storage
Array •
RedHat-151.103
N5K-access-70
ESX
UCS
or GENERIC
SERVER
M97-74
FC + FICON
CNA Adapters
VMs
VM to DataStore
Mapping
FCoE + iSCSI
MDS
or NEXUS
ISL/FCIP
MDS or NEXUS
STORAGE PORT
& LUN(s)
End-to-End Visibility
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Visualize Compute: Cisco UCS
UCS-FI-6120XP
MDS9710-16GBIT
Windows
2k8-R2-151.102
172.20.151.101
UCS-FI-6120XP
•
Blade Inventory
•
Module and Port view
UCS
or GENERIC
SERVER
MDS9710-16GBIT
FC + FICON
CNA Adapters
ESX
Mapping Service Profile
Storage
Array
RedHat-151.103
VMs
•
FCoE + iSCSI
UCS FI or MDS
or NEXUS
ISL/FCIP
MDS or NEXUS
STORAGE PORT
& LUN(s)
End-to-End Visibility
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Visualize Virtual and Physical Fabrics
N5K-access-71
M97-75
Windows
2k8-R2-151.102
172.20.151.101
Storage
Array
RedHat-151.103
N5K-access-70
ESX
UCS
or GENERIC
SERVER
Switch and Port Events
•
Average & Peak TX/RX
•
Switch CPU/Memory
•
Performance monitoring
of ISLs, Trunks, FCIP
•
Discards and Errors
M97-74
FC + FICON
CNA Adapters
VMs
•
FCoE + iSCSI
MDS
or NEXUS
ISL/FCIP
MDS or NEXUS
STORAGE PORT
& LUN(s)
End-to-End Visibility
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Visualize Storage Systems
N5K-access-71
M97-75
Windows
2k8-R2-151.102
172.20.151.101
Storage
Array
RedHat-151.103
N5K-access-70
UCS
or GENERIC
SERVER
Array port mapping
•
Host to LUN mapping
•
Array Capacity View
•
Array Inventory
•
Capacity Report
M97-74
FC + FICON
CNA Adapters
VMs
ESX
•
FCoE + iSCSI
MDS
or NEXUS
ISL/FCIP
MDS or NEXUS
STORAGE PORT
& LUN(s)
End-to-End Visibility
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
Configure: Wizards and Templates
Fabric Provisioning
• Wizards & Templates
OS Upgrade Wizard
Backup/Roll Back
Data Replication
FCIP I/O Acceleration
Provision Virtual
Port Channel
Provision
Zone
Network
Storage
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
63
Diagnose
Database
Crashes
Event Filtering and De-dup
Misconfiguration Analysis
Rule Based Fault Notification
Health Score
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
64
Measure
Port inventory
Port Capacity
Array Capacity
FORECAST
Identify
Bottlenecks
Identify Orphaned
Ports
Top Array
Ports & Hosts
VIEW + REPORT CAPACITY and PERFORMANCE
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
65
DCNM Resources
http://cisco.com/go/dcnm
Videos
Release Notes
Datasheets
Configuration Guides
Installation and Licensing Guide
Programmable Guide
http://cisco.com/go/license
Evaluation and Permanent Licensing
License transfer to another DCNM server
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
66
Wrap Up
Complete Your Online Session Evaluation
•
Please complete your Online
Session Evaluations after each
session
•
Complete 4 Session Evaluations &
the Overall Conference Evaluation
(available from Thursday) to receive
your Cisco Live T-shirt
•
All surveys can be completed via
the Cisco Live Mobile App or the
Communication Stations
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
CiscoLive.com/Online
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
68
Continue Your Education
•
Demos in the Cisco campus
•
Walk-in Self-Paced Labs
•
Lunch & Learn
•
Meet the Engineer 1:1 meetings
•
Related sessions
BRKDCN-1309
© 2017 Cisco and/or its affiliates. All rights reserved. Cisco Public
69
Q&A
Thank You