White Paper
Key Factors Driving Refresh of the Cisco MDS
9500 Series with the MDS 9700 Series
Over the last decade, the Cisco® MDS 9500 Series of director-class switches has
been part of a resilient, high-performing, operationally efficient SAN. Over the same
period, the demand for storage, bandwidth, and scale have also increased steadily. If
current trends continue, we can expect unprecedented growth as we step into the era
of virtualization, application proliferation, and affordable “all-flash” arrays. That is why
it makes more business sense to migrate the Cisco MDS 9500 product line to the
newer, more scalable and feature-rich Cisco MDS 9700 Series.
The most significant factors driving a technology refresh to the 9700 Series can be
summarized as:
• End-of-life considerations
• Increased bandwidth requirements
• SAN consolidation
• Stricter guaranteed uptime requirements
• Operational ease
• Scaling up versus scaling out
In this white paper, we describe each of these factors in detail and finally summarize
why a refresh with the 9700 Series is so relevant in the current scenario.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 1 of 14
Contents
End-of-Life Considerations ..................................................................................................................................... 3
Increased Bandwidth Requirements ...................................................................................................................... 4
Stricter Guaranteed Uptimes .................................................................................................................................. 6
Operational Ease ..................................................................................................................................................... 7
Scale Up Versus Scale Out ..................................................................................................................................... 7
A Classic Use Case Of Consolidation .................................................................................................................... 9
Customer Testimonials ......................................................................................................................................... 12
Summary ................................................................................................................................................................ 12
Six Simple Steps For Migration ............................................................................................................................ 13
FAQ ......................................................................................................................................................................... 14
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 2 of 14
End-of-Life Considerations
End-of-life announcements have already been made for a number of Cisco MDS 9500 modules that support up to
4-Gbps Fibre Channel (4GFC), and these modules stopped shipping in August 2015. The 8-Gbps Fibre Channel
(8GFC) module variants have imminent end-of-life dates. Evidently, an important decision is upon SAN owners as
to whether to fill up the empty slots or leave them empty forever. Some considerations that can help SAN owners
make a prudent and future-ready decision are:
●
It doesn’t make good investment sense to fill up old technology in their current inventory.
●
Cisco MDS 9500 devices have depreciated heavily in book value over the years.
●
Older hardware is prone to failures, as it is all subject to mean time between failures (MTBF).
●
Return materials authorizations (RMA) may have longer lead times due to challenges in sourcing parts from
vendors.
●
A major concern for SAN managers is reliability and uptime, which are both at risk with hardware that is
soon to be end of life.
●
It is highly advisable to consider a SAN refresh alongside a SAN update to avoid having to manage different
generations of hardware and software in the data center, which can be very costly.
As seen in Table 1, the entire Cisco MDS 9500 line is soon to be marked end of life/end of sale to make way for
the newer Cisco MDS 9700 products, which sufficiently address current market demands and also help ensure that
Cisco continues to invest in technology that empowers our customers to address the demands of the future.
Table 1.
MDS 9500 Director-Class Hardware End-of-Life/End-of-Sale Dates
Product ID
End-of-Life
Announcement
End-of-Sale Date
Last Shipment Date End-of-Support
Date
DS-C9509 (MDS 9509 director-class switch)
October 28, 2013
April 28, 2014
July 27, 2014
April 30, 2019
DS-C9506 (MDS 9506 director class chassis) and
associated parts
October 31, 2015
October 31, 2016
January 29, 2017
April 30, 2022
DS-C9513 (MDS 9513 director-class chassis) and
associated parts
October 31, 2015
October 31, 2016
January 29, 2017
April 30, 2022
DS-X9248-256K9 (48-port 8GFC advanced FC
module)
October 31, 2016
May 1, 2017
July 30, 2017
April 30, 2022
DS-X9232-256K9 (32-port 8GFC advanced FC
module)
October 31, 2016
May 1, 2017
July 30, 2017
April 30, 2022
DS-X9316-SSNK9 (16-port storage services
module)
October 31, 2016
May 1, 2017
July 30, 2017
April 30, 2022
DS-X9530-SF2AK9 (9500 Supervisor-2A module)
October 31, 2016
May 1, 2017
July 30, 2017
April 30, 2022
DS-13SLT-FAB3 (9513 crossbar switching fabric3 October 31, 2016
module)
May 1, 2017
July 30, 2017
April 30, 2022
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 3 of 14
Increased Bandwidth Requirements
Over time, data centers have added more demanding applications that require not only faster I/O but a much
higher throughput. In order to satiate the surging I/O demands from applications such as virtual desktop
infrastructure (VDI), virtual exchange and databases, big data analytics, and mixed virtual workloads, both compute
and storage are being updated in earnest. Some major developments in that direction has involved investing only
in consolidated servers based on blade hosted servers and server virtualization going forward. Similarly, all-flash
arrays and solid-state drives (SSDs), which were quite expensive in the recent past, have become more
commercially viable, resulting in rapid adoption. Figure 1 illustrates these trends.
Figure 1.
Technology and Market Factors Driving a SAN Refresh
Server consolidation has also resulted in the consolidation of host bus adapter (HBA) ports in order for a pool of
virtual servers to have high-speed end-to-end SAN connectivity to the data storage. Server and storage are being
refreshed every so often to meet these demands for faster I/O. The SAN shouldn’t end up becoming a bottleneck.
Hence, it makes sense to upgrade SAN ports to 16-Gbps Fibre Channel (16GFC) capabilities, rather than just
pushing the bar to 8GFC. To guarantee such throughputs between SANs separated geographically over large
distances and relying on transporting Fibre Channel (FC) over dense wavelength-division multiplexing (DWDM),
the Cisco MDS 9710 Multilayer Director offers an industry-leading dedicated FC credit of up to 4095 per port.
Let’s look at a typical example that has nearly doubled the bandwidth needed in traditional SAN deployments.
Desktop as a service (DaaS) and VDI could be realized due to the fact that servers are now increasingly being
virtualized and running on racks of blades that offer higher compute consolidation at a much lower footprint and
use shared resources. Now the scaled virtual servers, on a physical blade, have also created the need for HBA
instances to support I/O for more server instances. This in turn has driven the need for denser and higher-speed
top-of-rack fabric switches, which still need to connect to the director-class switches for redundant switched paths
to the storage targets. Hence, the bandwidth requirements of the inter switch link (ISL) have increased to
accommodate greater numbers of multiplexed I/O transactions. 8GFC will soon become insufficient to
accommodate the rapidly increasing demand for ISL bandwidth, and thus it makes more sense to move to 16GFC,
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 4 of 14
which can be operated at 8GFC if needed and at the same time wouldn’t require hardware replacement when
moving to 16GFC.
As the data center keeps pace with storage and server refresh every few years, it’s also pertinent to refresh your
SAN. The Cisco MDS 9700 Series modular switch also has the capacity, owing to its resilient, redundant, and
extensively designed backplane that churns out a total of 24 Tbps, to accommodate higher FC and FC over
Ethernet (FCOE) speeds that are well defined in the roadmap. Looking at the trend of how the needle moved from
1 Gbps to 16 Gbps and the adoption patterns, we can predict that the bandwidth needs will soon push for speeds
up to 32-Gbps FC and 40 Gigabit Ethernet FCOE, at least in the fabric core. The Cisco MDS 9700 Series design
guarantees that the backplane will continue to provide you upgrade opportunities when needed without having to
undergo a complete tech refresh, at least for the next decade (Figure 2).
The architecture guarantees true 16GFC throughputs from ingress to egress, due to the fact that there’s no
oversubscription anywhere in the frame path. Also, the fact that there’s no local switching ensures that the port-toport latency is deterministic and consistent among all director class switches in the industry today. When compared
to the inherent application latency of several microseconds, such low latency is several orders of magnitude less
and doesn’t contribute to the overall latency in the workloads.
Figure 2.
Unique differentiators for Cisco MDS 9710
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 5 of 14
Stricter Guaranteed Uptimes
When it comes to director-class switches, high availability and redundant failure modes are a nonnegotiable
requirement. The Cisco MDS 9700 Series of director-class switches goes several notches further in this
department compared to its predecessor, the Cisco MDS 9500 Series. Every component in the switch has
redundancy, helping provide uptimes of 99.999%. Everything from supervisors to fabric modules to the power
supplies has redundancy support, helping provide the overall throughput in both normal and failed modes of
operation. The advanced small form factor optics, power supplies, fabric modules, switching modules, supervisors,
and fans are all hot swappable, for minimal operational impact in replacing failed components (Figure 3).
Figure 3.
MDS 9710 Redundancy and High Availability
The design enables three fabric modules to provide the per-slot bandwidth of 768 Gbps to drive a full line rate of
16GFC per port. There’s an option to add a total of six fabric modules that together provide the industry-leading
bandwidth of 24 Tbps, which is sufficient to support the speed evolution of the director class in the future and at
same time provide N+1 redundancy.
Each power supply delivers a total power of 3000W, and it is possible to connect a total of eight power supplies,
providing an unparalleled combined input source and power module redundancy in the chassis. Three power
supplies are sufficient to provide power for a fully loaded chassis with 384 ports, and the addition of one more
power supply provides an N+1 redundancy mode of operation. This provides investment protection by allowing
room for adding higher-speed and higher-density modules in the future without having to refit new power supplies,
requiring chassis redesign.
The replacement of failed components requires that normal operations not be disrupted throughout the
maintenance window. The supervisors, fans, power supplies, SFPs, and fabric modules help ensure a fabric that
has zero downtime, even during unavoidable maintenance windows, which is a legacy that is carried forward from
the 9500 line of products.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 6 of 14
Operational Ease
Storage networks that power the largest and most demanding enterprise data center applications are adding more
and more capabilities, making operations proportionately more challenging. However, the Cisco MDS 9700 Series
allows you to consolidate your director-class storage footprint by 2:1 when compared with a Cisco MDS 9509, thus
cutting the operational overhead in half (Figure 4).
Figure 4.
SAN Consolidation with the Cisco MDS 9710
Not only does consolidating the ports into a single chassis save space, but the ability for the chassis to be refitted
to accommodate the higher scale and speed requirements of the future without having to do forklift upgrades, is a
major savings in future real estate costs. Effectively, the cooling needs are also halved, as no additional cooling
provisions are required to house the 9700 chassis, with its superior heat dissipation design.
The Cisco MDS 9700 Series will continue to be supported by the enhanced suite of Cisco NX-OS software
releases, which will provide much better management, housekeeping, call home, and power-on self-provisioning
features, reducing the management needs of an increasingly complex SAN. Owing to the much larger supervisor
memory and CPU capacity, the 9700 Series can successfully host a lot of CPU-intensive management operations.
The new breed of NX-OS in turn obviates the need for supervisors with higher memory and CPU configurations,
which are available only on the Cisco MDS 9700.
Scale Up Versus Scale Out
Data centers are poised to handle record growth in data and application proliferation. This projected growth is
driving the need to scale the SAN capabilities. An ever larger number of devices will connect to the SAN, and as a
result the ability of the Cisco MDS 9500 Series to support fabric login (FLOGI), name servers, device aliases,
domain lists, zones, and VSANs will soon fall short. To support a larger base of devices, all-flash arrays, and
virtualized hosts, the switches in the SAN need to either scale up or scale out. Clearly, scaling out is an inefficient
way that increases not only capital expenditures (CapEx) but also operating expenses (OpEx) at the time of initial
rollout and periodically thereafter for the upkeep of the SAN devices. A onetime CapEx investment is a much more
affordable alternative by scaling up the existing Cisco MDS 9500 inventory to MDS 9700 series.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 7 of 14
Figure 5.
Cisco MDS 9710 Scale Up Possibilities
The Cisco MDS 9700 Series offers unmatched scale that can support not only your current requirements but also
the future needs of your data center for manageable, incremental, and planned growth of your SAN. Also, to
counter the exhaustion of FC domains, larger numbers of VSANs should be able to accommodate much larger
mutually exclusive groups of devices that can be managed independently using the smart zoning capability. Each
VSAN will also accommodate a much larger number of devices by virtue of the increased size of FLOGI and Fibre
Channel Name Server (FCNS) databases that can be scaled. A much bigger device alias scale will allow you
individually tag, locate, and manage every device connected to your VSAN.
In addition, the Cisco MDS 9700 Series has a roadmap that calls for introducing newer chassis with higher modular
capacity and more powerful and improved supervisors to support the growth in capacity without having to
implement Inter-chassis links (ICL) that are inherently burdened with latency issues. This roadmap offers the
potential of scaling out as well without adding a significant burden to your real estate budget.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 8 of 14
A Classic Use Case Of Consolidation
A customer needed to add more server scale and new storage with higher capacity. This was difficult with the
current 8GFC capacity that the Cisco MDS 9500 Series was supporting (Figure 6). Instead, they needed to
upgrade to 16GFC and also triple the ISL capacity, from 32 Gbps to 96 Gbps, in the core and increase it by 2.5
times, from 36 Gbps to 80 Gbps, at the edge.
Figure 6.
SAN configuration before upgrade
This meant they needed to add 2 times the current capacity in the core and 1.5 times the current capacity at the
edge (Figure 7). With the current 24 Port 8GFC modules they had, there was no option of doing so as the
maximum capacity expansion possible was to 48 Port 8GFC modules, and that would, at best, have only doubled
their capacity instead of trebling it.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 9 of 14
Figure 7.
SAN configuration after upgrade
Table 2.
1:1 Consolidation Example for 3x Bandwidth at Core
Make / Role
Used Slots
Empty Slots
Current Capacity
Additional
Capacity Needed
at 3X
Additional
Capacity Needed
at 2.5X
Available
Headroom
1x 9513 core
6
4
1152 Gbps
2304 Gbps
1728 Gbps
1536 Gbps*
1x 9710 core
2
6
1536 Gbps
1920 Gbps
1344 Gbps
4608 Gbps**
Table 3.
1:1 Consolidation Example for 2.5x Bandwidth at the Edge
Make / Role
Used Slots
Empty Slots
Current Capacity
Additional
Capacity Needed
at 3X
Additional
Capacity Needed
at 2.5X
Available
Headroom
1x 9513 edge
11
0
2112 Gbps
4228 Gbps
3168 Gbps
0 Gbps
1x 9710 edge
3
5
2304 Gbps
4032 Gbps
2976 Gbps
3840 Gbps**
*Accounting for the variant for 48P Advanced 8GFC with 1.5:1 oversubscription.
**This can potentially go up to 1.5 Tbps per slot with 32GFC modules.
Note:
The calculations in the tables are based on port FC speed, not real FC speed.
As shown in Tables 2 and 3, by swapping the heavily populated 9513 to the 9710, the bandwidth expansion need
of 2X was easily met, with more headroom left to increase bandwidth by another 2X if required in the future. Even
for a maxed-out MDS 9513 at the edge, a bandwidth expansion of 1.8X could be achieved, surpassing the 1.5X
originally needed.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 10 of 14
In order to accommodate the increased scale of servers and storage without incurring any increase in the power
and space budgeted for the data center, consolidation was the only answer. It was estimated that an overall
increase of around 20 percent in the overall power budget and a third of the rack space budget would be required
to accommodate growth. So the refresh had to guarantee at least these savings to make the scale-out of servers
and storage viable.
Table 4.
N:1 Consolidation Example to Save up to 55 Percent in OpEx
Make / Role
Used Slots
Empty Slots
Current Capacity
Available
Headroom
Current Power
Power Savings
6x 9513 edge
38
28
7296 Gb
10752 Gb*
14398W
–
2x 9700 edge
10
6
7680 Gb
4608 Gb**
8540W
40%
4x 9513 edge
44
0
8448 Gb
0
14964W
–
2x 9710 edge
11
5
8448 Gb
3840 Gb**
6620W
55%
*Accounting for the variant for 48P Advanced 8GFC with 1.5:1 oversubscription.
**This can potentially go up to 1.5 Tbps per free slot when 32GFC / 40G FCoE ports are introduced
Note:
The calculations in the table are based on line FC speed, not real FC speed.
As shown in Table 4, by consolidating the Cisco MDS 9513 by 3:1 to the 9700 Series, keeping the total capacity
intact, a power savings of around 55 percent was achieved, along with a two-thirds reduction in rack space.
If only a 2:1 consolidation was possible of two fully loaded 9500 Series switches, they could still achieve a 40
percent savings in power, along with an increase in capacity headroom of 45 percent, at the same time reducing
rack space by half. Even with a 1:1 swapping, purely for capacity expansion, they could still achieve power savings
of around 11 percent.
The entire refresh change window for 18 switches in two fabrics lasted for 8 hours for each fabric, for a total of only
16 hours. It was preceded by a carefully laid out planning and pre-implementation phase that lasted only about 4
weeks and started with staging the new 9700 Series chassis and drawing the new cable plan. It was ensured that
business continuity was unaffected during the 4 weeks leading up to the actual change window. Apart from backup,
which was temporarily halted during the change window, all applications, such as Windows/Linux servers,
ClearCase versioned object bases (VOBs), and Microsoft Exchange, were up and running.
The refresh resulted in the consolidation of 18 Cisco MDS 9513 switches into 12 Cisco MDS 9710 switches, which
helped reduce overall OpEx by as much as 66 percent. These savings were facilitated by power savings of up to
40 percent, total space savings of 66 percent by freeing up a total of 84 RU, an increase in true capacity of 150 to
200 percent, a nearly 30 percent reduction in resources dedicated to managing switches, a built-in option of further
doubling capacity by populating empty slots with next-generation modules in the future, SAN investment protection
for at least 10 years with up to 128GFC in the future without the need for extensive hardware upgrades, and the
migration of existing service contracts and software licenses that were still in force to the new install. In addition, a
newer OS version with all known issues fixed meant fewer service-affecting issues and callouts, and finally the end
users have a high overall level of customer satisfaction, as new services can be rolled out quickly with no increase
in price for almost 10 years.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 11 of 14
Customer Testimonials
Read here how one of the largest IT service providers in Norway, also an existing Cisco MDS 9500 Series
customer, chose to consolidate its SAN and at the same time extend capabilities by embracing the 9700 Series
platform.
Read here how a major professional networking service provider placed its faith in the Cisco MDS 9700 Series to
meet the demands of increasing scale.
Summary
SAN is at the core of services hosted in the data center, and any risks to the longevity of the fabric must be
addressed immediately. Older equipment that has already depreciated in its book value is costlier to maintain and
can fail as it nears end of life. It is only prudent to consider extending service contracts that are soon to expire for
newer equipment that excels in the areas of bandwidth, throughput, resiliency, high availability, scale, and
operational ease and to protect your investment and consequently have more capital value, rather than expending
significant contract fees on older equipment.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 12 of 14
Six Simple Steps For Migration
1.
Go through initial setup on the new MDS switch and do the basic configuration (admin userid/password, mgmt.
port IP address, etc.).
2.
Copy the running configuration from the existing MDS switch to a tftp server (copy run tftp-server).
3.
Rename and edit the configuration file on the tftp server and remove all interface and zoning information and
leave the VSAN configuration (as well as any other management/security configuration you want to continue to
use). You cannot use the old interface configurations, since there are probably port, port-group, and line card
differences. If you are confident enough, you can try to keep the interface configurations, but remove or update
interface speeds, remove dedicated/shared commands, etc.
4.
Copy the modified configuration to the new MDS switch using “copy tftp-server run,” and then do a “copy run
start.”
5.
Connect an ISL from the existing MDS switch to the new MDS switch, and bring up the interface. Once it is up,
verify that the zoning information is migrated from the existing MDS switch to the new MDS switch.
6.
At this point, you can either start migrating devices one at a time from the existing MDS switch to the new
MDS switch (assuming that there is enough ISL bandwidth between the existing and new switches) or do a
cutover from the existing switch to the new switch.
Note:
A simple illustration of steps for migrating a single director is provided above. SAN migration can be more
complex depending on the network design and how applications are zoned with the storage luns, along with
backup, replication and cabling considerations. It is highly recommended to seek expert advice from professionals,
such as Cisco TAC or in some cases Cisco Advanced Services, wherever required.
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
Page 13 of 14
FAQ
Q. Why can’t I continue to update my SAN in phases with the new Cisco MDS 9700 Series until the 9500 Series
goes into end of support, rather than going in for a refresh?
A. Scaling out definitely is an option to address your SAN growth needs. However, somebody from Cisco
Advanced Services can advise you on how to stagger your refresh pattern. The objective should be to scale
up and at the same time reduce operational overheads. Just-in-time refresh is also not recommended, as
demands from new services may suddenly outpace your SAN infrastructure growth.
Q. I’m currently using the Fibre Channel over IP (FCIP) for SAN extension via the Cisco MDS 9000 16-port IPS
module on the 9500 Series SAN gateways. How do I move up to the 9700 Series and continue to use the
FCIP SAN extension, given that at present there’s no FCIP module on the 9700 Series?
A. An FCIP module for the 9700 Series is expected by Q1CY16. Currently you can use MDS 9250i IPS 2RU
switch, which provides 2 10-Gbps IPS ports. In addition, this fixed switch provides 40 Fibre Channel ports to
connect to SAN switches as well as Storage arrays for Data Migration Service (DMM) or Replication services.
For more information refer here.
Q. Can a 14RU 9513 be replaced in the same rack with a 14RU 9710? Or will I need to re-plan the cooling and
cabling requirements?
A. Provided that the cabinets can accommodate the extra 6 inches of depth and have sufficient power supply,
both models have identical dimensions in length and breadth and in the number of rack units required and can
hence be a simple replacement. Simple baffles can be fitted to change the air flow from side to side to back to
front. In majority of the cases, there may not be any need to re-plan cooling and cabling.
Printed in USA
© 2015-16 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public.
C11-735999-01
02/16
Page 14 of 14