® IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones IBM Redbooks Product Guide This IBM® Redbooks® Product Guide describes the IBM System Storage® SAN768B-2 and SAN384B-2 fabric backbones. IBM System Storage SAN768B-2 and SAN384B-2 fabric backbones are among the industry's most powerful Fibre Channel (FC) switching infrastructure offerings. They provide reliable, scalable, high-performance foundations for mission-critical storage. These fabric backbones also deliver enterprise connectivity options to add support for IBM FICON® connectivity, offering a high-performing and reliable FICON infrastructure with fast and scalable IBM z Systems™ servers. The IBM System Storage SAN b-type family switch, which is designed to support Fibre Channel connectivity for servers and storage, introduces the next generation of fabric backbones with Gen 5 Fibre 1 Channel and Fabric Vision technology to bring a long-term solution for mission-critical applications that require secure, high-performance, and low-latency storage networks. This enables organizations to continue using their existing IT investments as their businesses grow, and solve their most difficult business challenges, unleashing the full potential of high-density server virtualization, cloud architectures, and next-generation storage. Figure 1 shows the SAN768B-2 and SAN384B-2. Figure 1. SAN768B-2 and SAN384B-2 IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 1 Did you know? SAN768B-2 and SAN384B-2 enable simpler, flatter, and low-latency chassis connectivity to reduce network complexity, management, and costs by using UltraScale chassis connectivity. You can simplify and centralize end-to-end storage area network (SAN) administration with comprehensive diagnostic tests, monitoring, and automation through Fabric Vision technology. SAN768B-2 and SAN384B-2 allow faster problem identification, SAN configuration and management with customizable IBM Network Advisor health, and performance dashboards. Architecture and key components SAN768B-2 and SAN384B-2 address key customer requirements while consolidating the SAN infrastructure. These fabric backbones with Gen 5 Fibre Channel technology are available in two modular form factors. Built for large enterprise networks and measuring 14U, SAN768B-2 has eight vertical blade slots to provide up to 512 16 Gbps universal (E, F, D, M, and EX) Fibre Channel ports using 64-port 16 Gbps Fibre Channel blades. Designed for midsize networks and measuring 8U, SAN384B-2 has four horizontal blade slots to provide up to 256 16 Gbps universal (E, F, D, M, and EX) Fibre Channel ports using 64-port 16 Gbps Fibre Channel blades. The Gen 5 Fibre Channel technology directors support 2, 4, 8, 10, and 16 Gbps Fibre Channel, FICON, Fibre Channel over Ethernet (FCoE), and 1/10 Gbps Fibre Channel over IP (FCIP). These Gen 5 Fibre Channel director models offer maximum investment protection by performing the following functions: Interfacing with existing 2, 4, 8, or 16 Gbps Fibre Channel ports Offering configurable 10 Gbps ports for metro optical connectivity FCoE connection exclusively on the SAN768B-2 Offering 1/10 FCIP for global SAN extension infrastructure Offering fabric-based encryption for data-at-rest to protect valuable digital assets Integrating with the IBM Network Advisor unified network management solution to bridge operational gaps across server, network, and storage administrators Built upon years of innovation and using the core technology of systems that perform at up to 99.999% uptime, the new generation of SAN b-type networking backbones helps minimize downtime and disruption for some of the world’s most demanding data centers. Both models can do the following functions: Support 2, 4, 8, and 16 Gbps Fibre Channel or FICON connections, 10 Gbps Fibre Channel inter-switch link (ISL) connections in metro optical connectivity or with 10 Gbps dense wavelength division multiplexing (DWDM) devices, in-flight encryption, and compression, 64 (4 × 16) Gbps inter-chassis link (ICL) connections and 1/10 Gbps FCIP. Leverage fabric-based encryption for data-at-rest. Integrate with advanced fabric services and management tools. Deliver compatibility with the 8 and 4 Gbps IBM System Storage b-type SAN director, switch, and router models. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 2 Performance Emerging and evolving critical workloads and higher density virtualization are continuing to push the limits of SAN infrastructures. SAN768B-2 and SAN384B-2 directors feature industry-leading Gen 5 Fibre Channel technology that delivers 16 Gbps line-speed performance and up to 10.2 Tbps of chassis bandwidth with the 64-port 16 Gbps high-density blade connectivity option to address next-generation I/O requirements. In addition, local switching capabilities ensure that data traffic within the same port group does not consume slot bandwidth, which maximizes the number of line-rate ports. Performance capabilities are as follows: SAN768B-2 o Up to 512 ports (equivalent to 640 with UltraScale ICLs) at 16 Gbps in a single chassis 10.2 Tbps chassis bandwidth 8.2 Tbps Fibre Channel 2.0 Tbps UltraScale ICL bandwidth o 512 Gbps bandwidth per slot SAN384B-2 o Up to 256 ports (equivalent to 320 with UltraScale ICLs) at 16 Gbps in a single chassis 5.1 Tbps chassis bandwidth 4.1 Tbps Fibre Channel/FICON ports 1 Tbps UltraScale ICL bandwidth o 512 Gbps bandwidth per slot Each SAN768B-2 and SAN384B-2 contains redundant control processor modules (active/passive) and core blades (active/active), plus slots for Fibre Channel port blades and advanced functionality. All Fibre Channel ports on the blades support full-duplex, non-blocking performance. These backbones are available with a number of selectable options. The base systems do not include any blades, but at least one blade is required for host and storage connectivity, and two blades are recommended for redundancy. Based on the different power consumption of each port or specialty blade, various configurations result in higher than 2000 W power usage; the customer is required to purchase a pair of optional power supplies. Available blades for SAN768B-2 and SAN384B-2 are as follows: 16 Gbps 32-port, 48-port Fibre Channel blades : These port blades provide 2, 4, 8, 10, or 16 Gbps on each port, which can be configured as E, F, M, EX, or D_ports. These advanced 16 Gbps blades deliver enhanced fabric resiliency and application uptime through advanced features that are enabled by the Condor3 application-specific integrated circuit, including support for native 10 Gbps Fibre Channel, in-flight encryption and compression, ClearLink diagnostic technology (D_Ports), and increased buffers or Forward Error Correction (FEC). Fully populated options are available. 16 Gbps and 8 Gbps 64-port Fibre Channel blade : This blade, now also available in 16 Gbps, allows midsize-to-large enterprise customers to deploy high-density, chassis-based solutions to minimize a physical footprint without compromising performance. The 64-port blades do not support FICON, but can be used in the same chassis-switching FICON traffic. Figure 2 shows the 16 Gbps 64-port blade (FC 3664). IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 3 Figure 2. 16 Gbps 64-port blade (FC 3664) 8 Gbps Enhanced Extension blade : This blade is a scalable and flexible platform combining Fibre Channel and advanced FCIP technology, which is designed for building or expanding a high-performance SAN extension infrastructure for disaster recovery, data protection, and data mobility storage solutions to move more data faster and further. It provides Internet Protocol Security (IPSec) over both 10 GbE ports versus the previous blade’s support on one port. It is ideal for open systems and mainframe disk and tape extension, multi-site synchronous and asynchronous disk replication, centralized SAN backup, recovery and archiving, and global data and storage resource migration, distribution, and sharing. This blade offers twelve 8 Gbps Fibre Channel ports, ten 1 GbE ports with a license option for up to two 10 GbE ports per blade; up to four blades per SAN768B-2/SAN384B-2 director. FCoE 10GbE 24-port blade: This new blade provides FCoE and Data Center Bridging (DCB) for end-of-row server connectivity, delivering line-rate 10 Gbps performance across 24 ports. It is supported in only the SAN768B-2 (2499-816). A maximum of one blade is allowed per director. Figure 3 shows the FCoE 10GbE 24-port blade (FC 3880). Figure 3. FCoE 10GbE 24-port blade (FC 3880) Scale-out network design Networks are evolving to adapt to rapid growth and change in the server and storage infrastructure. UltraScale chassis connectivity leverages optical ICLs to connect up to 10 SAN768B-2 and SAN384B-2 backbones up to 2 kilometers apart, enabling flatter, faster, and simpler fabrics that increase consolidation while reducing network complexity and costs. UltraScale ICLs enable scalable core-edge and active/active mesh chassis topologies. These high-density chassis topologies reduce inter-switch cabling and free the ports for servers and storage, thus maximizing overall port density in a lower amount of rack space. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 4 Fabric Operating System and management software The Fabric Operating System (FOS), included with each SAN768B-2 and SAN384B-2 Gen 5 Fibre Channel base system, contains all the functions that are necessary to operate a base system. These systems require FOS V7.1 or later to take advantage of the advanced functions that are delivered through 1 Fabric Vision technology. SAN768B-2 and SAN384B-2 base systems are always enabled with the following tools: Web Tools: Enable graphical user interface (GUI)-based administration, configuration, and maintenance of fabric switches and SANs. Zoning: Segments a fabric into virtual private SANs to restrict device communication and apply certain policies only to members within the same zone. Full Fabric: Enables E_Ports and allows SAN768B-2 and SAN384B-2 to connect to other switches. Virtual Fabrics: Allow a physical switch to be partitioned into independently managed logical switches, each with its own data, control, and management paths. Enhanced Group Management (EGM): Enables extra device-level management functionality for SAN b-type products when EGM is added to the element management. Also allows large consolidated operations to groups of devices (such as firmware downloads and configuration uploads and downloads). Enterprise software bundle The enterprise software bundle is a bundle of FOS features on top of the base FOS functionality that is included in the hardware base for both the SAN768B-2 and the SAN384B-2. It includes the following features: Adaptive Networking : This service is a set of features that provides users with tools and capabilities for incorporating network policies to ensure optimal behavior of a large SAN. The FOS V7.0 release (or later) supports two types of quality of service (QoS) features with the 16 Gbps fabric backbones: Ingress Rate Limiting and SID/DID-based prioritization. Advanced Performance Monitoring : Helps identify end-to-end bandwidth usage by host/target pairs and provides for capacity planning. Extended Fabrics: Extend SAN fabrics beyond the Fibre Channel standard of 10 kilometers (km) by optimizing internal switch buffers to maintain performance on ISLs that are connected at extended distances. Fabric Watch: Constantly monitors mission-critical switch operations for potential faults and automatically alerts administrators about problems before they become costly failures. Fabric Watch includes Port Fencing capabilities. ISL Trunking: Enables Fibre Channel frames to be distributed efficiently across multiple ISLs between two IBM b-type SAN fabric switches and directors while preserving in-order delivery. Both b-type SAN devices must have trunking activated. SAN768B-2 and SAN384B-2 add enhanced ISL Trunking support using 16 Gbps ports and enable Fibre Channel packets to be distributed across up to eight 16 Gbps-capable ISLs for a combined bandwidth of up to 128 Gbps. Server Application Optimization : This license optimizes overall application performance for physical servers and virtual machines by extending virtual channels to the server infrastructure. Application-specific traffic flows can be configured, prioritized, and optimized throughout the entire data center infrastructure. Fabric Vision: This license offers a collection of breakthrough features and tools that are bundled into an optional licensed feature. It includes Flow Vision and Monitoring and Alerting Policy Suite (MAPS) advanced technologies and capabilities. It requires FOS V7.2 or later. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 5 Advanced capabilities can be enabled with extra optional licenses that are available: FICON with control unit port (CUP) activation: Provides in-band management of the supported SAN b-type switch and director products by system automation for IBM z/OS®, performance data for IBM Resource Measurement Facility™ (IBM RMF™), and to provide unsolicited status to IBM MVS™. This support provides a single point of control for managing connectivity in active FICON I/O configurations. To enable in-band management on multiple switches and directors, each chassis must be configured with the appropriate FICON CUP feature. Up to four CUPs per chassis can be enabled. Integrated routing : Allows any 8 or 16 Gbps Fibre Channel port in SAN768B-2 and SAN384B-2 to be configured as an EX_Port that supports Fibre Channel Routing (FCR), providing improved scalability and fault isolation, along with multi-vendor interoperability. Inter-Chassis License with eight 16Gbps 2 km QSFPs: This license activates ICL ports on the core blades of the 16 Gbps platforms. One ICL license enables only half of the ICL ports on the core blades of a SAN768B-2 platform or all of the ICL ports on the core blades of a SAN384B-2 platform. Each feature ships quantity eight 16 Gbps 2 km QSFP transceivers that are capable of providing connectivity up to 2 km. Inter-Chassis License with sixteen 16Gbps 100 m QSFPs: This license activates ICL ports on the core blades of the 16 Gbps platforms. One ICL Ports on Demand (POD) license enables only half of the ICL ports on core blades of a SAN768B-2 platform or all ICL ports on the core blades of a SAN384B-2 platform. Each feature ships sixteen 16 Gbps 100 m QSFP transceivers that are capable of providing connectivity up to 100 m. 16Gbps 2 km QSFP: This feature provides one 16 Gbps QSFP, designed to provide Fibre Channel connections at distances up to 2 km. Inter-chassis QSFP bundle : Provides sixteen 16 Gbps, QSFPs supporting connectivity up to 100 meters from the switching backplane of one half of an 8-slot chassis to the other half, or to a 4-slot chassis. Inter-Chassis License conversion : Is used to convert an ICL activation that is transferred from an 8 Gbps model as part of the model conversion process. Enterprise ICL license : Supports up to 3840 × 16 Gbps universal Fibre Channel ports (using 16 Gbps 48-port blades), up to 5120 × 8 Gbps universal Fibre Channel ports (using 8 Gbps 64-port blades), ICL ports (32 or 16 per chassis, optical QSFP) connected up to nine chassis in a full-mesh topology or up to 10 chassis in a core-edge topology. Connecting five or more chassis through ICLs requires an Enterprise ICL license. These optional advanced SAN Extension functionality licenses are available also: 8 Gbps Advanced Extension activation license : Enables two advanced extension features, FCIP trunking and adaptive rate limiting (ARL) on the SAN768B-2 or SAN384B-2 systems. The FCIP trunking feature allows multiple IP source and destination address pairs (defined as FCIP circuits) through multiple 1 GbE interfaces to provide a high-bandwidth FCIP tunnel and failover resiliency. The ARL feature provides a minimum bandwidth guarantee for each tunnel with full usage of the available network bandwidth without impacting throughput performance under high traffic load. 8 Gbps Extension blade 10 GbE activation license : Enables up to two 10 GbE ports on the 8 Gbps extension blades or 8 × 10 Gbps Fibre Channel ports on any eight ports of a 16 Gbps port blade. With this license, two extra operating modes, in addition to 1 GbE port mode, can be selected. Either two 10 GbE ports or ten 1 GbE and one 10 GbE ports can be configured on an 8 Gbps extension blade when this license is activated. One feature code should be installed and activated for each port blade to be enabled for 10 Gbps Fibre Channel. 8 Gbps FICON Accelerator activation license : Uses advanced networking technologies, data management techniques, and protocol intelligence to accelerate FICON disk and tape read-and-write operations over geographically extended distances, while also maintaining the integrity of command and acknowledgment sequences. This is ideal for data migration, disaster recovery, and business IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 6 continuity solutions beyond 300 km, and it supports emulation for IBM z/OS Global Mirror (formerly Extended Remote Copy or XRC) and tape pipelining for FICON tape and virtual tape. IBM Network Advisor simplifies Gen 5 Fibre Channel management and helps users proactively diagnose and resolve issues to maximize uptime, increase operational efficiency, and reduce costs. The wizard-driven interface dramatically reduces deployment and configuration times by allowing fabrics, switches, and ports to be managed as groups. Customizable dashboards graphically display performance and health indicators come ready to use, including all data captured using Fabric Vision technology. To accelerate troubleshooting, administrators can use dashboard playback to quickly review past events and identify problems in the fabric. In addition, dashboards and reports can be configured to show only the most relevant data, enabling administrators to more efficiently prioritize their actions and maintain network performance. IBM Network Advisor V12.0 or later is required for supporting transitions to cloud environments. IBM Network Advisor V12.1 or later is required for supporting Fabric Vision license capabilities. Optimized data center connectivity over distance Connecting distributed data centers enables data mobility for advanced data protection solutions. The SAN768B-2 and SAN384B-2 fabric backbones include integrated metro and global SAN extension capabilities, which can provide application agility and support flexible business continuity and disaster recovery solutions. The b-type director family with Gen 5 Fibre Channel technology enables high-speed replication and backup solutions over metro or WAN links with native Fibre Channel (10/16 Gbps) and optional FCIP (1/10 GbE) extension support. Extending Fabric Vision technology between data centers enables organizations to move more data faster and minimize the impact of disruptions and outages for non-stop business operations. The integrated metro connectivity includes in-flight compression and encryption to optimize bandwidth, and helps reduce the risk of unauthorized access. Benefits are as follows: Unleash the full potential of cloud storage with outstanding scalability, performance, and reliability Reduce network complexity, management, and costs with ultra-scale chassis connectivity Simplify and centralize end-to-end storage area network (SAN) administration Protect investments in existing SAN fabrics and automation tools while reducing costs and minimizing disruption Speed problem identification, SAN configuration, and management with IBM Network Advisor dashboards Maximize performance for I/O-intensive and bandwidth-intensive applications Optimize data center connectivity with integrated, high-performance metro and global connectivity Simplified Management and Robust Network Analytics Fabric Vision technology includes: Monitoring and Alerting Policy Suite (MAPS): Provides a new, easy-to-use solution for policy-based threshold monitoring and alerting. MAPS proactively monitors the health and performance of the SAN infrastructure to ensure application uptime and availability. By leveraging pre-built, rule-based and policy-based templates, MAPS simplifies fabric-wide threshold configuration, monitoring, and alerting. Administrators can configure the entire fabric (or multiple fabrics) at one time using common rules and policies, or customize policies for specific ports or switch elements. Fabric Performance Impact (FPI) Monitoring: Uses predefined thresholds and alerts in conjunction with MAPS to automatically detect and alert administrators to severe levels of latency and identifies IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 7 slow drain devices that might impact the network. This feature uses advanced monitoring capabilities and intuitive MAPS dashboard reporting to indicate various latency severity levels, pinpointing exactly which devices are causing or impacted by a bottlenecked port. Dashboards: Provides integrated dashboards that display an overall SAN health view, along with details about out-of-range conditions, to help administrators easily identify trends and quickly pinpoint issues occurring on a switch or in a fabric. Configuration and Operational Monitoring Policy Automation Services Suite (COMPASS): Simplifies deployment, safeguards consistency, and increases operational efficiencies of larger environments with automated switch and fabric configuration services. Administrators can configure a template or adopt an existing configuration as a template and seamlessly deploy the configuration across the fabric. In addition, they can ensure that settings do not drift over time with COMPASS configuration and policy violation monitoring within IBM Network Advisor dashboards. ClearLink Diagnostics : Ensures optical and signal integrity for Gen 5 Fibre Channel optics and cables, simplifying deployment and support of high-performance fabrics. ClearLink Diagnostic Port (D_Port) is an advanced capability of Gen 5 Fibre Channel platforms. Flow Vision: Enables administrators to identify, monitor, and analyze specific application flows in order to simplify troubleshooting, maximize performance, avoid congestion, and optimize resources. Flow Vision includes these items: Flow Monitor: Provides comprehensive visibility into flows within the fabric, including the ability to automatically learn flows and nondisruptively monitor flow performance. Administrators can monitor all flows from a specific host to multiple targets or logical unit numbers (LUNs), from multiple hosts to a specific target or LUN, or across a specific ISL. Additionally, they can do LUN-level monitoring of specific frame types to identify resource contention or congestion that is impacting application performance. Flow Generator: Provides a built-in test traffic generator for pretesting and validating the data center infrastructure (including route verification and integrity of optics, cables, ports, back-end connections and ISLs) for robustness before deploying applications. Forward Error Correction (FEC): Enables recovery from bit errors in ISLs, enhancing transmission reliability and performance. Credit Loss Recovery : Helps overcome performance degradation and congestion due to buffer credit loss. The Fabric Vision license is included with FOS V7.2 or later. After the switch is upgraded to FOS V7.2.0 or later and the Fabric Vision license is enabled, the Fabric Watch configuration and any Fabric Watch-related features are no longer supported. Fabric Vision technology, an extension of Gen 5 Fibre Channel, provides a breakthrough hardware and software solution that helps simplify monitoring, maximize network availability, and dramatically reduce costs. Featuring innovative monitoring, management, and diagnostic capabilities, the SAN768B-2 and SAN384B-2 fabric backbones with Fabric Vision technology enables administrators to avoid problems before they impact operations, helping organizations meet Service Level Agreements (SLAs). Specifications This section lists the specifications for SAN768B-2 and SAN384B-2. Product characteristics Table 1 lists the product characteristics for SAN768B-2 and SAN384B-2. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 8 Table 1. Product characteristics for SAN768B-2 and SAN384B-2 (part 1 of 4) Item Description Product number SAN768B-2 (2499-816) SAN384B-2 (2499-416) Base machine Single chassis: Up to 512 (SAN768B-2) or 256 (SAN384B-2) 16 Gbps universal (E, F, D, M, and EX) Fibre Channel ports using 64-port 16 Gbps Fibre Channel blades. Multi-chassis with UltraScale ICL ports: Up to 6144 16 Gbps Fibre Channel ports; UltraScale ICL ports (32 or 16 per chassis, optical QSFP) connect up to nine chassis in a full mesh topology or up to 12 chassis in a core-edge topology. Each SAN768B-2/SAN384B-2 chassis that connects to four or more SAN768B-2/SAN384B-2 chassis via UltraScale ICLs requires an Enterprise ICL license. The base chassis includes these components: One director chassis with a door, two vertical cable management combs, redundant (active/standby) control processor blades, redundant active/active core-switching blades, redundant power supplies, fans, and a ship group that includes an installation guide, CD-ROM (with manuals), two rack-power distribution unit (PDU) power cords, service tools and wrap tools, an RJ-45 serial cable, wrist strap, cable retainer kit, and SFP extraction tool. In addition to the above components, the base chassis of SAN768B-2 includes three blower fans and the option for two additional power supplies. The ship group also includes a 14U rack-mounting kit (BR120). As for SAN384B-2, the base chassis includes two blower fans and a 1U exhaust duct kit. The base FOS (v7.0 or later) software includes these items: Web Tools, Advanced Zoning, Full Fabric, Virtual Fabrics, EGM, and an enterprise software bundle (Fabric Watch, Extended Fabrics, ISL Trunking, Advanced Performance Monitoring, Server Application Optimization, Adaptive Networking, and Fabric Vision). Note: No port or advanced functionality blades are included in the base. Customers must order the 64-port, 48-port, or 32-port blades, or the supported specialty blades, and populate them with transceivers. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 9 Table 1. Product characteristics for SAN768B-2 and SAN384B-2 (part 2 of 4) Fibre Channel interfaces 16/10/8/4/2 Gbps ClearLink diagnostic technology D_Port (ClearLink diagnostic port), E_Port, EX_Port, F_Port, and M_Port (mirror port); optional port type control FICON interfaces 16/8/4/2 Gbps FCIP interfaces 1 GbE and 10 GbE Hot-swap components Redundant (active/standby) control processor modules (CPs), core routing modules, power supplies, fan modules, all Fibre Channel port blades, Extension blades, SFPs, and QSFPs Transceivers 16 Gbps: 64-port blades require hot-pluggable 100 m or 2 km QSFP connectors; 4×16 Gbps SWL, MPO 1×12 ribbon cable connector. (66 m OM3, 100 m OM4). QSFPs support only 4/8/16 Gbps (not 2 Gbps or 10 Gbps). 16 Gbps: 32-port and 48-port 16 Gbps blades require hot-pluggable SFP+ and LC connectors, 16 Gbps short-wavelength laser (SWL), long-wavelength laser (LWL), and extended long-wavelength laser (ELWL). 10 Gbps: 32-port and 48-port 16 Gbps blades require hot-pluggable SFP+ and LC, 10 Gbps SWL, and LWL. The 24-port FCoE blade requires hot-pluggable, 10 GbE SFP+ with any combination of Short-Reach (SR) and Long-Reach (LR) optical transceivers. 8 Gbps: 16 Gbps 64-port blades require hot-pluggable QSFP connector 4×8 Gbps SWL, MPO 1×12 ribbon cable connector (150 m OM3, 190 m OM4) 8 Gbps: 32-port and 48-port 16 Gbps blades, and extension blades, require hot-pluggable SFP+ and LC, 8 Gbps SWL, LWL, and ELWL. 8 Gbps: 64-port blades require hot-pluggable mSFP and mSFP LC connector, and 8 Gbps SWL only. 4 Gbps: 8 Gbps Enhanced Extension blades need copper or SFPs to be used with IP ports and LC, and 4 Gbps SWL and LWL. 1 GbE: Hot-pluggable optical SFP, SWL, and LWL, and GbE copper SFP (8 Gbps Extension blade GbE ports only). UltraScale ICL QSFP: CR16-8 and CR16-4 require hot-pluggable QSFP and MTP connector, and 4×16 Gbps SWL (50 m OM3, 100 m OM4, and 2 km QSFP for long distances). The Fibre Channel distance is subject to fiber optic cable and port speed. Fiber optic cable Fiber optic cables with LC are required and available in various lengths in single-mode and multimode formats. Power cords Jumper cables are included for installation. Country-specific power cords must be ordered for desktop/stand-alone installations. Warranty One year. 24×7 same-day maintenance service options are available. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 10 Table 1. Product characteristics for SAN768B-2 and SAN384B-2 (part 3 of 4) Optional features Blades: 32- and 48-port 16 Gbps Fibre Channel switch blades, no SFPs (3034 or 3035) 32- and 48-port 16 Gbps Fibre Channel switch blades, populated with 16Gbps SW SFPs (3634 or 3636) 32- and 48-port 16 Gbps Fibre Channel switch blades, populated with 16Gbps LW SFPs (3635 or 3637) 64-port 8 Gbps Fibre Channel switch blade, populated with 64 mSFPs (3864) 64-port 16 Gbps Fibre Channel switch blade, populated with 16 QSFPs (3664) FCoE 10GbE 24-port blade, populated with 24 x 10GbE SFPs (3880) Enhanced Extension 22-port blade, no SFPs (3891) Licenses: Inter-chassis kit (100 m or 2 km), FICON with CUP activation, Advanced Extension activation, Extension blade 10 GbE activation, FICON Accelerator activation, Integrated Routing, and ICL Enterprise Connectivity. Other: SFPs, fiber optic cables, and upgrade power supplies Scalability Full fabric architecture of 239 switches Performance FC: 2.125 Gbps line speed, full duplex, 4.25 Gbps line speed, full duplex, 8.5 Gbps line speed, full duplex, 10.53 Gbps line speed, full duplex, 14.025 Gbps line speed, full duplex, auto-sensing of 2, 4, 8, and 16 Gbps port speeds, and 10 Gbps and optionally programmable to fixed port speed ISL trunking Frame-based trunking with up to eight 16 Gbps ports per ISL trunk. Up to 128 Gbps per ISL trunk. Exchange-based load balancing across ISLs with DPS included in Fabric OS. Chassis bandwidth Slot bandwidth Local switching bandwidth 512 Gbps (data rate) UltraScale ICL bandwidth SAN768B-2: 10.2 Tbps per chassis (512 ports × 16 Gbps data rate + 2.048 Tbps UltraScale ICL bandwidth) SAN384B-2: 5.1 Tbps per chassis (256 ports × 16 Gbps data rate + 1.024 Tbps UltraScale ICL bandwidth) 512 Gbps for the 16 Gbps 32-port blade: 32 ports × 16 Gbps (data rate) 768 Gbps for the 16 Gbps 48-port blade: 48 ports × 16 Gbps (data rate) 1024 Gbps for the 16 Gbps 64-port blade: 64 ports × 16 Gbps (data rate) SAN768B-2 (2499-816): 2.048 Tbps. 32 UltraScale ICL ports provide the equivalent of 128 x16 Gbps ports. Each UltraScale ICL port provides 64 Gbps bandwidth over a QSFP (4×16 Gbps) link. SAN384B-2 (2499-416): 1.024 Tbps. 16 UltraScale ICLs provide the equivalent of 64 16 Gbps ports. Each UltraScale ICL port provides 64 Gbps bandwidth over a QSFP (4×16 Gbps) link. Both models: Frame-based trunking is enabled between four UltraScale ICLs. DPS distributes exchanges across all frame trunks. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 11 Table 1. Product characteristics for SAN768B-2 and SAN384B-2 (part 4 of 4) Switch latency Locally switched port latency is 700 ns. Blade-to-blade latency is 2.1 µsec. Encryption/compression is 5.5 µsec per node. Forward Error Correction (FEC) adds 400 ns between E_Ports (enabled by default). Maximum frame size 2112-byte payload Frame buffers 8192 per 16-port group on 32-port and 64-port blades and up to 8192 per 24-port group on 48-port blades, dynamically allocated Data traffic types Fabric switches supporting unicast USB 1 USB port per control processor for firmware download, support save, and configuration upload/download Fabric services Fabric Vision technology; Monitoring and Alerting Policy Suite (MAPS); Flow Vision; Advanced Performance Monitoring (APM) (including Top Talkers for E_Ports, F_Ports, and Fabric mode), Adaptive Networking (Ingress Rate Limiting, Traffic Isolation, and QoS), Bottleneck Detection, Advanced Zoning (default zoning, port/WWN zoning, and broadcast zoning), Dynamic Fabric Provisioning (DFP), Dynamic Path Selection (DPS), Extended Fabrics, Enhanced BB credit recovery, Fabric Watch, FDMI, Flow Vision, Frame Redirection, Frame-based Trunking, FSPF, Integrated Routing, IPoFC, ISL Trunking, Management Server, Monitoring and Alerting Policy Suite (MAPS), Configuration and Operational Monitoring Policy Automation Services Suite (COMPASS); NPIV, NTP v3, Port Fencing; Registered State Change Notification (RSCN), Reliable Commit Service (RCS), Server Application Optimization (SAO), Simple Name Server (SNS), and Virtual Fabrics (Logical Switch, Logical Fabric) Extension Supports DWDM, CWDM, and FC-SONET devices, Fibre Channel, in-flight compression (LZO), and encryption (AES-GCM-256) BB credit recovery, FCIP, Adaptive Rate Limiting (ARL), data compression, Fast Write, read/write Tape Pipelining, and QoS. FICON FICON cascading (Fabric OS: SAN768B-2, SAN384B-2). Support for lossless DLS, FICON CUP, and Advanced Accelerator for FICON (FICON Global Mirror and XRC emulation and read/write Tape Pipelining). The 8 Gbps and 16 Gbps 64-port blades must be in a logical switch that is not being used for FICON connections; they do not support FICON. Special-purpose blades The Enhanced Extension Blade (3891) provides SAN extension over IP networks (Twelve 8 Gbps Fibre Channel ports, and ten 1 GbE ports with license option for up to two 10 GbE ports per blade; Configuration and Operational Monitoring Policy Automation Services Suites (COMPASS); up to four blades). The FCoE blade (3880) provides Fibre Channel over Ethernet (FCoE) and Data Center Bridging (DCB) for end-of-row server connectivity, delivering line-rate 10 Gbps performance across 24 ports. (Up to one blade per SAN768B-2.) IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 12 High availability Table 2 lists the high availability features for SAN768B-2 and SAN384B-2. Table 2. High availability features Item Description Architecture Passive backplane, redundant active/passive control processor, redundant active/active core switching blades, and redundant WWN cards Chassis power Two 2000 W AC power supply modules (100 - 240 V auto-sensing) and 2N redundancy. SAN768B-2 supports two additional power modules. Cooling SAN768B-2: Three blower assembly modules (two are required for operation.) SAN384B-2: Two blower assembly modules (one is required for operation.) Solution availability Designed to provide 99.999% uptime capabilities. Hot-pluggable redundant power supplies, fans, WWN cards, processors, core switching, port blades, and optics, online diagnostic tests, and nondisruptive firmware download and activation. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 13 Management Table 3 lists the management features for SAN768B-2 and SAN384B-2. Table 3. Management features Item Description Management software HTTP, simple network management protocol (SNMP) v1/v3 (Fabric Element (FE) Management Information Base (MIB), Fibre Channel MIB), and Secure Shell (SSH), auditing and syslog, Advanced Web Tools, advanced performance monitoring, and IBM Network Advisor - Enterprise or IBM Network Advisor – Professional Plus, command-line interface (CLI), SMI-S compliant, Administrative Domains, and trial licenses for add-on capabilities Security AES-GCM-256 encryption on ISLs, DH-CHAP (between switches and end devices) and FCAP switch authentication, and FIPS 140-2 L2-compliant, HTTPS, IPSec, IP filtering, LDAP with IPv6, OpenLDAP, Port Binding, RADIUS, User-defined role-based access control (RBAC), Secure Copy Protocol (SCP), Secure RPC, SFTP, SSH v2, SSL, Switch Binding, TACACS+, and Trusted Switch Management access 10/100/1000 Ethernet (RJ-45) per control processor and in-band over Fibre Channel, serial port (RJ-45) and one USB per control processor module, and call-home integration that is enabled through IBM Network Advisor Diagnostic tests Built-in flow generator, ClearLink optics, and cable diagnostic tests, including electrical/optical loopback and link traffic/latency/distance, POST and embedded online/offline diagnostic tests, including environmental monitoring, FCping and Pathinfo (FC traceroute), flow mirroring, frame viewer, nondisruptive daemon restart, port mirroring, optics health monitoring, power monitoring (16 Gbps blades-only), RAStrace logging, and Rolling Reboot Detection (RRD) IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 14 Supported environments Table 4 lists the supported environments for SAN768B-2 and SAN384B-2. Table 4. Supported environments* Item Description Servers supported IBM Power Systems™ IBM System p® IBM z Systems Selected Sun and HP servers Other Intel processor-based servers with Linux, and Microsoft Windows 2008 and Windows 2012 Operating systems supported Microsoft Windows 2008 and Windows 2012 Red Hat Linux and Red Hat Linux Advanced Server SUSE Linux and SUSE Linux Enterprise Server (SLES) IBM AIX® storage system Other selected operating systems Storage products supported IBM XIV® Storage System IBM System Storage DS8000® storage servers IBM Storwize® V3700, V5000, and V7000 IBM System Storage SAN Volume Controller IBM Security Key Lifecycle Manager 2.0 or later Other selected storage systems FC switches supported System Storage and TotalStorage SAN b-type and m-type directors, switches and routers, and other directors, switches, and routers that are manufactured by Brocade * For the most current information, see the product details in the IBM System Storage Interoperation Center (SSIC) at http://www.ibm.com/systems/support/storage/ssic/interoperability.wss. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 15 Mechanical specifications Table 5 lists the mechanical specifications for SAN768B-2 and SAN384B-2. Table 5. Mechanical specifications Item Description Enclosure Rear panel-to-door airflow. SAN384B-2 ships with a 1U exhaust shelf. Mounting Rack-mountable in a standard 19-inch EIA cabinet Size SAN768B-2 Width: 43.74 cm (17.22 in.) Height: 62.23 cm (24.50 in., 14U) Depth (without door): 61.29 cm (24.13 in.) Depth (with door): 73.20 cm (28.82 in.) SAN384B-2 Weight Width: 43.74 cm (17.22 in.) Height: 35.60 cm (14.00 in., 8U) plus 4.00 cm exhaust shelf (1.75 in., 1U) Depth without door: 61.29 cm (24.13 in.) Depth with door: 73.20 cm (28.82 in.) SAN768B-2 103.38 kilogram (kg) (227.90 pounds (lbs)) for 512-port configuration fully populated 37.30 kg (82.20 lbs) for chassis SAN384B-2 69.00 kg (152.00 lbs) for 256-port configuration fully populated 25.40 kg (56.00 lbs) for chassis IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 16 Operating environment Table 6 lists the operating environment for SAN768B-2 and SAN384B-2. Table 6. Operating environment Item Description Temperature Operating: 0° to 40°C (32° to 104°F) Non-operating: –25°C to 70°C (–13°F to 158°F) Humidity Operating: 5 - 85% relative humidity (RH) non-condensing at 40°C (104°F) Non-operating and storage (non-condensing): 10 - 90% at 70°C (158°F) Altitude (operating) Up to 3000 meters (9842 feet) Shock Operating: 20 g, 6 ms, and half sine Non-operating: 33 g, 11 ms, and half sine Vibration Operating: 0.5 g p-p and 5 - 500 Hz Non-operating: 2.0 g p-p and 5 - 500 Hz Heat dissipation SAN768B-2 Maximum: 512-port configuration (fully loaded with QSFPs in ICL ports), 7462 BTU/hr SAN384B-2 CO2 emissions Maximum: 256-port configuration (fully loaded with QSFPs in ICL ports), 4182 BTU/hr SAN768B-2 8.9 metric tonnes per year (max: with 384 ports using eight 48-port blades SAN384B-2 Airflow 4.9 metric tonnes per year (max. with 192 ports using four 48-port blades) Rear panel-to-door airflow IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 17 Power Table 7 lists the power specifications for SAN768B-2 and SAN384B-2. Table 7. Power specifications Item Description Supported power range Voltage Range: 85 - 264 V ac Auto-volt Nominal: 100 - 240 V ac Power 85 - 132 V ac: 1000 W 180 - 264 V ac: 2000 W In-rush current 20 Amps maximum, peak Frequency 47 - 63 Hertz (Hz) Power consumption SAN768B-2 - 1952 W fully loaded SAN384B-2 - 1064 W fully loaded Notes 1. For the most current information, see the Fabric Vision technology solution brief: http://ibm.co/1GWNDKc Related information For more information, see the following resources: IBM System Networking SAN b-type family web page: http://ibm.com/systems/networking/switches/san/b-type IBM Offering Information web page (announcement letters and sales manuals): http://www.ibm.com/common/ssi/index.wss?request_locale=en On this page, enter SAN768B-2 or SAN384B-2, select the information type, and then click Search. On the next page, narrow your search results by geography and language. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 18 Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law : INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurement may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. © Copyright International Business Machines Corporation 2013-2015. All rights reserved . Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 19 This document was created or updated on July 8, 2015. Send us your comments in one of the following ways: Use the online Contact us review form found at: ibm.com/redbooks Send your comments in an e-mail to: redbooks@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. HYTD Mail Station P099 2455 South Road Poughkeepsie, NY 12601-5400 U.S.A. This document is available online at http://www.ibm.com/redbooks/abstracts/tips1127.html . Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml. The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: AIX® DS8000® FICON® IBM® MVS™ Power Systems™ Redbooks® Redbooks (logo)® Resource Measurement Facility™ RMF™ Storwize® System p® System Storage® System z® XIV® z/OS® z Systems™ The following terms are trademarks of other companies: Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. IBM System Storage SAN768B-2 and SAN384B-2 Fabric Backbones 20