Data Center Global Telecommunications Cabling Standards Landscape and Latest Developments Jonathan Jew - J&M Consultants, Inc. Chair TIA TR-42.6 Telecom Administration Vice-chair TIA TR-42.1 Commercial Buildings Co-chair BICSI Data Center Subcommittee Vice-chair USTAG ISO/IEC JTC 1 SC 25 WG 3 Interconnection of IT Equipment www.j-and-m.com WHY USE DATA CENTER TELECOM STANDARDS? Point-to-Point Non-Standard Cabling Unorganized, Single-Use Structured Standards Based Cabling - Organized, Reusable, Flexible Standard/Structured vs. Proprietary/Point-to-Point Cabling • • • • Better availability Saves money Faster deployment Support for multiple current & future protocols What are the standards? • ANSI/TIA-942-A Telecommunications Infrastructure Standard for Data Centers (US, approved April 2012, published Aug 2012; Addendum 1 approved Feb 2013, available now) • CENELEC EN 50173-5 Information Technology – Generic Cabling Systems Part 5: Data Centres (2007, European Union). Addendum 2 approved for publication in 2012 • ISO/IEC 24764 Information Technology – Generic Cabling for Data Centre Premises (2010, international). Addendum 1 approved for publication in 2014 • ANSI/BICSI-002 Data Center Design and Implementation Best Practices (2011, international). Revision in progress (~2014). New standards & updates • BICSI, ISO/IEC, CENELEC, and TIA are all very busy revising and developing new standards • They work in conjunction with each other and organizations such as IEEE BICSI-002 Data Center Best Practices • ANSI/BICSI-002 Data Center Design and Implementation Best Practices Standard • Developed by 150+ committee members worldwide (mostly users, engineers, consultants) • Meant to supplement, not replace existing data center cabling standards by ISO/IEC, CENENLEC, TIA, and others • Under revision expected late 2014/early 2015 • Large document (~400 pages) covering a wide range of subjects - electrical, mechanical, architectural, fire protection, security, structural, commissioning, telecom, etc. TIA-942-A-1 Data center fabrics • Switch fabrics provide high-bandwidth any-to-any connectivity required for cloud computing • Fabrics can and should be built using the standards based cabling • More cabling and higher bandwidth needed for fabrics than traditional switch architecture Traditional 3-Tier Fat-Tree / Leaf & Spine Switch Fabric Balanced Twisted Pair Cabling • Cat 5e & 6 for 1 Gigabit Ethernet to 100 m • Cat 6A, 7, or 7A for 10G Ethernet to 100 m • ISO/IEC requires and other standards recommend Cat 6A minimum • Cat 8 in development for 40GBase-T up to 30 m (adequate for 80% of server connections); 25Gbase-T also under consideration Switch-to-Server Link Length % of data center links 30 meters 80% 40 meters 90% 100 meters 100% Optical Fiber Cabling • Laser optimized 50/125 um multimode fiber (OM3 or OM4) less expensive than singlemode fiber for lengths <=150 m • 40G uses 8 MM fibers and MPO connector • 100G uses 20 MM fibers and either two 12 fiber MPOs or one 24 fiber MPO connector • 4-lane/8-fiber 100G (Mar 2015) will cost less, use less power, and is expected to support at least 70m over OM3, 100m over OM4. Optical Fiber Cabling • 16-fiber and 32-fiber MPOs being developed by TIA (TIA-604.18 FOCIS-18) • More efficient use of fibers for 40G & 4-lane 100G which use 8 fibers (12-fiber MPOs have 4-unused fibers) • 400G Ethernet will probably use 32 multimode fibers Summary • Standards-based structured cabling optimizes cost, flexibility, and availability • Standards are continuously updated: – CENELEC: European Union – TIA: US, Canada, and US-based companies outside US – ISO/IEC: most other countries • BICSI standards are international best practices that work with other standards • Consider future needs to support 10/40/100G and data center fabrics needed for clouds – Cat 6A, 7, or 7A for balanced-pair cable – OM4 for multimode fiber terminated on MPOs (with multiples of 8 fibers) • New 4-lane 100G Ethernet (Mar 2015) • New Cat 8 / 40GBase-T (Feb 2016) Thank you Jonathan Jew President J&M Consultants, Inc. Email: jew@j-and-m.com Co-chair BICSI data center subcommittee – www.bicsi.org Co-editor TIA-942-A, Editor TIA-942-A-1 Chair TIA TR-42.6 telecom administration subcommittee Vice-Chair TIA TR-42.1 commercial building cabling Editor ISO/IEC TR 14763-2-1 telecom administration identifiers US National Committee Project Manager ISO/IEC 24764 data center standard Data Center & Administration Section Editor – ISO/IEC 14763-2 cabling planning & installation Supplementary Information Differences between standards TIA ISO/IEC CENELEC Balanced Pair backbone Cat 3 min Cat 6A recommended Cat 6A, 7, 7A Cat 6, 6A, 7, 7A Balanced Pair Horizontal Cat 6 min Cat 6A recommended Cat 6A, 7, 7A Cat 6, 6A, 7, 7A Multimode Fiber OM3 min OM4 recommended OM3 min OM2 min Coaxial cable 734/735 coax None None MMF connector LC for 1 or 2/port MPO for 3+/port LC for 1 or 2/port MPO for 3+/port LC or MPO SMF connector at EDA LC for 1 or 2/port MPO for 3+/port LC for 1 or 2/port MPO for 3+/port LC-PC SMF connector at Entrance Rm LC for 1 or 2/port MPO for 3+/port Angled-LC Angled-LC ISO/IEC Category 8 • ISO/IEC TR 11801-99-1 – Balanced cabling for 40Gbps channels – (2014-2015) meant for IEEE • Characterizes 30 meter channels (26 m with 2x2m patch cords) to support 40 Gbps using existing Cat 6A, 7, 7A, and new Cat 8.1 and Cat 8.2 components • Cat 8.1 for Class I channels is backward compatible with Cat 6A • Cat 8.2 for Class II channels compatible with Cat 7/7A • IEEE 40GBase-T target is 30 m for Cat 8, unknown for Cat 6A, 7, 7A • Parallel effort in TIA underway Ethernet channel lengths over multimode fiber Fiber Type 1G 10G 40G 100G 100G # fibers 2 275 m 2 26 m 8 - 20 - 8 - 550 m 82 m - - - 800 m 300 m 100 m 100 m 70 m* 1040 m 550 m 150 m 150 m 100 m* OM1 OM2 OM3 OM4 Distances in red are specified by manufacturers but not in IEEE standards. *IEEE 802.3bm Standard for 4-lane 100G target 1Q2015 BICSI-002 Subjects Covered • • • • • • • • Site Selection Architectural and Structural Design Electrical Systems Mechanical Systems (i.e., HVAC) Fire Protection and Security Building Automation Systems Commissioning Maintenance BICSI-002 Subjects Covered • Telecommunications – Access Providers & Entrance Facilities – Telecom Spaces – Cabinets & Racks – Cabling Pathways – Telecom Cabling – Field Testing – Telecom Administration • Information Technology