High Power Distribution for High Efficiency in Data Centers June 2012 High Power Distribution for High Efficiency in Data Centers Executive Summary With the growth of virtualization and cloud computing, a new approach to power distribution for high density server installation saves floor space, simplifies cabling, and increases electrical efficiency, while reducing the weight on the raised floor. The traditional power distribution system for data centers in North America is a 480/277V three phase power system supplying PDMs which convert the 480 voltage to the 208V and 120V single phase branch circuit utilized by IT equipment. Today, virtually all IT equipment is designed for global compatibility and can operate at 230V or 240V. The next level of power density is being stretched to 277V distribution. In general, operating IT equipment at its highest possible voltage is most efficient. This paper describes different distribution architectures to achieve power density in today’s new data centers. Basic configuration The traditional power distribution system for data centers in North America is a 480/277V three phase power system supplying PDMs which convert the 480 voltage to the 208V and 120V single phase branch circuit utilized by IT equipment as shown in Figure 1. 480/277 V 480/277 V UPS 208/120 V PDM RPP Figure 1 The increase in server installations drawing 15kW to 35kW per rack places tremendous strain on the basic configuration of the distribution system. The increase in power density can lead to an almost 40% of floor space encroachment with equal or more weight loading due to deployment of PDMs. More PDMs mean multiple breakers and related cabling driving up capital cost plus increased cooling costs. This problem is unique to North America whereas in EMEA region the power distribution is simplified to gain maximum efficiency. In the recent past consideration for alternative voltage power distribution was impractical because a significant portion of data centers operated at 120V. However, today’s power supplies and rack based equipment can operate at 208V and higher. The increased power density has created global compatibility for IT equipment, meaning it can operate on North American 120/208V or European 230V and 240V systems. Alternative power distribution configuration The alternative power distribution system shown below is what EMEA region is using as a basic power distribution configuration. A 400/230V distribution configuration is illustrated in Figure 2 below. 480/277 V 400/230 V UPS HPP Figure 2 In the above block diagram power is distributed at 230V (line to neutral) instead of 208V (line to line) eliminating the need for the step down transformer before the server loads in High Power RPP (HPP). why don’t we raise the bar? The next level of an alternative power distribution of 415/240V configuration is illustrated in Figure 3 below. 480/277 V 415/240 V UPS HPP Figure 3 The block diagram power is distributed at 240V (line to neutral) instead of 230V (line to neutral) again eliminating the need for the step down transformer before the server loads but increasing power density. Configuration Voltage (L-N) Current Power (kW) Basic – 120V 120 25 9 Alternative – 400V 230 25 17.25 Alternative – 415V 240 25 18 This table summarizes basic and alternative power distribution configurations with the basis of three phase. Power = Volts x Amps x 3 holding the current constant at for example 20A. It is clearly evident that power density increases significantly from basic 120V distribution to alternate 230V and doubles at 240V power distribution. While we observe the higher power density we almost avoid the indirect issue of over sizing the PDM transformer due to elimination of PDM completely in the configuration. The transformer over sizing occurs generally because there is no clear picture of long term power consumption for each group of server racks. How can you define the peak load? Also this type of alternate power configuration offers increased power density without adding a single breaker to the distribution with less current, higher efficiency and lower cost of ownership. 1 For any circuit size at higher voltage there is less current running through cables resulting in lower heat losses. Combined elimination of the PDM transformer and related heat losses with reduced cable heat losses results in reduced cooling costs, hence the reduced cost of ownership. Most recently yet another alternate power distribution configuration is being considered in North America as shown in Figure 4 below. 480/277 V 480/277 V UPS RPP Figure 4 In this power distribution configuration the server loads are at 277V line to neutral. For our example, this configuration can yield 20.7kW for 25A circuit reaching a new horizon of power distribution. This configuration offers similar benefits as described for 400V and 415V configuration with higher power density. The current US standards already include the voltage and many consultants and engineers are familiar with this system. However, there are some disadvantages such as: • Not a common configuration • Higher cost due to little high volume equipment available to run on 277V • New designs of IT equipment and connectors above 250V What about interrupting capacity? There are clear advantages of alternate power distribution configuration and at the same time increased available fault current at the branch circuit level due to elimination of a PDM transformer. Let us assume 100kVA power source with and without transformer with the application of simple short circuit current calculations. With basic configuration (with PDM in the distribution, hence secondary voltage at 208V) Full load current = 100,000V/208V = 480.7A Assuming the transformer impedance is 3% and, for the sake of clarity and simplification, let’s assume there are negligible line impedances between the transformer secondary and the fault. Available short circuit current SC at secondary = 100%/3% x 480.7A = 16,023.3A 2 This scenario is good and we can use max 22 kAIC branch breakers as they are available in the size that will allow 42 pole distribution. Now in the case of an alternate power distribution configuration (415/240V for example) the available short circuit will be: Full load current = 100,000 VA/240V = 416.6A Available short circuit current SC = 100 x 416.6 = 41,666.6A Cyberex high power distribution The Cyberex line of High Power Panel (HPP) are designed to meet your alternate power distribution configuration. To address the demanding power density, HPP can be deployed for 480/277V, 415/240V or the 400/230V configuration with ample access for installation with numerous input options. The modular design addresses all of the above concerns. The HPP is equipped with fusedswitch panelboards to offer up to 200 kAIC at the branch circuit level in the absence of the PDM transformer. This innovative design allows you to maintain all the benefits of the alternate power distribution configurations discussed in this paper without the loss of premium floor space. Cyberex® HPP designed with ABB Panelboard Cyberex® HPP designed with Cooper Bussmann Panelboard 3 Special cases and concerns System neutral – The requirement of neutral and whether to over size or not completely depends on the total harmonic distortion produced. Information is available to calculate the size of the neutral wire. Breakers – Most circuit breakers in North America are not rated at the alternate configuration voltages. The EMEA region has addressed this concern for breakers and most recently North American manufacturers have introduced compact breakers and panelboards with global attitude in applications that are rated and certified at higher voltages. Conveniently, Cyberex HPP addresses this concern with a fused-switch panelboard and ABB finger safe panelboards as does Square D and GE. Connectors – Appropriate voltage rating for the connectors are available in North America to meet alternate power distribution configurations discussed in this paper. Occasional NEMA style connectors can be easily replaced by IEC style connectors. Existing Installation – There will be a need for a transformer for bypass so it may be cost prohibitive to deploy alternate power distribution of 400V and 415V. The best approach would be to use the alternate power distribution alongside the basic or standard power distribution which is highly desirable for the data center poised for expansion through high efficiency and power density. Conclusion The high traffic internet explosion has driven new approaches to grow power density and efficiency in the data center through power distribution. The use of proposed 230V, 240V or 277V instead of traditional 120V can increase efficiency, and reduce floor space, as well as operating costs. The alternate power distribution proposed can be deployed alongside the traditional power distribution combined with UPS and servers. 4 About Cyberex From the clean environment of a data center to the harsh conditions of an offshore oil platform, Cyberex products meet the challenge of providing clean, dependable electric power to industry, government, and the service sector. Loss of data can mean losing millions of dollars every minute. In this critical industry, the need for stable power is paramount. The loss of a single data record can mean the permanent loss of business, and Cyberex products provide power quality solutions for you. Cyberex products provide peace of mind–the peace of mind that comes from knowing your equipment is safe and secure from the consequences of unpredictable power. By designing the world-class standard in power equipment, Cyberex products have become a leader in customdesigned digital static transfer switches, power distribution and uninterruptible power systems. Company profile For more than a quarter century, the people of Power Solutions have been providing innovative, imaginative solutions in response to customer needs. Power Solutions is dedicated to providing mission critical, power quality products and services manufactured and marketed under the following leading brands – Cyberex®, Current Technology®, Joslyn®, United Power® and JT Packard. Products like the Cyberex Digital Static Transfer Switch and Cyberwave UPS lead the industry in quality, technology and functionality. Standalone power distribution systems, premium hardwired surge protection solutions and 24x7 global field services are also offered. 5 Thomas & Betts Power Solutions 5900 Eastport Blvd. • Richmond, VA 23231-4453 USA Tel: (804) 236-3300 • Toll free: (800) 238-5000 • Fax: (804) 236-4841 tnbpowersolutions.com/cyberex © 2013, Thomas & Betts Power Solutions, LLC. • Thomas & Betts Power Solutions, LLC., is a wholly owned subsidiary of Thomas & Betts Corporation. Cyberex® is a registered trademark of Thomas & Betts International, Inc. • Specifications are subject to change without notice. • Visit our website for latest revisions. 051313