Circuit Breakers in Data Centers: The Hidden Danger

advertisement
Circuit Breakers in Data Centers:
The Hidden Danger
How to efficiently protect critical IT equipment by choosing
the most reliable overcurrent protection device for your
Rack Power Distribution Unit.
By Dr. Jim Anderson and Mike Jansma
Contents
Introduction .................................................................................. 1
How Are Servers Kept On All the Time ......................................... 1
Overcurrent Protection Devices ................................................... 1
The Problem with Fuses in a Data Center..................................... 3
Circuit Breakers and PDUs: A Match Made In Heaven ................. 3
All PDU Circuit Breakers Are Not Created The Same .................... 4
Conclusion .................................................................................... 6
About the Authors .................................................................... 7
About Enlogic ........................................................................... 7
Introduction
How much does a modern data center cost? Millions? Hundreds of millions? The total cost of building a
data center is increasing every day. Why do these buildings cost so much?
The answer is simple: the job that they do is hard to do perfectly. A modern data center is designed to
provide a secure, reliable home to hundreds (or even thousands) of servers – servers that run mission
critical software for companies all over the world. One part of a data center's job is to provide reliable,
clean, power to each of the servers in the data center.
Here in the 21st century, that part of the job would be relatively simple, right? In our homes, when we
flip switches things happen: lights turn on, the computer powers up, or the TV flickers on. However,
think about all of the times that you've lost power or how often the lights flicker. A data center is
designed to prevent these events.
When it comes to the job of providing power to servers, data center designers have two primary tasks to
accomplish: to provide reliable power and avoid unsafe operating conditions. But doing this is more
difficult than you might imagine.
How Are Servers Kept On All the Time?
To ensure that servers always have power, data center designers go to great lengths. They often build
two completely separate and redundant electrical power distribution systems along with multiple types
of power backup solutions. These expensive and complex solutions ensure that if power to the data
center is ever lost, backup systems can quickly provide the needed energy. Additionally, if there is ever a
failure of one of the redundant power systems, the secondary system can provide the required
electricity until the failed system is repaired.
What makes all of the expensive design so fascinating is that it all comes down to a single component to
ensure that all of the servers continue to receive the energy required in order to stay powered up: the
rack power distribution unit (rack PDU). The rack PDU receives electricity from the data center's
redundant power delivery systems and it is responsible for delivering that energy to the individual
servers and IT equipment. If the rack PDU fails, then the attached servers will fail.
The rack PDU is a data center operator's last line of defense against accidental circuit overloads and
harmful faults. In many countries, the rack PDU includes design features that are required by electric
code and mandatory industry standards requiring protection against these events. As such, rack PDU
vendors incorporate solutions to for these occurrences.
Overcurrent Protection Devices
While most servers and IT equipment have become more energy efficient, the total energy demand
required per unit has risen dramatically over the past 15 years from an average power density of 1-2 kW
to an average of 4-6 kW per rack. Some high density IT racks have power loads of 20 kW and above.
Page 1 of 7
LT-51000-A2
These increasing rack power densities have led to dramatically larger input power feeds to the rack PDU
devices.
Rack PDUs are no longer built similarly to the power strip under your desk. Today, the most common
power feed is a single phase, 208-240V input rated at 30-32A (depending on regional codes). Some have
three-phase inputs with ratings as high as 23 kW and higher.
At these high power densities, the amount of heat generated in the electrical conductor wires becomes
substantial and dangerous if not limited to thermal design specification. These thermal limits are
strained as the data center hot aisle temperatures become warmer due to higher power density.
Thermal limits are additionally burdened with energy efficiency initiatives such as raising the operating
temperature due to the new data center industry thermal standards (set about by ASHRAE). When
electrical circuits are exposed to overcurrent or excessive temperatures, the conductor insulation melts
and there is a real danger of smoke, fire, and electrical shorting faults, all of which create a danger to
data center workers.
Today’s rack PDU requires Overcurrent Protective Devices (OCPD) to mitigate the overcurrent risks in
the rack PDU and IT equipment. As more IT equipment is consolidated into fewer, more densely packed
IT equipment racks, it has become critical to isolate potential electrical faults within one server rack in
order to protect upstream power distribution equipment.
There are 2 major types of OCPD used in today's rack PDU: supplemental protectors and branch-rated
OCPD. Supplemental protective devices are devices that serve a single purpose: limit the total current
output of a circuit. However, they are prone to inconsistent protection characteristics, highly sensitive to
the surrounding thermal environment, and incapable of protecting circuits in failure situations. These
types of highly disruptive faults pass by the supplemental protector (often the device fails) and usually
result in upstream circuit breaker tripping to contain the fault with additional servers and IT equipment
placed at risk for surge damage and dropped loads. These simple overcurrent protective devices might
conform to and be labeled with industry standards including UL1077, UL489A, or other local standards.
Conversely, branch rated circuit OCPD devices provide comprehensive protection against simple circuit
current overload and isolation protection for harmful equipment circuit fault conditions. As the power
density of IT racks climb, these fault protection devices become more critical to both reliability and
safety. In many cases, protection is required by electric codes and industry standards. When a short
circuit or ground fault failure occurs, the branch rated OCPD can normally respond by opening the circuit
to isolate the fault to only the attached equipment. Often these OCPD devices are labeled to conform to
branch rated standards such as UL. Figure 1 shows some typical examples of the OCPD used in IT rack
PDUs.
To further complicate the selection process, there are multiple technologies used in both supplemental
and branch rated OCPD devices. Simple thermal fuses are the most basic technology, while resettable
circuit breakers are the most common OCPD device utilized in most rack PDU devices. The most
professional-grade rack PDUs will use non-thermal based, hydraulic-magnetic (hy-mag) circuit breakers
to provide the most comprehensive overcurrent and fault protection within the rack PDU.
Page 2 of 7
LT-51000-A2
The Problem with Fuses in a Data Center
How does a PDU keep the data center servers safe from dangerous overcurrent and fault conditions?
There are many ways for a PDU to do this, but the least expensive method (in terms of cost to PDU
vendors) is by the use of thermal fuses.
Fuses are a special type of resistor that exists for one purpose: to fail when exposed to electrical
currents beyond the designed capacity. A fuse works by flowing the electrical current through a piece of
metal designed to melt when the current exceeds the rated limit. When an overcurrent condition
occurs, the metal melts and the circuit is prevents electrical flow. The servers will lose power; however,
if the fuse melts quickly enough the servers will not be damaged or destroyed. Because the fuse is
designed to melt as a function of temperature, the use of fuses in thermally hot environments requires a
“derating” of the current carrying capacity. In a data center, this means that a fuse rated for 16A or 20A
might melt (or fail) at electrical currents lower than the designed. Failure to account for this derating
results in a tripping of the fuse during normally safe operation.
Fuses have other negative attributes for use in a data center. Sure, a fuse can step in and save the
servers in an overcurrent event, however, it's what you have to do after such an event that makes fuses
no longer an acceptable solution for data center application. Once a fuse has blown, it must be replaced.
This replacement is dangerous for the user. As a result, the entire rack PDU must be powered down.
Often this type of replacement must be performed by a licensed electrician. And the electrician must
have the correct fuses on hand to replace the blown fuse.
Considering that if a current overload condition occurs, you'll probably have fuses in all of the PDUs that
are connected to that power source blow, you may be looking at an expensive and time consuming
repair job. As long as those blown fuses are still installed in PDUs, all of the servers are running on a
single backup power source. If a fault occurs to that power source, all of the attached severs will be lost.
While fuses are inexpensive, recovering from a blown fuse can be devastating in terms of cost and
downtime in the data center environment.
Circuit Breakers and PDUs: A Match Made In Heaven
The good news is that there is a much better solution: circuit breakers. Circuit breakers detect
overcurrent (or fault) situations and break the electrical circuit providing power. How they do this and
how well they do it is what distinguishes one PDU from another.
There are several differences between fuses and circuit breakers. One of the most important differences
is that unlike a fuse, when a circuit breaker trips because a circuit overload, no replacement is required –
it can be reset manually or automatically by data center operators.
In a circuit breaker, an internal pilot device senses a fault current and trips the device. When a circuit
overload condition occurs, contacts in the circuit breaker have to open to interrupt the circuit. To
separate the metal contacts mechanically, stored energy within the breaker opens the circuit.
Page 3 of 7
LT-51000-A2
All PDU Circuit Breakers Are Not Created the Same
So circuit breakers are better than fuses in a PDU, end of story, right? Well, no. There are different types
of circuit breakers available. Different types of circuit breakers have different costs and physical size.
When vendors build rack PDUs, they have to make a cost / benefit decision about which circuit breaker
they will use. The type of circuit breaker has a significant impact on how well servers are protected
against fault conditions and accidental tripping.
Two different types of circuit breakers are often used in PDUs, and although these two different types of
circuit breakers perform the same job, they do it differently based on performance characteristics


Thermal-based circuit breakers
Hydraulic-magnetic (hy-mag) circuit breakers
Thermal Fuse
UL248 type
Thermal Circuit Breaker
UL489 Thermal type
Hydraulic Magnetic Circuit
Breaker
UL489 Hydraulic-Magnetic type
Branch Rated
OCPD
Supplemental
Protector
Figure 1. Typical OCPD found in IT Rack PDUs.
Thermal-based circuit breakers are also known as thermal circuit breakers or thermal-magnetic circuit
breakers. These thermal-based circuit breakers are made using a bi-metallic strip. This strip has two
different pieces of metal welded together; each meant to expand when in contact with the heat of an
electric current. With two different kinds of metal, the expansion rate for each is different. The thermal
based circuit breaker depends on the heat generated by the passing electrical current; the greater the
heat, the more severely the metal strip bends.
When there is an electric overload, the heat generated bends the strip enough to pull the circuit level to
the off position, protecting the wiring from damage. When a thermal circuit breaker is reset, it is likely
to trip again if the overload condition still exists.
Page 4 of 7
LT-51000-A2
Breaker Derating as % of Nameplate
The problem with using a thermal circuit breaker in a PDU is that they are susceptible to unintended
tripping when used in high temperature environments. Because the PDU is contained in a rack and is
exposed to high temperatures, the PDU's thermal circuit breaker can accidentally trip and lose power.
Figure 2 illustrates how the trip setting of a typical thermal circuit breaker changes due to ambient
temperature; for many rack PDUs, the trip setting can vary by more than 15%.
130%
120%
110%
100%
90%
80%
70%
-30
-20
-10
0
10
20
30
40
50
60
Ambient Temperature, C
Figure 2. Temperature Derating Curve for Typical Thermal Circuit Breaker.
Another challenge when using a thermal circuit breaker is the tripping temperature changes by more
than 15% over the rack PDU's operating temperature specification! Thermal circuit breaker vendors
recommend a PDU load current at 80% or less of the rated current to account for the variation in
temperature that the external rack temperature can cause a trip. This restricts total power output per
rack, and results in unplanned downtime conditions.
An alternative type of circuit breaker used in a PDU is the type used by Enlogic: the hydraulic-magnetic,
or hy-mag, circuit breaker. The hy-mag breaker is independent of temperature as a tripping mechanism
Hy-mag breaker's tripping mechanism does not use thermal elements (bimetal) and is not affected by
ambient rack temperature. Rather, the current load is connected through a solenoid coil. At currents
above the circuit breaker rating, the coil generates magnetic flux necessary to force a spring mechanism
to open the electrical circuit and prevent current flow through the breaker. What this means is that the
tripping current is related to the amount of current only; the hy-mag breaker will keep consistent
tripping characteristics across the entire range of data center ambient temperatures.
When a circuit breaker trips, the data center operator wants the affected servers powered up as quickly
as possible. This is where another big difference between fuses and circuit breakers, and further
between thermal and hydraulic-magnetic circuit breakers becomes clear.
When a circuit breaker trips or fuse blows, users should explore why the event occurred and take action
to prevent reoccurrence. However, in the case of a fuse: the entire rack PDU must be powered down
and specially trained personnel must replace the blown fuse.
In the case of the resettable circuit breaker: the circuit failure is often isolated to just one portion of the
rack PDU output. Users can independently eliminate the fault condition and reset the circuit breaker
Page 5 of 7
LT-51000-A2
without powering down the remainder of the rack PDU. Further, circuit breakers don’t require specially
trained personnel to reset the device after failure.
When a hydraulic-magnetic circuit breaker is tripped, it provides the ability to be instantaneously reset
once you've corrected the cause of the circuit failure. This is not the case with thermal circuit breakers.
With thermal circuit breakers, once the bimetal has been bent due to the tripping event, it does not
immediately return to its original position. Instead, this circuit breaker remains tripped until the bimetal
is cooled. The time from the circuit breaker tripping until it can be reset is called the reset time. Reset
time can vary between 3–20 minutes depending on the breaker.
Conclusion
Currently, there are PDUs in data centers without any fuses or circuit breakers. These PDUs leave servers
exposed and vulnerable to the fluctuating power system. With ever-growing rack power density, and
increasingly variable power loads in the IT rack, the need for comprehensive circuit overload and fault
protection is increasingly important.
The two types of protection available include fuses and circuit breakers. Fuses are an older technology
that requires problematic downtime and specialized effort to replace. Circuit breakers come in two
primary types: thermal and hydraulic-magnetic circuit breakers.
Thermal circuit breakers are susceptible to the ambient heat generated in a rack environment and are
subject to accidental tripping. Hydraulic-magnetic circuit breakers, such as those used in Enlogic PDUs,
provide comprehensive protection against circuit overload and isolation of faults. Unlike the thermal
breakers, Enlogic’s hydraulic-magnetic breakers are unaffected by heat and can be reset immediately
after being tripped.
Page 6 of 7
LT-51000-A2
About the Authors
Dr. Jim Anderson - Dr. Anderson is the Director of Product Line Management for Enlogic. He is
responsible for ensuring that Enlogic creates the correct products with the correct features. Dr.
Anderson received his Ph.D. in Computer Science from Florida Atlantic University and his MBA from the
University of Texas at Dallas. He has worked as a Director of Product Line Management for a number of
very large firms including Verizon, Alcatel, Siemens, and Boeing as well as working at a number of
successful startups. Dr. Anderson is also a senior member of the Institute for Electrical and Electronics
Engineers (IEEE) and holds a senior management position in the U.S. branch of the organization.
Mike Jansma – Mike Jansma is Co-founder and Chief Marketing Officer of Enlogic. As a technology
innovation leader, Mike has an exceptional history of listening to customers, digging into complex
problems, and providing innovative solutions for real customer pains in energy technology and power
distribution fields. To date, Mike holds 7 technology patents with additional technology patents
pending.
Prior to founding Enlogic, Mike received his Bachelors and Masters degrees in Mechanical Engineering
from The Illinois Institute of Technology and an MBA from Washington University in St. Louis. Mike
began his career in Energy Technology at the US Department of Energy’s Argonne National Laboratory.
Later he held technical and marketing leadership positions at ABB and Schneider Electric/APC.
About Enlogic
Enlogic is a new company that makes an entire family of world class 4th generation PDUs. Founded by
the engineers who created the first three generations of PDUs that are on the market, Enlogic's PDUs
are revolutionary in how reliable they are and how easy they are to both install and support. Enlogic's
PDUs are amazingly slender in order to not interfere with any hot-swappable server components, they
use hydraulic-magnetic circuit breakers and are temperature rated up to 55-60 degree C in order to
ensure proper operation in today's increasingly hot data center environment, and they feature Enlogic's
novel field replaceable network control card that permits equipment changes without having to power
down connected servers. You won't find these features in anyone else's PDUs, including any APC PDU.
To find out more about what are arguably the world's best 4th generation PDUs, contact your local
Enlogic sales representative in order to request more information or even better, to set up an
appointment to see what makes Enlogic's PDUs the right choice for your data center.
Page 7 of 7
LT-51000-A2
Download