WHITE PAPER #51
POWER EQUIPMENT AND DATA
CENTER DESIGN
EDITORS:
David Loucks, Eaton
Stephen McCluer, Schneider Electric
CONTRIBUTORS:
Lynn Simmons, Dell
John Collins, Eaton
George Navarro, Eaton
Dusty Becker, Emerson Network Power
Bill Campbell, Emerson Network Power
Harry Handlin, GE Digital Energy
Mark Szalkus, GE Digital Energy
Brad Thrash, GE Digital Energy
Pamela Lembke, IBM
Kevin Bross, Intel
Shaun Harris, Microsoft
Jim Spitaels, Schneider Electric
PAGE 2
Table of Contents
I.
Introduction and scope .......................................................................................................................................................... 3
II.
Categories and definitions..................................................................................................................................................... 3
III.
Power Component Requirements ......................................................................................................................................... 5
IV.
About The Green Grid ..........................................................................................................................................................51
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 3
I.
Introduction and scope
This paper begins with a description of a power system diagram and a discussion of its importance. A complete
power system diagram for a specific data center is essential for discussion on all aspects of designing an
efficient power system.
The paper then discusses various topics that impact efficiency, including power system configurations and how
a data center is “grown” over time. Following this, the focus becomes more specific and different power system
components are discussed in detail. A diagram showing measurement and control points is provided for each
component as a point of reference.
Timetables (current, 2014, and 2014+) for implementation of the core tenets of this guide are provided for
each component and are intended to serve as a quick reference for data center designers. For example,
someone planning on upgrading UPSs can refer to the topic on instrumentation in the timetable for UPSs and
learn what specific features are recommended to be implemented in the year 2014.
Wherever practical the text will distinguish between direct current (DC) and alternating current (AC) power
distribution. However, when covering more general topics associated with power, the distinction may not be
made and both DC and AC power solutions should be given consideration.
II.
Categories and definitions
The following categories and components will be reviewed in their own sections of this paper. Each section
category/component will contain an overview of the section, general information about efficiency, then more
detail (including specific efficiency metrics).
The End to End Power System
The end to end power system includes all of the components that are a part of the power distribution path, that
is, the path to move electricity from the building utility service to the IT load.
UPS
The uninterruptible power supply (UPS) converts unconditioned power to provide conditioned power to critical
loads without interruption. It contains an energy storage system, such as a bank of batteries, which supply
power to the load when utility power is unavailable. This discussion includes double conversion AC UPSs, line
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 4
interactive AC UPSs, AC UPS eco-mode operation, and DC UPSs. Note that although a data center commonly
has multiple UPSs, they are modeled as a single block. DC UPSs are also modeled as a single block, even
though they may actually consist of multiple rectifier modules and separate batteries. Where appropriate,
isolation is indicated in the block diagrams. Voltage-converting AC UPSs may alternatively incorporate an
autotransformer in their bypass or output path. For more detail on UPS types, see IEC 62040-31.
Transformers
An isolation transformer is an electromagnetic device with multiple windings per phase that converts an AC
voltage into another AC voltage and simultaneously provides galvanic isolation by transmitting energy between
windings through magnetic inductive coupling (induction). The transformer winding size and type is selected to
provide the required voltage transformation level and may also provide secondary benefits including improved
safety ground performance, AC-DC (battery) isolation, harmonic cancellation through phase-shifting, and
transient voltage reduction through grounding.
An autotransformer is similar in material and function. However, it will be physically smaller and cost less than
an equivalent isolation transformer. It changes voltage, but it does not have any additional power quality
characteristics, such as galvanic isolation and harmonic cancellation, which are found in the isolation
transformer.
PDU
A Power Distribution Unit (PDU) is an electrical distribution cabinet, free-standing or rack-mounted, whose main
function is to provide a required point of power distribution. The PDU houses circuit breakers that are used to
create multiple branch circuits from a single feeder circuit, and can also contain transformers, electrical panel
boards, surge protection devices, and power monitoring/controls.
ICTE PSU
A typical power supply unit (PSU) for Information and Communication Technology Equipment (ICTE) is designed
to convert (rectify) alternating current (ac) voltage from the mains supply to several direct current (dc) voltages,
both positive and negative, typically + 12V,-12V,+5V,+5V standby and +3.3V. Switched mode power supplies
(SMPS) are the predominant form. Some PSU models are available with DC inputs.
1
http://webstore.iec.ch/webstore/webstore.nsf/artnum/044928!opendocument
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 5
Because the diverse collection of output voltages results in widely varying current draw requirements, most
modern computer power supplies actually consist of several different switched mode power supplies. Each
produces just one voltage component and each is able to vary its output based on component power
requirements. All are linked together to shut down as a group in the event of a fault condition.
Generator set
A generator is made up of an engine (sized in kW) which spins an alternator (sized in kVA) to produce AC
electric power. Several generator sets may be connected to work together (in parallel) to provide the required
power for the connected load. Generators may be portable or fixed and are available in a variety of sizes
ranging from a few kW or kVA to several MW or MVA each. Generators can be connected to data center power
systems using either transfer switches, paralleling switchgear, or both, depending on the needs and design of
the installation.
III.
Power Component Requirements
The End to End Power System
Creation of a detailed power system design specification is the beginning point for any discussion of the impact
of a data center’s power components on overall efficiency of the data center. A specification should address
questions such as capacity constraints, space constraints, and both upstream and downstream power
distribution. The data center power source capacity (current) and configuration (voltage), as well as the
requirements of the expected compute loads (servers, storage, network, etc.), must be well defined and
understood.
The diagrams that follow below are for AC systems. The DC systems are not shown in the diagrams, but are
similar. Typically the difference lies in removing the inverter. Static bypass and maintenance bypass would also
be removed from the diagram and replaced with additional rectifiers.
A common means of describing a data center power system is the power system diagram, often referred to as
a one-line diagram in three-phase AC systems or DC systems. See Figure III-1 for a simplified example. Other
input and output voltage ranges are also available, including medium voltage.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 6
208/120V
Non-essential Loads
(lighting, etc.)
480/277V
Critical
Loads
Gen
Maintenance Bypass
480/277 VAC
PDU
PSUs
PDU
PSUs
PDU
PSUs
Static Bypass
Gen
ATS
UPS
34 kV
fuse
Utility
480/277 VAC
Utility
Transformer
Essential Loads
(mechanical,
cooling, etc.)
Main Switchboard
Figure III-1: Simplified Diagram – 60 Hz Data Center Power System with Static UPS
Each piece of power equipment is a fundamental building block in the larger power system design that a data
center depends on for performance and features. It is the arrangement of these blocks into a functional
system that matters the most to data center performance objectives and requirements.
The power system diagram can be used to show the entire power system from end to end and in some cases
can do so all on one concise sheet. At one end is the incoming power source. At the other end are the loads.
Power system diagrams, though electrical in nature, can provide information about system and subsystem
function and location. Functional details called out in the power system diagram can include system
redundancy, the type of energy storage elements, emergency backup generators, maintenance bypass devices,
protection devices (circuit breakers, breaker panels, and fusing), and metering points.
A power system diagram can also include monitoring and measuring devices that are critical to data center
efficiency. The diagram needs to be well defined and understood. This applies equally to both existing and new
data centers. Building management or other system management software applications contain what amounts
to a power system diagram with cooling and compute layouts added.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 7
Availability and Efficiency
For the most critical loads, operational availability is the primary objective for the physical infrastructure. All
data center power designs are based on an underlying assumption that power disturbances (such as
transients, voltage fluctuations, outages, and other power quality abnormalities) are inevitable, whether
originating from the electric utility or from within the facility. Any of these conditions can damage equipment
and/or interrupt operations. Therefore equipment to mitigate such disturbances is built into every design.
The addition of these devices increases energy consumption through added efficiency losses. Power system
component layout and level of redundancy affect the losses within the power system. Built-in redundancy is
perceived to be necessary to achieve maximum availability. To achieve greater levels of redundancy, more
components are used to provide the same amount of backup power. These additional components, even when
operating in standby mode, consume energy and therefore reduce the overall data center efficiency, as defined
by PUE2 or DCiE3. Designers should be aware of the trade-off between redundancy/availability and efficiency.
They should also consider solutions in which the power system can be reconfigured to provide the needed
power and redundancy within the required time, while not needlessly operating standby equipment. Some
common configurations are discussed here.
A-B (“Dual Bus”) Systems
This power system architecture is configured with two sides, A and B, each operating at less than 50 percent
load. Each side can include multiple UPSs. Either side can handle 100 percent of the system load. If one of the
sides has a problem, the load is connected or switched to the other operational side. The tie of the two sides
requires a combination of switching devices and synchronization of all components downstream of the UPS.
Switches can reside upstream of the UPS for maintenance isolation purposes. Figure 1-2 shows an example of
AC system redundancy, designed to allow a high level of system availability even during maintenance or
component failure. Extra redundancy at strategic areas inside your power system may lead to a higher
availability.
2
Definition of PUE, http://www.thegreengrid.org/sitecore/content/Global/Content/white-papers/The-Green-
Grid-Data-Center-Power-Efficiency-Metrics-PUE-and-DCiE.aspx
3
(see definition of PUE above)
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 8
Side A
Side B
Automatic
transfer
switch
Automatic
transfer
switch
Generator
Utility
t
e
x
t
Distribution switchgear
Noncritical
loads
Essential
mechanical
loads
UPS
t
e
x
t
Distribution switchgear
tie
breaker
UPS
t
e
x
t
e
x
t
Generator
Utility
Maintenance
bypass
Noncritical
loads
Essential
mechanical
loads
UPS
UPS
t
e
x
t
e
x
t
Maintenance
bypass
2-input
critical load
UPS
Uninterruptible power supply
PDU
Power distribution unit
2-input
critical load
PDU
PDU
Static transfer
switch
1-input
critical load(s)
Figure III-2: Simple Diagram - A-B Redundant power System
The tie points can be of a continuous or momentary type with momentary being the more typical approach. A-B
systems can be implemented either with a central bypass module or a distributed bypass. The centralized
bypass module uses uninterruptible power supplies without internal bypass functionality. The distributed
bypass has the bypass integrated with the individual parallel UPS. Distributed bypass is less common in
current installations, but is coming of age and will be increasingly used. Note that even though this discussion
describes a dual power path system, more than two paths are permissible.
Isolated Redundant Systems
Also known as a catcher system, this configuration can include any number of UPSs and includes a standby
UPS prepared to back up the primary UPS. The redundancy is provided by virtue of a transfer to the catch side.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 9
One method is to use the standby UPS to power the static bypass of each connected UPS, as shown inFigure
III-3. Another method is to put a static transfer switches between the output of each UPS and its load, thereby
introducing a single point of failure The standby UPS must be able to instantaneously pick up the load of one or
more faulted UPS systems, which can result in a potential step load from zero to over 100 percent of rated
capacity. In either scenario, the standby UPS runs at idle almost all of the time, consuming energy but
providing no actual conditioning or protection until needed.
Utility
Generator
Standby UPS (2N capacity)
Static Bypass
Automatic
Transfer
Switch
Primary UPS 1 (N capacity)
Static Bypass
To PDU
Group 1
Maintenance Bypass
Primary UPS 2 (N capacity)
Static Bypass
To PDU
Group 2
Maintenance Bypass
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 10
Figure III-3: One-line representation of one of several isolated redundant (catcher) system configurations
Emerging Power Systems
Beyond the systems described above, other technologies are emerging that may provide more mainstream
solutions in the future. These types of systems, most conducive to cloud technologies, move the redundancy
out of the site power distribution line and into other areas, such as software or mirrored data centers.
Reliability may be affected from a power system standpoint but is addressed elsewhere in the company’s
larger data center picture.
Several examples of this exist today. At a basic level, ITE can be fed with two feeds, one on a UPS and one with
utility only on the other. Battery backup function can be moved from the central UPS location to inside each
server. Backup time can be reduced to less than 5 minutes. This can reduce overhead for a full UPS, but can
lead to issues when dealing with batteries that need to be replaced prior to a server’s end of life. A
combination between the two is available: at the power supply level, the main AC input can come directly from
the utility (the “efficiency input”), with second PSU input of a DC battery backup (the “availability input”), where
batteries are racked near the ITE unit.
Further outside the box come newer energy storage technologies. Lead acid batteries are the most common
today, but other battery chemistries (such as lithium-ion) may become available that aid in more efficient,
reliability, or economical storage. Other technologies like supercapacitors (or ultracapacitors) or flywheels
could help with removal of batteries all together.
Accuracy in measurement
Any level of accuracy can be measured today. However, to measure with high accuracy is quite costly.
Individual component built-in meters are typically not accurate enough for good PUE measurement (built in
meters are around 95 to 98 percent accurate), but externally available bolt on type meters can be attached to
obtain the desired level of accuracy. However, because of costs, this accuracy of metering is neither common
practice on purchased powertrain components or at an installed data center facility.
The accuracy of the measurement varies by use. If the measurements are used for tracking changes in
performance within a facility, the built in meters are generally accurate enough. If certification or comparison to
other facilities is required, more accurate supplemental meters should be installed at appropriate points in the
power path.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 11
Core Tenets Timetable
Instrumentation
2012:
Typically, products are metered individually. Total facility power is measured manually at a meter or
monitored via a utility bill. All products are be instrumented to measure volts and amps, enabling the
calculation of watts, VA and apparent power factor.
2014:
Metering should progress further down the powertrain (for example, from UPS output to PDU output to
power supply input). Accuracy of measurements should improve to 1 percent.
2014+:
Metering should measure energy as well as power. Accuracy of measurements should improve to 0.5
percent if individual components in the power path will be measured in real time.
Discoverability
2012:
Modbus systems are configured on installation. System Network Mapping Protocol (SNMP) devices can
be discovered, but are more often configured on installation.
2014:
SNMP V3 will be more widely implemented.
2014+:
SNMP V3 will be used for discovering all components along the power chain. All components will be
discoverable and auto announced, able to be integrated in with the smart grid.
Scalability
2012:
In general, power paths have more redundancy than required and so are not optimized for efficiency.
2014:
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 12
All power chain components should be operating at the highest efficiency for the load range used.
Equipment will become more automated and adapted to self-optimize its efficiency by scaling pieces or
group of components.
2014+:
Provisioning power quality (that is, how and where the power is supplied from) should be based on the
equipment type to reduce efficiency losses and cost where not required. Align rack power sizing with
typical uses.
Enhanced management infrastructure
2012: None
2014: None
2014+:
Power equipment will use SNMP V3 as the de facto protocol to communicate between itself and the
centralized management agent.
Ability to be policy driven
2012:
Components in the power chain
Measured data from instrumented units is available or calculated by the system and displayed on the
system.
Each piece of equipment has its own timestamp, utility pricing and carbon awareness. This is statically
known when available.
2014:
Components in the power chain
All equipment obtains a timestamp from one location. Accuracy of the timestamp improves over time.
One second accuracy is required for efficiency purposes. More accuracy is required for fault analysis.
All measured and calculated values are available to the centralized management console, including but
not limited to volts, amps, watts, VA, apparent power factor.
Energy (kWh) is calculated by the centralized management console.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 13
Utility pricing and carbon awareness
Available in real time
2014+:
Components in the power chain

The centralized management console will calculate, report, and use the average, minimum and
maximum values of volts, amps, watts, VA, power factor, energy.

The centralized management console will take all inputs to optimize the overall efficiency of the
power chain, specifically using PUE, predicting the most optimal path to efficiency.

Utility pricing and carbon awareness

Future forecasted; adjustments made to components of powertrain (shift workloads, shift to
generator, etc.) based on forecast.
Standardized metrics
Metrics should match with Data Center Sustainability Model
2012:
End to end efficiency for the critical power path (Building entrance to IT load) <90 percent for the data
center usage pattern
2014:
End to end efficiency for the critical power path of 90 percent Efficiency for the data center usage
pattern
2014+:
End to end efficiency for the critical power path of 92 percent Efficiency for the data center usage
pattern
Applicable Standards
EPA Energy Star products for business and government:
http://www.energystar.gov/index.cfm?c=products.pr_find_es_products
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 14
UPS
Efficiency Topics
Modes of operation
UPS topology classifications and efficiency performance is guided by what’s referred to as “normal” mode of
operation. A double conversion UPS processes power through the rectifier (AC to DC) conversion and the
inverter (DC to AC) conversion while a standby UPS largely operates its bypass circuit, passing somewhat
unconditioned power from the input AC utility source to the output load. Recent development in UPS
technology has introduced system architecture and controls that enable advanced UPS performance. Many
UPSs that are fundamentally designed to perform as a double conversion can also operate in standby mode to
raise the system efficiency, known generically as energy saving or economy mode (“eco-mode”)4. Energy saving
mode offers less power protection by nature, but is higher efficiency because it is not processing the same
amount of power as a traditional double conversion UPS. There may be tradeoffs in levels of protection vs.
efficiency that come with multimode operation so that it is necessary to choose the level that is appropriate for
the data center requirement.
Another example of more efficient modes of operation is the multi-module UPS. The UPS can place unneeded
UPS modules in a low power standby mode.
Another type of multi-mode operation does not cross between UPS classifications or types, but falls in the
category of system provisioning. A UPS can be designed and configured to optimize its operation real-time to
improve system efficiency. The two simple examples described have datacenter infrastructure equivalents: a)
datacenters throttling back cooling in synchronization with heat load and b) server consolidation setting
compute loads to operate at higher duty cycles and reduce overhead losses.
The first example of UPS subsystem provisioning involves cooling. Advances in UPS controls have afforded the
ability for the UPS to control its internal fan or cooling load as a function of operating conditions, output load
and/or ambient temperature. Cooling can represent about 1 percent of a UPS’s fixed losses, or power
consumed that does not provide useful work to the critical output load. If fans can be throttled pack during
lighter loads, system efficiency can be maintained at an optimal level under different operating conditions.
4
http://www.thegreengrid.org/en/Global/Content/white-papers/WP48-EvaluationofEcoModeinUninterruptiblePowerSupplySystems
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 15
The same applies to cooling load as a function of ambient temperature - UPSs designed to operate in ambient
temperatures of 40 degrees C can made to operate cooling fans at slower speeds when the ambient that UPS
operates in is well below 40 degrees C.
System Airflow and Cooling
Fan loads represent a big portion of a UPS’s inefficiency. Ventilation is a particular design aspect of a given
UPS and should take fan airflow into consideration in order to provide precise thermal management.
Understanding a UPS fan arrangement and reasons for its design and construction can help identify system
inefficiencies. Look for underlying thermal management that shares fan airflow across multiple power train
sections. At times the number and size of fans relates to a design target for fan redundancy. The use of large
fans that provide a broader airflow stream can provide degrees of redundancy similar to multiple smaller fans
and reduce the system fan load.
Environmental Conditions
Ambient temperature - allow the measured or monitored environmental parameters to dictate levels of
performance.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 16
Measurement and Control Diagram
The following block diagram serves as a basis for illustrating different components of the CIM model for an AC
UPS:
Outputs:
Inputs:
Phase A (V, I, W)
Phase A (V, I, kW, kWh)
Phase B (V, I, kW, kWh)
Phase C (V, I, kW, kWh)
AC
Bypass
Input
AC
Output
Phase C (V, I, W)
Phase A (V, I, kW, kWh)
Phase B (V, I, kW, kWh)
Phase B (V, I, W)
Mode (normal, energy
saving, etc.)
AC
Utility
Input
Status
Phase C (V, I, kW, kWh)
AC UPS
DC+ (Vdc, Idc)
DC- (Vdc, Idc)
DC Input
Mode Inhibit
Control Input
Figure III-4: AC UPS – Measurement and control points
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 17
Core Tenets Timetable
Instrumentation
2012:
Nearly all new UPSs are fully instrumented to provide real time operating power and performance data.
ENERGY STAR® for data centers requires monitoring at the output of the UPS for PUE. Efficiency for
most UPSs is statically known based on lab measurements, but might not be gathered in real time.
Attribute
Notes
AC Bypass
Ingress power connection to the AC UPS from external “ac bypass”
Input
source
AC Utility
Ingress power connection to the AC UPS from external “ac utility”
Input
source (can include a generator source)
DC Input
Ingress power connection to the AC UPS from external “dc input”
source; could be a battery source or other type of DC source
Control Input
AC Output
Egress power connection to the AC UPS to external “ac output” load
bus
Status
Table III-1: Current AC UPS “Connection Point” Type Values
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 18
2012: (continued)
Location
AC Utility Input
AC Bypass
Input
AC Output
DC Input
Attribute
RMS Voltage
RMS Current
Real Power
Energy = Real Power
Frequency
* Time
Interval
RMS Voltage
RMS Current
Real Power
Real Power * Time
Frequency
Interval
RMS Voltage
RMS Current
Real Power
Real Power * Time
Frequency
Interval
Average Voltage
DC Current
Average Power
Battery Time
Units
Volts
Amperes
Kilowatts
Kilowatt-hrs
Hz
Volts
Amperes
Kilowatts
Kilowatt-hrs
Hz
Volts
Amperes
Kilowatts
Kilowatt-hrs
Hz
Volts
Amperes
Kilowatts
Minutes
Remaining
Table III-2: 2012 AC UPS “Connection Point” Metered
Values
2014:
Meters should be accurate to +/- 1 percent. Efficiency is measured in real time and accurate to +/- 1
percent. Instrumentation should be added for the following metered values:
Location
Attribute
Units
AC Output
UPS Output Energy
kW
AC Utility Input
Ambient
Temperature
Degrees C
AC Utility Input
Humidity
%
AC Output
True Power Factor
%
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 19
Table III-3: Additional AC UPS “Connection Point” Metered Values for 2014
2014+:
Meters should be accurate to +/- 0.5 percent. Efficiency is measured in real time accurate to +/- 0.5
percent. Instrumentation should be added for display and alarm of the following metered values:
Location
Attribute
Units
AC Utility Input
RMS Under voltage
Volts
RMS Over voltage
Volts
AC Bypass
Input
RMS Under voltage
RMS Over voltage
AC Output
Volts
Volts
RMS Under voltage
Volts
RMS Over voltage
Volts
Overload
Kilowatts
Efficiency
%
Availability
Time
System
Table III-4: Additional AC UPS “Connection Point” Metered Values for 2014+
Discoverability
2012:
UPS reports the items listed below:
Attribute
Definition
Units
ID
Name/Model
Text String
Notes
Table III-5: 2012 AC UPS “Connection Point” Status Values
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 20
2014:
The UPS should additionally report the attributes below, and report all attributes in real time, at least
every minute:
Attribute
Definition
Units
Notes
Location
Physical location
Datacenter
Security Sensitivity
Floor #
Security Sensitivity
Room #
Security Sensitivity
Table III-6: Additional AC UPS “Connection Point” Status Values for 2014
2014+:
The UPS should report all attributes in real time, at least every second.
Scalability
2012:
There is currently no standard for UPS scalability.
2014:
A UPS should be designed to add or remove capacity in real time for operation at the optimal efficiency
point (without shutting down the load, hot scalable) or automatically self-configuring.
2014+:
UPS efficiency curves should be as flat as possible, operating efficiently at any load. This allows data
centers to move away from modules. Efficiency should be 95 percent or higher when operating between
25-75 percent load.
Note that scalability affects reliability. For more information see IEEE 493, “Gold Book”.
Enhanced management Infrastructure
None specific to UPS.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 21
Ability to be policy driven
2012:
Available operating modes for the UPS are statically known and controllable. Modes include double
conversion, line interactive, eco-mode, bypass, offline – hot standby, offline – cold standby, and off.
2014:
Above operating modes are pollable and controllable by the centralized management console. Efficiency
is estimated at the UPS level by the unit and sent to a centralized management console.
2014+:
Operating modes are announced and controllable; Units communicate instantaneous monitoring and
prediction of efficiency in each mode and can be adjusted based on this data.
Standardized metrics
For UPSs, metrics should harmonize with data center sustainability:
2012:
No current efficiency standards are available for UPSs worldwide. UPS efficiency should be measured
according to the standard IEC 62040-3 Annex I (2010).
2014:
UPSs used in data centers should meet the efficiency requirements as outlined by the EU code of
conduct or Energy Star for UPSs. Energy saving mode UPSs should be used if conducive to the business
type.
2014+:
UPSs used in data centers should choose backup technologies based on TCO, materials and
sustainability.
Applicable Standards
For efficiency:

IEC 62040
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 22

EPA Energy Star for UPSs,
http://www.energystar.gov/index.cfm?c=new_specs.uninterruptible_power_supplies
For reliability:

IEEE 493
For design and safety:

UL1778
For electromechanical design, specific to US:

NEMA PE1
Transformers
Efficiency Topics
Transformers can appear in many places within the power train. For purposes of this discussion, the focus is
on “stand-alone” transformers rather than transformers embedded in the equipment (such as UPSs or PDUs),
as the performance of those transformers is factored into the performance of the device in which they are
installed.
Currently data centers have numerous isolation transformers because of the need to reduce the size and
weight of conductors. Primarily for safety reasons, the loads within a data center presently tend to be operated
at lower voltages (100 – 240 Vac). As the number and size of the loads increases, the current draw increases.
Higher currents require larger conductors. Larger conductors require more space and weigh more. Since
electrical power is proportional to system voltage multiplied by current, at some point it becomes more cost
effective to distribute power at higher voltages since the current levels (and therefore conductor sizes) will be
smaller. In alternating current systems, a transformer is used to “transform” the voltage level.
A more efficient practice would be to use fewer transformers or to use autotransformers throughout the data
center. This is done by using end devices that will tolerate operating at higher voltages. However, this can pose
a safety issue at the rack because of lower impedance and higher fault currents. Designers need to be aware
of the tradeoffs when determining the voltage for their data center equipment.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 23
Measurement and Control diagram
The following block diagram serves as a basis for illustrating different components of the CIM model for a
transformer.
Outputs:
Inputs:
Phase A (V, I, kW, kWh)
Phase A (V, I, kW, kWh)
Phase B (V, I, kW, kWh)
Transformer
Primary
Transformer
Secondary
Phase C (V, I, kW, kWh)
Phase B (V, I, kW, kWh)
Phase C (V, I, kW, kWh)
Temperature
Transformer
Pressure/Vacuum
Liquid Level
Figure III-5: Transformer – Measurement and control points
Core Tenets Timetable
Instrumentation
Transformers can be metered for many reasons. These include providing signals for protection, data for
diagnostics, and data for energy management. The type of transformer instrumentation selected is typically the
result of a trade-off between the cost of the instrumentation versus the importance of the transformer, or of
the importance of the data or loads fed by the transformer. The instrumentation selected may include the
monitoring of current, voltage, and the various signals derived from those two (power, energy, power factor,
distortion, etc.), as well as temperature. In the case of dry-type transformers, the temperature monitoring may
be done at multiple points on cores (a single transformer has one core per phase) or windings. In the case of
liquid-filled transformers, temperature monitoring is usually only done at the top of the fluid.
Typically liquid level and tank pressure or vacuum are also monitored in liquid-filled transformers.
Transformers with forced cooling, either air and/or liquid, may have additional monitoring of the motor driving
the fans and/or pumps. This fan/pump monitoring may include voltage, current, and values from those signals.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 24
Liquid-filled transformers are typically not found inside a data center and are typically larger than those used
for individual PDUs.
Large, critical transformers will typically also have sensitive transformer differential-current monitoring where
the total current flowing into the transformer is compared to the total current flowing out of the transformer,
adjusted by the turns ratio. This particular monitoring, called transformer differential protection, looks for small
differences in the calculated value versus the actual value of the ratio of input to output current. Small
differences can indicate a short within the transformer winding. This particular type of monitoring may not be
cost efficient in a typical data center, however.
2012:
Attribute
Input
Notes
Ingress power connection to the transformer from an external primaryside source (can include a generator source)
Output
Egress power connection from the transformer to an external
secondary-side load
Temperature
Egress transformer system internal temperature reading
Pressure or
Egress transformer system pressure reading
vacuum
Liquid Level
Egress transformer system liquid level status
Table III-7: 2012 Transformer “Connection Point” Type Values
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 25
2012: (continued)
Definition
Units
RMS Voltage
Volts
RMS Current
Amperes
Real Power
Kilowatts
Attribute
Input
Energy = Real Power
* Time Interval
Kilowatt-hrs
Frequency
Output
Hz
Volts
RMS Voltage
RMS Current
Amperes
Real Power
Kilowatts
Real Power * Time
Kilowatt-hrs
Interval
Frequency
Hz
Temperature
Degrees C
Pressure
PSI
Liquid Level OK
Logic High
System
Table III-8: 2012 Transformer “Connection Point” Metered Values
Typically, only input or output power is measured. High accuracy measurements at utility input should be
more accurate than compute room level.
Attribute
Definition
Units
System
Liquid Level OK
Logic High
Notes
Table III-9: 2012 Transformer “Connection Point” Status Values
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 26
2014:
Measure both input and output power to calculate efficiency. Efficiency measurements can not only
monitor for savings, but can also monitor for degradation and harmonic currents. Measurement does
not need to be done in real time, but should be measured periodically, either automatically or manually.
2014+:
None.
Discoverability
2012:
Transformers, at the basic core and coil level, are inherently non-reporting and do not announce energy
or performance levels or location. However, some subset of the total installed base of transformers
does have separate instrumentation installed that will report certain electrical and mechanical
properties. This instrumentation typically is not manufactured by the same company as the transformer,
but may be integrated into the transformer by the transformer OEM.
The data reported from this instrumentation may be presented in the form of a display and keypad
mounted on or near the transformer equipment or it could be in the form of mechanical gauges with
mechanical contacts that open or close when certain limits are reached. These contacts can be
supplied with a wetting voltage and the signal wired into a data center monitoring system through a
digital input module. More advanced systems may use serial communications from the transformer
monitor to the data center. For those systems that do have communications capability, the physical
connections and protocols range from vendor proprietary to well documented open standards.
As a result, the integration of transformer data almost always requires a custom integration effort. This
effort varies depending on what combination of different vendor's electrical and mechanical monitoring
systems are installed on any particular transformer. For a list of transformer attributes that might be
reported, refer to the appendix.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 27
2014:
Attribute
Definition
Units
Notes
Location
Physical location
Datacenter
Security Sensitivity
Floor #
Security Sensitivity
Room #
Security Sensitivity
Table III-10: Proposed discoverability attributes
2014+:
None
Scalability
Although transformers can be paralleled to increase available capacity as long as the turns ratios, phase shift
(from primary to secondary), and phase rotation are matched between units, this is almost never done in
practice. However, there is a particular type of distribution system called a network (either spot or grid) that
does rely on paralleled transformers, but these are coupled with special types of protective devices called
network protectors that monitor the direction, type (real vs. reactive), and magnitude of current flowing through
each transformer.
A system could be built with network protectors in anticipation of paralleling transformers. However, paralleling
transformers increases available fault current and may over-duty the electrical distribution system unless it is
installed with the thought that future upstream transformers may be paralleled.
2012:
In the United States, low voltage (600 V and less on either winding) dry-type distribution transformers
manufactured on or after January 1, 2007 are now required to meet NEMA TP1 efficiency levels.
Likewise in the US, all transformers with rated primary voltage of between 601 and 34500 volts and a
rated output voltage of 600 volts and less manufactured on or after January 1, 2010 are required to
meet the efficiency standards outlined in 10 CFR Part 431 5.
5http://www1.eere.energy.gov/buildings/appliance_standards/commercial/distribution_transformers_finalrule
.html
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 28
2014:
The U.S. Department of Energy is examining higher efficiency transformers. Several Candidate
Standards Levels (CSL) have been proposed. Currently interest is focusing on the CSL3 as a likely
candidate for the future energy efficiency target, replacing the current TP-1 (CSL1) transformer,
although the publish date of this document has not been decided. Fewer and higher efficiency
transformers (NEMA TP1 or equivalent) should be used. Increasing the voltage within the datacenter
increases this efficiency.
2014+:
Systems designed from the outset as a spot network may provide more scalability and higher reliability
than conventional primary selective systems; because the system can be designed as N+1 (or more)
without having to add any switching. Adding transformers increases capacity and redundancy, but it also
adds available fault current. A distribution system designed with that in mind will be braced and rated
for the higher fault currents. Efficiency can be increased by using fewer series isolation transformers or
replacing them with autotransformers.
Enhanced management Infrastructure
2012:
As mentioned previously, transformers currently require external hardware to report into a management
infrastructure. This hardware could be specified (but typically is not) to report into an IT monitoring
system.
2014:
None
2014+:
None
Ability to be policy driven
This does not apply to transformers.
Standardized metrics
2012:
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 29
As a device that attempts to transport electrical energy as efficiently as possible, one design goal is to
reduce transformer losses and thereby increase transformer efficiency. Transformer efficiency is
defined as the ratio of the useful power leaving the transformer divided by total power entering the
transformer; typically, the larger the transformer, the higher the efficiency. Today, transformers usually
do not include high accuracy power monitoring of both the input (primary) and output (secondary)
power, other than the transformer current differential protection mentioned previously. (Note that this
differential monitoring neither calculates nor displays transformer efficiency for the operator to use.)
2014:
None.
2014+:
Transformer efficiency should meet the appropriate CSL standard that will be defined by industry.
Applicable Standards
For efficiency: NEMA TP1
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 30
PDU
A Power Distribution Unit (PDU) is defined here as a free standing electrical distribution cabinet, whose main
function is to provide a required point of power distribution. The PDU houses circuit breakers that are used to
create multiple branch circuits from a single feeder circuit, and can also contain transformers, electrical panel
boards, and surge protection devices.
PDU transformers
A secondary function of some PDUs is to convert voltage. The AC voltage–converting PDUs contain either an
isolation transformer or an autotransformer to step the AC distribution voltage down. See the Transformers
section for more information.
DC voltage converting PDUs contain DC/DC converters. For simplicity, multiple PDUs are modeled as a single
block. Where appropriate, isolation is indicated in the block diagrams.
Typical Power Distribution Unit Specifications:

Rating:
30 - 300kVA

Transformer type: Isolation transformer or Autotransformer

Harmonic tolerance type:

High efficiency option:

Input Voltages:

Output Voltages:
K4, K13, or K20 (optional based on need)
(TP1 or better)
208,480, or 600VAC
208/120VAC
PDU Types
A large data center typically contains a large number of PDUs. A single site can contain over 100 PDUs. The
location and number of distribution points depends on the power system layout and design and can take on
many more forms depending on their location and function in the overall power system, as shown in Figure 1-2.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 31
To IT Loads
Input
Metering
Optional
Transformer
Output
Metering
To
rack-mounted
distribution
To remote
power panels
Figure III-6: Representative single-line diagram of a floor-mounted PDU
Examples of PDUs include Remote Power Panels (RPP), Power Distribution Racks (PDR), Rack Power Modules
(RPM), and zero U electrical strips (ePDU). The PDU naming convention used will vary depending on the
equipment vendor. The following generic definitions serve as a guide in determining where a particular PDU
type resides and what its purpose may be:
Remote Power Panel (RPP)
A remote power panel is a distribution panel that is subfed from an upstream panel, usually from a floor-mount
PDU. This PDU maximizes use of datacenter space by locating the point of distribution of moderately high
current capacity close to compute loads. The device is basically a transformerless PDU that meets a particular
form factor requirement. Generally equipped with standard panelboards, this cabinet accommodates branch
circuit breakers. Like its transformer- based relative, it can be configured with single or dual source feeds to
accommodate an A-B bus and dual corded downstream equipment.
Rack Power Distribution Unit (RPDU )
An RPDU should not be confused with a floor-standing PDU. Instead of using circuit breakers the RPDU
consists of multiple receptacles into which the power cords of the ITE can plug. It is packaged in a standard
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 32
ICT equipment rack form factor so that it can be located either in the same rack/cabinet or in-row close to the
IT equipment racks that it is powering. RPDU’s are typically cord-connected and available in a variety of form
factors. Some are rack-mounted and take up 1-4U of shelf space. Others mount outside of the U area within a
rack or cabinet (for example, mounted vertically in the rear of the rack or cabinet). RPDUs are also known by
such names as “rack power module”, “rack PDU”, “zero-U PDU”, “enclosure PDU”, and “cabinet distribution
unit”. Because it is the final connection point to the ITE, an RPDU can provide the finest level of power
consumption granularity in the data center (except for what can be derived for the individual ITE itself).
Similar to the larger PDU, the device can be highly configurable and may include a dual feed option, remote
switching, rack-level power monitoring, or even individual cord circuit monitoring..
Efficiency Topics
PDUs that contain transformers should follow the requirements laid out in the Transformers section of the
document.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 33
Measurement and Control diagram
Outputs:
Inputs:
Output 1 (V, I, W)
Phase A (V, I, kW, kWh)
Phase B (V, I, kW, kWh)
Output 2 (V, I, W)
AC Input 1
AC Outputs
Phase C (V, I, kW, kWh)
:
:
Circuit N (V, I, W)
Phase A (V, I, kW, kWh)
Phase B (V, I, kW, kWh)
Output 3 (V, I, W)
AC Input 2
(optional)
Source 1 available (T/F)
Phase C (V, I, kW, kWh)
PDU
Source 2 available (T/F)
Output 1 enable (T/F)
:
:
Output N enable (T/F)
Status
Control
Inputs
AC Input Select Override
Output 1 enabled (T/F)
:
:
Output N enabled (T/F)
Figure III-7: PDU Measurement and control points
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 34
Core Tenets Timetable
Instrumentation
For PDUs that contain transformers, instrumentation similar to that in the transformer section should be
implemented.
For PDUs that do not contain transformers, the inputs and outputs can be reduced to the power connections
on the input and output locations; temperature, pressure and liquid level are no longer required.
2012:
Attribute
Input
Notes
Ingress power connection to the transformer from an external primaryside source (can include a generator source)
Output
Egress power connection from the transformer to an external
secondary-side load
Temperature
Egress transformer system temperature reading
Pressure or
Egress transformer system pressure reading
vacuum
Liquid Level
Egress transformer system liquid level status
Table III-11: 2012 Transformer “Connection Point” Type Values
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 35
2012: (continued)
Attribute
Definition
Units
Input
RMS Voltage
Volts
RMS Current
Amperes
Real Power
Kilowatts
Energy = Real Power
Kilowatt-hrs
* Time Interval
Frequency
Hz
RMS Voltage
Volts
RMS Current
Amperes
Real Power
Kilowatts
Real Power * Time
Kilowatt-hrs
Output
Interval
Frequency
Hz
Temperature
Degrees C
Pressure
PSI
Liquid Level OK
Logic High
System
Table III-12: 2012 Transformer “Connection Point” Metered Values
Typically, only input or output power is measured. High accuracy measurements at utility input should be
more accurate than compute room level.
Attribute
Definition
Units
System
Liquid Level OK
Logic High
Notes
Table III-13: 2012 Transformer “Connection Point” Status Values
2014:
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 36
Measure both input and output power to calculate efficiency. Efficiency measurements can not only
monitor for savings, but can also monitor for degradation and harmonic currents. Measurement does
not need to be done in real time, but should be measured periodically, either automatically or manually.
2014+:
None.
Discoverability
2012:
Some models of all types of PDUs are discoverable on a network. This feature is typically available at an
extra cost.
2014:
All PDUs, regardless of their type or what they contain, should be discoverable
Attribute
Definition
Units
Notes
Location
Physical location
Datacenter
Security Sensitivity
Floor #
Security Sensitivity
Room #
Security Sensitivity
Table III-14: PDU discoverability attributes
2014+:
None
Scalability
2012:
None
2014:
None
2014+:
None
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 37
Enhanced management Infrastructure
2012:
DUs with transformers currently require external hardware to report into a management infrastructure.
This hardware could be specified (but typically is not) to report into an IT monitoring system.
2014:
None
2014+:
None
Ability to be policy driven
2012:
Some models of rack power distribution equipment have the ability to be monitored, capped, or
otherwise controlled.
2014:
None
2014+:
All models of PDUs should have the ability to be monitored and controlled at the output.
Standardized metrics
For PDUs with transformers, follow the information listed in the transformer section.
Applicable Standards
For efficiency: NEMA TP1
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 38
Newer power distribution technologies under consideration
Most data centers today use alternating current (AC) power systems, which distribute electricity somewhere
between 100Vac and 600Vac throughout the facilities. However, a growing number of direct current (DC)
advocates are promoting the use of DC power in the data center.
Higher voltage direct current (HVDC) power distribution configurations (<600Vdc) have the potential to reduce
energy consumption and increase efficiency in the data center. Higher voltage direct current power systems
may involve fewer components, which can result in higher reliability and lower total cost of ownership when
compared to AC power systems.
Efficiency
Efficiency was studied in The Green Grid via White Paper #16 Quantitative Analysis of Power Distribution
Configurations for Data Centers 6. The more detail on the following information can be found in that white
paper.
A typical North American AC system drops the incoming AC voltage down through a series of conversions to
480Vac. At that point a double conversion uninterruptible power supply (UPS) converts AC to DC for battery
charging, typically resulting in losses of 3% or greater. An inverter then converts the voltage back to AC,
typically adding 1% to 3% losses for this second conversion. A power distribution unit (PDU) - which, if it has a
transformer, is a third conversion, typically with 1% to 2% losses - distributes AC voltage to power supply units
(PSUs) in each of the various IT equipment loads. The PSU finally converts the voltage to 12Vdc, currently with
6% to 10% losses.
A typical HVDC system first uses a rectifier to convert the incoming 480 VAC to 380VDC. Then a PSU converts it
directly to 12Vdc. Eliminating the extra conversions means that HVDC power distribution configurations
generally need no inverters and fewer step-down converters or intermediate voltages.
Reliability
With fewer individual components, HVDC systems are less complex and likely to be more reliable than AC
systems. In addition to conversion-related hardware, HVDC removes other elements, such as the sensing
6
http://www.thegreengrid.org/en/Global/Content/white-papers/Quantitative-Efficiency-Analysis
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 39
mechanisms that AC systems must use to ensure that equipment is in synchronization. Paralleling of sources
in a DC system is simplified because synchronization is unnecessary.
The reduction of complexity can also save space. Cost savings can also be realized due to fewer components
in the UPS (and possibly in the PSU), reduced cable losses, and a reduced cable cross-sectional area, which
requires less copper.
All that said, AC represents far more of a known quantity. Additionally, other alternative AC solutions in
development increase end to end efficiency. HVDC is a relatively new concept and, while well-understood,
there are not nearly as many scaled-up, production-level examples of HVDC power distribution configurations in
operation. However, many groups worldwide have launched HVDC demonstration sites. In Japan, verification
tests were performed on an HVDC system in 2009 with the cooperation of domestic and international IT
equipment manufacturers. The test results showed the HVDC system, when compared to the unmodified
existing AC system, had higher availability and end to end efficiency.
Please see The Green Grid’s white paper #31 Issues relating to the adoption of higher voltage direct current
power in the data center for further information and issues relating to the adoption of HVDC power system in
the data center.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 40
Appendix
Transformers
An isolation transformer (Iso Xfmr) is an electromagnetic device with multiple windings per phase that converts
an AC voltage into another AC voltage and simultaneously provides galvanic isolation by transmitting energy
between windings through magnetic inductive coupling (induction). The transformer winding size and type is
selected to provide the required voltage transformation level as well to provide secondary benefits including
harmonic cancellation through phase-shifting and transient voltage reduction through grounding.
Main
Protective
Device
Optional
Transformer
To IT Loads
Main
Protective
Device
Tie
Optional
Transformer
To IT Loads
Figure III-8: One-line representation of Secondary Selective System Using Two Winding Transformer
Transformer windings are available in a variety of connection types including zig-zig, wye, delta, and T (Scott).
Each of the windings on an isolation transformer can be different connection types. For example, a commonly
available transformer has a delta-connected input (called the primary winding) and a wye-connected output
(secondary winding).
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 41
Figure III-9: Delta/Wye 3-Phase Transformer (ANSI transformer symbols shown)
Referring to Figure III-9: Delta/Wye 3-Phase Transformer (ANSI transformer symbols shown), notice that in
addition to the three connections to the right side winding (secondary winding), that wye-connected winding
has a fourth connection. The fourth connection, called the neutral, is typically grounded. A neutral connection
provides additional transformation ratios since loads can be connected either phase-to-phase (e.g. A-B, B-C,
etc.) or phase-to-neutral (e.g. A-N, B-N, etc.). When loads are connected to the transformer secondary phase to
phase, again referring to Figure III-9: Delta/Wye 3-Phase Transformer (ANSI transformer symbols shown), the
ratio of the secondary voltage to the primary voltage is defined by the ratio of the number of turns of wire in
each winding, as follows:
VAB N1

VA ' B ' N 2
Equation III-1
where:
VAB
is primary phase-to-phase voltage
VA′B′
is secondary phase-to-phase voltage
N1/N2
is turns ratio (ratio of number of turns on primary winding divided by the
number of turns on the secondary winding
When those same loads are connected phase-to-neutral, the voltage supplied will be equal to the phase-tophase voltage magnitude divided by the square root of 3.
VA ' N 
VA ' B '
3
Equation III-2
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 42
where:
VA′N
is primary phase-to-neutral voltage
VA′B′
is secondary phase-to-phase voltage
Besides having a convenient additional voltage tap, the neutral connection of a transformer is a common point
to bond the electrical distribution system to earth ground. Creating a reference between the distribution system
and earth is an important safeguard to reduce the likelihood of transient phase to ground overvoltages caused
by intermittent ground faults. Since circulating ground currents can flow if the neutral is bonded to ground at
multiple locations, using a transformer with a delta primary is a common method of blocking ground (also
known as zero-sequence) current flowing through the transformer. For this reason, wye/wye transformers
should not be used except under certain circumstances7 or when mandated by local utilities8. If wye/wye
transformers are being considered, it is recommended that a power systems engineer familiar with the
possible problems be consulted prior to completing the design.
7
Wye-wye transformers can have very low zero sequence impedances, causing higher than expected ground
fault currents. Unless 5-legged core designs are used, unbalanced loads can result in excessive transformer
enclosure heating.
8
Some utilities may mandate a wye-wye transformer at the service entrance to reduce the maximum voltage
that can appear across an upstream protective device.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 43
Figure III-10: Measured Loss-versus-Loading for 75 kVA TP-1 Transformer with Linear and Non-Linear Loading
Another factor that affects the performance of a transformer is the presence of non-linear currents. In locations
like data centers that include non-linear single-phase loads, and where those loads do not use power factor
(harmonic) correcting power supplies, third harmonic currents generated by those loads can become trapped
in delta windings of transformers. This trapping is beneficial in that these currents do not leave the transformer
to affect upstream sources, but detrimental in that they consume available capacity from that transformer.
These additional third harmonic currents increase heating within the transformer by a factor of the square of
the magnitude of the harmonic current within the winding.
Additionally, since eddy current losses within the core are proportional to the square of the frequency of the
harmonic, third harmonic currents generate nine times greater core losses when compared to the fundamental
frequency current. As these third-harmonic currents are produced only from single-phase, but not three-phase,
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 44
power supplies, one mitigating technique is to specify three phase power supplies. Of course another is to
specify power factor (harmonic) corrected power supplies. If you must deal with the harmonics generated from
legacy equipment, another way of mitigating third-harmonic currents is to select a different type of transformer
winding.
Figure III-11: Delta/Zig-Zag 3-Phase Transformer (ANSI transformer symbols shown)
One type of transformer winding that cancels third-harmonic currents is called a zig-zag winding (see Figure
III-11Figure III-11: Delta/Zig-Zag 3-Phase Transformer (ANSI transformer symbols shown)). Unlike delta
windings, zig-zag windings do not simply trap the third harmonic currents within the winding. Instead, they rely
on a particular property that third harmonic current appears as zero sequence current. Unlike phase current
that is displaced by 120 degrees between each phase, each third harmonic phase current is 360 degrees out
of phase with the other third-harmonic phase currents. The result is that each is in phase with the others.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 45
Note that in a zig-zag winding, each phase winding is split into two parts. One half of a winding is wound on the
same core as one other phase, except that this other phase is reversed in polarity. The result is that the two
magnetic fields of in-phase currents combine and cancel, preventing any magnetic induction through the core.
There are no eddy current losses and no current trapped circulating around a delta winding. The result is much
lower loss and therefore higher efficiency.
Removing these third-harmonic currents (which are part of a larger group of harmonic currents called triplen
currents) may also reduce losses within electrical structures. This is because triplen currents sum in the
neutral path. Not only will there be higher resistive losses in that conductor, there can also be inductive heating
effects with any surrounding metal. If the neutral conductor is routed in the same conduit as the other three
phase conductors, the magnetic induction cancels and no induction occurs as no currents flow in the conduit
or structure.
However, what may happen within electrical distribution equipment is that the neutral bus may be mounted
closer to the walls of the equipment enclosure than one or more of the remaining phase conductors. The
harmonic currents flowing through that bus or cable inside the enclosure will not be canceled as effectively by
other currents flowing in any other phase. As a result these triplen currents can inductively couple to the
conduit, busway housing, or equipment itself, and induce inductive heating losses within that equipment. The
symptoms of this appear on infrared photographs that show a hot spot on an enclosure, conduit, or busway
wall that exceeds the temperature of the conductor itself that is contained within the enclosure. It is the
inductive heating effects within the metal structure (enclosure, conduit, etc.), caused by the inductively
coupled harmonic currents flowing through the neutral path, that heat this metal hotter than the enclosed
conductors. The hottest spot will occur at a point where the impedance of the enclosure, conduit, or busway
housing increases, such as at a splice or threaded or pressure junction between two pieces of metal.
Regardless of the winding type selected, one particular special transformer arrangement involves paralleling
multiple transformers feeding a common load. This configuration is called a network connection. In this layout,
multiple sources feed a common load. This provides redundancy and scalability. It is important in a spot
network configuration that each parallel transformer be identical in winding type, transformation ratio, and
impedance to insure that each transformer shares its portion of the load. As shown in Figure III-12: One-Line
Representation of Spot Network. (ANSI transformer symbols shown), a special protective device called a network
protector is required. Unlike conventional overcurrent protective devices, a network protector must monitor the
direction of current flowing as well as the magnitude of reactive power.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 46
Ordinary protective devices like fuses or circuit breakers only examine the magnitude of current, but don't care
if the current is real or reactive, nor do they care about the direction of flow for either or both of those currents.
If, for example, the upstream feeder were open, the network transformer would still be excited through backfeeding of reactive power from the load bus. In many networks, this is considered an acceptable mode of
operation, but requires that the network protector distinguish acceptable transformer excitation current (which
will be reactive) from fault current flowing backwards through the transformer (which will have a real
component as well).
Typical Feeder
Primary
Secondary
Network
Protector
Loads
Network Transformer
Figure III-12: One-Line Representation of Spot Network. (ANSI transformer symbols shown)
An autotransformer (Auto Xfmr) is an electromagnetic device with a single winding per phase that converts one
AC voltage to a different AC voltage, but does not provide galvanic isolation between the two windings as the
two windings are wired in series. Autotransformers are generally specified when their smaller size and more
efficient operation (compared to similarly rated isolation transformers) are desired.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 47
HV
Secondary
Primary
HV
Secondary
Primary
LV
LV
Step-Down(Buck)
Step-Up(Boost)
Figure III-13: Single-Phase Autotransformer. Also known as "buck" or "boost" transformers depending on whether
application is step-down or step-up, respectively. (ANSI transformer symbols shown)
Primary
A (H1)
B (H2)
LV
HV
A (L1)
C (H3)
LV
HV
B (L2)
N
LV
HV
C (L3)
N
Secondary
Figure III-14: Three-Phase Autotransformer wired as a step down transformer. (ANSI transformer symbols shown)
Autotransformers are smaller and more efficient because the secondary winding must only be sized to provide
the current needed at a low voltage (typically values like 16, 24 or 32 volts). This secondary voltage is then
wired in series with the primary. If the secondary winding is wired such that the input and output voltages add,
the transformer is called a "boost" transformer and the output voltage is raised above the input voltage. If the
winding is wired to subtract, the transformer is called a "buck" transformer and the output voltage reduced
below the input voltage.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 48
Unlike a two winding transformer, the kVA rating of the autotransformer will just be the voltage increase or
decrease times the rated current; and for a three phase unit times the square root of 3. For instance, an
autotransformer that provides a voltage increase from 400 to 480 volts must step the voltage up by 80 volts. A
single-phase 7.5 kVA autotransformer can provide 94 amperes at 80 volts. Wired in a 3-phase connection, with
an output voltage of 480 volts, three 7.5 kVA autotransformers providing 94 amperes per phase provide the
equivalent of a 75 kVA two winding transformer (480 x 94 x 1.732 versus 80 x 94 x 1.732)! In this case the
autotransformer is only 1/10th the size (and cost) of an equivalently sized two-winding transformer.
Additionally, that smaller transformer will have lower total losses than an equivalently rated two-winding
transformer 10 times larger.
For small transformation ratio changes (480 to 400V or 208 to 240V for example), the physical size of the
autotransformer (and therefore its losses) will be smaller than an equivalently rated two-winding transformer.
Remember that an autotransformer does not provide galvanic isolation between primary and secondary and
also tends to have a lower impedance compared to similarly rated two-winding transformers. Be sure to include
these factors when calculating required bus bracing, interrupting rating, and when performing arc flash
calculations.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 49
Transformer Attributes
Name
Value
1
kVA capacity
2
Primary Winding connection
3
Primary Voltage
4
Primary Basic Impulse Level (BIL)
5
Primary Current Rating
6
Primary conductor type
(copper or aluminum)
7
Primary termination area
(air terminal chamber [ATC], throat, close-coupled)
8
Secondary Winding connection
(1-phase, 3-phase wye, 3-phase delta, T)
9
Secondary Voltage
(1-phase, 3-phase wye, 3-phase delta, T)
10
Secondary BIL
11
Secondary Current Rating
12
Secondary conductor type
(copper or aluminum)
13
Secondary termination area
(ATC, throat, close-coupled)
14
% Efficiency @ FL
15
No Load Losses
16
% Impedance (%Z)
17
Temp Rise @ FL
18
Insulation Temp Rise Rating
19
Weight
20
Dimensions
21
Insulation Type
(Dry Type [VPE, VPI, Cast], Mineral Oil, Vegetable Oil, Silicon)
22
IEEE Cooling Class
(see Cooling class Tab)
23
Forced Air Cooling Fan voltage
24
Forced Air Cooling Fan current
25
Forced Fluid Cooling pump voltage
26
Forced Fluid Cooling pump current
27
Design
(Substation, Unit Substation, Pad Mounted, Pole Mounted, Distribution)
28
Taps
(e.g. 2.5%, 2 Full Capacity Above Normal (FCAN) and 2 FCBN)
29
Enclosure material
30
Enclosure color
31
Enclosure ingress protection rating
US: NEMA 1, NEMA 12, ROW: IEC IP ratings
32
Temperature sensing
Dial Type Thermometer, RTDs, Thermocouples
33
Liquid Level Gauge
Y, N, N/A
34
Pressure Vacuum Gauge
Y, N, N/A
Table III-15: Transformer attributes
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 50
IEEE Std C57.12.00-2010 and IEC 60076-2: 1993 both define designations used to describe how
transformers are cooled. Earlier standards used notations such as OA (open air), FA (forced air), FFA (future
forced air), etc. These are now considered obsolete designations and should be replaced with these IEEE and
IEC standard designations. As outlined in Table 2 from IEEE Std. C57.12.00-2010, the previous designations
can be converted to present designations. Examples include the following:
Previous Cooling Type Designations
Current Cooling Type Designations*
OA
ONAN
FA
ONAF
OA/FA/FA
ONAN/ONAF/ONAF
Table III-16: Transformer designations – previous and current
* First Character - Internal cooling mechanism in contact with windings
O - Mineral Oil with fire point < or = to 300 deg C
K - Insulating liquid with fire point > 300 deg C
L - Insulating liquid with no measureable fire point
Second Character - Circulation method for internal cooling medium
N - Natural convection flow through cooling equipment and in windings
F - Forced circulation through cooling equipment (i.e. cooling pumps) and natural convection flow through windings
D - Forced circulation through cooling equipment, directed from the cooling equipment into at least the main windings
Third Character - External cooling medium
A - Air
W - Water
Fourth Character - Circulation mechanism for external cooling medium
N - Natural Convection
F - Forced Circulation - Fans (air cooling) or Pumps (water cooling)
Table III-17: Transformer designations – letter definition
Transformer Type
Defining Characteristic
Substation
HV and LV bushings on top of tank or enclosure
Unit Substation
HV and LV bushings on side of tank or enclosure
Pad Mount
Both sets of bushings (HV and LV) on same side under lockable cover
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.
PAGE 51
Pole
Distribution
Transformer hangs on utility pole
Typically smaller, low voltage units with simple lug HV and LV connections
Table III-18: Transformer types
IV.
About The Green Grid
The Green Grid Association is a non-profit, open industry consortium of end users, policy makers, technology
providers, facility architects, and utility companies collaborating to improve the resource efficiency of
information technology and data centers throughout the world. With more than 150 member organizations
around the world, The Green Grid seeks to unite global industry efforts, create a common set of metrics, and
develop technical resources and educational tools to further its goals. Additional information is available at
www.thegreengrid.org.
The Green Grid Confidential – Members-Only Content
2012 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted, or stored
in any retrieval system of any nature without the written permission of the copyright owner.