WHITE PAPER #23
PROPER SIZING OF IT
POWER AND COOLING
LOADS WHITE PAPER
CONTRIBUTORS:
JOHN BEAN, APC
RON BEDNAR, EMERSON NETWORK POWER
RICHARD JONES, CHATSWORTH PRODUCTS
ROBB JONES, CHATSWORTH PRODUCTS
PHIL MORRIS, SUN MICROSYSTEMS
DAVID MOSS, DELL
DR. MICHAEL PATTERSON, INTEL
JOE PRISCO, IBM
WADE VINSON, HP
JOHN WALLERICH, DELL
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 2
TABLE OF CONTENTS:
I.
Introduction.................................................................................................................................................3
II.
Today’s Data Centers.................................................................................................................................3
III.
Nameplate Power Rating Misconceptions................................................................................................3
IV.
Nameplate Versus Actual...........................................................................................................................5
V.
Cooling Load Calculations..........................................................................................................................6
VI.
Energy Efficiency Focus – New Tools.........................................................................................................6
VII.
Tool Links.....................................................................................................................................................8
VIII.
Proper Use of Calculating Tools.................................................................................................................9
IX.
Summary/Conclusion.................................................................................................................................9
X.
References...............................................................................................................................................10
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 3
I. INTRODUCTION
Between 2000 and 2006, energy requirements for data centers doubled, and they are on track to double
again by 2011. In the same time period, typical per-rack heat densities went from 1 kW to 7 kW, and they
are estimated to exceed 20 kW per rack by 2010. This places tremendous importance on a data center
facility planner’s ability to properly budget for future power and cooling infrastructure requirements. In the
past, facilities planners used IT equipment nameplate ratings values as a method to plan power and cooling
capacity requirements, which resulted in over-built, under-utilized, inefficient infrastructure. Today’s energy
costs and efficiency demands necessitate a more accurate method of determining those requirements. This
white paper has been developed to introduce the reader to the many new and highly accurate software tools
available for estimating power and cooling capacity requirements.
II. TODAY’S DATA CENTERS
The inaccuracy of data centers’ standard process for estimating the power consumption of IT equipment
has led to the under- and over-provisioning of circuits, power distribution units (PDUs), and uninterruptible
power supplies (UPSs) as well as the cooling systems needed to remove the energy dissipated. This can have
a significant impact on power distribution efficiencies, total power consumption, service availability, and
operational costs.
In an attempt to simplify this process and to help improve power consumption accuracy, the majority of IT
vendors have made power and cooling estimation tools available for their various equipment models and
configurations. Prior to the proliferation of those tools, the American Society of Heating, Refrigerating, and
Air-Conditioning Engineers (ASHRAE) suggested the use of the “thermal report,” which provided the end
user with power-draw and cooling-load estimates—heat removal and airflow rates—for minimal, typical, and
maximum load configurations1. But even that significant step forward was still subject to errors based on
different server configurations and the exact complement of hardware options (CPU, hard drives, memory, I/O
configurations, etc.) The minimum baseline has since been expanded upon by system manufacturers.
III. NAMEPLATE POWER RATING MISCONCEPTIONS
Prior to the ASHRAE thermal report, the buyer or data center architect had little-to-no information on the
expected power consumption of a server, so the default value was initially the nameplate value. Yet the
term “nameplate” is often misused. When discussing server power draw and cooling loads, nameplate is
frequently used (albeit incorrectly) to describe the value from the server data sheet for the power supply
(generally the output of the power supply listed in watts). The technically correct description of “nameplate
value” is the value from the system rating label (see Figure 1).
The nameplate on a piece of IT equipment is a marking from a third-party agency that indicates that the
equipment complies with a set of requirements set forth by a product safety standard such as IEC 60950-1
(Information technology equipment - Safety - Part 1: General requirements). Figure 1 shows an example of a
nameplate found on a piece of IT equipment.
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 4
FIGURE 1. EXAMPLE NAMEPLATE
The nameplate includes, at a minimum, rated input voltage, rated input amperage, kilovolt-ampere (kVA),
frequency, and phase information. It is important to note that the wattage does not appear on the nameplate.
The purpose of the nameplate is twofold:
• To give information necessary for the preparation of the electrical installation (e.g., circuit
breakers, receptacles, wiring methods) to ensure adherence with electrical codes.
• To provide assurance to the installer or user who plugs the equipment into the mains that the
equipment is suitable for installation at a particular location within the data center.
Manufacturers derive the nameplate value in many different ways to ensure compliance with a product
safety standard. The value could be based on the line cord assembly that is used to provide power to the IT
equipment or the DC output power rating on the power supply unit (PSU).
The nameplate value chosen may be a result of the cost of manufacturing: many OEMs standardize their
power supplies on a smaller number of PSUs, which can result in substantial gaps between nameplate values
and actual power consumed for a given piece of IT equipment. Also, power consumption can vary based
on device configurations—additional components such as memory can increase the actual power draw of
equipment. The same power supply may be used in a server with low-voltage CPUs and minimal memory as in
the same server model that is configured with the highest memory and CPU configurations.
Regardless of how they are chosen, nameplate values are generally accepted as representing power
consumption levels that exceed actual power consumption of equipment under normal usage. Therefore,
these over-inflated values do not support realistic power prediction.
Unfortunately, nameplate value (regardless of whether it is the incorrect use of the data sheet PSU
description or the proper definition above) is still frequently used by data center planners in the design,
planning, and deployment of power and cooling systems. This practice often results in excessive capital
expense and operational costs due to lower efficiencies that result from overestimating resource
requirements. Many of these planners mistakenly use the nameplate value because it is either on the data
sheet when they are ordering or always available and clearly visible on any piece of IT equipment when
installed.
There is also a misconception that devices upstream from the IT equipment (such as PDUs and/or UPSs)
must be sized to accommodate the sum of all the nameplate ratings. As discussed above, the nameplate
rating generally does not reflect actual power consumption and using it as an indicator leads to grossly
over-provisioned power distribution systems. Typically, any electrical loads hardwired into a system are, by
code, limited to 80% of the upstream components’ capacity. However, because the server has a plug, it is
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 5
considered “portable” and the management of the load then falls to the owner. In this case, a better design
practice would be to use the peak-measured or maximum-measured manufacturer’s reported draw for the
specific IT equipment configuration and ensure that the sum of those loads does not exceed 80% of the
rating of the upstream panel. Also, it is important to check with the local code authority having jurisdiction
for any specific ruling that would preclude this practice. There are some specific geographies, instances,
and tools that can allow higher loadings when the diversity of the load (frequency of equipment peak power
occurring at the same time) is known, but these are outside the scope of this paper. In general, the electrical
loading should be sized based on IT equipment peak measured or maximum measured power consumption
levels and not nameplate values.
IV. NAMEPLATE VERSUS ACTUAL
51 Servers, PSU Rating vs peak draw
Actual
"50%"
PSU Nameplate (Watts)
1400
1200
1000
800
600
400
200
0
0
200
400
600
800
Peak draw (Watts)
FIGURE 2. PSU NAMEPLATE RATING VERSUS PEAK POWER DRAW
The above figure shows 51 servers and their power supplies tested by the U.S. Environmental Protection
Agency3. The nameplate rating (Y axis) is plotted against peak draw (X axis). Consider, for example, the large
group of 1000 W PSUs in the figure. There were 23 of these, which indicates the manufacturer’s propensity to
use “standard-size” components at the next-greater size rather than exactly matching the expected load.
Also noteworthy is the range of actual power draw found for these servers. The low end was roughly 160
watts compared to a high end of about 580 watts—all using the same 1000 W power supply. With only the
knowledge of the nameplate value in hand, many data center managers use a method of de-rating in order to
estimate actual power consumption. In doing so, they establish actual power consumption as a percentage of
the nameplate value. Some use a simple “nameplate divided by 2” (nameplate/2) convention for estimating
power loads. If that method were accurate, then the peak draw values would be 50% of the PSU nameplate
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 6
values and fall on the 50% line in Figure 2, yet review of Figure 2 clearly shows this is not the case. This
example shows how the nameplate/2 practice could result in incorrect power and cooling load estimates,
as well as the over-sizing (or under-sizing) of both electrical systems and cooling systems. Some of the data
points fall on or near the 50% line, but the number that do is far less than the number that do not.
The issue is further complicated by the redundancy (or lack thereof) in PSUs. A typical server operation with
redundant PSUs has both PSUs operational and carrying 50% of the load. In the case of one failing, the
functioning PSU can rapidly ramp to 100% of the server load. Two important aspects should be noted. First,
the total connected load from the dual PSU “nameplates” is now double the non-redundant load and can add
to far greater over-sizing if not clearly understood. Second, dual power supplies will typically increase server
consumption, due to the PSU being lower down the utilization curve, which can mean lower efficiency. This
added power needs to be factored in. Clearly, rule-of thumb methods such as nameplate/2, PSU divided by 2,
and others cannot account for this level of detail.
V. COOLING LOAD CALCULATIONS
Data center planners and managers also use IT equipment’s power predictions to calculate cooling load. All
of the inaccuracies involved in using nameplate data for power load calculations are repeated in the cooling
load analysis. In sizing the cooling infrastructure, both the cooling load (kW or BTU/hr) and airflow rates (m3/
hour of CFM) need to be determined. To further complicate the problems created by using nameplate data,
planners and managers use an estimated, historical, or rule-of-thumb differential temperature (deltaT) across
the servers. From that deltaT and the estimated power load, they calculate an airflow rate. Unfortunately, that
airflow rate is based on two inaccurate estimates (nameplate and an assumed deltaT).
VI. ENERGY EFFICIENCY FOCUS – NEW TOOLS
As a result of the focus on energy efficiency, most IT vendors now provide power measurement and power
estimation tools that can help to identify a more accurate power consumption value for their IT equipment.
That value is based either on actual measurements (a power meter) or an estimation tool (hardware options
and utilization).
Ideally, measuring the power consumed by IT equipment provides the most accurate method for determining
the power and cooling load that IT equipment places on a data center. IT equipment manufacturers provide
power meters that are internal and external to IT equipment. These can be used to show trends in power
consumption over long periods of time under many workloads. Measuring power consumption allows for realtime measurement of application intensity and other factors that have an impact on power consumption,
such as inlet ambient temperature and/or operating voltage.
Vendors’ power and cooling estimation tools are best for planning power and cooling loads for new equipment
being placed in the data center. The values found in vendor planning tools are typically the result of using
this approach, with an added margin for error that will yield much more useful power consumption estimates.
Application loading or utilization within the planning tool is offset from a peak power consumption level based
on some type of proxy application, such as an industry benchmark or stress program. It is the deviation from
actual usage to this proxy that adds one level of inaccuracy. Another factor that increases inaccuracy is the
variation of power consumption in electronic components that vary from lot-to-lot. Although vendors’ tools
typically yield results that are far better than common de-rating methods, their results tend to be somewhat
conservative as compared with actual power measurements from the IT equipment.
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 7
An additional benefit of power and cooling estimation tools is that some tools provide more accurate
estimations of the specific airflow rates for the server configuration, utilization, and inlet air temperature. The
airflow rates—combined with the heat released by the equipment—will provide the information that planners
need to size the cooling infrastructure.
Table 1 below is a typical example of the results obtainable by using a vendor estimation tool compared to the
value that would likely have been derived from using a nameplate de-rating estimate.
Equipment Design Input
Tool Output
1U server
Power
370 watts
208V AC power (power factor .98)
Flow rate
32 CFM
Dual socket quad core processor
Weight
46 lbs
8 GB memory
Current
1.78 A
2 disk drives
deltaT
37.3°F
Dual power supplies
Operating condition: 25°C server
inlet temperature, typical workload
TABLE 1. TOOL EXAMPLE
Using the configuration above, the sample vendor’s power estimate is 370 watts. Per the vendor’s guidance,
this estimate would still be conservative, even for a virtualized load that typically operates at a higher level of
processor utilization. General-purpose servers running at lower utilization levels would measure up to 20%
less in terms of total power consumed.
The tool estimates a worst-case configuration for the same server to be 490 watts under full load, but that
power estimate would increase to 555 watts if the server were running a more processor-intensive load,
such as HPC (high performance computing) cluster computing. Furthermore, if the system were running a
processor-intensive load at a maximum inlet temperature of 35°C (95°F), the power estimate would increase
to 620 watts due to the electrical inefficiencies that result from running at high temperatures. This server has
a power supply nameplate value of 670 watts. If, for example, the server power supply was actually delivering
650 watts of DC power, and it was running at 85% efficiency, then it would be consuming 765 watts and
releasing approximately 2610 BTUs (1 W of load requires 3.412 BTUs to cool) of heat into the room.
This example demonstrates that there are many parameters that affect power consumption, including, but
not limited to, server configuration, processor selection, memory amount and configuration, number of drives,
peripherals, I/O, processor utilization, and inlet temperature. The sophistication of vendor estimation tools
continues to grow and provide the end user with an increasingly accurate method of predicting power and
cooling loads.
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 8
VII. TOOL LINKS
As mentioned above, actual power measurements are ideal, but many IT vendors produce power estimation
tools and calculators that can be used for planning purposes.
Cautionary note – Updates to calculators may occur without notice. While The Green Grid will attempt to keep
the reference links below as up-to-date as possible, readers should contact the supplier to verify they have
the latest version.
Company
Link
Comments
HP
TOOL:
HP also has a white paper on
http://h30099.www3.hp.com/configurator/powercalcs.asp
the tool (see second link)
WHITE PAPER:
http://search.hp.com/redirect.html?type=REG&qt=powe
r+calculator&url=http%3A//h20000.www2.hp.com/bc/
docs/support/SupportManual/c00881066/c00881066.
pdf%3Fjumpid%3Dreg_R1002_USEN&pos=1
Dell
PLANNING FOR ENERGY REQUIREMENTS:
This link includes instructions
http://www.dell.com/calc
and Data Center planning
tools
IBM
Calculator for the blades and modular product lines (x86
architecture server systems and rack storage), the tool can
be downloaded at:
http://www-03.ibm.com/systems/bladecenter/resources/
powerconfig/
For the Power Processor-based server systems, an online
tool is available at the following link:
www.ibm.com/systems/support/tools/estimator/energy
Sun Power
Tool:
Calculators
http://www.sun.com/solutions/eco_innovation/
powercalculators.jsp
Cisco Power
Tool:
Calculator
https://tools.cisco.com/cpc/authc/forms/CDClogin.
Registration required
fcc?TYPE=33619969&REALMOID=06-00071a10-621813a3-a831-83846dc90000&GUID=&SMAUTHREASON=0&
METHOD=GET&SMAGENTNAME=$SM$GDuJAbSsi7kExzQD
RfPKUItt%2bPcjKOjTGlbtk%2fRp7BdNYLiP9lyOBjXBU5PAxIX
D&TARGET=http%3A%2F%2Ftools.cisco.com%2Fcpc%2F
TABLE 2. POWER/COOLING ESTIMATION TOOLS
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 9
VIII. PROPER USE OF CALCULATING TOOLS
Remember that many of these vendor estimation tools are estimators. Because there are so many factors
that affect power draw, it is nearly impossible to exactly match a tool to the actual results that will be seen in
every data center. If more exact power or cooling planning information is needed, there is no substitute for
real-time measurement using the particular system installed in a controlled fashion and running its normal
workload and utilization levels.
The tools may be used for electrical and cooling load planning instead of nameplate or maximum measured
values, and they will yield far more accurate predictions than those derived from other estimating methods.
However, detailed attention should be paid to individual server configurations and their power systems and
potential system upgrades. For example, changes to the memory configuration or additional PCI cards over
the life of the server could affect the amps from an electrical circuit, the heat released, and the server airflow
or deltaT.
Finally, when using tool estimators for data center planning and design purposes, keep in mind that while
cooling systems can handle short increases in heat load in the room, an increase in amperage draw from a
server that is over the limit of a circuit breaker will create a problem far more quickly. Servers can temporarily
run on an increased intake air temperature, and typically a cooling system has the ability to react to that short
temperature spike and provide extra cooling. If a spike in amperage draw over the circuit breaker limit occurs,
that server and others around it may go down due to tripped breakers in the rack-level power distribution or in
a floor PDU.
Know what safety factors have been designed into your data center. Use the online tools to get good
estimates for power draw and heat load, but also measure your units once installed to verify that the tools
were correct.
IX. SUMMARY/CONCLUSION
Measuring power consumed by IT equipment provides the most accurate method for determining the
power and cooling load that IT equipment places on a data center. However, vendors provide power and
cooling calculators that can be used to determine a more accurate estimation of load based on hardware
configuration and utilization.
The Green Grid highly recommends the use of power meters and power calculators for ensuring that
equipment resource infrastructures are sized properly to accommodate the actual electrical and heat loads.
Due to planning calculator tools’ inability to reflect your specific application, the accuracy of your planning/
deployment process can be increased by factoring in your understanding of the differences between
the tools’ estimates and your own application-based measurements. If future capacity requirements are
unknown, refer to ASHRAE 90427, Datacom Equipment Power Trends and Cooling Applications (2005)2.
In summary, using nameplate values to estimate power resource requirements or space-cooling loads can
produce highly unreliable results, including substantial under-sizing (miscalculating) or over-sizing (using
the label as is without understanding hardware options and equipment utilization) of electrical systems and
cooling systems. Miscalculations also can result in loss of service or excessive cost of operations. The more
accurate the power consumption data that can be established, the more efficient and reliable the electrical
distribution systems can be. In addition, accurate data helps improve service availability and optimize
operational costs for power and cooling systems
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD
PAGE 10
X. REFERENCES
1. ASHRAE 90431, Thermal Guidelines for Data Processing Environments (2004)
https://eweb.ashrae.org/eweb/DynamicPage.aspx?Site=ASHRAE&WebKey=69c74d61-facd-4ca4-ad838063ea2de20a&listwhere=(prd_etab_ext%20LIKE%20’%25datacom%25’)
2. ASHRAE 90427, Datacom Equipment Power Trends and Cooling Applications (2005)
https://eweb.ashrae.org/eweb/DynamicPage.aspx?Site=ASHRAE&WebKey=69c74d61-facd-4ca4-ad838063ea2de20a&listwhere=(prd_etab_ext%20LIKE%20’%25datacom%25’)
3. United States Environmental Protection Agency, Energy Star for Workstations
http://www.energystar.gov/index.cfm?c=new_specs.enterprise_servers
4. Find an ENERGY STAR Compliant Computer: http://www.energystar.gov/index.cfm?fuseaction=find_a_
product.ShowProductGroup&pgw_code=CO
5. ENERGY STAR Framework Document
www.energystar.gov/ia/partners/prod_development/revisions/downloads/computer/Workstation_
Framework_Document081705.pdf
XI. ABOUT THE GREEN GRID™
The Green Grid is a non-profit trade organization of IT professionals formed to address the issues of power
and cooling in datacenters. The Green Grid seeks to define best practices for optimizing the efficient
consumption of power at the IT equipment and facility levels, as well as the manner in which cooling is
delivered at these levels. The association is funded by four levels of membership, and activities are driven
by end-user needs. The Green Grid does not endorse any vendor-specific products or solutions but seeks to
provide industry-wide recommendations on best practices, metrics, and technologies that will improve overall
data center energy efficiencies. www.thegreengrid.org
©2009 The Green Grid. All rights reserved. No part of this publication may be used, reproduced, photocopied, transmitted or stored in any retrieval system of any
nature, without the written permission of the copyright owner.
Version 1.0 Release Date: TBD