Rack, Row, and Room-Based Cooling Systems
System Design and Total Cost of Ownership
Table of Contents
1. Executive Summary...........................................................................................................................3
2. Preamble...........................................................................................................................................3
3. 3.1 3.2
3.3 3.4 3.5 3.6 Basic Setup Data.................................................................................................................................4
Site.............................................................................................................................................................. 4
System Topology / Availability...................................................................................................................... 4
Computer Room.......................................................................................................................................... 5
Air Conditioning System............................................................................................................................... 6
Energy Supply and Distribution.................................................................................................................... 7
Energy Flow – Energy Consumers................................................................................................................. 7
4. 4.1 4.2 Physical and Technical Basics..............................................................................................................8
Mass Transport of Heat Transfer Media......................................................................................................... 8
Cooling...................................................................................................................................................... 12
5. 5.1 5.2 5.3 System Design / Optimization..........................................................................................................16
Passive Cabinet Rear Door Cooling Unit...................................................................................................... 17
Row-Based Cooling Unit............................................................................................................................. 21
Conventional Room-Based Cooling............................................................................................................ 26
6. 6.1 6.2 6.3 6.4 Costing............................................................................................................................................30
Investment................................................................................................................................................. 30
Overview of Operating Data for Air Conditioning Systems.......................................................................... 31
Energy Costs.............................................................................................................................................. 32
Maintenance Costs..................................................................................................................................... 34
7. TCO – 10-Year Comparison...............................................................................................................34
Table of Figures.......................................................................................................................................... 35
Abbreviations and Acronyms...................................................................................................................... 36
List of References........................................................................................................................................ 37
2
1. Executive Summary
When building a new data center or retrofitting an existing site, data center operators can choose from several
ventilation options. This paper considers three: rack, row, and room-based cooling. These options have features
in common. The same fundamental physical laws apply to all three. Moreover, where design and operation are
concerned, these options differ only in terms of details.
What differentiates the three solutions is their individual advantages and disadvantages. To what extent these
advantages and disadvantages are relevant depends on the project under consideration.
Row-based cooling, for instance, tends to be used in smaller projects (up to 20 racks). Room-based cooling is best
suited for lower load densities, while rack-based cooling is better suited for higher loads. The break-even point above
which rack-based cooling is better suited heavily depends on a project’s basic conditions.
Generally speaking, the following basic conditions should always be implemented in a cooling system to ensure
cost-effective operation:
Temperatures should be kept at a maximum within the system
Dry or free cooling should be used*
Keep temperature difference high and mass/volume flow low throughout the system
Install complete hot/cold aisle separation within the IT room
For availability reasons, cooling systems usually include redundancy units in each subsystem. Put redundancy
units (especially fans and pumps) on and operate all units at low partial load.
The decision for the optimum ventilation system must take into account not just the total cost of ownership (TCO),
but also all project-specific boundary conditions. In the data center considered in this paper – and the three load
cases analyzed: 3, 6 and 9 kW per cabinet – row-based cooling is somewhat less suited than rack and room-based
cooling. The break-even point above which rack-based cooling is better suited stands at just over 10 kW per rack.
The optimum solution has a PUE (mech) of 1.1 for each load if basic conditions are assumed.
Since, however, the optimum solution always depends on a project’s specific circumstances, this paper is merely
intended as a guide where cooling system optimization and design are concerned.
2. Preamble
This study aims to demonstrate how three solutions for recirculating air conditioning in a data center – rack, row, and
room-based cooling – can be designed and optimized for a low TCO over the course of a 10-year operation period.
The study will be carried out at a 1,000 kW nominal load model data center in Frankfurt/Main, Germany. The infrastructure analyzed has been designed for this 1,000 kW nominal load, but in this paper the TCO is calculated for three
partial load scenarios: 30, 60 and 90 percent load.
The cooling system (pumps, heat exchangers, chillers, etc.) will be designed identically for all three scenarios (rack,
row, and room-based cooling). Thus, only the effects of the ventilation on the overall system will be demonstrated.
Only additional costs incurred by the respective recirculating air conditioning solution are considered in the investment.
Any project will have its own boundary conditions. Many parameters affect the TCO. If the boundary conditions are
changed, the TCO will thus also be different. This may lead to a different ranking and choice of ventilation design in
different circumstances.
Given this, the following statements are intended only as guidelines for design and system optimization.
* This paper compares 3 ventilation systems only, recooling is set at dry cooling; adiabatic or wet cooling would be more energy efficient
than dry, but comes with higher invest and other disadvantages.
3
3. Basic Setup Data
3.1 Site
Supply air temperature is set slightly below the upper limit of the recommended envelope for class 1 equipment
according to American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) TC9.9.
Unit
Site
Location
Frankfurt/Main
Glycol concentration in outer circuit
IT equipment
%
34
111
Sea level
AMSL
IT air temperature difference
Kelvin
13
Nominal load per rack
kW
10
Number of racks
Quantity
100
Total nominal load
kW
1,000
Air conditioning system
Maximum supply air/room temperature
°C
25
Availability
Uptime
TIER
III
Other consumers
With UPS in percent of the nominal load
%
3%
Without UPS in percent of the nominal load
%
3%
Partial loads
Partial load 1
%
30%
Partial load 2
%
60%
Partial load 3
%
90%
Figure 1: Site/basic design data
3.2 System Topology / Availability
The infrastructure systems shall be designed in accordance with the Uptime Institute TIER III category, or category
C of the BITKOM "Planning Guide for Reliable Data Centers" (BITKOM, German Federal Association for Information
Technology, Telecommunications and New Media, 2012):
Category C
MSHV
NSHV
UPS
Cold
production
UPS
UV
UV
ULKG
ULKG
STS
IT Component
IT Component
IT Room
Figure 2: Topology in accordance with Category C; source: BITKOM
3.3 Computer Room
Leaving aside the need for a ventilation system, the IT room for a 1 MW model data center at 10 kW/rack would look
as follows:
4
A total of 100 racks
Rack dimensions of 2,200H x 1,200D x 800W
Arranged in 10 rows for every 10 racks
Continuous in accordance with the hot aisle/cold aisle principle
Figure 3: Room layout for cabinet door cooling (hot/cold aisles in red/blue)
A 500 mm clearance is required above the racks for cabling.
Figure 4: Vertical section of computer room
A room with an IT area of 262 sqm and a height of 2.7 m is required.
5
3.4 Air Conditioning System
As standard, a chilled water based cooling system shall be presupposed in this WP, as illustrated in the presentation "Effiziente Kühlsysteme für Hochleistungsrechner" [Efficient Cooling Systyes For High-Performance Computers]
(Koch, 2013, p. 6).
off
IT Room
Cooling System
Cooling Water (+ Glycol)
Cooling
IT Room
Air Circulation
Computer Equipment
Figure 5: Conventional chilled water-based cooling air-conditioning system
Thermal load produced by electronic components is generally transferred into air as a first heat transport medium.
By means of air as a transport medium, heat load is transported out of the chassis, the racks, and sometimes even
out of the room and the building.
On the path out of the building, thermal load is transferred into water, which has significantly better heat transport
characteristics than air. For rack/row/room cooling, heat load is transferred from air into a liquid medium at the rack/
row/room level.
Heat transport by water is usually separated into two circuits: a secondary circuit for transport within the building
towards its perimeter, operated with pure water, and a primary circuit for transport outside the building-operated
with a frost-resistant mix of glycol and water. Both circuits are connected via a plate heat exchanger. At the end of
the heat transport path the load is discharged into the atmosphere by means of a chiller with a free cooling function.
In order to only compare the three ventilation systems, an identical cooling system for all ventilation options is assumed. This system is equipped with optimized components (EC fans, speed controlled pumps, etc.) and operated
at optimal conditions (all units “ON” under partial load, etc.). Figure 6 illustrates how the unit is equipped in the data
center example.
Air conditioning system
Unit
Value
Cooling water pumps
Approximate necessary volume flow
m3/h
Pump type
Volume flow per unit
Nominal power consumption
Number (N+1)
Installed volume flow
165
WILO IL-E65/150
m3/h
64
kW
5.46
Quantity
4
m3/h
256
Plate heat exchanger, primary and secondary circuit
Nominal load
Dimensions for nominal load
Air conditioning system
Approximate necessary volume flow
Value
Volume flow per unit
Nominal power consumption
Number (N+1)
Installed volume flow
167
WILO IL-E65/150
m3/h
64
kW
5.46
Quantity
4
m3/h
256
Chiller (Liebert HPC-M series, G model)
kW
1,150
K
2
Necessary cooling capacity
kW
Unit type
Number (N+1)
Figure 6: Units in the air-conditioning system
m3/h
Pump type
Cooling capacity per unit
6
Unit
Glycol pumps
1,250
HPC FG0 030
kW
350
Quantity
5
3.5 Energy Supply and Distribution
3.5.1 Cable Paths, Switching Systems
In accordance with standards, cable paths should be designed for a maximum voltage drop of 3 percent (based on
nominal voltage).
If a TIER topology and real partial load operation at 30, 60, and 90 percent IT load are considered, in the majority of
cases, two paths are used under basic conditions – each with approximately 15, 30 and 45 percent load. This method
reduces conduction losses to 0.9, 1.8 and 2.7 percent in both paths depending on the load. These percentage values
will be included in total energy consumption calculations for three load levels.
3.5.2 UPS Systems
The latest technologies allow a very high level of efficiency; even when operating at partial load. This depends, however, on the stability of the power grid, ranging from 95 percent (poor grid) to 98 percent (good grid) over a very
wide load range.
In this paper, a TIER III topology is used along with two power supply paths, only one of which is protected with a UPS.
Under normal conditions, only half the load will flow along the UPS path.
As stated above, UPS losses will be between 2 and 5 percent (98 / 95 percent efficiency) of the UPS load. As only half
of the load will be backed by a UPS in TIER III setup, UPS losses will be 1.5 percent compared to full load.
3.6 Energy Flow – Energy Consumers
From an energy flow standpoint, electrical energy is introduced into the building and the computer rooms, fully
converted to thermal energy in accordance with the conservation of energy principle, consequently transported back
out of the rooms and the building in various heat transfer media.
A simplified schematic of the main energy-consuming infrastructure is shown below:
Figure 7: Schematic diagram of data center infrastructure
The red arrows above indicate the flow of electrical energy arriving at the IT and Telco equipment, while the blue
paths symbolize heat that is exiting the building.
At current utility prices, utility bills represent a major share of the TCO of a data center.
Two essential factors in turn determine the energy efficiency of an air-conditioning system, which consumes the
majority of energy within the data center infrastructure:
Moving such large flows of heat transfer media away from the load (IT systems, etc.) and out of the building using
as little energy input as possible.
7
In terms of the transfer points that are always available (except for direct free cooling) from one heat transfer
medium to another, temperature loss should be kept to an absolute minimum so that cooling can occur at the
highest possible temperature when the thermal load reaches the outside of the building, consequently allowing
for the maximum possible proportion of free cooling.
Optimizing an air-conditioning system to make it more energy efficient however is nearly always in direct conflict
with reducing investment costs. The lowest possible TCO is always achieved with some sort of a compromise between energy efficiency optimization and low levels of investment.
4. Physical and Technical Basics
4.1 Mass Transport of Heat Transfer Media
4.1.1 Physical Basics
Two physical formulas apply for thermal load removal via mass transport of heat transfer media:
Mass Transport:
The energy ( ) that must be absorbed to move an ideal liquid or gaseous medium (loss-free propulsions required) is
determined from the volume flow ( ) and total pressure loss ( ) on the transport route using the following physical
formula:
Heat Transport:
In the cooling system, a heat transfer medium is used to transport a specific thermal load.
Volume flow ( ) is determined by the thermal load ( ) and temperature range (the difference between the flow and
return temperatures), where c is the specific thermal capacity of the medium, as follows:
Mass flow ( ) is related to the volume flow by the density ( ):
Mass Transport of a Heat Transfer Medium:
The two formulas above lead to the following general formula for ideal liquid and gaseous media with loss-free propulsions:
Conclusion:
Although this formula only shows the theoretical optimum, the following rules also apply to real conditions (nonideal medium, lossy propulsions, etc.):
Keep ΔT as high as possible
Keep Δp as low as possible, but in conflict with investment costs (large pipe widths, etc.)
8
4.1.2 Media Transport in the Data Center Air Conditioning System
Recirculating Air Conditioning:
In this portion of the heat transport path, air is used as the heat transfer medium. To transfer the heat load in air from
the heat sources to the first transfer point (i.e. into a liquid) fans are used, and these fans consume electrical power.
This power draw depends on the total back pressure along the flow path.
In standard air conditions (101.325 Pascal and 25°C), the formula may also be presented as follows:
At 15 / 20°C the factor changes to 0.0812 / 0.0826; this is because air density is highly temperature-dependent.
A theoretical minimum value of 0.42 percent for fan propulsion is based on typical values of a 50 Pa pressure drop
on the entire air path and a 10 Kelvin operating temperature range.
Real values for air conditioning units are higher by at least a factor of 5 (2 percent of the load for fan propulsion) due
to components and other consumers that are not loss-free (controls, etc.) in optimum operating points. In unfavorable operating points, real values increase by a factor of up to 50 (20 percent of the load for fan drive). In both cases,
the temperature range is 10 K.
Secondary/Cooling Water Circuit:
Here, a certain cooling water volume flow must be displaced in order to transport the thermal load collected out
of the computer rooms to the building envelope; generally, a plate heat exchanger is used as the transfer point to a
frost-proof external circuit. The pump's operating energy must be absorbed to achieve this.
The physical properties for water ( = 1,000 kg/m3) give the following:
or
A theoretical minimum value of 0.20 percent for the pump drive is based on a standard value of 0.5 bar pressure drop
in the circuit as a whole, and a 6 Kelvin operating temperature range.
Primary/Glycol Circuit:
A certain glycol volume flow must also be displaced in order to transport the thermal load from the building envelope
to the heat exchanger. The pump's operating energy must be absorbed to achieve this.
The proportion of glycol contained in the mixture will vary depending on climate zone and frost protection requirements. At 34 percent glycol with a media temperature of 20°C, the mixture illustrates the following characteristics
for density and thermal capacity: 1,050 kg/m3 and 3.64 kJ/kg K.
For pump capacity, this results in:
or
9
A theoretical minimum value of 0.22 percent for the pump drive is based on a standard value of 0.5 bar pressure drop
in the circuit as a whole, and a 6 Kelvin operating temperature range.
Heat Exchange:
Here, a certain air flow must again be displaced by the heat exchanger register in order to release the thermal load
from the external circuit into the atmosphere. Fan propulsion energy must again be absorbed to achieve this.
The formula for air applies again in this situation, whereby temperatures and air densities can vary enormously over
the course of the year.
A theoretical minimum value of 0.04 percent for fan propulsion is based on a standard value of 5 Pa pressure drop
on the entire air path, and a 10 Kelvin operating temperature range. Real values for air conditioning units are usually
higher due to components and other consumers that are not loss-free (controls, etc.).
On the other hand, the thermal lift of the warm exhaust air is generally used in the heat exchangers, which reduces
fan propulsion energy requirements.
Conclusion:
Theoretically – when speaking of ideal media, loss-free propulsions, etc. – around 1 percent would need to be absorbed for media transport (depending on pipe widths, cable paths, etc.) over the entire transport path of the thermal load from the load out of the building.
Real components are not loss-free and are usually operated at non-ideal operating points. Real values are several
times higher.
4.1.3 Components and Operating Conditions
Fans:
In theory, the characteristic curve of fan speed/volume flow plotted against power consumption follows a cubic (third
power) course; in reality, however, this relationship is around the power of 2.5. In the example below, from "Optimierte
Energieeffizienz durch geregelte Kaltgangeinhausungen" [Optimized energy efficiency through regulated
cold aisle enclosures] (Emerson Network Power - Racks and Solutions, 2011), the relationship between output and
volume flow at 30 percent speed is 0.2 W per m3/h and, when at a speed of 90 percent it is 0.8 W per m3/h.
Figure 8: Characteristic curve of a typical EC fan
10
Primary and Secondary Circuit Pumps:
Modern pump drives are similar in their characteristic curves to modern fans. Power consumption is non-linear
and depends, to a large extent, on volume flow. Partial load operation is much more energy efficient than full load
operation.
Figure 9: Typical pump curves (WILO Stratos GIGA 65/1-38/3,8)
Conclusion:
Modern fans and pumps should be operated at partial load as much as possible, since energy efficiency is noticeably
higher with this method of operation than in full load operation.
Partial load operation is usually possible for two reasons:
The data center infrastructure is generally designed redundantly; at least N+1, often 2 N.
Redundant fans and pumps should also be "ON" in controlled operation.
The design is based on a nominal load, which is only rarely reached in actual operation.
11
4.2 Cooling
4.2.1 Temperature Levels in the Cooling System
Generally speaking, a large part of the thermal load is produced in the data center (i.e. within the ITC components
being used, such as servers, storage, network switches, etc.) at a very high temperature that is well above 50°C.
This is the case for microprocessors and other components.
Only a small part of the load is produced in low-density components at a relatively low temperature of about 20°C.
Most current ITC components are air-cooled. A moderate exhaust air temperature is generated in the return air; this
temperature mainly depends on the technology being used.
The temperature of the supply air delivered by the air conditioning system also has a direct influence. In today's existing systems, this can be just over 10°C and is well over 20°C in the most up-to-date, energy efficiency-optimized
systems.
Currently, air conditioning systems operate on the air side within a small temperature range of 12/15°C for legacy IT
systems, and remain within a temperature range of 27/47°C for blade servers.
Figure 10: Typical temperatures in the data center air conditioning system
Following the transfer to the medium of water, temperatures in the cooling water circuit are between 5/10°C and
20/28°C.
In the primary circuit, the temperature must be 1 to 2 Kelvin lower to compensate for plate heat exchanger losses.
Depending on the climate zone and temperatures over the course of the year, certain conditions will apply for the
release of the thermal load into the atmosphere:
Dry cooling is sufficient (ambient temperature is a few Kelvin colder than cooling circuit)
Adiabatic or hybrid cooling is required (ambient temperature is similar to cooling circuit)
Mixed operation combining free and compressor-type cooling is used
Pure compressor-type cooling is required to raise the temperature level in the cooling circuit above the ambient
temperature.
From pure dry cooling to pure compressor-type cooling, consumption increases enormously, particularly the energy
input required to release the heat into the atmosphere.
12
4.2.2 Cooling via the Freecooling Chiller
Conventional chillers should be used with a free cooling function depending on the ambient temperature of the three
variants described above (adiabatic and wet cooling systems are an option, but not examined within this study).
For many cities, the ambient temperature profile is recorded in climate charts. For dry cooling systems, the "dry bulb"
temperature is relevant. For wet cooling systems, the "wet bulb" temperature is important.
The following diagrams have been created using the Emerson Network Power design software "Hirating", Version 7.8.
Figure 11: Annual "dry bulb" temperature profile, Frankfurt
Compressor operation is not necessary at ambient temperatures below a certain temperature threshold (left of the
green line in the image below) and the cooling capacity can be supplied with a high level of energy efficiency using
the pure free cooling function.
In the case of higher ambient temperatures (between the green and the red line), free cooling is no longer sufficient.
Part of the cooling capacity must be generated in compressor operation, since free cooling does not contribute
anymore past a certain ambient temperature (red line). Thus, the entire cooling capacity must be generated in compressor operation.
Figure 12: Cooling based on ambient temperature
13
Partial Load [kW]
Partial load operation 170kW load
Ambient Temperature [°C]
Figure 13: Chiller power consumption with cooling water at 20/26°C; 170 kW partial load operation
At high ambient temperatures, the power consumption of the chiller is determined by the COP compressor. Around
25–30 percent of the thermal load is required for energy usage. On cold days, the compressor is off and the chiller is
in free cooling mode consuming very little energy for the fans. In the transitional range, the chiller is in mixed mode,
which combines both free cooling and compressor cooling.
The annual average shows a distinctively higher energy efficiency than pure compressor operation. This attribute,
according to the COP compressor (or EER unit), is known as the "Seasonal Performance Factor" (SPF).
At high cooling water temperatures, real SPF values of well over 10 are reached. The inverse would mean that far less
than 10 percent of the thermal load is being consumed to increase temperature using the cooling unit/heat pump
in the chiller.
The transition between free cooling, mixed operation, and full compressor cooling mode is within a temperature
range of >350 h/K for Frankfurt*. In return, the number of hours of mixed operation and full compressor cooling
mode is reduced accordingly.
4.2.3 Components and Operating Conditions
Chiller:
The total chiller load is made up of the IT load and all other thermal loads (i.e. recirculating air conditioning systems,
power distribution, UPS, etc.) inside the building.
The design of the chiller is based on a 1,000 kW IT nominal load, plus at least 20 percent extra for losses in the UPS,
power supply lines, recirculating air conditioning system fans, lighting, and heat input from outside in the summer.
Therefore, the load is at least 1,200 kW in N+1.
FG0 030
Outdoor air temperature
Glycol
40.0°C
Unit fluid flow
59.6 m3/h
Sea level
111 m
Inlet fluid temperature
26.0°C
Refrigerant
R410A
Outlet fluid temperature
20.0°C
Unit power supply
ETHYLENE GLYCOL 35%
400 V/3 ph/50 Hz
Unit performances
Model
SPL (@ 1m, f.f, re.ISO3744) (with nominal airflow)
79.5 dB(A)
Cooling capacity
374.2 kW
PWL (re.ISO3744) (with nominal airflow)
99.5 dB(A)
Unit power input
129.1 kW
Unit OA
212 A
2.90
Unit FLA
249 A
Outlet fluid temperature
20.0°C
Unit LRA
Inlet fluid temperature
26.0°C
Width
FG0 030
Unit EER
493 A
5,750 mm
Figure 14: Liebert HPC Chiller FG0 030 data sheet
* i.e. raising the cooling water temperature by 1 Kelvin increases the number of annual hours of free cooling by approximately 350.
14
At 374 kW nominal output, 4+1 units are thus required for 1200 kW maximum load.
As such, in actual operation at 30, 60 or 90 percent IT load, the units are run under appropriate partial load.
City
Frankfurt
Unit
FG0 030
Unit fluid flow
Fluid
54.0 m3/h
ETHYLENE GLYCOL 30%
Requested thermal load
170.0 kW
Annual absorbed energy SCH
87,909 kWhe
Annual absorbed energy CH
283,398 kWhe
Annual energy saving
195,489 kWhe
Electrical saving
69%
Figure 15: Annual energy consumption of the Liebert FG0 030 at 20/26°C in partial load operation
At approximately 50 percent partial load (170 kW load), a Super Chiller will consume 88.4 MWh annually.
This is equivalent to an SPF of 16.8. The inverse shows that power consumption of the Super Chiller is around 6 percent of the load in the annual average.
This average annual power consumption, relative to the load, depends on the cooling water temperature, as shown
below:
Figure 16: Typical chiller – power consumption relative to load
15
5. System Design / Optimization
The operating conditions for three air recirculation variants and three load categories are shown in the diagram
below:
Figure 17: Schematic diagram of operating data
In the upper area of the diagram, the feeding in and supply of electrical energy (in red: output in kW) to all components and systems is shown from right to left. For all scenarios, "other loads" in front of and behind the UPS, each
using 30 kW, are taken as constant and independent of the load.
The ICT systems are shown on the bottom left side of the diagram. These fully convert the 900 kW, 600 kW or 300
kW electrical power consumed in the three partial load scenarios to thermal load.
This thermal load is then discharged through the medium of air and arrives in approximately 288,000, 192,000, or
96,000 m3/h of return air at 35°C (a supply air temperature of 25°C and a mean temperature range of 10 Kelvin for
the ICT systems are assumed) at each recirculating air conditioning unit.
At the bottom of the diagram, the heat transfer media in the conventional air conditioning system are shown. Above
this, temperatures are depicted in green (flow line/supply air below, return line/return air above); the volume flows
are shown in blue and the thermal loads in brown.
In the following sections, the boxes that represent the units each contain essential design and efficiency parameters.
16
5.1 Passive Cabinet Rear Door Cooling Unit
Product descriptions and specifications in this section can generally be found in the product brochure Knürr® DCD –
Cooling door for maximum energy efficiency: 35 kW cooling capacity (Emerson Network Power EMEA, 2013).
5.1.1 Product
Knürr DCD is an air/water heat exchanger that is integrated into the rear door of a server cabinet. The heat exchanger
is suitable for absorbing heat loads from server cabinets of up to 35 kW. It can be configured in such a way that no
heat is released from the installation area.
Figure 18: Knürr DCD rear door cooling unit
Cold air site
Housing
Steel sheet (powder-coated)
Operating temperature
10°C – 35°C (50°F – 95°F) (other temperatures up on request)
Maximum humidity
8 g/kg
Outlet air temperature (ASHARE specification)
18°C – 27°C (64.4°F – 80.6°F)
Air temperature difference
15 K – 20 K
Cold water side
Cooling capacity
35 kW
Inlet cold water temperature
12°C – 18°C (53.6°F – 64.4°F) (other temperatures up on request)
Outlet cold water temperature
18°C – 24°C (64.4°F – 75.2°F) (other temperatures up on request)
Maximum pressure
10 bar (145 psi)
Pipe connection ON/OFF
1" external thread (on edge) (DIN ISO 228 - 1)
Figure 19: Knürr DCD specification
5.1.2 Design
Recirculating Air Conditioning System:
One Knürr DCD per rack
As standard, there is no volume flow controller in the Knürr DCD. Volume flow is permanently set to the IT
nominal load, although in partial load this results in higher pressure losses in the secondary circuit.
An optional volume flow controller (e.g. per row of cabinets) would incur additional costs – a total of 50,000
EUR for every 10 rows. This additional investment would pay for itself through energy savings in the case of the
secondary circuit pumps over an extended period of time.
The cold room concept is supported and a cold temperature is maintained throughout the whole room.
17
5.1.3 Operating Data
Figure 20: Nominal data for Knürr DCD at 1.8 m3/h cooling water volume flow
Description
Total air recirculation load
Minimum volume flow
Maximum performance, cooling unit
Load
30%
60%
90%
100%
kW
300
600
900
1,000
m3/h
73,846
147,692
221,538
246,154
kW
10
10
10
10
100
100
100
100
100%
Cooling units in operation
Load in normal operation
Load per cooling unit
Air vol. flow/unit
%
30%
60%
90%
kW
3.0
6.0
9.0
10.0
m3/h
738
1,477
2,215
2,462
kW
0
0
0
0
m3/h
1.8
1.8
1.8
1.8
CW – flow line
°C
22.5
21.0
19.5
19.0
CW – return line
°C
23.9
23.9
23.8
23.8
Power consumption/unit
Cooling water vol. flow/unit
Figure 21: Operating data for passive cabinet rear door cooling unit
5.1.4 Effect on Building and Computer Room
Figure 22: Room layout for cabinet door cooling system
If assumed that the hot aisles are reduced to approximately 1,000 mm width instead of 1,200 mm, the required floor
space remains unchanged in relation to the basic scenario.
18
Figure 23: Vertical section of the floor for rack-based cooling
In preferred "back-to-back" layout (in accordance with the "cold aisle/hot aisle concept"), large air flows must be transported from the aisle behind the racks to the front of the racks without a large amount of pressure loss.
This requires more "air space" above the racks, increasing the required ceiling height by 300 mm in relation to the
basic scenario. A raised floor is not necessary here, since the pipework may be located under the ceiling.
5.1.5 Overview of Operating Data
The essential operating parameters for the three load categories are as follows:
Figure 24: Overview of operating data for rear door cooling unit at 30 percent load
19
Figure 25: Overview of operating data for rear door cooling unit at 60 percent load
Figure 26: Overview of operating data for rear door cooling unit at 90 percent load
Even at the highest of the three load categories, the PUE (mech) does not noticeably increase.
20
5.1.6 Rack-based cooling features
Benefits:
No fan propulsion energy required in the cooling unit
No raised floor or suspended ceiling required
Low height between floors
No partitions or closed intermediate ceilings required
No additional floor space required, as hot aisle width usually can be reduced from 1200 mm (basic scenario) to
app. 1000 mm (with rear door cooling units)
The cold room concept is implemented, with the hot area limited to within the racks, and the whole room representing the cold area.
Optimum PUE (mech) at high thermal load density
Disadvantages:
A large number of cooling units (one for each IT rack) and water connections are required
Relatively low cooling water temperature required
An additional unit (air handler) may be needed in order to filter the air and keep the humidity within the permitted range (costs are not considered in this paper).
5.2 Row-Based Cooling Unit
Product descriptions and specifications in this section can generally be found in the "Knürr DCD – Modular Rack
Cooling from 6 kW to 60 kW" product brochure (Emerson Network Power EMEA, 2013).
5.2.1 Product
The Knürr DCL is a rack-based cooling unit in chilled water configuration to be installed side-by-side with computing
racks. Knürr DCL can be used as a closed air loop rack cooling device (L version) or as a hybrid (H) with perforated
front doors connecting air flow to the room. This versatility is achieved by using interchangeable side panels to alter
the air flow pattern.
The hybrid and closed loop versions support the "cold room" concept. Air is drawn by fans from the rear of the Knürr
DCL through the heat exchanger, cooled down, and discharged through the front of the rack.
Knürr® DCL-H
Knürr® DCL-H 1-1 combination
*30/34 kW per DCL-H for 2,000/2,200 mm height version
Figure 27: Knürr DCL-H hybrid solution
21
DCL30
Nominal cooling capacity
Air flow
Water flow
Maximum water pressure
34 kW
5,000 m3/h (3,237 CFM)
6,000 m3/h (3,885 CFM)
4.5 m3/h (20 CFM)
5.0 m3/h (22 CFM)
10 bar (145 PSI)
10 bar (145 PSI)
5
6
5 x 170 W
6 x 170 W
300 x 1,000 (1,100) (1,200) x 2,000 [mm]
300 x 1,000 (1,100) (1,200) x 2,200 [mm]
10.72 l / 2.83 gal.
11.93 l / 3.15 gal.
Number of fans
Fans power consumption
Dimensions (W x D x H)
DCL34
30 kW
Heat exchanger internaI fluid volume
Figure 28: Knürr DCL-H specification
5.2.2 Design
Recirculating Air Conditioning System:
DCL–H in accordance with TIER III
Redundancy is maintained per row of cabinets
5.2.3 Operating Data
CR0342C
Unit inlet air temperature
41.7°C
Fluid
Unit inlet air relative humidity
24.0%
Inlet fluid temperature
20.0°C
Outlet fluid temperature
26.0°C
Unit airflow
ESP
Sea level
6,000 m3/h
0 Pa
111 m
Unit fluid flow
Unit power supply
WATER
1.02 l/s
230 V/1 ph/50 Hz
Unit performances
Unit
CR0342C
Unit power input
Net total cooling capacity
25.6 kW
Unit EER
Net sensible cooling capacity
25.6 kW
Internal filter class (EN779 std)
SHR
1.11 kW
23.10
No
1.00
Width
300 mm
Off coil air temperature
28.7°C
Depth
1,200 mm
Off coil air relative humidity
49.2%
Height
2,222 mm
72.1 dB(A)
Weight
188 kg
Fluid pressure drop coil+connections
29 kPa
Room SPL (@ 2m, f.f)
CW coils
Quantity
1 n°
Unit fluid flow
1.02 l/s
Unit fluid side pressure drop
35 kPa
Valve pressure drop
5 kPa
CW fans
Quantity
Type
Power supply
Power input
6 n°
Normal
230 V/1 ph/50 Hz
6 x 0.18 kW
Operating ampere
6 x 1.37 A
Full load ampere
6 x 1.40 A
Locked rotor amp.
6 x 0.10 A
Fan input voltage
Figure 29: Nominal data for Knürr DCL-H row-based cooling unit
22
10.0 V
Description
Total air recirculation load
Minimum volume flow
Maximum performance, cooling unit
Load
30%
60%
90%
100%
kW
300
600
900
1,000
m3/h
73,846
147,692
221,538
246,154
kW
25.6
25.6
25.6
25.6
40
50
50
50
29%
47%
70%
78%
Cooling units in operation
Load in normal operation
Load per cooling unit
Air vol. flow/unit
%
kW
7.5
12.0
18.0
20.0
m3/h
2,000
2,900
4,250
4,750
0.57
kW
0.09
0.15
0.39
m3/h
1.8
1.8
2.6
2.9
CW – flow line
°C
21.3
19.2
17.7
17.2
CW – return line
°C
25.3
25.0
23.7
23.2
Power consumption/unit
Cooling water vol. flow/unit
Figure 30: Design and operating data for row-based cooling system
At 30 percent load, many row-based cooling units (two per row of cabinets) enter standby mode, as it is not economically efficient enough to reduce fan speed, and thus cooling capacity, below a minimum rpm rate.
5.2.4 Effect on Building and Computer Room
Figure 31: Room layout for row-based cooling system
With in-row-cooling units added, as sketched in figure 31, aisle will get longer in this scenario, and the required floor
space increases from 262 to 300 sqm – 38 sqm more than in the basic scenario.
Figure 32: Vertical section of the floor for row-based cooling
The ceiling height remains unchanged for the basic scenario, as hot air is returned to the cold aisle through the rowbased cooling unit.
23
5.2.5 Overview of Operating Data
The essential operating parameters for the three load categories are as follows:
Figure 33: Overview of operating data for row-based cooling system at 30 percent load
Figure 34: Overview of operating data for row-based cooling system at 60 percent load
24
Figure 35: Overview of operating data for row-based cooling system at 90 percent load
At the highest load category, the PUE somewhat increases, as both fan speed and power consumption, and (due to
lower chilled water temperature) annual chiller energy consumption disproportionately increases.
5.2.6 Row-Based Cooling System Features
Benefits:
No raised floor or suspended ceiling required
Low height between floors
The cold room concept is implemented, with the hot area limited to within the racks, and the whole room representing the cold area.
Disadvantages:
More cooling units; redundancy maintained for each row of cabinets
Relatively low cooling water temperature required
An additional unit (air handler) may be needed in order to filter the air and keep the humidity within the permitted range (costs are not considered in this paper).
25
5.3 Conventional Room-Based Cooling
Product descriptions and specifications in this section can generally be found in the "Liebert® PCW – Cool the Cloud"
(Emerson Network Power, 2012) product brochure.
5.3.1 Product
Liebert PCW recirculating air conditioning units are ideal for data centers in the 200 kW to 4–6 MW range, that use
chilled water as a cooling fluid.
The cooling system comprises the PCW cooling unit as well as free cooling chillers, delivering the best efficiency to
ensure data center reliability and availability.
Figure 36: Liebert PCW recirculating air conditioning unit
Version
Unit
Extended Down
PH046
PH066
PH081
PH091
PH111
PH136
PH161
PH201
Gross power overall [kW]
35.8
51.9
63.7
76.9
87.2
116.2
138.4
162.9
Sensible net output [kW]
34.5
50.7
61.9
72.6
84.6
111.8
128.2
150.6
Power consumption [kW]
1.31
1.19
1.79
1.85
2.55
3.69
4.50
5.39
Sensible net EER value
26.3
42.6
34.6
39.2
33.2
30.3
28.5
27.9
11,500
16,100
20,000
19,500
28,000
30,800
34,500
40,400
Air flow [m3/h]
Water temperatures
10°C – 15°C
Return air
24°C 50% RH
Figure 37: Liebert PCW Extended Down - specification
5.3.2 Design
Recirculating Air Conditioning System designed as follows:
CRAC units according to thermal load and redundancy as well as dimensions of the climate clasp
Raised floor for cold air distribution or alternatively suspended ceiling for return air
Hot/cold separation via cold/hot aisle containment
26
5.3.3 Operating Data
The following example illustrates the PH201EL model, since it is the most economical solution that meets cooling
capacity and installation area requirements.
PH201EL – Extended Height, Downflow Down, EC Fan 2.0
Unit inlet air temperature
38.2°C
Fluid
Unit inlet air relative humidity
25.0%
Inlet fluid temperature
22.0°C
Outlet fluid temperature
28.0°C
Unit airflow
53,138 m3/h
ESP
5 Pa
Sea level
111 m
WATER
Unit fluid flow
9.19 l/s
Unit power supply
400 V/3 ph/50 Hz
Unit performances
Unit
PH201EL
Unit power input
10.43 kW
Net total cooling capacity
219.4 kW
Unit net sensible EER
Net sensible cooling capacity
219.4 kW
Internal filter class (EN779 std)
SHR
21.00
F5
1.00
Width
Supply air temperature
25.2°C
Depth
890 mm
Supply air relative humidity
52.5%
Height
1,970 mm
72.9 dB(A)
Weight
1,125 kg
Room SPL (@ 2m, f.f)
3,350 mm
CW coils
Quantity
1 n°
Unit fluid flow
9.19 l/s
Unit fluid side pressure drop
68 kPa
Fluid pressure drop coil+connections
51 kPa
Valve pressure drop
17 kPa
CW fans
Quantity
4 n°
Type
Normal
Power supply
400 V/3 ph/50 Hz
Power input
4 x 2.60 kW
Operating ampere
4 x 3.96 A
Full load ampere
4 x 5.00 A
Locked rotor ampere
4 x 0.10 A
Fan input voltage
10.0 V
Figure 38: Liebert PH201EL performance data
Description
Total air recirculation load
Minimum volume flow
Maximum performance, cooling unit
Load
30%
60%
90%
100%
kW
300
600
900
1,000
m3/h
73,846
147,692
221,538
246,154
kW
219
219
219
219
5
6
6
6
27%
46%
68%
76%
Cooling units in operation
Load in normal operation
Load per cooling unit
Air vol. flow/unit
%
kW
60.0
100.0
150.0
166.7
m3/h
15,000
24,500
36,700
40,400
kW
0.43
1.31
3.71
4.83
m3/h
13.4
14.7
22.3
24.6
CW – flow line
°C
24.5
23.5
22.8
22.6
CW – return line
°C
28.5
29.5
28.8
28.6
Power consumption/unit
Cooling water vol. flow/unit
Figure 39: Operating data for recirculating air conditioning units
Figure 39 also illustrates how a recirculating air conditioning unit can go into standby mode at 30 percent load, as it
is not economically efficient to reduce cooling capacity (and fan speed) below a certain threshold.
27
5.3.4 Effect on Building and Room Plan
Figure 40: Room layout for room-based cooling system
The floor space required for the IT equipment (racks) remains unchanged. The climate clasp adds 60 sqm facility
management (FM) space compared to the basic scenario.
Figure 41: Vertical section of the floor for room-based cooling
In the diagram above, the ceiling height has increased by 900 mm for the raised floor at this thermal load density and
room geometry, with a limit of 5 m/s for the airspeed in the raised floor.
5.3.5 Overview of Operating Data
The essential operating parameters for the three load categories are as follows:
Figure 42: Overview of operating data for room-based cooling at 30 percent load
28
Figure 43: Overview of operating data for room-based cooling at 60 percent load
Figure 44: Overview of operating data for room-based cooling at 90 percent load
5.3.6 Room-Based Cooling System Features
Benefits:
Highest chilled water temperature compared with the two alternatives (i.e. low energy usage for cooling)
No cooling water pipework is present in the computer room
Optimum PUE (mech) at low thermal load density
Disadvantages:
Raised floor required or, alternatively, a suspended ceiling
A greater height between floors
Enclosure required
Cold room concept not supported since most areas of the room are warm
29
6. Costing
6.1 Investment
Direct share and direct effects on investment costs if the recirculating air conditioning system is selected are as
follows:
Investment
Rack
Row
Room
0
0
112,700
Raised floor
EUR
Floor space
sqm
0
0
322
Enclosure
EUR
0
0
55,000
Cold aisles
Quantity
0
0
5
EUR
500,000
425,000
180,000
Recirculating air conditioning
Units
Cooling water pipework
Connections
Units
Quantity
100
50
6
EUR
50,000
35,000
21,800
Quantity
100
50
6
EUR
550,000
460,000
369,500
0
IT area
EUR
0
112,500
Floor space
sqm
0
45
0
Technical area
EUR
0
0
150,000
Floor space
sqm
0
0
60
Height between floors
EUR
39,300
0
144,900
Floor space
sqm
262
300
322
m
0.3
0.0
0.9
EUR
39,300
112,500
294,900
EUR ('000)
550
573
520
Additional height between floors
Building
Total for existing construction (without height between floors)
Figure 45: Comparison of investment for rack, row, and room-based cooling systems
Rack and row-based cooling systems have integrated hot/cold separation; enclosures are needed for room-based
cooling as well as raised flooring.
Row and room-based cooling systems require additional floor space, whilst rack and room-based cooling systems
need a greater height between floors.
30
6.2 Overview of Operating Data for Air Conditioning Systems
Under the assumed basic conditions, the following operating conditions have been determined for the three air
recirculation variants corresponding to the three partial loads:
Unit
30%
60%
90%
m3/h
73,846
147,692
221,538
Supply air temperature
°C
25
25
25
Return air temperature
°C
38
38
38
m3/h
180.0
180.0
180.0
Flow line
°C
22.5
21.0
19.5
Return line
°C
23.9
23.9
23.8
Partial load operation
Air recirculation
Total volume flow
Cooling water circuit
Volume flow
Plate heat exchanger
Load
kW
339
639
939
K
0.18
0.62
1.34
m3/h
55.7
105.0
154.4
Flow line
°C
26.0
24.8
23.3
Return line
°C
20.0
18.8
17.3
kW
378
694
1,017
Temperature loss
Glycol circuit (34%)
Volume flow
Freecooling chiller
Load
Figure 46: Overview of operating data for rear door cooling
In the case of rack-based cooling, the volume flow in the secondary circuit is assumed to be unregulated. For the
smallest load (3 kW per rack), the secondary circuit is generally designed for less than the standard 6 Kelvin, since
heat exchangers require a minimum volume flow for efficient heat transfer.
Unit
30%
60%
90%
m3/h
80,000
145,000
212,500
Supply air temperature
°C
25
25
25
Return air temperature
°C
38
38
38
129.6
Partial load operation
Air recirculation
Total volume flow
Cooling water circuit
Volume flow
m3/h
72.0
90.0
Flow line
°C
21.3
19.2
17.7
Return line
°C
25.3
25.0
23.7
Plate heat exchanger
Total load
kW
335
639
953
K
0.16
0.62
1.38
m3/h
55.0
105.1
156.7
Flow line
°C
26.1
24.5
22.3
Return line
°C
20.1
18.5
16.3
kW
373
694
1,033
Temperature loss
Glycol circuit (34%)
Volume flow
Freecooling chiller
Load
Figure 47: Overview of operating data for row cooling
31
Unit
30%
60%
90%
m3/h
75,000
147,000
220,200
Supply air temperature
°C
25
25
25
Return air temperature
°C
38
38
38
133.7
Partial load operation
Air recirculation
Total volume flow
Cooling water circuit
Volume flow
m3/h
66.8
87.9
Flow line
°C
24.5
23.5
22.8
Return line
°C
28.5
29.5
28.8
Plate heat exchanger
Total load
kW
333
639
957
K
0.16
0.62
1.38
m3/h
54.7
105.1
157.2
Flow line
°C
29.3
28.9
27.4
Return line
°C
23.3
22.9
21.4
kW
372
694
1,035
Temperature loss
Glycol circuit (34%)
Volume flow
Freecooling chiller
Load
Figure 48: Overview of operating data for room-based cooling
6.3 Energy Costs
The annual energy costs for the site under consideration with regards to the three load categories are determined
using an assumed energy price of 125 EUR per MWh (12.5 cents/kWh), as follows:
EUR per MWh
125
Average power consumption
Partial load operation
Unit
30%
60%
90%
Air conditioning system
kW
0.0
0.0
0.0
Quantity
100
100
100
Consumption per unit
kW
0.00
0.00
0.00
Cooling water pumps
kW
9.1
9.1
9.1
Quantity
4
4
4
kW
2.26
2.26
2.26
6.2
Air recirculation units
Number of units
Number of units
Consumption per unit
kW
0.5
2.4
Quantity
4
4
4
Consumption per unit
kW
0.12
0.59
1.54
Cooling unit
kW
21.9
46.4
79.0
%
5.8%
6.7%
7.8%
kW
31.5
57.8
94.3
Glycol pumps
Number of units
Average power consumption (1/JAZ)
Air conditioning system
Power supply
UPS losses
kW
5.1
9.6
14.2
Power distribution losses
kW
3.3
12.5
28.1
Power supply
kW
8.4
22.2
42.3
Total power consumption
kW
39.8
79.9
136.5
EUR ('000)
43.6
87.5
149.5
Annual energy costs
Figure 49: Power consumption and energy costs for rear door cooling
32
EUR per MWh
125
Average power consumption
Partial load operation
Unit
30%
60%
90%
19.5
Air conditioning system
Air recirculation units
Number of units
Consumption per unit
Cooling water pumps
Number of units
Consumption per unit
kW
3.6
7.5
Quantity
40
50
50
kW
0.09
0.15
0.39
4.0
kW
0.9
1.6
Quantity
4
4
4
kW
0.23
0.40
1.00
6.4
kW
0.5
2.4
Quantity
4
4
4
Consumption per unit
kW
0.12
0.59
1.60
Cooling unit
kW
21.4
48.0
87.6
%
5.7%
6.9%
8.5%
kW
26.4
59.5
117.5
Glycol pumps
Number of units
Average power consumption (1/JAZ)
Air conditioning system
Power supply
UPS losses
kW
5.0
9.6
14.4
Power distribution losses
kW
3.2
12.6
28.7
Power supply
kW
8.3
22.2
43.1
Total power consumption
kW
34.6
81.7
160.6
EUR ('000)
37.9
89.4
175.8
Annual energy costs
Figure 50: Power consumption and energy costs for row cooling
EUR per MWh
125
Average power consumption
Partial load operation
Unit
30%
60%
90%
22.3
Air conditioning system
Air recirculation units
Number of units
Consumption per unit
Cooling water pumps
Number of units
Consumption per unit
kW
2.2
7.9
Quantity
5
6
6
kW
0.43
1.31
3.71
4.3
kW
0.8
1.5
Quantity
4
4
4
kW
0.19
0.38
1.08
6.5
kW
0.5
2.4
Quantity
4
4
4
Consumption per unit
kW
0.12
0.59
1.61
Cooling unit
kW
12.7
26.0
49.7
%
3.4%
3.8%
4.8%
kW
16.1
37.7
82.7
Glycol pumps
Number of units
Average power consumption (1/JAZ)
Air conditioning system
Power supply
UPS losses
kW
5.0
9.6
14.4
Power distribution losses
kW
3.2
12.2
27.7
Power supply
kW
8.2
21.8
42.2
Total power consumption
kW
24.2
59.6
124.9
EUR ('000)
26.5
65.2
136.8
Annual energy costs
Figure 51: Power consumption and energy costs for room-based cooling
In the case of smaller thermal loads, room-based cooling is best. However, for higher loads, rack-based cooling is
most efficient.
For subsequent consideration of TCO, the cost of energy is assumed to be constant; yet, in practical scenarios, particularly in Germany, an annual price increase would need to be factored in to obtain more precise figures.
33
6.4 Maintenance Costs
The expected maintenance costs over a period of 10 years, including precautionary replacement of wearing parts
(fans, filter mats, etc.), if available, are approximately as follows:
Maintenance
Number of units
Total
Rack
Row
Qty.
100
50
Room
6
EUR ('000)
100
350
150
Figure 52: Maintenance costs for recirculation air conditioning system
The passive cabinet rear door cooling units do not have fans or motor-driven valves. This simplifies maintenance
requirements considerably since wearing parts do not need to be replaced. Overall, despite the fact that there are a
larger number of units, maintenance costs are significantly reduced.
7. TCO – 10-Year Comparison
As previously stated in the Preamble, this study considers only the additional investment costs incurred as a direct
result of procuring the recirculating air conditioning solution.
This comparison is thus based only on a partial TCO, comprised of the sum of these additional costs, including energy
and maintenance costs over a 10-year period.
All air recirculation solutions would be housed in the same building envelope. Additional costs, due to a greater
height between floors, would thus not be incurred.
TCO
Rear door
Unit
30%
60%
Invest
EUR ('000)
550
550
550
Energy costs over 10 years
EUR ('000)
436
875
1,495
Partial load operation
90%
Maintenance costs over 10 years
EUR ('000)
100
100
100
TCO over 10 years
EUR ('000)
1,086
1,525
2,145
Unit
30%
60%
Invest
EUR ('000)
573
573
573
Energy costs over 10 years
EUR ('000)
379
894
1,758
TCO
Partial load operation
Rows
90%
Maintenance costs over 10 years
EUR ('000)
310
310
310
TCO over 10 years
EUR ('000)
1,262
1,777
2,641
Unit
30%
60%
Invest
EUR ('000)
520
520
520
Energy costs over 10 years
EUR ('000)
265
652
1,368
TCO
Partial load operation
Room
90%
Maintenance costs over 10 years
EUR ('000)
150
150
150
TCO over 10 years
EUR ('000)
935
1,322
2,037
Figure 53: TCO comparison for rack, row, and room-based cooling with 30/60/90 percent load
In this example, the break-even point between rear door and room-based cooling is just over 10 kW per rack.
Row cooling is more expensive in terms of overall costs and tends to be used for smaller projects (up to approximately
20 racks).
In a particular project with specific boundary conditions, the decision to use a certain ventilation system may not be
taken based only on the “lowest TCO” criterion, but considering all project-specific boundary conditions.
34
Table of Figures
Figure 1: Figure 2: Figure 3: Figure 4: Figure 5: Figure 6: Figure 7: Figure 8: Figure 9: Figure 10: Figure 11: Figure 12: Figure 13: Figure 14: Figure 15: Figure 16: Figure 17: Figure 18: Figure 19: Figure 20: Figure 21: Figure 22: Figure 23: Figure 24: Figure 25: Figure 26: Figure 27: Figure 28: Figure 29: Figure 30: Figure 31: Figure 32: Figure 33: Figure 34: Figure 35: Figure 36: Figure 37: Figure 38: Figure 39: Figure 40: Figure 41: Figure 42: Figure 43: Figure 44: Figure 45: Figure 46: Figure 47: Figure 48: Figure 49: Figure 50: Figure 51: Figure 52: Figure 53: Site/basic design data.............................................................................................................................. 4
Topology in accordance with Category C; source: BITKOM............................................................. 4
Room layout for cabinet door cooling (hot/cold aisles in red/blue).......................................... 5
Vertical section of computer room.................................................................................................... 5
Conventional chilled water-based cooling air conditioning system.......................................... 6
Units in the air conditioning system.................................................................................................... 6
Schematic diagram of data center infrastructure.......................................................................... 7
Characteristic curve of a typical EC fan........................................................................................... 10
Typical pump curves (WILO Stratos GIGA 65/1-38/3,8)....................................................................... 11
Typical temperatures in the data center air conditioning system.............................................. 12
Annual "dry bulb" temperature profile, Frankfurt.......................................................................... 13
Cooling based on ambient temperature............................................................................................ 13
Chiller power consumption with cooling water at 20/26°C; 170 kW partial load operation.... 14
Liebert HPC Chiller FG0 030 data sheet................................................................................................ 14
Annual energy consumption for the Liebert FG0 030 at 20/26°C in partial load operation..... 15
Typical chiller – power consumption relative to load.................................................................. 15
Schematic diagram of operating data............................................................................................... 16
Knürr DCD rear door cooling unit.................................................................................................... 17
Knürr DCD specification........................................................................................................................ 17
Nominal data for Knürr DCD at 1.8 m3/h cooling water volume flow........................................ 18
Operating data for passive cabinet rear door cooling unit......................................................... 18
Room layout for cabinet door cooling system.............................................................................. 18
Vertical section of the floor for rack-based cooling.................................................................. 19
Overview of operating data for rear door cooling unit at 30 percent load............................ 19
Overview of operating data for rear door cooling unit at 60 percent load............................ 20
Overview of operating data for rear door cooling unit at 90 percent load............................ 20
Knürr DCL-H hybrid solution............................................................................................................... 21
Knürr DCL-H specification..................................................................................................................... 22
Nominal data for Knürr DCL-H row-based cooling unit................................................................ 22
Design and operating data for row-based cooling system.......................................................... 23
Room layout for row-based cooling system................................................................................... 23
Vertical section of the floor for row-based cooling................................................................... 23
Overview of operating data for row-based cooling system at 30 percent load...................... 24
Overview of operating data for row-based cooling system at 60 percent load...................... 24
Overview of operating data for row-based cooling system at 90 percent load...................... 25
Liebert PCW recirculating air conditioning unit............................................................................. 26
Liebert PCW Extended Down - specification....................................................................................... 26
Liebert PH201EL performance data...................................................................................................... 27
Operating data for recirculating air conditioning units............................................................. 27
Room layout for room-based cooling system................................................................................. 28
Vertical section of the floor for room-based cooling................................................................ 28
Overview of operating data for room-based cooling at 30 percent load................................. 28
Overview of operating data for room-based cooling at 60 percent load................................. 29
Overview of operating data for room-based cooling at 90 percent load................................. 29
Comparison of investment for rack, row, and room-based cooling systems.......................... 30
Overview of operating data for rear door cooling...................................................................... 31
Overview of operating data for row cooling................................................................................. 31
Overview of operating data for room-based cooling................................................................... 32
Power consumption and energy costs for rear door cooling .................................................. 32
Power consumption and energy costs for row cooling ............................................................. 33
Power consumption and energy costs for room-based cooling .............................................. 33
Maintenance costs for recirculation air conditioning system.................................................. 34
TCO comparison for rack, row, and room-based cooling with 30/60/90 percent load........... 34
35
Abbreviations and Acronyms
COP
Coefficient of performance
CRAC
Computer room air conditioning unit
CW
Cooling water
EC fan
Electronically commutated fan
EER
Energy efficiency ratio
EUREuro
FM
Facility management
ICT
Information and communication technology
IT Information technology
PUE
Power usage effectiveness
RACU
Recirculating air conditioning unit
rpm
Revolutions per minute
SPF
Seasonal performance factor
TCO
Total cost of ownership
UPS
Uninterrupted power supply
36
List of References
BITKOM - German Federal Association for Information Technology, Telecommunications and New Media
(17 April 2012). Reliable Data Centers planning aid. Berlin-Mitte, Berlin, Germany.
Emerson Network Power – Racks and Solutions (2011).
Optimierte Energieeffizienz durch geregelte Kaltgangeinhausungen. [Optimized energy efficiency through
regulated cold aisle enclosures.] Arnstorf, Bavaria, Germany.
Emerson Network Power (October 2012).
Liebert PCW – Cool the Cloud. Piove di Sacco, Italy.
Emerson Network Power EMEA (November 2013).
Knürr DCD – Kühltür für höchste Energieeffizienz: 35kW Kühlleistung. [Cooling door for optimum energy
efficiency: 35 kW cooling capacity.] Arnstorf, Bavaria, Germany.
Emerson Network Power EMEA (November 2013).
Knürr DCL – Modulare Rack-Kühlung von 6 bis 60kW. [Modular Rack Cooling from 6kW to 60 kW.] Arnstorf,
Bavaria, Germany.
Koch, D. P. (22 October 2013).
Effiziente Kühlsysteme für Hochleistungsrechner. [Efficient cooling systems for high-performance computers.]
Arnstorf, Bavaria, Germany.
37
Ensuring The High Availability
Of Mission-Critical Data And Applications.
About Emerson Network Power
Emerson Network Power, a business of Emerson (NYSE:EMR), delivers
software, hardware, and services that maximize availability, capacity,
and efficiency for data centers, health care, and industrial facilities.
A trusted industry leader in smart infrastructure technologies,
Emerson Network Power provides innovative data center infrastructure
management solutions that bridge the gap between IT and facility
management and deliver efficiency and uncompromised availability
regardless of capacity demands. Our solutions are supported globally
by local Emerson Network Power service technicians.
Learn more about Emerson Network Power products and services at
www.EmersonNetworkPower.eu
Locations
Emerson Network Power
Global Headquarters
1050 Dearborn Drive
P.O. Box 29186
Columbus, OH 43229, USA
T +1 614 8880246
Emerson Network Power
Racks and Solutions
Europe Middle East And Africa
Mariakirchener Straße 38
94424 Arnstorf
Germany
T +49 8723 27 0
F +49 8723 27 154
knuerr@emerson.com
Emerson Network Power
United Kingdom
George Curl Way
Southampton
SO18 2RY, UK
T +44 (0)23 806
1 0311
F +44(0)23 8061 0852
While every precaution has been taken to ensure accuracy and completeness in this
literature, Emerson assumes no responsibility for content, and disclaims all liability
for damages resulting from use of this information or for any errors or omissions.
Technical data are subject to change without notice.
1101.150.126
MKA4L0ENWP Rev.1.03-06/2014
Globe Park
Fourth Avenue
Marlow Bucks, SL7 1YG
Tel: +44 1628 403200
Fax: +44 1628 403203
UK.Enquiries@Emerson.com
Follow us on Social Media:
EmersonNetworkPower.eu
Emerson. Consider it Solved, Knürr®, Liebert®, Emerson Network Power and the Emerson Network Power logo are trademarks or service marks of Emerson Electric Co. or its affiliated companies.
©2014 Emerson Electric Co. All rights reserved.