Project Milestone Meeting
City of Gresham, Oregon
May 30, 2014
IT AIRE DATA CENTER COOLING
SYSTEM EVALUATION &
RESULTS
Babak Lajevardi, Ph.D. Candidate
Joseph F. Junker, P.E.
Karl R. Haapala, Ph.D.
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Project Objective and Timeline
2


Objective: To compare the energy efficiency and
performance of City of Gresham data center before
and post installation of IT Aire cooling system.
Timeline
 Summer
2013: Monitoring system implementation
 Fall 2013-Spring 2014: Monitoring of cooling systems
 Winter 2014-Spring 2014: Analysis of results
 Spring 2014: Technical article and white paper
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Data Center Energy Consumption
3

Worldwide data center energy use [EPA, 2010]
 About
240 billion kWh annually
 Roughly 1.3% of the world total

U.S. data center energy use [EPA, 2010]
 About
80 billion kWh annually
 Roughly 2% of the U.S. total
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Data Center Energy Consumption
4
Total energy use (kWh)
300
US Datacenters
250
Wordwide
Datacenters
200
150
100
50
0
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
2001
2000
1999
Year
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Strategies for Data Center Energy Reduction
5

Technology development and implementation
 Optimized
rack layout
 Hot aisle-cold aisle containment
 New cooling system equipment solutions

Energy and thermal measurement and monitoring
 Allows
evaluation of energy efficiency performance
 Gives insight to thermal management inefficiencies
 Gives insight into strategic investment for cost reduction
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Data Center Air Flow
6
Partition
Q, T, RH (return)
Q, T, RH (exit)
Hot Aisle
IT Rack
Q, T, RH (intake)
Cold Aisle
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Q, T, RH
(supply)
Data Center Monitoring
7

Gresham City Hall
wireless monitoring
network



AC unit power
Outdoor air
AC unit
Rack
Node
Dry bulb temperature
Relative humidity (RH)
Current transducers
Roof
Receiver
Cold aisle
Supply air/Rack intake air

Data loggers


Record with interval
of one minute
Save a copy to an OSU
FTP address every 24
hours
Rack power
Rack power
Rack exit air
Energy Efficiency Center &
Return air
Data room
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Hot aisle
Outdoor Temperature
Old System
Week avg. 41.55°F
Week st. dev. 3.86°F
60
55
50
45
40
35
30
25
20
IT Aire System
Week avg. 42.07°F
Week st. dev. 4.61°F
Saturday
Friday
Thursday
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Wednesday
Tuesday
Monday
Sunday
Saturday
Friday
Thursday
Wednesday
Tuesday
Monday
Sunday
Outdoor Temperature (°F)
8
Outdoor Relative Humidity
IT Aires System
Week avg. 77.58%
Week st. dev. 8.45%
Old System
Week avg. 78.21%
Week st. dev. 7.42%
100
80
60
40
20
0
Saturday
Friday
Thursday
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Wednesday
Tuesday
Monday
Sunday
Saturday
Friday
Thursday
Wednesday
Tuesday
Monday
Sunday
Outdoor Relative Humidity (%)
9
Current Data Center Efficiency Metrics
10

Power Usage Effectiveness (PUE) [The Green Grid, 2007]
Pinf. +PIT
PUE=
PIT
3
Uptime Institute
PUE
2.5
EPA
2
Digital Reality Trust
1.5
Lawrence Berkeley
National Laboratory
Trendline
Linear
(Series5)
1
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004
2003
2002
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Power Use Effectiveness (PUE)
11
Old System
Week avg. 1.25
Week st. dev. 0.06
1.5
IT Aire System
Week avg. 1.05
Week st. dev. 0.03
1.4
PUE
1.3
1.26
1.25
1.2
1.26
1.25
1.24
1.26
1.24
1.1
1.07 1.07
1.02 1.05
1.01
1.06
1
Saturday
Friday
Thursday
Wednesday
Tuesday
Monday
Sunday
Saturday
Friday
Thursday
Wednesday
Tuesday
Monday
Sunday
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
1.06
PUE as a Function of Wet Bulb Temperature
12
Old system
1.50
ITAire
PUE
1.40
1.39
1.30
1.25
1.26
1.25
1.20
1.29
1.33
1.24
1.14
1.10
1.05
1.00
1.05
1.05
1.34
1.17
1.19
1.08
1.01
31-35 36-40 41-45 46-50 51-55 56-60 61-65 66-70
Wet Bulb Temperature (°F)
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
PUE as a Function of Wet Bulb Temperature
Humidity Ratio (Pounds of
moisture/Pounds of dry air)
13
IT Aire: 1.1-1.2
Old System: 1.3-1.4
IT Aire: 1.0-1.1
Old System: 1.2-1.3
ASHRAE
Envelope
Dry Bulb Temperature °F
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Current Data Center Efficiency Metrics
14

Rack Cooling Index (RCI) [M. K. Herrlin, 2005]
(
)
(
)
é å Tintake -Tmax-rec
ù
Tintake >Tmax-rec
ú ´100%
RCI HI = ê1(Tmax-all -Tmax-rec )n
ê
ú
ë
û
é å Tmin-rec -Tintake
ù
Tintake <Tmax-rec
ú ´ 100%
RCI LO = ê1(Tmin-rec -Tmin-all )n
ê
ú
ë
û

ASHRAE Guideline [ASHRAE, 2011]
 Recommended:
 Allowable:
64°F < Tintake< 77°F
59°F < Tintake< 90°F
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Rack Cooling Index (ASHRAE Guideline)
15
Intake Temperature (°F)
100
ITAire System
Old System
Allowable
90
80
Recommended
70
60
Old System
RCIHI=100%
RCILO=92.5%
50
IT Aire
RCIHI=100%
RCILO=100%
40
0
2000
4000
6000
Rack Intake Number
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
8000
10000
Data Center Air Flow
16
Air recirculation
Q, T, RH
(supply)
Q, T, RH (return)
Q, T, RH (exit)
Hot Aisle
Q, T, RH (intake)
Air bypass
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Cold Aisle
Current Data Center Efficiency Metrics
17

Return Temperature Index (RTI) [Herrlin, 2007]
(
é T
-TSupply
Return
RTI= ê
DTRack
ê
ë

) ùú
ú
û
Supply and Return Heat Indices (SHI, RHI) [Sharma and
Bash, 2002]
SHI=
Tintake -TSupply
Texit -TSupply
RHI=
Treturn -TSupply
Texit -TSupply
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Return Temperature Index (RTI)
18
120%
Old System
Week avg. dev. 63%
IT Aire System
Week avg. dev. 35%
100%
Saturday
Friday
Thursday
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Wednesday
Tuesday
Monday
Sunday
Saturday
Friday
Thursday
Wednesday
Tuesday
80%
Monday
90%
Sunday
RTI
110%
Supply Heat Index (SHI)
19
1
Old System
IT Aire
SHI
0.8
0.6
0.4
0.2
Lower SHI, Less recirculation
0
Saturday
Friday
Thursday
Wednesday
Tuesday
Monday
Sunday
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Return Heat Index (RHI)
20
Higher RHI, Less by pass air
1
RHI
0.8
0.6
Old System
0.4
IT Aire
0.2
0
Saturday
Friday
Thursday
Wednesday
Tuesday
Monday
Sunday
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Cooling Energy (IT Aire vs. Old System)
21
W
e
e
k
100
91.5
Energy (kWh)
80
86.6
85.4
85.2
a
v
g
.
60
40
IT Aire
22.3
20
0
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Old
20.0
Saturday
4
23.2
Friday
Thursday
Average energy saving of 67.2 kWh
Wednesday
Monday
Sunday
s
t
d
e
v
.
23.6
22.9
20.6
18.6
Tuesday
4
1
.
5
5
°
F
W
e
e
k
93.3
92.0
87.8
Cooling Energy Reduction (IT Aire vs. Old System)
100%
80%
60%
40%
20%
0%
W
e
e
k
a
v
g
.
78.6%
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
Saturday
4
Friday
Thursday
Average energy reduction of 75.7%
74.8%
73.1%
Wednesday
Tuesday
s
t
d
e
v
.
Monday
4
1
.
5
5
°
F
W
e
e
k
78.6%
73.3%
75.9%
75.6%
Sunday
Cooling System Energy Reduction
22
Next Steps
23

Continued system monitoring (Until Spring 15)
Elucidate seasonal variation effects
 Large data repository for higher resolution of PUE prediction
 Enables statistical models for efficiency improvements
 Provides in-depth insights into performance


Upcoming milestones
Presentation at IIE ISERC, May 31-June 4, 2014
 Presentation at SME NAMRC, June 9-13, 2014
 Whitepaper, June 30, 2014 (draft mid-June for review)

Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
PUE as a Function of Wet Bulb Temperature
Humidity Ratio (Pounds of
moisture/Pounds of dry air)
24
IT Aire: 1.1-1.2
Old System: 1.3-1.4
IT Aire: 1.0-1.1
Old System: 1.2-1.3
ASHRAE
Envelope
Dry Bulb Temperature °F
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
25
Acknowledgments
We gratefully acknowledge the funding support
of this project from Portland Development
Commission (PDC), Oregon Best, and Energy
Trust through Oregon Best Commercialization
Program.
We wish to express our appreciation from the
assistance we received from the City of Gresham
and IT Aire.
References
26










R. Sharma, C. Bash, and C. Patel, “Dimensionless Parameters for Evaluation of Thermal Design and Performance of Large-scale Data
Centers,” in 8th AIAA/ASME Joint Thermophysics and Heat Transfer Conference, 2002, pp. 1–11.
R. Brown, “United States Environmental Protection Agency Energy Star Program, Report to congress on server and data center
energy efficiency, public law, 109-431,” 2008.
Y. Joshi and P. Kumar, Energy efficient thermal management of data centers. Springer, 2012.
“The green grids opportunity: decreasing data center and other IT energy usage patterns,” The Green Grids, Technical Report,
2007.
L. Stahl and C. Belady, “Designing an alternative to conventional room cooling,” presented at the IEE conference publication, 2001,
pp. 109–115.
B. Kenneth, “Heat Density Trends in Data Processing, Computer Systems, and Telecommunications Equipment,” UpTime Institute, White
paper, 2005.
C. Bash, C. D. Patel, and R. K. Sharma, “Dynamic thermal management of air cooled data centers,” presented at the Thermal and
Thermomechanical Phenomena in Electronics Systems, 2006, pp. 445–452.
G. Koutitas and P. Demestichas, “Challenges for Energy Efficiency in Local and Regional Data Centers,” Journal of Green
Engineering, vol. 1, p. 32.
W. Tschudi, E. Mills, S. Greenberg, and P. Rumsey, “Measuring and Managing-Data Center Energy Use,” Ernest Orlando Lawrence
Berkeley NationalLaboratory, Berkeley, CA (US), 2005.
S. Law, “Gresham startup says it can slash energy costs at power-thirsty server farms,” Portland Tribune, 2013.
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering
References
27











V. Avelar, Azevedo, and A. French, “PUETM: A comprehensive examination of the metric,” The Green Grid, White paper 49, 2012.
J. Bruschi, P. Rumsey, R. Anliker, L. Chu, and S. Gregson, “Best practices guide for energy-efficient data center design,” U.S.
Department of Energy, Energy Efficiency & Renewable Energy, Federal Energy Management Program, [Washington, DC], 2010.
M. K. Herrlin, “Rack cooling effectiveness in data centers and telecom central offices: The rack cooling index (RCI),” TransactionsAmerican Society of Heating Refrigerating and Air conditioning Engineers, vol. 111, p. 725, 2005.
T. ASHRAE, “9.9 (2011) Thermal guidelines for data processing environments–expanded data center classes and usage guidance,”
Whitepaper prepared by ASHRAE technical committee (TC), vol. 9, 2011.
M. K. Herrlin, “Improved data center energy efficiency and thermal performance by advanced airflow analysis,” presented at the
Digital Power Forum, 2007, pp. 10–12.
M. K. Herrlin, “Airflow and cooling performance of data centers: two performance metrics,” ASHRAE transactions, vol. 114, no. Part
2, 2008.
M. K. Patterson, “Energy efficiency metrics,” in Energy Efficient Thermal Management of Data Centers, Springer, 2012, pp. 237–
271.
A. Vijayaraghavan and D. Dornfeld, “Automated energy monitoring of machine tools,” CIRP Annals-Manufacturing Technology, vol.
59, no. 1, pp. 21–24, 2010.
J. W. Gardner and V. K. Varadan, Microsensors, MEMS and smart devices. John Wiley & Sons, Inc., 2001.
C.-Y. Chong and S. P. Kumar, “Sensor networks: evolution, opportunities, and challenges,” Proceedings of the IEEE, vol. 91, no. 8, pp.
1247–1256, 2003.
J. Yick, B. Mukherjee, and D. Ghosal, “Wireless sensor network survey,” Computer networks, vol. 52, no. 12, pp. 2292–2330, 2008.
Energy Efficiency Center &
Industrial Sustainability Laboratory
School of Mechanical, Industrial, and Manufacturing Engineering