TOWARD A GREENER HORIZON:
P R O P O S E D E N E R G Y S AV I N G C H A N G E S
T O M F C F D ATA C E N T E R S
Naji Alamrony
nalamron@uwaterloo.ca
watitis.uwaterloo.ca
@watitisconf
#watitis2015
AGENDA
• Objectives and Motivation
• Energy and Peak power reduction
Reducing IT load
Reducing Data Center energy waste
• Current State of MFCF Data Centers
• Proposed actions
• Conclusion
#watitis2015
OBJECTIVES
• Propose changes to current data centers
Reduce energy consumption
• Minimal implementation cost
• Significant rewards
• Getting UW support/involvement
#watitis2015
PEAK POWER VS ENERGY
• Energy determines
Size of energy bill
Battery life
• Peak power determines
Capacity
• Electrical
• Cooling
#watitis2015
M O T I VAT I O N
• Monetary
Energy bills
• Environmental
greenhouse gases emission
• Capacity
Efficient use of the datacenter building
Cost of building new data center
#watitis2015
E N E R G Y C O N S U M P T I O N C O M PA R I S O N
number of servers
Electricity consumption share
(billion kWh)
7% 1%
1%
3.3,1,4%
22%
40%
14.1, 19%
37.5, 49%
20.5, 27%
30%
Small and Medium Server Rooms
Enterprise/Corporate Data Centers
Small and Medium Server Rooms
Enterprise/Corporate Data Centers
Multi-Tenant Data Centers
Hyper-Scale Cloud Computing
Multi-Tenant Data Centers
Hyper-Scale Cloud Computing
High-Performance Computing
Ref: Data Center Efficiency Assessment, NRDC, August 2014
#watitis2015
High-Performance Computing
MFCF ENERGY CONSUMPTION
Estimated daily kWh of MFCF
Data centers
• Estimated Energy
cost from $10,000
to $15,000/Month
4500
4000
kWh
3500
3000
2500
2000
1500
1000
500
0
start
IT
#watitis2015
goal
DataCenter
RESTRICTIONS
•
•
•
•
•
Cost  start cheap but effective
High availability
Performance
Virtualizability
Capacity (Data center)
Cooling
Electricity
Spatial  using existing data center
#watitis2015
REDUCING PEAK POWER/ENERGY
• Reduce total load
•
•
Reduce IT load
Reduce Data Center load
•
•
Cooling
Electrical
• Increase data center efficiency
•
Reduce PUE
𝑇𝑜𝑡𝑎𝑙 𝑙𝑜𝑎𝑑
𝑃𝑈𝐸 =
𝐼𝑇 𝐿𝑜𝑎𝑑
#watitis2015
BACKGROUND – IT LOAD
P O W E R , E F F I C I E N C Y V S . U T I L I Z AT I O N
Reference: The case for energy-proportional computing, Barroso, 2007
#watitis2015
BACKGROUND – IT LOAD
Efficiency of Power supplies
100%
Efficiency (%)
95%
90%
85%
80%
75%
NeoPower PSU
70%
Corsair PSU
65%
Dell 80plus
60%
55%
50%
0%
20%
40%
60%
80%
100%
Loading (% of rated output power)
#watitis2015
120%
BACKGROUND - IT
M A C H I N E S ’ U T I L I Z AT I O N – 1 D AY
#watitis2015
BACKGROUND - IT
M A C H I N E S ’ U T I L I Z AT I O N – 2 W E E K S
#watitis2015
REDUCING IT LOAD
• New machines:
Use properly sized machines
•
•
•
•
RAM
CPU
Storage (SSD vs. HDD)
Power supply
Use efficient class machines
• Blade server
• Micro server
#watitis2015
R E D U C I N G I T L O A D ( C O N T. )
• When using blades consider
cooling capacities (up to 30kW/rack)
SSD drives is recommended
More blades on a chassis
more efficient
#watitis2015
R E D U C I N G I T L O A D ( C O N T. )
• Improve utilization
External power management policies
• ON/OFF policy
• Suspend polices
Internal power policies
• Enable c-states (sleep modes)
• Enable p-states (DFVS)
Virtualization
• Provides elasticity for cluster
• Consolidate light weight services
#watitis2015
I N C R E A S E D ATA C E N T E R E F F I C I E N C Y
REDUCING COOLING LOAD
• Measured air flow
Input (vertical) = output (horizontal)
• Tight hot/cold aisle containment
Open aisle (currently used)
Cold aisle containment (better)
Hot aisle containment (recommended)
• Higher cold aisle temperature
ASHRAE recommends up to 27 ℃
#watitis2015
H O T A I S L E C O N TA I N M E N T
#watitis2015
#watitis2015
ASHRAE PSYCHROMETRIC CHART
#watitis2015
S A F E T E M P E R AT U R E L I M I T S
Device
CPU
Memory
GPU
Hard drive
I/O devices
#watitis2015
Temperature Specs ℃
60 - 70
85 - 105
60 - 80
50 - 60
50 - 60
FREE COOLING
• Use of free cooling
Air-side economizer
Water-side economizer
#watitis2015
C O O L I N G D E G R E E - D AY
• Cooling degree-days (CDD)
A measure of how much (in degrees), and for how long (in
days), the outside air temperature was above a certain level
(base temperature)
• CDD is proportional to cooling energy
• Waterloo CDDs
Base temp
18℃
23℃
27℃
CDD
352
94
13
From http://www.degreedays.net/
#watitis2015
P O W E R D I S T R I B U T I O N WA S T E
• Require infrastructure change:
Higher voltage distribution (480, 220v vs. 120)
DC distribution
Reduce electricity conversion stages
#watitis2015
MFCF SERVER ROOMS
• M3-3101 (~ 50kW)
#watitis2015
• MC-3015 (~ 100 kW)
C U R R E N T S TAT E O F M F C F D ATA C E N T E R
• Lack of power saving measures
No air flow Management/Analysis
No efficiency measure (PUE/DCiE)
No power-usage monitoring
No use of machines utilization logs
#watitis2015
PROPOSED ACTION
"To measure is to know - if you cannot
measure it, you cannot improve it”
- Kelvin
#watitis2015
P R O P O S E D A C T I O N S ( C O N T. )
• Determine data center capacity
Start with CFD study
• Monitor energy utilization to
Get baseline efficiency PUE, DCiE, or DCP
Determine highest Energy consumers
Plan load distribution across datacenter
• Implement Hot aisle containment
• Increase cold aisle temperature (23 ℃)
#watitis2015
P R O P O S E D A C T I O N S ( C O N T. )
• Monitor machine utilization pattern
• Determine:
Comatose servers
ON/OFF and suspend policies
Virtualizable machines
• High availability clusters (elasticity)
• Consolidating services
#watitis2015
P R O P O S E D A C T I O N ( C O N T. )
• Utilize machine’s remote management
To apply ON/OFF and suspend policy
To provide machines’ owners remote control
• Investigate free cooling options
#watitis2015
SUGGESTION FOR THE UNIVERSITY
O F WAT E R L O O
• Move electricity bill payment down to faculty
level
• Incentive
• Free cooling worth investigation
#watitis2015
CONCLUSION
• Many ways to reduce energy
• Start with cheaper, easier, most rewarding
Weed out comatose server
Define and implement ON/OFF policies
Use Virtualization
Tighten hot aisle containment
Increase cold aisle temperature (23 to 27 ℃)
Utilize economizers (air-side, water-side)
• Utilize measures for comparison:
PUE, CDD …
#watitis2015
TO B E C O N T I N U E D … .
• I am hoping that by next year, I will be able
to present a different kind of presentation.
• One that includes data, results and actual
changes to improve energy efficiency
• Ultimately leading to a greener future
#watitis2015
Questions?
• Email: nalamron@uwaterloo.ca
Ext. 36847
#watitis2015
This is an optional area for you to put a
message or caption for your photo. Delete it if
you don’t need to use it.
#watitis2015