GREEN COMPUTING
CS 595
LECTURE 12
4/1/2015
Number one concern of large scale data centers?
• Efficiency
affects:
accessibility
reliability
sustainability
costs
Can we affect datacenter efficiency with software?
- yes!
OUTLINE
• Power consumption in large datacenters
• Alternative energy sources
• Green hardware design best practices
•
•
•
•
•
•
•
Server level hardware
DC/DC, AC/DC conversions
Rack power distribution
UPS
Cooling
Building power distribution
Green software design best practices
•
Resource virtualization
• Storage
• Computational
POWER CONSUMPTION IN LARGE DATACENTERS
•
United States: 91 Billion kWh
•
Datacenter power usage effectiveness (PUE):
•
•
•
•
•
•
•
•
•
•
(2013)
ESIS – Consumption for supporting infrastructure
EITS – Consumption for IT power substations
ETX – Building Transformer losses
EHV – High voltage cable losses
ELV – Low voltage cable losses
PUE =
𝑬𝑺𝑰𝑺+𝑬𝑰𝑻𝑺+𝑬𝑻𝑿+𝑬𝑯𝑽+𝑬𝑳𝑽+𝑬𝑭
𝑬𝑰𝑻𝑺 −𝑬𝑪𝑹𝑨𝑪 −𝑬𝑼𝑷𝑺 −𝑬𝑳𝑽+𝑬𝑵𝒆𝒕𝟏
EF – Consumption from fuel (natural gas, oil)
ECRAC – CRAC consumption
EUPS – Loss at UPS
Enet1 – Network room energy from type 1 unit substitution
PUE?
•
For every (PUE)Watt consumed at the datacenter, 1 Watt is delivered to the IT load
• 2014 Global Datacenter average PUE:
• Amazon: 1.45
• Microsoft: 1.13
• Google: 1.1
• Yahoo: 1.07
• Facebook: 1.07
1.7
POWER CONSUMPTION IN LARGE DATACENTERS
• Datacenter energy breakdown:
• Computing: 52%
•
•
•
•
•
CPU – 15%
Server Power supply – 14%
Other Server Components – 15%
Storage – 4%
Networking – 4%
• Support Systems: 48%
•
•
•
•
•
Cooling – 38%
UPS – 5%
Building Power Distribution/Transformer – 3%
PDU – 1%
Lighting – 1%
POWER CONSUMPTION IN LARGE DATACENTERS
• Average and maximum power densities:
• 10 years ago:
• 100W – 175W per sq ft.
• Modern data center:
• Average: 225W
• Maximum: 400W
• Previous slide: 5,000 sq ft. datacenter
• 2mW of energy to dissipate!
• Is this a large datacenter?
• 2011: Clemson University completes its new datacenter:
• 50,000sq ft.
• 25,000sq ft. -office space
• 25,000sq ft. - rack space
• 4.5mW at full capacity (current)
ALTERNATIVE ENERGY SOURCES
SOLAR
• Solar Constant: amount of incoming solar electromagnetic radiation
• ~1.36 kW/m2 (maximum)
• Average solar panel efficiency: 14%
• 19w/ft2 (best condition)
• Closer to 10w/ft2
• 5,000 sq ft. datacenter:
• 2mW  200,000 sq ft of solar panels! (4.59 acres) lol?
• Semprius (solar startup): 33.9%
• 1.89 acres 
• Energy storage?
• Datacenter placement?
• Cloudy?
ALTERNATIVE ENERGY SOURCES
WIND
• Advantages:
• Generate 250W – 1.8MW per turbine
• Green, renewable energy
• Disadvantages:
• Unpredictable
• Noise
• Aesthetics
ALTERNATIVE ENERGY SOURCES
GEOTHERMAL
•
Cooling, Cooling, Cooling!
•
Approach 1:
•
Climate cooling
• Strategic placement of datacenter that allows outside air temperature to help regulate inside
temperature.
•
Approach 2:
•
Ground cooling
• Average temperature at 20ft below earth’s surface: 54F
• Datacenters can use closed loop venting ducts underground to dissipate the heat produced by
equipment.
GREEN HARDWARE DESIGN BEST PRACTICES
SERVER LEVEL HARDWARE - CPU
• CPUs with lower TDP:
• Intel Ivy Bridge Xeon:
• 22nm manufacturing process
•
• E3-1280 – 4 core w/HT (8 threads)
• Max TDP: 69W
• E5-2600 V2 – 10 core w/HT (20 threads)
• Max TDP: 130
AMD Piledriver Opteron:
• 32nm manufacturing process
• 6366 HE – 16 core
• Max TDP: 85W
• 6386 SE – 16 core
• Max TDP: 140W
• Goal: Increase Flops/Watt
GREEN HARDWARE DESIGN BEST PRACTICES
SERVER LEVEL HARDWARE - CPU
• Performance per watt: rate of computation that can be delivered by a CPU
for every Watt of power consumed.
• www.green500.org:
• #1: 5.27 GFlops/W
• #1 - #10 average: 3.91 GFlops/W
• #500: 22.11 MFlops/W
•
Is this green?
• Google Nexus 6:
•
~100 MFlops/W
CPU + GPU + Android System
GREEN HARDWARE DESIGN BEST PRACTICES
SERVER LEVEL HARDWARE - STORAGE
•
HDD vs SSD:
•
WD Caviar Blue 500 GB SATA 6GB/s
• Read/Write: 6.8W
• Idle: 6.1W
• Read/Write performance: 130MB/s
•
Seagate Cheetah 15k 146GB SAS
• Read/Write: 17.2W
• Idle: 14.4W
• Read/Write performance: 186MB/s
•
OCZ Vertex 3 240GB SATA 6GB/s
• Read/Write: 1.25W
• Idle: 0.53W
• Read/Write performance: >500MB/s
•
Cost per GB?
•
•
HDD: ~$0.05/GB
SSD: ~$0.47/GB
GREEN HARDWARE DESIGN BEST PRACTICES
SERVER LEVEL HARDWARE - RAM
• DDR2 vs DDR3
• DDR2 – 1.8v
•
•
1066 MT/s
80nm manufacturing process
• DDR3 – 1.25v – 1.65v
•
•
2133 MT/s
20nm (Samsung, 2012)
• DDR4 (September 2012):
• 1v – 1.2v
•
Up to 4266 MT/s (megatransfers/sec)
• 20nm current manufacturing process
GREEN HARDWARE DESIGN BEST PRACTICES
SERVER LEVEL HARDWARE - MOTHERBOARD
• 85% of power consumed by motherboard:
• Voltage Regulators
•
Converts voltage provided by power supply into appropriate voltages needed by the CPU,
RAM, Chipsets, etc.
•
•
Also used to “clean up” DC voltage from the power supply.
Linear voltage regulator:
•
Solid state semiconductor to regulate voltages
•
•
~60% efficient
Switching voltage regulator:
•
Circuit that transitions between full regulated out, low-current output, and off
• Generates less heat  ~95% efficient
•
Use motherboard with switching voltage regulators for higher efficiency.
GREEN HARDWARE DESIGN BEST PRACTICES
SERVER LEVEL HARDWARE - NETWORK
• Use of power efficient networking components:
• Switches
• Routers
• NICs
• Devices can be scaled depending on demand
• Device low power mode
• Turn off ports
• Goal:
• Increase Bandwidth/Watt in communication components
GREEN HARDWARE DESIGN BEST PRACTICES
AC/DC, DC/DC CONVERSIONS
• AC/DC conversions:
• Translates non-polarized alternating current (building power feed) to polarized direct
current (used by IT components)
• Not 100% efficient
•
Heat generated equates to loss
•
Also must be dissipated requiring more power consumption!
• AC/DC conversion locations:
•
•
Building feed to UPS
Server Power supply
•
•
EPA Energy Star rating – 80% efficiency across all rated power output.
Switching voltage regulator (previous slides)
GREEN HARDWARE DESIGN BEST PRACTICES
RACK POWER DISTRIBUTION
• Problem with current rack deployment?
• Rack mount servers connected to UPS
•
Power transforms at UPS and again at the server
• Solution (Google):
•
Remove unneeded AC/DC, DC/AC transformations
•
•
•
AC from building line feed connects to UPS in the rack
UPS performs AC/DC transformation
Servers connect directly to UPS’s DC output
GREEN HARDWARE DESIGN BEST PRACTICES
UPS
• Uninterruptible Power Supply:
• Stores power to ensure IT component operations in the case of short power outages
• Also used to “clean” electricity from outside source
• Efficiency: 92% - 95%
• Arranged in a 2N configuration for fault tolerance.
• Example: 20,000sq ft. datacenter at 120w/sq ft.
•
Electricity cost: $0.10/kWh
•
95% efficient vs 92% efficient UPS
•
$72,000 annual difference in electricity costs
• Generated $72,000 worth of heat!!
GREEN COMPUTING
CS 595
LECTURE 12
4/1/2015
GREEN COMPUTING
• Current system extremely wasteful
• Need energy to power
• Need energy to cool
•
1000 racks, 25,000 sq ft, 10MW for computing, 5MW to dissipate heat
• Need a system more efficient, less expensive strategy with immediate impact
on energy consumption
DATA CENTERS
• Focus by green computing movement on data centers (SUVs of the tech world)
• 6,000 data centers in US
• Cost: $4.5 B (more than used by all color TVs in US)
• In 2007, DOE reported data centers 1.5% of all electricity in US
• Greenhouse gas emission projected to more than double from 2007 to 2020
DATA CENTERS
• Within a few years, cost of power for data center was expected to exceed
cost of original capital investment
GOAL
• Fed. Gov. wanted data center energy consumption to be reduced by at least
10%
• Same as energy consumed by 1M average US households
DATA CENTER METRICS
• Metrics
• SPECPowerjbb benchmark and DCiE from Green Grid
• Green Grid – group of IT professionals
• Power Usage Effectiveness PUE
PUE = Total facility power/IT equipment power
• Data Center infrastructure Efficiency metric DCiE
1/PUE
FUTURE VISION
• Sources of computing power in remote server warehouses
• Located near renewable energy sources – wind, solar
• Usage shifts across globe depending on where energy most abundant
CURRENT APPROACHES
• Some “low hanging fruit” approaches
• Orient racks of servers to exhaust in a uniform direction
• Higher fruit - Microsoft
• Built near hydroelectric power in WA
• Built in Ireland - can air cool, 50% more energy efficient
• Countries with favorable climates: Canada, Finland, Sweden and Switzerland
CURRENT APPROACHES
• Google – trying to reduce carbon footprint
Carbon footprint includes direct fuel use,
purchased electricity and business travel,
employee commuting, construction, server manufacturing
• According to Google, its data centers use ½ industry’s average amount of power
• How? Ultra efficient evaporative cooling (customized)
• Yahoo (are they back??)
• Data centers also carbon-neutral because of use of carbon offsets
CURRENT APPROACHES
• US government
• EPA has phase-one of Energy Star standards for servers
• Measure server power supply efficiency and energy consumption while
idle
• Must also measure energy use at peak demand
• Green Grid consortium
• Dell, IBM, Sun VM-Wear AMD
• Green500 – 500 most green supercomputers
DATA CENTER PRODUCT SPECIFICATION
COMPLETION
• 2009
• Servers v1.0
• 2011
• Data Center Buildings Program
• 2012
• UPS v1.0 (uninterruptable power supply)
• 2013
• Servers 2.0
• Storage v1.0
• 2014
• Large Network Equipment v1.0
• 2015
• Data Center Cooling Equipment v1.0
CURRENT APPROACHES
• Replace old computers with new more energy-efficient
• But manufacturing through day-to-day uses energy
• Dell - reducing hazardous substances in computers, OptiPlex
50% more energy efficient
• Greenest computer company – VirtualBoxImages
• What is “Greenest computer ever” ?
• Is MacBook air (pro) greenest?
• https://www.apple.com/macbook-pro/environment/
GOALS FOR FUTURE
1.
2.
3.
Consider energy to manufacture, operate, dispose of wastes
4.
Benefit of digital alternative to physical activities
Sense and optimize world around us
Predict and respond to future events by modeling behavior (grown in
performance)
•
•
E-newspapers, online shopping
Personal energy meter??
THE CASE FOR ENERGYPROPORTIONAL COMPUTING
BARROSO AND HOLZLE (GOOGLE)
INTRO
• Energy proportional computing primary design goal for servers
• Cooling and provisioning proportional to average energy servers consume
• Energy efficiency benefits all components
• Computer energy consumption lowered if:
• Adopt high-efficiency power supplies
• Use power saving features already in equipment
INTRO
• More efficient CPUs on chips based on multiprocessing has helped
• But, higher performance means increased energy usage
SERVERS
• Servers
•
•
•
•
Rarely completely idle
Seldom operate at maximum
10-50% of max utilization levels
100% utilization not acceptable for meeting throughput, etc. – no slack time
SERVERS
• Completely idle server waste of capital
•
Difficult to idle subset of servers
• Servers need to be available
•
•
•
Perform background tasks
Move data around
Help with crash recovery
• Applications can be restructured to create idle intervals
•
Difficult, hard to maintain
• Devices with highest energy savings, highest wake-up penalty, e.g disk spin up
ENERGY EFFICIENCY AT VARYING UTILIZATION
LEVELS
• Utilization – measure of performance normalized to performance at peak
loads
• Energy efficient server still consumes ½ power when doing almost no work
• Power efficiency – utilization/power value
• Peak energy efficiency occurs at peak utilization and drops as util. decreases
• At 20-30% utilization, efficiency drops to less than ½ at peak performance
TOWARD ENERGY-PROPORTIONAL MACHINES
•
Mismatch between servers’ high-energy efficiency characteristics and behavior
•
Designers need to address this
•
Design machines that consume energy in proportion to amount of work performed
•
•
•
No power when idle (easy)
Nearly no power when little work (harder)
More as activity increases (even harder)
CPU POWER
• Fraction of total server power consumed by CPU changed since 2005
• CPU no longer dominates power at peak usage, trend will continue
• Even less when idle
• Processors close to energy-proportional
• Consume < 1/3 power at low activity (70% of peak)
• Power range less for other components
•
< 50% for DRAM, 25% for disk drives, 15% for network switches
DYNAMIC RANGE
• Processors can run at lower voltage frequency mode without impacting
performance
• No other components with such modes
•
Only inactive modes in DRAM and disks
• Inactive to active mode transition penalty (even it only idle to submilliseconds)
• Servers with 90% dynamic range could cut energy by ½ in data centers
• Lower peak power by 30%
• Energy proportional hardware reduce need for power management software
DISKS - INACTIVE/ACTIVE
• Penalty for transition to active from inactive state makes it less useful
•
•
•
•
Disk penalty 1000 higher for spin up than regular access latency
Only useful if idle for several minutes (rarely occurs)
More beneficial to have smaller penalty even if higher energy levels
Active energy savings schemes are useful even if higher penalty to transition because in low
energy mode for longer periods
CONCLUSIONS
• CPUS already exhibit energy proportional profiles, other components less so
• Need significant improvements in memory and disk subsystems
• Such systems responsible for larger fraction of energy usage
• Need energy efficient benchmark developers to report measurements at
nonpeak levels for complete picture
GREEN INTROSPECTION BY K.
CAMERON
HISTORY OF GREEN
• In the 1970s
•
•
•
•
Energy crisis
High gas prices
Fuel shortages
Pollution
• Education and action
• Environmental activism
• Energy awareness and conservation
• Technological innovation
GIFTS FROM THE 70S
• Energy crisis subsided
• In the meantime advances in computing responsible for:
• Innovation for energy-efficient buildings and cars
• Identified causes and effects of global climate change
• Grassroots activism, distributing info about energy consumption, carbon emission, etc.
• The same computing technologies pioneered by hippie geeks (???) are the
problem now
WHAT HAPPENED NEXT
• Call to action within IT community (what about the 80s??)
• In 1990s
• General-purpose microprocessors built for performance
• Competing processors
•
•
ever-increasing clock rates and transistor densities
fast processing power and exponentially increasing power consumption
• Power wall at 130 watts
• Power is a design constraint
BETTER, BUT ALSO WORSE?
• To reduce power consumption
• Multicore architectures – higher performance, lower power budgets
• But
• Users expect performance doubling every 2 years
• Developers must harness parallelism of multicore architectures
• Power problems ubiquitous – energy-aware design needed at all levels
MORE PROBLEMS
• Memory architectures consume significant amounts of power
• Need energy-aware design at systems level
•
Disks, boards, fans switches, peripherals
• Maintain quality of computing devices, decrease environmental footprint
• Can’t rely on nonrenewable resources or toxic ingredients
THOSE DATA CENTERS
• IT helping in data centers
• Reducing energy with virtualization and consolidation
• Need to address chip level device to heating/cooling of building
• Need metrics
TRADE-OFF
• How often to replace aging systems?
• 2% of solid waste comes from consumer electronic components
• E-waste fastest growing component of waste stream
• In US 130,000 computers thrown away daily and 100 M cell phones annually
• Recycle e-waste (good luck)
• Use computers as long as possible?