Revolutionizing Data
Center Efficiency
Presented by Kenneth G. Brill,
Executive Director, Uptime Institute
Agenda
• Background
• Revolutionizing Data Center Efficiency
•
Findings
•
Primary drivers of poor efficiency
•
Recommendations
•
2,012 goal
Uptime Institute
Our Mission Is to Continuously Increase
Uptime and Global IT Productivity Through
Benchmarking and Collaborative Learning
What We Do
• User-written, vendor-neutral white papers,
standards and specifications
• Symposiums, design charrettes, seminars and
focus groups
• Certifications (tier level and site resiliency)
• Knowledge community organization and
facilitation
Site Uptime Network® 08 NA Members
Colgate Palmolive
AIG
Confidential (3)
A.G. Edwards
Consonus
Allstate
Deere & Company
American Express
Depository Trust
America Online
Dept. of Veterans
AT&T
Affairs
Bank of America
Bank of Montreal (Canada) Discover Financial
DST Systems
Barclays
eBay, Inc.
Bayer
EVERTEC
BellSouth
FannieMae
Boeing
Fidelity Investments
BP
GE
Capital Group
GMAC-RFC
Caterpillar
GTECH
Chevron
Hewitt Associates
Cisco Systems
Hewlett-Packard
Cincinnati Bell Tech
Highmark
Citigroup
Household Int’l (HSBC)
Clorox
Iron Mountain
JC Penney
Johnson & Johnson
Kaiser Permanente
Lehman Brothers
LexisNexis
Lockheed Martin
Lowe’s Companies
MasterCard
Mayo Clinic
Microsoft
Mitre
Nationwide
Office Depot
Pfizer
Philadelphia Stock Exch
Planet Internet
Progressive Insurance
Rackspace Managed Host
Rogers Comm (Canada)
Royal Bank of Canada (Canada)
Sabey
Savvis
SIAC
Shell
Social Security Administration
Target
Thomson Legal & Regulatory
TD Bank (Canada)
United Airlines
United HealthCare
United Parcel Service
USAA
US Bancorp
Verizon Communications
Version Wireless
Visa
Wachovia Bank
Washington Mutual
Wells Fargo
McKinsey Report:
Revolutionizing Data Center Efficiency
• Uptime Institute gratefully acknowledges Will Forest and
McKinsey and Company for their contributions to data center
efficiency
• First presented at the Institute’s 08 Symposium, the original
Report is available for free download at
www.uptimeinstitute.org/resources
• This presentation retains all original findings, poor efficiency
drivers, recommendations and the implementation goal
• The supporting material is sometimes different
• Corporation Average Datacenter Efficiency (CADE) is better
defined with a new four year example
McKinsey Report
Finding #1
• “Data center facilities spend (CapEx and
OpEx) is large and quickly growing…in many
technology intensive industries…”
• “Some intensive data center users will face
meaningfully reduced profitability if current
trends continue”
For IT-Intensive Businesses,
Fundamental IT Economics Have Changed
• Facilities OpEx is 8% of IT’s budget and is growing
at 20% annually
•
Finding these costs may require forensics as
significant facility costs (especially depreciation) may
not be within IT
• If IT’s annual budget growth remains at 6%,
Facilities cost growth will drive out new application
development
• If IT’s budget growth is increased, corporate
profitability will be meaningfully reduced
Data Center Accounts For A Quarter Of
IT’s Annual Budget
Application
Development
20
Development 40
Maintenance
20
Facilities
IT Budget
Data Center
100%
8%
25%
Infrastructure
&
Operations
60%
End Users
15
Hardware,
Storage, Ops
Network
(LAN/WAN)
15
Other
5
Source: McKinsey; breakdown shown is typical of an IT intensive business
17%
IT Has Become A Major Portion Of
Corporate Fixed Assets
• IT capital assets now represent 50% or more of
total fixed assets in other than heavy
manufacturing or basic industries
• And data center facilities are now becoming
50% of the 50%
Source: Harvard Business Review article
INSTALLED VOLUME SERVERS - US
1,000,000 Additional Physical Servers Per
Year Results In Construction Boom
18,000
16,000
14,000
13.6% CAGR
12,000
9.9%CAGR
CAGR
9.9%
10,000
8,000
6,000
4,000
2,000
0
2000
2001
2002
2003
2004
2005
2006
YEAR
Sources: EPA 2007 Report to Congress, Vernon
Turner/IDC, Koomey 2007
2010
7.5% CAGR
27%
100
6.3% CAGR
12%
DATA CENTER ENERGY CONSUMPTION
BILLION KWH
120
15.7% CAGR
13.8% CAGR
80
10
Power
Plants
Data Center Energy Consumption
Is Growing!
60
40
20
0
2000
2001
2002
2003
2004
2005
2006
2007
2010
YEAR
Source: Uptime Institute Network members: Top Quartile sites – avg 32 K SF
Source: Uptime Institute Network members: Average sites – avg 33 SF
Sources: EPA 2007 Report to Congress, Vernon Turner/IDC, Koomey 2007
Servers Aren’t As Cheap As
They First Appear
• Tier IV data center space, power and cooling costs
per server*
•
Cap Ex investment per server
•
Charge back
•
Electricity
$15,400
2,020/yr $8,080/4 years
470/yr
1,880/4 years
* IT acquisition cost per server of $1,500 to
$2,500
Site Costs Per $2,500 Server (Assumes Low Cost
location And Institute Best Practices)
Site Costs (USA)
CapEx per Server*
Annual Expense
Site Depr’n
Electricity
Site Operations
Total per Server
GHG per server
* 55% site asset utilization
Site Concurrent Maint./Fault Tolerance
Tier II
Tier III
Tier IV
$8,300 % $14,000
% $15,400 %
550
42
950
50
1,000
44
420 32
350 26
$1,320 100
420 23
500 27
$1,870 100
470 33
550 23
$2,020 100
4 tons
4 tons
4 tons
Embedded IT Watts Per $1,000 Of
Server Spending
(Logarithmic scale)
Root Cause: Power Efficiency Lagging
Moore’s Law Results In Rising Site TCO
Summary Of Finding #1
“Meaningfully Reduced Profitability”
• Increasing facility costs (forensics may be required)
have unfavorably changed IT’s fundamental economics
•
•
Choke out new application development or
Meaningful reduction in corporate profitability
• Servers aren’t necessarily cheap
• An unintended consequence of Moore’s Law success
has now become a “disruptive technology”
• Root cause of the economic change has been “invisible”
to most senior executives
McKinsey Report
Finding #2
• “For many industries, data centers are
one of the largest sources of Greenhouse
Gas (GHG) emissions”
Data Centers Are Major Energy Consumers
Attracting Government Attention
• Public Law 109-431 mandated study to determine US
data center energy consumption
• EPA Report to Congress, August 2007
•
Data centers consumed 1% of US electricity production in
2000
•
•
•
2% in 2005
•
Fastest growing industrial segment in the economy
Projected 3% in 2010
Comparable to energy consumption of all televisions, but
growing much more rapidly
Data Centers Are THE Most Energy
Intensive Asset In Most Companies
• Just three data centers out of 3,400 street
addresses account for 10% of total energy
consumption for a major financial
• Data center energy intensity is 20 to 100 times
that of an office building
• A single data center can consume the energy
equivalent of 25,000 homes
Summary Of Finding #2
Greenhouse Gasses
• The energy required to power and cool a single
“cheap” server emits 4 tons of GHG per year
• 15 million servers in 2010 equals 60,000,000
tons annually of GHG
• Governments can’t help but become very
concerned with data center energy efficiency
McKinsey Report:
Primary Drivers Of Poor Efficiency
• “Poor demand and capacity planning and
management within and across functions…”
• “Significant failings in asset management”
• “Boards, CEOs, and CFOs are not holding CIOs
accountable…”
Server Assets Are
Dramatically Underutilized
100
90
Source: McKinsey Disguised Client
80
70
60
50
40
30
20
10
0
0
10
20 30
40
50
Average Daily Server Utilization in Percent
Facility Capacity Utilization in
Percent
Installed Data Center Power And Cooling
Capacity Is Underutilized
100
80
60
40
20
0
2,000
4,000
6,000
8,000
Installed Facility Capacity in kW
10,000
Application And Infrastructure Decisions Typically
Fail To Include True Total Costs
True Application TCO
Application dev labor and licenses
Main and support
Servers, network,
and other hardware
DC utilization
(availability,
redundancy, DR,
infrastructure,
facilities)
True Infrastructure TCO
Hardware
cost (OpEx)
Software (OpEx)
Maintenance
(labor & parts)
Network & connectivity
DC utilization
(availability,
redundancy, DR,
infrastructure,
facilities)
Not Typically Considered in Business Case TCO for Go/No Go Decisions
Source: McKinsey analysis, EPA Report, Uptime Institute
Managerial And Economic Sophistication
Has Not Kept Up
• In 1975-1985, mainframes with 70% to 80%
utilization handled 80% of compute demand
• Today, 80% of compute demand is handled
by distributed systems with 5% to 30%
utilization
Lack Of Data Center CapEx Oversight
Has Resulted In
• Failure to free up existing capacity by decommissioning
comatose servers
• Anticipating unknown business requirements results in
overbuilding
• Sizing being based on highest case demand
• Sweet spot costs and alternate solutions not being
understood/considered
• Inadequate cross functional, financial analysis and
specialized project management skills leading to major
items being missed resulting in delays and overruns
Summary:
Primary Drivers Of Poor Efficiency
• “Poor demand and capacity planning and
management within and across functions…”
• “Significant failings in asset management”
• “Boards, CEOs and CFOs are not holding CIOs
accountable…”
McKinsey Report Recommendations
• “Rapidly mature and integrate asset
management capabilities…”
• “Mandate the inclusion of true total cost of
ownership … in business case justification of
new products and applications to throttle
excess demand”
• “Formally move accountability for data center
facilities and operations to the CIO and appoint
internal ‘Energy Czar’…”
McKinsey Report:
2012 Goal
• Double data center efficiency by 2012 as
measured by CADE (Corporate Average
Datacenter Efficiency
Energy Czar IT Initiatives
• Kill comatose servers
•
•
100% kill of 15% accumulated comatose servers
Implement formal de-commissioning program using
ITIL to document, bill back and audit
• Virtualize
•
•
40% of applications
5 to 1 collapse
• Buy 20% more energy efficient hardware
•
•
More efficient power supplies and better fans
Rightsize memory
Energy Czar Facilities Initiatives
• 12% improvement in site energy efficiency
•
Measure site infrastructure energy overhead
(Green Grid 1/DCiE or PUE if all forms of energy
are included over a 12 month period) and correct
any surprises
•
•
•
•
•
Correctly set cooling unit set points
Truly implement hot/cold aisle concepts
Eliminate humidification/de-humidification
Turn off unneeded cooling units
If available, increase waterside free-cooling
Baseline And Fourth Year
Server Qty, IT And Utility kW And GHG
Baseline
2012
Year
No Chg
Czar
One App/One Server
Virtualized Apps
Active Applications
6,200
0
6,200
10,900
0
10,900
6,500
4,400
10,900
Comatose Servers
1,100
3,000
0
IT Plug Load – kW
Utility Load – kW
2,200
4,800
6,000
11,800
3,100
5,300
242*
130*
4 Year CO2 emissions
* Cumulative over four years in thousands of tons
Cumulative Four Year Outcome
Financial
2008 - 2012
No Chg
Czar
IT server CapEx expenditures
$31
$20
Site asset CapEx expenditures
112
0
54
33
$197
$53
4 Year IT + Site OpEx expenses
Total CapEx + OpEx
All numbers in millions of dollars
Corporate Average Datacenter Efficiency
(CADE) ver 1.0
CADE
=
IT
Efficiency
IT Asset
IT Energy
Utilization x Efficiency
x
Site
Efficiency
Site Asset
Site Energy
Utilization x Efficiency
CADE Calculation -- Baseline Year
Server Based Compute Load
• IT Efficiency
•
IT asset utilization* (10% on active apps, 8.5% overall due
to comatose servers)
•
IT energy efficiency (assume 5% for base year)
• Facility Efficiency
•
•
Site asset utilization* (55%)
Site energy efficiency** (46%) (use a 12 month moving
average)
• CADE = 8.5% x5% x55% x46% x100 = 11
* Asset utilization = IT Plug load/site asset capacity
** Same as GreenGrid’s DCiE except including all forms of
energy running average over 12 months
CADE: Baseline And Fourth Year
Baseline
2012
Year
No Chg
Czar
IT asset utilization
9%
8%
15%
IT energy efficiency
5%
5%
76%
Site asset utilization
55%
75%
77%
Site energy efficiency
46%
51%
58%
11
15
500
CADE
Energy Czar
Fourth Year And Cumulative Results
• McKinsey Report Goal for 2012 was a doubling of data
center efficiency
•
Continuation of current trends fails (CADE is slightly
improved from 11 to 15, minimum CADE goal would have
been 33)
•
Energy Czar program more than succeeds
• CADE goes from 11 to 500 for an efficiency increase of
46 times
• Financially, $144 million is saved over four years
• 112,000 tons of GHG emissions are avoided
Revolutionizing Data Center Efficiency
Summary
• Findings
• Primary drivers of poor efficiency
• Recommendations
• 2012 goal
Questions?