Data Center Efficiency Metrics:
mPUE™, Partial PUE, ERE, DCcE
Dan Azevedo, Symantec
Jud Cooley, Oracle
Michael Patterson, Intel
Mark Blackburn, 1E
Join The Green Grid Technical Forum conversation on Twitter!
As you are tweeting, use #TGG11.
Presenters
Dan Azevedo, Symantec
Director of Data Center Architecture, Strategy & Innovation
Board Member and Metrics & Measurements Chairman, The Green Grid
Jud Cooley, Oracle
Senior Director, Design for the Data Center
Alternate Board Member, The Green Grid
Michael K. Patterson, Intel
Senior Power & Thermal Architect
Technology & Strategy Chairman, The Green Grid
Mark Blackburn, 1E
Co-Founder & Chief Technologist
Metrics & Measurements Vice Chairman, The Green Grid
Copyright © 2011, The Green Grid
The United States of America, European
Union, and Japan have successfully
completed harmonization efforts around
one data center energy efficiency metric
Power Usage
Effectiveness (PUE)
U.S. Data Center Industry Session
13 Jan 2010
• Attendees:
• Session Objectives:
 Data Center Efficiency Metrics: Develop common definitions for key
metrics and seek consensus on guiding principles for metrics
 Measurement Protocols: Define current status and gaps and coordinate
plans for further development
 Tools: Review status and coordinate development
Copyright © 2011, The Green Grid
U.S. Data Center Industry Session
Taskforce Results
• Released two memos:
 Data Center Industry Leaders Reach Agreement on Guiding Principles
for Energy Efficiency Metrics (Feb 2010)
 Recommendations for Measuring and Reporting Overall Data Center
Efficiency – Version 1 Measuring PUE at Dedicated Data Centers (Jul
2010)
US Consensus Achieved with Public Announcement for PUE with Guidance and
Refinement
• Work in Progress
 Recommendations for Measuring and Reporting Overall Data Center
Efficiency – Version 2 Measuring PUE in a Mixed Use Facility
Copyright © 2011, The Green Grid
Global Metric Harmonization Meetings
26 Mar 2009, 02 Feb 2010, 26 Oct 2010, 15 Feb 2011
• Attendees:
 Ministry of Economy, Trade and Industry
 Green IT Promotional Council
 The U.S. Department of Energy‟s Save Energy Now and Federal Energy
Management Programs
 The U.S. Environmental Protection Agency‟s ENERGY STAR Program
 The European Commission Joint Research Center Data Centers Code of
Conduct
 The Green Grid
• Session Objectives:
 Share global lessons and practices with an objective of arriving at a set of
metrics, indices, and measurement protocols which can be formally endorsed or
adopted by each participant organization to improve data center energy
efficiency globally.
Copyright © 2011, The Green Grid
Global Metric Harmonization Meetings
Taskforce Results
• Released two memos:
 The United States of America, European Union and Japan
Reach Agreement on Guiding Principles for Data Center Energy
Efficiency Metrics (Feb 2010)
 Global Taskforce Reaches Agreement on Measurement
Protocols for PUE – Continues Discussion of Additional Energy
Efficiency Metrics (Feb 2011)
The United States of America, European Union and Japan Successfully Complete
Harmonizing PUE
• Work in Progress:
 Prioritized further areas of work to drive Data Center Energy
Efficiency (see appendix)
8
Copyright © 2011, The Green Grid
Influences Specific to PUE
Global
Harmonization
of Metrics
U.S. Industry Data Center
Taskforce
The Green Grid
Copyright © 2011, The Green Grid
Partial PUE (pPUE)
Rationale for Partial PUE
• PUE can be used to compare data center designs
• Often PUE's are presented that do not take into account all of
the infrastructure components in order to highlight one
particular portion of a data center
– “Container only” PUE
– “Cooling system” PUE
– “Power delivery” PUE
• These comparisons have value, but are not truly PUE
• “Partial PUE” provides a formal language for this
• Partial PUE is abbreviated “pPUE”
Copyright © 2011, The Green Grid
pPUE Definition
• PUE = Total Facility Energy divided by the IT
Equipment Energy
• This takes into account energy use within a facility
• Partial PUE is for energy use within a boundary
pPUE = Total Energy within a boundary divided
by the IT Equipment Energy within that
boundary
Copyright © 2011, The Green Grid
Example 1
• PUE = 600/475 = 1.26
• pPUE of Container = 500/475 = 1.05
Copyright © 2011, The Green Grid
Example 2
PUE = 2700/2000 = 1.35
PPUE1 = 1700/1500 = 1.13
PPUE2 = 700/500 = 1.40
• Zones 1 and 2 have the same chiller power,
but different IT loads
• So their Partial PUE's are different
Copyright © 2011, The Green Grid
Notations
• N is for “non-IT” power
• I is for “IT” power
• PUE =
(N0 + N1 + N2 + I1 + I2)
(I1 + I2)
pPUE1 =
pPUE2 =
(N1 + I1)
(I1)
(N2 + I2)
(I2)
Zone 0 has no IT, so pPUE is
undefined
Copyright © 2011, The Green Grid
pPUE Relative to PUE
•
•
Zones 1 and 2 each hold a portion of the IT load:
(I1)
(I2)
r1 =
r2 =
(I1 + I2)
(I1 + I2)
PUE =
(N0)
(I1 + I2) + (r1 * pPUE1) + (r2 * pPUE2)
(The proof is in the White Paper, note that r1 + r2 = 100%)
The PUE is the overhead seen by all zones plus the Partial
PUE of each zone multiplied by the percentage of that zone's
contribution to the overall IT load
Copyright © 2011, The Green Grid
What It Means
•
PUE =
(N0)
(I1 + I2)
+ (r1 * pPUE1) + (r2 * pPUE2)
• If one of the pPUE's is bad, work on that
• But if one of the r's is very large, the payback for working on
that is great
Copyright © 2011, The Green Grid
Example 3
This is the same as example 2 except
that Zone 1 is 10 times as large
• PUE = 20,000/15,500 = 1.29
• pPUE1 = 17,000/15,000 = 1.13
• pPUE2 = 700/500 = 1.40
• r1 = 15,000/15,500 = 0.97
• r2 = 500/15,500 = 0.03
Copyright © 2011, The Green Grid
Summary
• Partial PUE can be used to describe the PUE contribution of
a subset of facility components
• Partial PUE refers to components within a boundary
• The boundary can be physical or logical, for example:
– All cooling components
– All power components
– All Tier II components
– All baseline load components
– All departmental components
• PUE is a valuable metric for monitoring and managing data
center power
• If only a portion of the facility is being analyzed, use partial
PUE
Copyright © 2011, The Green Grid
Energy Reuse
Effectiveness (ERE)
“I am re-using waste heat from
my data center on another part
of my site and my PUE is 0.8!”
Copyright © 2011, The Green Grid
“I am reusing waste heat from my data
center on another part of my site and my
PUE is 0.8!”
• While re-using excess energy from the data center
can be a good thing to do, it should not be rolled into
PUE
• The definition of PUE does not allow this
• There is a metric to do this! ERE
Copyright © 2011, The Green Grid
ERE Definition
Total Energy
PUE =
IT Energy
Cooling + Power + Lighting + IT
PUE =
IT
Total Energy - Reused Energy
ERE =
IT Energy
Cooling + Power + Lighting + IT - Reused
ERE =
IT
Copyright © 2011, The Green Grid
ERE Alternate Development
Define energy reuse factor (ERF) as:
Reuse Energy
ERF =
Total Energy
Then:
ERE = (1 - ERF)  PUE
And finally:
Cool + Pwr + Light + IT - Reused
ERE =
 (1  ERF)  PUE
IT
ERF and PUE are mathematically related, but differ and
need to be defined and reported clearly.
Copyright © 2011, The Green Grid
PUE and ERF Ranges
1  PUE  
0  ERF  1
0  ERE  
Cooling + Power + Lighting + IT
PUE =
IT
Reuse Energy
ERF =
Total Energy
Cool + Pwr + Light + IT - Reused
ERE =
IT
Copyright © 2011, The Green Grid
Data Center Boundary
Consideration
and
ERF
Cooling
Waste
E
A
Utility
IT
F
B
UPS
C
PDU
D
Data Center
Boundary
A+B
PUE =
D
Copyright © 2011, The Green Grid
Use of ERE
waste
Non-DC
space
H
Cool
G
A
Absorp
Chiller
E
I
Utility
F
ICT
B
UPS
A+B
PUE =
D
C
PDU
A + B-F
ERE =
D
Both PUE & ERE valid metrics
Copyright © 2011, The Green Grid
D
Data Center
Boundary
Improper Use of ERE
waste
H
Cool
G
A
AbsorpC
hiller
E
I
Utility
F
ICT
B
UPS
C
PDU
D
A + B-F
Data Center
A+B
ERE =
PUE =
Boundary
D
D
PUE is still the right metric; note that “A” (and PUE) will be less than in the design
w/o added recovery and absorption chiller
Copyright © 2011, The Green Grid
Comparison with PUE
• One view of PUE is that is the “tax” or burden in energy costs you
must pay above the IT load to run the data center; ERE allows the
same vision
• PUE = 1.0 means 100% of the energy you bring in to the data center
goes to the IT
• ERE = 1.0 means you only need to bring into the site an amount
equal to 100% of the IT energy to support the data center
• We need both!
• Case 1
Case 2
• PUE = 2.0
PUE = 1.2
• ERF= 0.55
ERF=0.25
• ERE = 0.9
ERE=0.9
• Case 1 focuses on PUE, Case 2 focuses on ERF
Copyright © 2011, The Green Grid
Data Center Compute
Efficiency (DCcE)
Unused Servers
• ~15% of servers powered on but not being used
• CPU utilization doesn‟t tell the whole story
• Virtualization doesn‟t cure the problem – it can
make it worse
• Removing unused servers will increase data
center efficiency
 Reduce power for physical servers
 Reduce resource usage for virtual servers
Copyright © 2011, The Green Grid
How to Find Unused Servers
• Servers are procured to provide a primary service
• If they are no longer providing this service then they are
no longer needed and should be decommissioned or
repurposed.
• Useful work <> CPU utilization
• Secondary & tertiary services can cause utilization




Virus scanning
Disk indexing / defragmentation
Backup
Etc..
Copyright © 2011, The Green Grid
Determining Primary Service Usage
• Tracking primary services resource utilization is
impractical – too many different primary service apps
• Tracking secondary & tertiary services is easier –
typically a well known small set of applications
• Primary service work = All work – secondary & tertiary
work
 True for CPU & I/O
• Some applications do not lend themselves to
measurement like this – e.g. terminal services
 Track incoming network sessions for „useful‟ processes
 Track interactive logons – always assumed to be useful
Copyright © 2011, The Green Grid
Calculating Primary Service Usage
• Over a time period
 If
– All CPU minus secondary & tertiary CPU > noise threshold
 Or
– All I/O minus secondary & tertiary I/O > noise threshold
 Or
– There have been incoming network sessions for primary services
 Or
– There has been an interactive logon
 Then
– Server was being useful
 Else
– Server was not being useful
Copyright © 2011, The Green Grid
Server Compute Efficiency (ScE)
• The proportion of samples that the server is
providing primary services over time (as a
percentage)
• Any server with an ScE of 0% over a prolonged
period is not being used and can be
decommissioned or repurposed
• Any servers with low ScE are worth investigation
– may be candidates for virtualization
Copyright © 2011, The Green Grid
Data Center compute Efficiency (DCcE)
• Data Center compute Efficiency (DCcE) aggregates ScE
across all servers in the data center
• Provides a benchmark against which to improve (like
PUE)
• DCcE is NOT a productivity metric – it does not measure
how MUCH work is done, just the proportion of work that
is useful
• DCcE CANNOT and SHOULD NOT be compared
between data centers due to the subjective
determination of secondary and tertiary processes
Copyright © 2011, The Green Grid
Thank you for attending
The Green Grid Technical Forum 2011
Join The Green Grid Technical Forum
conversation on Twitter!
As you are tweeting, use #TGG11.