Data Center Efficiency Metrics: mPUE™, Partial PUE, ERE, DCcE Dan Azevedo, Symantec Jud Cooley, Oracle Michael Patterson, Intel Mark Blackburn, 1E Join The Green Grid Technical Forum conversation on Twitter! As you are tweeting, use #TGG11. Presenters Dan Azevedo, Symantec Director of Data Center Architecture, Strategy & Innovation Board Member and Metrics & Measurements Chairman, The Green Grid Jud Cooley, Oracle Senior Director, Design for the Data Center Alternate Board Member, The Green Grid Michael K. Patterson, Intel Senior Power & Thermal Architect Technology & Strategy Chairman, The Green Grid Mark Blackburn, 1E Co-Founder & Chief Technologist Metrics & Measurements Vice Chairman, The Green Grid Copyright © 2011, The Green Grid The United States of America, European Union, and Japan have successfully completed harmonization efforts around one data center energy efficiency metric Power Usage Effectiveness (PUE) U.S. Data Center Industry Session 13 Jan 2010 • Attendees: • Session Objectives: Data Center Efficiency Metrics: Develop common definitions for key metrics and seek consensus on guiding principles for metrics Measurement Protocols: Define current status and gaps and coordinate plans for further development Tools: Review status and coordinate development Copyright © 2011, The Green Grid U.S. Data Center Industry Session Taskforce Results • Released two memos: Data Center Industry Leaders Reach Agreement on Guiding Principles for Energy Efficiency Metrics (Feb 2010) Recommendations for Measuring and Reporting Overall Data Center Efficiency – Version 1 Measuring PUE at Dedicated Data Centers (Jul 2010) US Consensus Achieved with Public Announcement for PUE with Guidance and Refinement • Work in Progress Recommendations for Measuring and Reporting Overall Data Center Efficiency – Version 2 Measuring PUE in a Mixed Use Facility Copyright © 2011, The Green Grid Global Metric Harmonization Meetings 26 Mar 2009, 02 Feb 2010, 26 Oct 2010, 15 Feb 2011 • Attendees: Ministry of Economy, Trade and Industry Green IT Promotional Council The U.S. Department of Energy‟s Save Energy Now and Federal Energy Management Programs The U.S. Environmental Protection Agency‟s ENERGY STAR Program The European Commission Joint Research Center Data Centers Code of Conduct The Green Grid • Session Objectives: Share global lessons and practices with an objective of arriving at a set of metrics, indices, and measurement protocols which can be formally endorsed or adopted by each participant organization to improve data center energy efficiency globally. Copyright © 2011, The Green Grid Global Metric Harmonization Meetings Taskforce Results • Released two memos: The United States of America, European Union and Japan Reach Agreement on Guiding Principles for Data Center Energy Efficiency Metrics (Feb 2010) Global Taskforce Reaches Agreement on Measurement Protocols for PUE – Continues Discussion of Additional Energy Efficiency Metrics (Feb 2011) The United States of America, European Union and Japan Successfully Complete Harmonizing PUE • Work in Progress: Prioritized further areas of work to drive Data Center Energy Efficiency (see appendix) 8 Copyright © 2011, The Green Grid Influences Specific to PUE Global Harmonization of Metrics U.S. Industry Data Center Taskforce The Green Grid Copyright © 2011, The Green Grid Partial PUE (pPUE) Rationale for Partial PUE • PUE can be used to compare data center designs • Often PUE's are presented that do not take into account all of the infrastructure components in order to highlight one particular portion of a data center – “Container only” PUE – “Cooling system” PUE – “Power delivery” PUE • These comparisons have value, but are not truly PUE • “Partial PUE” provides a formal language for this • Partial PUE is abbreviated “pPUE” Copyright © 2011, The Green Grid pPUE Definition • PUE = Total Facility Energy divided by the IT Equipment Energy • This takes into account energy use within a facility • Partial PUE is for energy use within a boundary pPUE = Total Energy within a boundary divided by the IT Equipment Energy within that boundary Copyright © 2011, The Green Grid Example 1 • PUE = 600/475 = 1.26 • pPUE of Container = 500/475 = 1.05 Copyright © 2011, The Green Grid Example 2 PUE = 2700/2000 = 1.35 PPUE1 = 1700/1500 = 1.13 PPUE2 = 700/500 = 1.40 • Zones 1 and 2 have the same chiller power, but different IT loads • So their Partial PUE's are different Copyright © 2011, The Green Grid Notations • N is for “non-IT” power • I is for “IT” power • PUE = (N0 + N1 + N2 + I1 + I2) (I1 + I2) pPUE1 = pPUE2 = (N1 + I1) (I1) (N2 + I2) (I2) Zone 0 has no IT, so pPUE is undefined Copyright © 2011, The Green Grid pPUE Relative to PUE • • Zones 1 and 2 each hold a portion of the IT load: (I1) (I2) r1 = r2 = (I1 + I2) (I1 + I2) PUE = (N0) (I1 + I2) + (r1 * pPUE1) + (r2 * pPUE2) (The proof is in the White Paper, note that r1 + r2 = 100%) The PUE is the overhead seen by all zones plus the Partial PUE of each zone multiplied by the percentage of that zone's contribution to the overall IT load Copyright © 2011, The Green Grid What It Means • PUE = (N0) (I1 + I2) + (r1 * pPUE1) + (r2 * pPUE2) • If one of the pPUE's is bad, work on that • But if one of the r's is very large, the payback for working on that is great Copyright © 2011, The Green Grid Example 3 This is the same as example 2 except that Zone 1 is 10 times as large • PUE = 20,000/15,500 = 1.29 • pPUE1 = 17,000/15,000 = 1.13 • pPUE2 = 700/500 = 1.40 • r1 = 15,000/15,500 = 0.97 • r2 = 500/15,500 = 0.03 Copyright © 2011, The Green Grid Summary • Partial PUE can be used to describe the PUE contribution of a subset of facility components • Partial PUE refers to components within a boundary • The boundary can be physical or logical, for example: – All cooling components – All power components – All Tier II components – All baseline load components – All departmental components • PUE is a valuable metric for monitoring and managing data center power • If only a portion of the facility is being analyzed, use partial PUE Copyright © 2011, The Green Grid Energy Reuse Effectiveness (ERE) “I am re-using waste heat from my data center on another part of my site and my PUE is 0.8!” Copyright © 2011, The Green Grid “I am reusing waste heat from my data center on another part of my site and my PUE is 0.8!” • While re-using excess energy from the data center can be a good thing to do, it should not be rolled into PUE • The definition of PUE does not allow this • There is a metric to do this! ERE Copyright © 2011, The Green Grid ERE Definition Total Energy PUE = IT Energy Cooling + Power + Lighting + IT PUE = IT Total Energy - Reused Energy ERE = IT Energy Cooling + Power + Lighting + IT - Reused ERE = IT Copyright © 2011, The Green Grid ERE Alternate Development Define energy reuse factor (ERF) as: Reuse Energy ERF = Total Energy Then: ERE = (1 - ERF) PUE And finally: Cool + Pwr + Light + IT - Reused ERE = (1 ERF) PUE IT ERF and PUE are mathematically related, but differ and need to be defined and reported clearly. Copyright © 2011, The Green Grid PUE and ERF Ranges 1 PUE 0 ERF 1 0 ERE Cooling + Power + Lighting + IT PUE = IT Reuse Energy ERF = Total Energy Cool + Pwr + Light + IT - Reused ERE = IT Copyright © 2011, The Green Grid Data Center Boundary Consideration and ERF Cooling Waste E A Utility IT F B UPS C PDU D Data Center Boundary A+B PUE = D Copyright © 2011, The Green Grid Use of ERE waste Non-DC space H Cool G A Absorp Chiller E I Utility F ICT B UPS A+B PUE = D C PDU A + B-F ERE = D Both PUE & ERE valid metrics Copyright © 2011, The Green Grid D Data Center Boundary Improper Use of ERE waste H Cool G A AbsorpC hiller E I Utility F ICT B UPS C PDU D A + B-F Data Center A+B ERE = PUE = Boundary D D PUE is still the right metric; note that “A” (and PUE) will be less than in the design w/o added recovery and absorption chiller Copyright © 2011, The Green Grid Comparison with PUE • One view of PUE is that is the “tax” or burden in energy costs you must pay above the IT load to run the data center; ERE allows the same vision • PUE = 1.0 means 100% of the energy you bring in to the data center goes to the IT • ERE = 1.0 means you only need to bring into the site an amount equal to 100% of the IT energy to support the data center • We need both! • Case 1 Case 2 • PUE = 2.0 PUE = 1.2 • ERF= 0.55 ERF=0.25 • ERE = 0.9 ERE=0.9 • Case 1 focuses on PUE, Case 2 focuses on ERF Copyright © 2011, The Green Grid Data Center Compute Efficiency (DCcE) Unused Servers • ~15% of servers powered on but not being used • CPU utilization doesn‟t tell the whole story • Virtualization doesn‟t cure the problem – it can make it worse • Removing unused servers will increase data center efficiency Reduce power for physical servers Reduce resource usage for virtual servers Copyright © 2011, The Green Grid How to Find Unused Servers • Servers are procured to provide a primary service • If they are no longer providing this service then they are no longer needed and should be decommissioned or repurposed. • Useful work <> CPU utilization • Secondary & tertiary services can cause utilization Virus scanning Disk indexing / defragmentation Backup Etc.. Copyright © 2011, The Green Grid Determining Primary Service Usage • Tracking primary services resource utilization is impractical – too many different primary service apps • Tracking secondary & tertiary services is easier – typically a well known small set of applications • Primary service work = All work – secondary & tertiary work True for CPU & I/O • Some applications do not lend themselves to measurement like this – e.g. terminal services Track incoming network sessions for „useful‟ processes Track interactive logons – always assumed to be useful Copyright © 2011, The Green Grid Calculating Primary Service Usage • Over a time period If – All CPU minus secondary & tertiary CPU > noise threshold Or – All I/O minus secondary & tertiary I/O > noise threshold Or – There have been incoming network sessions for primary services Or – There has been an interactive logon Then – Server was being useful Else – Server was not being useful Copyright © 2011, The Green Grid Server Compute Efficiency (ScE) • The proportion of samples that the server is providing primary services over time (as a percentage) • Any server with an ScE of 0% over a prolonged period is not being used and can be decommissioned or repurposed • Any servers with low ScE are worth investigation – may be candidates for virtualization Copyright © 2011, The Green Grid Data Center compute Efficiency (DCcE) • Data Center compute Efficiency (DCcE) aggregates ScE across all servers in the data center • Provides a benchmark against which to improve (like PUE) • DCcE is NOT a productivity metric – it does not measure how MUCH work is done, just the proportion of work that is useful • DCcE CANNOT and SHOULD NOT be compared between data centers due to the subjective determination of secondary and tertiary processes Copyright © 2011, The Green Grid Thank you for attending The Green Grid Technical Forum 2011 Join The Green Grid Technical Forum conversation on Twitter! As you are tweeting, use #TGG11.