Pursuing Operational Excellence
Better Data Center Management
Through the Use of Metrics
Objective
Use financial modeling of your data center
costs to optimize their utilization.
Data Centers are Expensive to Build
•
A 1MW 7,500 ft2 “white space” commercial
data center will cost about $16.7M to build.
•
The building to house that data center will be
about 23K ft2. Total cost > $700/ft2.
•
Compare that to $100 - $150 / ft2 for a custom
home in middle America.
Data Centers Are Expensive to Operate
• Monthly electric bills for a 1MW data
center average about $82.6k ($991K/yr.)
(At 85% occupancy, with a 1.5 PUE)
• Network charges are very high as well.
• The data center infrastructure is one of the
fastest growing cost of deploying IT.
Data Center Cost Accounting
Do you know how much your data center is
costing you?
Data Center OpEx
• The best financial modeling strategy for data
center OpEx is to boil down all operational
costs into a monthly recurring cost (MRC)
expressed in $/kW for every data center or
colocation facility you operate in.
• This is your “cost of compute”.
Components of Data Center OpEx
• Depreciation
• Maintenance of infrastructure components
(Including preventative maintenance)
• Utilities (power, water, …)
• Fuel (e.g. diesel for generator tests)
• Security
• Salaries (for data center infrastructure
personnel, dedicated security personnel)
Know your “cost of compute”
• Question you are answering:
– How much is it costing me to deploy a rack full
of IT gear at a given location?
• Boil everything down to MRC in $/kW
Cost of Compute
•
Know this number for every data center you
own and for every colocation vendor you use.
•
Easier to compute for dedicated data centers.
•
More difficult but possible to compute for
shared use buildings.
OpEx Financial Accounting Issues
•
•
MRC in $/kW is not constant over time.
–
There are seasonal variations for cooling loads.
–
Design targeted PUEs are attained at full loads
For a given location, MRC in $/kW will
decrease as IT HW occupancy increases.
Typical Cost of Compute
•
For a large Enterprise (economies of scale apply) in a
concurrently maintainable data center
•
•
$235 - $300 / kW MRC
For small / medium Enterprise (no economies of scale) in
a non-concurrently maintainable DC
•
$300 - $400 /kW MRC; much higher for a concurrently
maintainable DC (which a small enterprise is unlikely to have)
Ways to Manage Cost of Compute in
Your Data Center
•
•
•
•
Capacity Planning
Energy Management
Cooling Management
Heat Management
Capacity Planning
One of best ways to control Costs of Compute is
to control capacity.
Benefits
• Improved quality of forecast over time
• Reduced CapEx if you can delay build out of
new data center
• Better decision support
Financial Control Over Space & Power
• Measure and publish utilization over time.
– Know your metrics: (avg. kW/rack, avg. ft2/rack)
• Set-up a space & power governance
program
– Must pre-authorize HW intake to data center
– Must follow-through with decommissioning
equipment.
Financial Control Over Energy
Management
• You are probably spending too much!
• You probably don’t know your PUE (Power
Utilization Efficiency)
• You are probably running too cold!
Energy Management Strategies
• Measure, track, and publish your PUE
• Use ASHRAE’s new recommended
operating temperatures.
• See Green Grid paper on the ROI of cooling
system improvements
Why PUE is Important
• 1 rack = 5kw
• 5kw rack x 8,760 hrs = 43,800kwh/ year
• 43,800 kwh x $0.07/ = $3,066 per rack annual
•
•
•
•
•
5,000 sqft data center with 140 racks
140 racks x $3,066 = $429,240 annual IT energy costs
At PUE of 2 = $858,000 Total DC energy
At PUE of 1.5 = $643,500 Total DC energy
Difference: $214,500
Energy Management ROI
• Generally speaking, for every 1.8°F that you
raise the temperature in your data center,
you save 2-4% of your total energy bill.
Seemingly Small Changes Can be Big
•
•
•
•
91 watts vs 70 watts processors = 21 watts / processor
21 watts x 2 processors
= 42 watts savings
42 watts x 8760hrs / 1000
= 368 kwh
368kwh x $0.07/kwh
= $25.75 per server
• 140 racks / 5000sqft DC
• 140 racks x 13 servers / rack
• 1680 x $25.75 x 2
13 servers / rack
= 1680 servers
$86,520
Cooling Management
Understanding how much cooling your data
center needs.
Convert
Tons of air to BTU
To find BTUs
1 kW
1 ton
Tons to kW
kW to tons
Formula
Example
12,000 BTUs = 1 ton of air
2500w x 3.41 = 8525 (BTU)
Watts x 3.41 = BTU
= .2843 tons
= 3.517kW
20 tons x 3.52 = 70kW
Tons x 3.52 = kW
70kW x .28 = 19.6 tons
kW x .28 = tons
Cooling – effectiveness
How does your cooling capacity stack up
against my IT load?
Load on the UPS in kW x .28 = tons of cooling.
Compare this against tons of cooling in your
data center.
Heat Management
DC looking to add another CRAC unit. Consider
heat containment first. Lets look at some simple
napkin math, pursuant to a containment solution.
CRAC
Containment
CapEx
$80k
$45k
OpEx (5 yrs.)
$50k
$ 0k
Total
$130k
$45k
Other Green Grid Recommendations
•
•
•
•
Install OEM variable speed drives (VSDs) in all computer room
air handlers (CRAHs). Upgrade older CRAH units with newer
more efficient models.
Improve rack airflow management by adding baffles and
blanking panels, which improve isolation of hot and cold air
aisles.
Reposition the CRAH temperature/humidity sensors from the
inlets of the CRAHs to the front of the IT equipment racks.
Adjust the temperature set-points of the CRAH sensors and the
chiller units
What About the Cost of Outsourcing
Is you organization benefiting from selectively
outsourcing some of your infrastructure?
There May be Financial Advantages
to Outsourcing
• Consider colocation
– You can probably not build and operate
anything smaller than a 2MW data center for
less than what you would pay for colocation.
(For the same level of redundancy & service
levels)
• Consider other outsourcing options
The Outsourcing Spectrum
BYODC
Application
Middleware
Operating
System
Virtual
Infrastructure
IT Physical
Infrastructure
Physical
Hardware
Data Center
Environment
Colocation
Managed Hosting
IaaS
PaaS
SaaS
Cost of Colocation
Location
Boston
Chicago
Dallas
NY/NJ
No VA
No CA
So CA
$/kW MRC
$300 - 400
$225 - 425
$200 - 500
$275 - 510
$250 - 400
$275 - 450
$300 - 650
Funding IT in Your Data Center
IT Funding Models
The IT funding model is IT's primary mechanism for gaining approval for spending on
behalf of the business, accounting for that spending, and communicating costs back
to business stakeholders.
IT funding models vary substantially across organizations, but they all seek to strike
three critical balances:
Balancing cost with service levels
Balancing maintenance with new investments
Balancing effective IT coordination with responsiveness to diverse business partner needs
Why Provide Financial Transparency
• Transparency improves IT accountability and
focuses the mission on what is important.
• Transparency supports maintaining proper
balance.
• Transparency helps the business determine
better buy versus build decisions, including
brokering external services.
Deliver Cost Transparency via
Showbacks or Chargebacks ?
How to Get Started
Recommendations you can implement now to improve or initiate your showback
or chargeback program.
• Create a service catalog.
Identify and align business services to technology capabilities currently
being delivered to business units.
• Gather service costing information.
Start with gathering data and information from the finance department
about the cost and expense of each technology-based product and
service.
• Relate services and costs.
Create a financial cost model of how technology services and costs are
related and allocated to business units.
• Make it plain.
Implement greater IT service costing transparency that plainly details the
relationships to budgets and/or P&L statements.
David Sipes, Global Data Center Manager,
SOCAL AFCOM Vice President of Operations
David.Sipes@Jacobs.com