AIT Partnership Group Ltd White Paper: Using Cold Aisle Containment in the Data Centre Retrofitting cold aisle containment to increase capacity and reduce energy use Tim Daniels 14/10/2013 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use Contents 1. Summary ......................................................................................................................................... 2 2. Traditional Approaches to Data Centre layout ............................................................................... 3 3. Cold Aisle Containment................................................................................................................... 4 4. Design of Cold Aisle Containment................................................................................................... 5 a. Fire Detection and Suppression .................................................................................................. 5 b. Hot Aisle Containment ................................................................................................................ 5 5. Maximising Efficiency...................................................................................................................... 6 6. Examples of Best Practice ............................................................................................................... 7 7. a. Case Study 1: The University of Nottingham (Hot Aisle Containment Example)........................ 7 b. Case Study 2: DIY Chain (Cold Aisle Containment Example) ...................................................... 8 References ...................................................................................................................................... 9 1 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 1. Summary Our ever increasing use and dependency on corporate and cloud software applications mean that data centres, which house the computing and storage equipment on which all this software runs, are continuously under growth pressure. As a consequence data centres use a large and growing amount of energy both to run the IT equipment and to cool it. The overwhelming majority of data centres rely on air-based cooling provided by airconditioning to keep IT equipment in the server racks below the industry standard temperature limit of 27⁰ C. In order to achieve this air temperature is closely monitored and controlled within the data centre. IT equipment running too hot will reduce its lifetime and cause early failure, while over cooling the data centre wastes energy and increases costs. A traditional style commonly used in building data centres has been to use an open-plan room, with cold air fed to the server racks via an under floor void and warm air re-circulated from the server racks back to the air-conditioning units at ceiling level. Some mixing of the warm and cold air flows used to be tolerated but cold aisle containment overcomes this problem. Cold aisle (or hot aisle) containment stops cool air mixing with warm air, typically reducing data centre energy costs by 10-20%. Examples of both cold aisle and hot aisle containment are provided in this white paper. One large UK DIY chain, reduced their cooling costs by 14% by introducing cold aisle containment in its data centre. It should be noted that this data centre was already well organised, and managed. Studies have shown that if data centres are badly configured then rearranging the cabinets, air flow and air conditioning systems, and improving their management, can reduce energy consumption by over 50%. Ref 1: Example of Cold Aisle Containment 2 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 2. Traditional Approaches to Data Centre layout Early data centres and IT equipment rooms were traditionally located within a building with little planning for airflow and equipment cooling. Equipment was placed into rows of standard sized 19 inch wide server racks. Brute force cooling methodologies were used to cool the room, usually with air-conditioning units placed around the periphery of the room, positioned along the ends of the rows to let the cool air flow along the rows. Hot spots tended to occur in heavily loaded racks in mid-aisle positions which were furthest away from the airconditioning. Overcooling some areas, along with powerful ventilation, was a necessary (costly) evil in order to avoid hot spots. During the 1990s raised flooring became widespread in commercial building design which gave great flexibility to office layout, allowing services to run under in an under-floor plenum (space) to any part of the room. Data centres used this concept too, delivering cold air via an under floor plenum space to specific ‘cold’ aisles in the data centre. These cold aisles are arranged with cabinet fronts on both sides of the aisle. The alternate rows, known as ‘hot’ aisles, receive hot air from the rear of the server cabinets on both sides. Hugely practical, the concept of hot and cold aisles with an under floor plenum was widely adopted by the industry and has served well for many years. Properly balanced, the cold air coming from the under floor plenum naturally creates a positive pressure in the cold aisle that flows into and over the server cabinets towards the hot aisle. Hotter air rises and flows back to the air-conditioning units creating a natural air circulation system within the data centre. Ref 2: Traditional Hot and Cold Aisle Concept However high-density compute servers and server ‘blades’ have replaced older equipment in-rack power usage has crept up from 1-2kW to towards 5-8kW per rack. Cooling needs have also grown in proportion with this increased power making it ever harder to keep servers within ASHRAE temperature limits. The challenge for data centres is that they are designed to last for many years, even decades, and yet the equipment within them changes every 3-5 years. The need to improve cooling effectiveness has become a key factor in order to cram in more high-density compute servers. More efficient cooling will also allow tighter temperature regulation, improved reliability, longer equipment life, lower running costs, and a reduced carbon footprint to comply with government initiatives. 3 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 3. Cold Aisle Containment For traditionally designed data centres the introduction of physical separation between the hot and cold aisles eliminates the mixing of hot and cold air streams, and instantly improves thermal efficiency. Separating the air flows improves server rack cooling and enables higher in-rack equipment densities. There are two main ways of separating the air; the first is to flood the room with cold air and contain the hot exhaust air, feeding this directly back to the air-conditioning systems; the second is to contain the cold air within the cold aisle (fed via an under-floor plenum) and allow the exhaust air to flow freely from the rear of the servers back to the air-conditioning systems. Some papers argue that hot air containment is more efficient but there is no universally proven right or wrong approach. The ‘right’ approach really depends on how the data centre has been physically designed. ‘Chimney’s’ fitted to the rear of the server cabinets can work well to contain the hot air and feed it back to the air conditioning systems, but is often difficult and expensive to retrofit to a live data centre. Since most data centres already feed cold air directly to the cold aisles via an under-floor plenum it is usually simpler and cheaper to enclose the cold aisles with roof panels and sliding doors to the end of each aisle. Ref 3: Cold Aisle Containment Using cold aisle containment has several benefits. Stopping the air mixing allows air temperatures to be more closely controlled at point of delivery, and therefore raised, which reduces energy costs. Increased operational temperatures can save even more when coupled with free cooling systems, since external air can be used for a greater percentage of the year due to higher set-point temperatures. Creating physical separation reduces the need to pressurise the cold aisle to eliminate warm air re-flow, reducing the power used by the airconditioning fans. As well as saving energy costs, equipment reliability also improves because temperatures remain more stable and this further reduces overall operational costs and downtime. 4 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 4. Design of Cold Aisle Containment Cold aisle containment is simply a barrier between the cold and hot aisles. This is achieved in various ways depending on current data centre design, efficiency needs, and the budget available. The simplest scenario is the use of plastic strip curtains (also known as butchers curtains) over the cabinet rows to separate hot and cold aisles. However they tend to be ill fitting and blow about in the breeze, creating a poor fit and reducing efficiency of the separation system. Ref 4: Components of Cold Aisle Containment A better solution is to fit light weight roof panels over the cold aisle and sliding doors at both ends of each cold aisle. Usually constructed of sheet metalwork, solutions can also include bespoke side panels to even up cabinet heights or fill in gaps where cabinets are missing. AIT uses a variety of solutions depending on project need. a. Fire Detection and Suppression The roof panels are usually hinged in order to allow them to open in event of fire, allowing existing fire detection and suppression systems to work effectively. In rare circumstances additional fire detection and suppression equipment is needed. Many variations of detection/suppression exist but suffice it to say we have always been able to find a practical solution to meet the customers’ (and regulatory) needs. b. Hot Aisle Containment We have concentrated on cold aisle containment in this white paper, but it is equally valid to install hot aisle containment to separate the cold and hot air. Many of the same principles apply. The choice of which to use is dictated by the current data centre layout. Hot aisles may be contained or chimney systems may be fitted to the rear of the cabinets to capture warm exhaust air. A hot aisle containment case study is given the section covering examples of best practice. 5 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 5. Maximising Efficiency Installing cold aisle containment, by design, changes the air-flow characteristics of the data centre. To maximise those benefits it is important to perform a data centre audit before installing the containment to highlight and plan for any other changes that must be done in conjunction with it. Since the cold air flow is now contained the fans within the computer room air-conditioning (CRAC) must often be adjusted to reduce air flow. Additional savings can often be made by updating the CRAC with new variable speed fan drives. Temperature set-points within the CRACs must also be adjusted to suit the new conditions, usually to increase the temperatures. ASHRAE now recommends data centre running temperatures up to 27⁰C, yet many still run in the 18-21⁰C range. More controlled air flows mean higher temperatures can be maintained without the danger of hot spots occurring. Sometimes the number of CRACs and their cycling order must also be adjusted for the new setup. The data centre may also want to retrofit measurement equipment to measure power use within the data centre at the CRAC and server-rack level (or distribution board), and measure temperature at various points within the cold and hot aisles. Since each cold aisle becomes enclosed, closer monitoring may be necessary than has previously been the case. An audit will also uncover under-floor plenum issues, such as reduced air flow due to cable stuffing, and recommend any floor vent re-configuration to maximise efficiency after installation of the cold aisle containment system. 6 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 6. Examples of Best Practice AIT Partnership Group has installed many cold aisle containment systems in the UK, within large data centres and smaller IT equipment rooms, for businesses and public sector organisations. a. Case Study 1: The University of Nottingham (Hot Aisle Containment Example) An AIT data centre audit uncovered efficiency savings that would save them an estimated £5,808 a year. Hot aisle containment (rather than cold) was installed as part of the solution due to the configuration of the data centre. The payback period for the project was estimated to be less than three years, while their efficiency rating improved from a PUE of 1.8 to a PUE of 1.5. (PUE = power usage effectiveness ratio). Chris Tadman, the data centre team leader at The University of Nottingham was very happy with his new hot aisle containment. “I estimate the hot aisle containment system made a 5% saving on our Primary Data Centre energy usage. It helps us to meet the expanding computing requirements of the University whilst reducing our energy bill and Carbon Footprint, both of which are very important targets for the University.” Read the full University of Nottingham Case Study Ref 5: One aisle of containment showing sliding door and bespoke side panels to create an airtight aisle. Ref 6: Bespoke doors to allow access to contained aisle. Ref 7: Another view of bespoke doors and metalwork to create the aisle containment chamber. 7 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use b. Case Study 2: DIY Chain (Cold Aisle Containment Example) AIT recommended and installed cold aisle containment in the data centre of Europe’s leading DIY chain. The effect was instant, a 4⁰C drop in temperature within the cold aisle. Once balancing and reconfiguration of the CRACs took place it was estimated that the customer would save nearly £40,000 per year, paying for the project in very little time. Read the full DIY Chain Case Study Ref 8: Data Centre hall with aisles on either side. Each aisle has cold aisle containment with a sliding door on each end. Ref 9: Inside each aisle, with the server rack fronts on both sides and cold aisle containment roof overhead Ref 10: On odd shaped aisle with in-filling panels on one side to create an airtight space. These can be removed and replaced with cabinets as the data centre expands. 8 White Paper: Using Cold Aisle Containment in the Data Centre October 14, 2013 Retrofitting cold aisle containment to increase capacity and reduce energy use 7. References Focused Cooling Using Cold Aisle Containment (Emerson) http://www.emersonnetworkpower.com/documents/enus/brands/liebert/documents/white%20papers/focused%20cooling%20using%20cold%20aisl e%20containment.pdf Hot Aisle vs Cold Aisle Containment (APC) http://www.apcmedia.com/salestools/DBOY-7EDLE8_R0_EN.pdf Cold Aisle Containment for Improved Data Center Cooling Efficiency (Rittal White Paper 506) http://www.rittal.us/literature/download.cfm?id=554&sbu=05 AIT Hot and Cold Aisle Containment http://www.ait-pg.co.uk/data-centre-management/data-centre-design-and-build/hot-coldaisle-containment/ AIT Partnership Group Ltd 1 Southern Court South Street Reading RG1 4QS 0845 293 2970 www.ait-pg.co.uk All rights reserved. The AIT Logo and name is a registered trademark of the AIT Partnership Group Ltd. All other company or product names may be trademarks or registered trademarks of their respective owners. Whilst every effort has been made to ensure accuracy, neither the author nor AIT takes responsibility for the information given herein. 9