Utility
advertisement
DC Power Distribution Alternatives for Datacom Applications BJ Sonnenberg Manager Business Development Data Center Trends Increasing rack power density Increasing drive for efficiency Increasing component integration on site ` AC Power >40kW New High Efficiency Topologies DC Power Future Higher Voltage Topologies New 48V DC Topologies < 2 kW Traditional Telecom 48V DC AC UPS Powered Configurations Forces re-evaluation of all available topologies 2 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Modularity & Scalability Data Center Power Protection and Distribution Basic Architectures Traditional AC UPS Maintenance Bypass PDU B PSU CDU DC/DC B UPS Bypass 480V MV Primary Dist. AC AC/DC DC/AC B 12V VR B B B B AC/DC Server Rack 300-400VDC 48V/380DC Modular System, Row-based DC UPS 480V CDU PSU AC/DC AC DC/DC B MV VR B Rectifier 12V Server PDU Rack Attributes of a DC UPS: Simple input and output distribution, easy to parallel ,no load balancing (derating) Minimal conversion stages grid to chip – high end-to-end efficiency and reliability High power quality - isolated from mains, no harmonics Safe voltages at point of use (48VDC systems) 3 Advantages of integrated, modular UPS systems: High power density in a compact footprint Reduced field wiring and copper content Targeted for row-based deployments; no need for dedicated power room System optimization challenge – best overall system topology for a specific application , not necessarily optimization of individual existing components. Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Source: Intel Improved Efficiency with DC Solutions Elimination of conversion stages and simplified distribution 4 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Megatrends in Data Center Distribution Evolution Renewable resources Microgrids 5 Facility distribution optimization Rack/server distribution optimization • Decrease utility power consumption – peak shaving and over time. Address all loads , not only compute loads (lighting , cooling , etc) Server virtualization • Maintain high availability • Best system utilization – rightsizing , no stranded power • Scalability • TCO Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Impact of Renewable Energy Sources - Example of Telecom Facility Transformation Vision Present Facility AC loads Utility Gen AC Batt DC UPS Ultimate AC compute loads AC UPS DC Facility AC loads Telco and dc compute loads Batt 48VDC Facility DC loads Many transition paths Utility Interim Facility AC loads AC AC compute Legacy loads Utility DC UPS Gen Batt 400VDC DC compute And telco loads DC Batt DC compute And telco loads Green Generator Facility DC loads DC DC Energy Station DC 48VDCTelco Legacy loads 6 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 400VDC DC facilitates and optimizes use of renewable resources Elements of an Optimized Power Distribution –Grid to Chip BEST COMPROMISE OF THE FOLLOWING Efficient facility distribution to equipment racks Ability to easily connect renewable energy resources Efficient equipment rack distribution Efficient server PCB distribution – optimize conversion stages Eliminate stranded power Operate at peak efficiency at any load Scalability Low TCO Low initial cost Reliability 7 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Why 380VDC - Voltage Range Selection Based on Several Criteria 1500 IEC High Long DC Voltage 800 Server PS 750 Ordinance (JPN) 600 NEC(U.S.) 600 450 575 [to 48] 428 (192cell) Validus 405 374 (168cell) 380 380 321 (144cell) US DC Demo 373(U.K.) 350 320 300 300 NEDO PJ. (Sendai) Cable Distance 900 BS (U.K.) Distribution efficiency 1000 420 380 354(EU) 324V (US 311(JPN) Operating bulk - voltage 260 200 Low 0 8 Law, Regulation, Code, and Stds. Distr. Gen. Telecom (Number of Cells) Demos (Reference) Rating voltage of parts and elements AC ICT input voltages ( Peak ) Edited from source: NTT FACILITIES, INC. Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Short Benefit of HVDC system ETSI Std. (draft) Which DC Voltage? Depends on facility size/application : 1. Scalable facility level distribution Sites/PODs requiring battery back-up - -48VDC “in row” solutions Facility/application with renewable energy availability – 380VDC – simpler interface 2. Server rack level distribution -48VDC or 380VDC for racks with power densities > 10kW 9 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Power Room Constraint As rack densities increase, power room size increases proportionally; i.e. a 10x increase in density results in a 10x increase in power room size. Power and Switchgear Room 10 No UPS Room, Increased computing space UPS / Battery Room Data Center Raised Floor Switchgear Row-based UPS Data Center Raised Floor High-Density Data Center @ 20kW per Rack Switchgear Room-based UPS Data Center @ 2kW per Rack Additional Data Center Raised Floor Space Data Center Raised Floor Row-based UPS architecture offers ~30% more computing capacity Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 380V DC Application Alternatives Facility level distribution Server rack distribution 11 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 380VDC System Components CONVERSION Communications System cabinet Rectifier FACILITY DISTRIBUTION Bus duct, ancillary switchgear RACK DISTRIBUTION Power strips connectors to servers Controller Storage , battery flywheel ,etc 12 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Power supply DC Rectifier and System Design Philosophy Modular approach selected for scalability and reliability Maintain same reliability and maintainability as 48VDC systems Provide input/output isolation to avoid protection coordination issues Minimize footprint – space savings High scalability and ease of expansion ( from 12.5kW up ) Efficiency gains through conversion improvement and facility design Ready to accept renewable energy sources Offer multiple ratings by use of standard building blocks and conversion cabinet configurations and a set of input/output interface accessories 13 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Advantage of High Modularity – Improved Efficiency with Energy Optimization Mode Active power management increases overall system efficiency Unneeded power conversion units (PCU’s) set to standby in low-load conditions Optimizes individual PCU load for greater system efficiency Useful for sites with variances in load Periodically cycles energized PCU’s for even run time 33% 33% 56% 56% 33% 33% 56% 56% 33% 33% Standby Standby 33% 33% 56% 56% 33% 33% Standby Standby 33% 33% Standby 56% Without Energy Optimization N+1 redundancy maintained With Energy Optimization 14 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 1U high 10KW Rectifier Feature Input voltage windows 260 to 530VAC Output power 10KW Power factor 0.99 THD <5% Output voltage 297 to 400V Max output current 30A Efficiency 95% Safety UL/CE 15 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Emerson’s 380V V0 Solution (Present) Feature Input 480VAC with 3 phase 4 wire Output 297 to 400V Provides up to 100KW in each rack Load insulation detection function Max 7 output load connection with load measurement Over load and on/off detection function Standard server rack 600(w)*1000(D)*2000(H) :mm 16 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Future Offer -System Sizing and Configuration Considerations Rectifiers space in cabinet (cooling,safety) DC output configurations – bus duct vs hard wire/Tier level AC input configurations – Tier level Maintainability level – facility level and DC UPS system Components rating limitation (today) Footprint optimization Ease of installation/expansion Tier 1/2/3 – N+M configuration with multiple inputs/outputs Tier 4 configuration – 2N 17 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 400VDC System Building Block Implementation Example 480VAC Battery or Loads A Loads B flywheel Include provision for internal bussing 1000A CB Current / Voltage Monitor 1000A CB C Draw-out or Plug-in Breakers Preferred Fused Disconnects acceptable 20 -22 Rectifiers 250kW N+1 400A CB 480VAC Distr. A Distr. B 40 MCCBs 40 MCCBs 8 brks up to 125A 8 brks up to 125A Individually monitored option Individually monitored option 400A CB Battery or Loads A Loads B flywheel Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 18 1. Any part of the system can be maintained without system shutdown 2. Distribution breakers can be added and wired without system shutdown 3. Top and bottom entry provided 4. All external wiring run in conduits – sufficient space to be provided for conduit terminations 5. Failure of either bus A or B should not affect the other bus 6. For single corded loads only one distribution is required 7. Rectifiers can be added or replaced without system shutdown 8. System can operate without battery 9. Individual Rectifier failure does not affect the output bus 10. All cabinet connections must be with conduit 11. UL / CSA / CE approved C Controller Output Connectivity Options –Bus Duct Advantage Bus duct Wire in conduit Distribution options: A Battery Flywheel 1.Fuses – shorter clearing time 2.Breakers – easier to operate 3.Bus duct plug-ins – space,scalability Server Rack 400VDC UPS Bus duct DistrA 400VDC UPS DistrB 800A CB limit 230kW A Battery Flywheel B Bus duct B Server Rack 19 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. A Battery Flywheel 400VDC UPS B Server Rack AC Input Configurations – Example of 250kW System Scalable to 750kW Utility Utility A OR 1200A 250kW 250kW 400A 250kW 250kW Utility 400A 250kW Utility 400A 250kW Utility Utility Utility Utility Utility Utility A B B A B A Utility Utility A B OR 400A 1200A 250kW TIER 2 250kW 250kW 250kW 400A 250kW 20 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 400A 250kW TIER 3 Partial 2N Concept Server Room or Electrical Room Server Room or Electrical Room Server Room 400VDC UPS N+1 Battery Flywheel Optional B A Battery Flywheel Future Battery Flywheel 400VDC UPS n=+1 400VDC UPS N+1 400VDC UPS Red. N+1 400VDC UPS Future N+1 21 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Server Rack Critical Server Rack Non critical Server Rack Future Facility Loads Server Rack Distribution Feed A Feed B 12- 40 servers Feed A Feed B Power strip 50A input ~14.5kW 12 positions Power strip 50A input ~14.5kW 12 positions Single strip can power up to 12 2U servers at 1200W – A or A+B or 24 1U servers at 600W Single strip can power up to 2 blade chassis at 14.5kW – A or A+B For A only configuration up to 20 2U servers,40 1U servers (two strips) – total rack power available – 29kW For A only configuration up to 4 blade chassis (two strips) – total rack power available – 29kW Blade chassis example HPc7000 today 3x2700W power supplies ~8kW max Typical load- 4-6kW Connector UL listed at 5A or 20A End of discharge voltage ~ 290VDC 22 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 380V DC Application Alternatives Facility level distribution Server rack level distribution 23 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 380VDC Sub-Rack Sun Container–UCSD POC Site 2 X 10kW 400VDC Controller Output Breakers 400V-48V Converters 24 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 48VDC or 380VDC Sub-Rack Concept Horizontal shelf mounting shown . Shelves can also be mounted vertically Building blocks: • 12.5-15kW 380VDC • 6000W 48VDC 25 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Rack Distribution Copper Content Assumptions: 1. 92 servers/rack at 275W-300W 2. Two power systems/rack – 13.5kW each, n+1 3. Rack power ~ 27kW 4. Currents calculated at end of battery discharge – 1.75VDC 26 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Rack Layout -Optimized 11x300W 11x300W servers 3375W servers 3375W Cross bar – 20” 12V – 625A @1.75V 48V - 156A @1.75 V 380V – 23A @1.75V 30A,480V, AC,3phase input plug 13.5kW 11x300W 11x300W servers 3375W servers 3375W 11x300W 11x300W servers 3375W servers 3375W 13.5kW 11x300W 11x300W servers 3375W servers 3375W Distribution bar – 47.25” 12V – 312 A 48V – 78A 380V – 12A Server plug 12V - 25A 48V – 6.25A 380V - ? (5A plug standard) 27 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Rack Layout –Full Bus Sizes 11x300W 11x300W servers 3375W servers 3375W Cross bar – 20” 12V –1250A @1.75V 48V - 312A @1.75 V 380V – 46A @1.75V 30A,480V, AC,3phase input plug 13.5kW 11x300W 11x300W servers 3375W servers 3375W 11x300W 11x300W servers 3375W servers 3375W 13.5kW 11x300W 11x300W servers 3375W servers 3375W Distribution bar – 47.25” 12V – 614A 48V – 156A 380V – 24A Server plug 12V - 25A 48V – 6.25A 380V - ? (5A plug standard) 28 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Back-up 29 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Example of Hybrid System Evolution in Enterprise Building Existing System Meter 100kW 350-800V Solar Array 480Vac 480Vac PV Inverter Eff 93% AC Distr. Utility 480Vac Load larger than PV output .Solar power fed back to utility. 480/208V PDU Lighting 208/120VA servers Solar to server eff = .93x.98x.92= .838 Utility to server eff = .98x.92=.90 Lighting and distribution losses not included 30 Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Eff – 98%? PS eff – 92%? Hybrid System Migration -Scenario 1 Meter 100kW 350-800V Solar Array 480Vac 480Vac PV Inverter AC Distr. Utility 480Vac NetSure ITM 48V DC UPS 380VDC Converter Already existing on site 48VDC Loads Solar to server eff = .93x.97x.95=.856 Utility to server eff = .97x.95=.92 Lighting and distribution losses not included 31 380VDC Loads Converter to 24VDC Lighting 24VDC Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Existing Emerson products PS eff – 95%? Hybrid System Migration -Scenario 2 100kW 300-400V Solar Array Replaces Existing PV Inverter DC/DC Regulator 380VDC output Eff- 97% 380VDC Converter No battery 480Vac AC Distr. Eff- 97% NetSure ITM With Battery 380VDC distr. backbone (already considered for buildings/residential applications) Converter to 24VDC Lighting 24VDC Utility 380VDC Loads 380-48VDC Converter PS eff – 95%? 48VDC Loads Vehicle charge station Appliances 32 Estimated efficiencies – detailed study in progress Solar to server eff = .97x.95 = .92 Utility to server eff = .97x.95 = .92 Lighting and distribution losses not included Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. 400VDC System Voltages Cell type VRLA -6 V block Wet cells Single cell Number of cells Number of blocks Nominal VDC Operating VDC @2.25VDC/cell 144 150 156 162 168 174 180 186 24 25 26 27 28 29 30 31 288 300 312 324 336 348 360 372 324.00 337.50 351.00 364.50 378.00 391.50 405.00 418.50 equalize 162 324 364.50 163 326 366.75 164 328 369.00 165 330 371.25 166 332 373.50 167 334 375.75 168 336 378.00 169 338 380.25 170 340 382.50 171 342 384.75 172 344 387.00 173 346 389.25 174 348 391.50 175 350 393.75 176 352 396.00 177 354 398.25 178 356 400.50 179 358 402.75 180 360 405.00 To optimize battery applications the voltage range should be: Nominal Min Max Typical operating voltage 33 Operating VDC @2.17VDC/cell 351.54 353.71 355.88 358.05 360.22 362.39 364.56 366.73 368.90 371.07 373.24 375.41 377.58 379.75 381.92 384.09 386.26 388.43 390.60 336VDC-352VDC 294VDC 400VDC 378VDC VRLA 386VDC Flooded cells Not counting cable voltage drop. Emerson Network Power, Energy Systems North America, Inc.; Proprietary and Confidential Information. Min VDC @ 1.75VDC/cell end of discharge 252.00 262.50 273.00 283.50 294.00 304.50 315.00 325.50 283.50 285.25 287.00 288.75 290.50 292.25 294.00 295.75 297.50 299.25 301.00 302.75 304.50 306.25 308.00 309.75 311.50 313.25 315.00 48VDC multiples 6.00 6.25 6.50 6.75 7.00 7.25 7.50 7.75 Outside ETSI spec
