Network Connectivity Care and Feeding of DC Power Plant and UPS Batteries www.phoenixbroadband.com MasterAgent Site Controller BatteryAgent Sensor Units Up To 6 Battery Strings RJ-11 “Daisy Chain” Battery String #1 Battery String #2 Overview • Present and future of battery backup • Battery theory • Battery failure causes • Load vs AC characteristics testing • Current preventative maintenance practices • Remote monitoring www.phoenixbroadband.com The Need for Batteries… Ancient technology, bright future • The market for batteries is growing, not shrinking: – Stationary batteries are >$3 billion/yr business; expected to be >$7 billion by 2010 (BCC Research Group) • • • • • Telecom, cell sites, cable headends Industrial, IT infrastructure Automotive & Hybrid vehicles, Alternative energy systems: Wind, solar, fuel-cell, etc, all need batteries for storage www.phoenixbroadband.com Batteries in Broadband • Legacy headend UPS power plants: – Develops backup source of single-phase or 3-phase AC – Typically up to 40 12V batteries in series (~500VDC) • Modern 48VDC headend power plants: – Newer equipment designed to operate from 48VDC – Typically 24 2-volt batteries in series • Outside plant standby power – Another story for another day www.phoenixbroadband.com Anatomy of a UPS Standby Mode Normal Mode AC IN Rectifier Rectifier Rectified AC input powers inverter and charges batteries DC Inverter Inverter AC OUT AC OUT Charge Current Batteries power inverter Batteries Batteries www.phoenixbroadband.com Some Battery Terminology • • • “Cell” – Simple form of energy storage device typically comprised of positive and negative plates, separators, electrolyte and a container. – This device can be placed in series with other cells to form a monobloc or battery – Lead-acid cells are typically about 2.1 vdc “Monobloc” (sometimes called a module) – A number of cells connected (typically in series) and packaged together a single container – What is commonly thought of as a 12vdc battery can also be thought of as a 6-cell monobloc “Battery” – Combination of “monobloc” modules placed in series or parallel, the total of which forms a battery www.phoenixbroadband.com The Lead-Acid Battery What’s in the box? www.phoenixbroadband.com Lead-Acid Battery Types Many sizes & shapes… • “Flooded” or “Wet” Cells – The cell plates (commonly a lead alloy)are suspended in a bath of liquid electrolyte (typically sulphuric acid) • “Gel” Cells – The liquid electrolyte is replaced with a thick gel electrolyte • “AGM” (Absorbed Glass Mat) Cells – The space between plates is filled with a mat-like material that holds liquid electrolyte • Gel and AGM are sealed-cell technologies – Maintenance free – Sometimes called VRLA (Valve Regulated Lead-Acid) www.phoenixbroadband.com Capacity Metrics • “Amp-Hours” (AH) – A constant that describes how long a cell can supply a specified amount of current before reaching its “end voltage”. This is the most common capacity metric. • “Cold Cranking Amps” (CCA) – The number of amps a new, fully charged battery can deliver at 0°F for 30 seconds, while maintaining a voltage of at least 7.2 volts, for a 12 volt battery. Used by the automotive industry, • “MCA & “CA” (Marine Cranking Amps/Cranking Amps) – The load in amperes which a battery at 32°F , can continuously deliver for 30 seconds and maintain a terminal voltage equal or greater than 1.2 volts per cell. Used by the marine industry www.phoenixbroadband.com Capacity Limitations Why it’s not a perfect voltage source • An ideal cell would have unlimited capacity. • Capacity is limited by non-ideal internal elements – Rmetal is a very low resistance comprised of strap, post, plate & electrolyte resistances – Relectrolyte is known as charge transfer resistance or contact resistance between plate and electrolyte – Rleakage is a very high resistance that causes self-discharge – C is the battery’s inherent capacitance which is about 1.5 farads per 100 AH capacity www.phoenixbroadband.com • As batteries age they lose some ability to deliver power • According to IEEE 450 “2002” when a battery has lost 20% of its capacity it is no longer viable Discharge Behavior • Initial “Coup de Fouet” – Sudden deep drop, then some recovery over several seconds • Linear voltage decay until “cutoff voltage” is reached. • Fast voltage drop after cutoff time • Deep discharge is bad • Excessive discharge rate is bad • The discharge rate must be kept within manufacturer’s ratings www.phoenixbroadband.com Charging Considerations • Ideal charger has 3 states: – Bulk: Constant current quick charge ‘till voltage rises – Absorption: Constant voltage ‘till current drops – Float: Low-current maintenance charge • Excessive charge current causes heat and “gassing” • Overcharging causes dry-out • Undercharging leads to sulphation • Charge rate and voltage are temperature dependent www.phoenixbroadband.com Why Batteries Fail “Treat them kindly” • Heat: – For every additional 15 degrees of heat over 77 deg F, lead acid battery life (regardless of type) is cut in half. • Overcharging: – Overcharging causes heat and ‘gassing’ – not good. • Undercharging: – Leads to sulphation of plates • Deep-discharging: – The first time a lead-acid cell is discharged by 80%, its life expectancy is halved • Mechanical Deterioration – Corrosion of straps & posts, sulphation of grids Field studies have shown VRLA batteries last approximately 3-8 years if treated properly www.phoenixbroadband.com The Battery Management Conundrum • Stationary batteries are expensive • Batteries need regular checking and maintenance to achieve their rated life • Operators are being driven to increase system availability while reducing maintenance costs • When budgets are cut, maintenance is the first to go www.phoenixbroadband.com Costs Availability Preventative Maintenance Practices “No Maintenance” • The most common practice • Batteries are replaced when they fail • The most costly practice • Power failures result in downtime & loss of revenue • The least cost-effective practice • Lack of vigilance can result in undetected deterioration • Lack of maintenance can result in catastrophic failure • A genuine job-security threat • A headend outage can be a career-ender www.phoenixbroadband.com Preventative Maintenance Practices “Rip n’ Tear” • Time based replacement – Based on projected 3-8 year life expectancy – Commonly called the “rip n’ tear” approach • The problem with rip n’ tear: – Replacement too early is costly and inefficient – Waiting too long to replace will cause loss of services & revenue • TB replacement is gambling!! www.phoenixbroadband.com Preventative Maintenance Practices “Periodic Maintenance” • Requires regular site visits – Quarterly • • • • • Visual inspection AC Characteristic measurement Voltages/Current Corrective action Manual data logging – Annual/ Semi/Tri annual (2-3 yrs) • Capacity testing (load) • Other similar to quarterly www.phoenixbroadband.com Battery Test Methods DC Load Testing • Designed to test battery capacity in amp-hours – A heavy load is placed on the battery and the time to discharge to the end-voltage is measured • Manual and intrusive testing – Any discharge event is a potential outage-causing event. • Expensive and time consuming – Requires special equipment and personnel • Should not be performed within 72 hours of a discharge event www.phoenixbroadband.com More on DC Load Testing … • IEEE recommends Time adjusted or Rate adjusted testing for capacity testing – Accomplished by taking batteries off line and testing with a constant load to specified terminal voltage – Time adjusted: Greater than one hour; no correction, except temperature – Rate adjusted: Less than one hour and requires the battery’s published specs and corrections for time and temperature www.phoenixbroadband.com Manual Test Methods ”AC Characteristics” Testing C1 www.phoenixbroadband.com C1 R1 R2 Remote vs Manual 3200 3000 Conductance • Commonly known as “impedance” or “conductance” testing • Characterizes the cell’s internal resistances • Non intrusive measurement (manual or remote) • Designed to provide battery “State of Health” (SOH) information • Can be performed with handheld instruments or via a remote monitoring system • Generally accepted as the best SOH test method 2800 Remote Manual 2600 2400 2200 2000 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 two day interval AC Characteristics Testing How it works… • Forces a known AC current through the battery terminals AC Current Source (Iac) AC Voltage Amplifier Gain = A – Causing a small AC voltage to be developed • AC signal can easily be separated from large DC component • AC voltage is amplified and measured Vac AC Voltage Meter (Vac) 1.0 amp – Rb = Vac/Iac – Example: If Iac = 1amp and Vac = 0.001volt, then Rb = 0.001 ohm Vcell Rb = R1 + R2 C1 C1 R1 R2 www.phoenixbroadband.com Current Preventative Maintenance Practices Manual Methods Summary • Manual testing is expensive & some tests are intrusive • Data logged manually and transferred to software program (MS excel) manually. • Quarterly tests do not provide enough data for meaningful trending analysis (and so will miss impending failures) • Provides a good opportunity for visual inspection www.phoenixbroadband.com The Need for Monitoring… • Mission-critical infrastructure elements must be monitored, maintained, and managed. • Major outages can be apocalyptic. www.phoenixbroadband.com Remote Battery Monitoring • Much more comprehensive means of looking at battery state of health • Provides operators with instant status update on entire enterprise • Reduces/eliminates unnecessary PM site visits • Provides asset management & inventory control • A more intelligent and cost effective means of determining battery replacement www.phoenixbroadband.com Remote vs Manual “The devil is in the details” Remote vs Manual 3200 Conductance 3000 2800 Remote Manual 2600 2400 2200 2000 1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 two day interval www.phoenixbroadband.com Remote Monitoring Legacy “Too much data – not enough information” • Cumbersome – Slow serial based communications – Alarm storms – Poor correlation and analysis capability • Expensive • Proprietary in nature • Complex installation and maintenance www.phoenixbroadband.com Remote Monitoring Today “User Friendly – Standards-based” • New technology has dramatically reduced cost & complexity • Standards based systems (HTML, TCP/IP, SNMP) • Intelligent reporting • Provides real time status of hundreds or thousands of battery plants instantaneously • Trend analysis and correlation • Information available to many vs few www.phoenixbroadband.com Web based Clients Battery Monitoring System Architecture Network Connectivity • • • • Monitoring systems are typically comprised of sensors, site controllers, and software Sensors make impedance measurements Communications between controller and NOC software uses SNMP Clients can use a browser to access the NOC software, or directly access the site controller. MasterAgent Site Controller BatteryAgent Sensor Units Up To 6 Battery Strings RJ-11 “Daisy Chain” Battery String #1 www.phoenixbroadband.com Battery String #2 The Battery Sensor • Connects to battery post • Measures battery temperature, voltage, & impedance • Low current: <10ma idling; typ 1 amp during test • Each sensor is addressed by the site controller • Site controller determines when tests are made and collects data www.phoenixbroadband.com The Site Controller • Manages multiple strings of sensors • Can be powered from battery string or from wall-transformer • Communicates with NOC via Ethernet • Sends alarm traps if any measurement is abnormal • Built-in web page • Built-in email client • Fully SNMP compliant www.phoenixbroadband.com Built-in Web Server • Site summary page with alarm color coding • String summary page with alarm color coding • Battery details page with individual battery real-time measurements • Complete provisioning of text labels and alarm thresholds via web – password protected • Provisioning can also be done via SNMP from NOC www.phoenixbroadband.com Benefits of remote monitoring “Continuous testing; It just makes sense” • Remote monitoring is the best way to determine comprehensive state of battery health • Real time visibility of enterprise DC power plants • Reduced maintenance costs (fewer site visits) • Fewer outages • More efficient use of resources during crises • Proactive vs reactive maintenance • Asset management/inventory control • Enterprise wide accessibility Costs Availability www.phoenixbroadband.com Benefits of remote monitoring “A more rational approach” • Eliminates the need for manual data logging and analysis • Eliminates data overload – provides useful information • Provides historical data for warranty claims • Consistent measurements (eliminates human errors) • Alarm notification and routing • Eliminates site access problems (manpower/security) www.phoenixbroadband.com Summary • • • • • Batteries are a growth industry, not a dying technology As batteries age they will fail to deliver expected run time Manual testing has proven to be at best only partially effective Remote monitoring combined with yearly inspection offers the most comprehensive and effective method for assessing battery replacement Remote monitoring will allow operators to be proactive thereby reducing the number of system outages and realizing significant savings in battery replacement www.phoenixbroadband.com