Minimizing the Risk of Battery Failure in Data Centers
A 2010 National survey on data center outages conducted by the Ponemon Institute found that 65% of outages were directly attributed to UPS and battery failure.
After working in the critical power industry for 18 years, there is one thing I can tell you for sure. The industry heavily relies on stationary battery systems to make it all work, and if you’re not paying due respect to battery maintenance, you may pay dearly for it. Data centers can spend hundreds of thousands of dollars creating highly reliable power delivery systems. These systems, at various locations of the design, only work if the batteries perform their intended function. Batteries would be considered the most “low tech” device in this complex scheme but also one of the most important. Today’s serious data centers employ technologies to virtually autonomously monitor, test, and evaluate battery systems. This data needs to be scrutinized and evaluated constantly to maintain properly operating battery plants. Let’s take a look at some common things we analyze to minimize the risk of battery failure.
Valve Regulated Lead Acid (VRLA) batteries are the most commonly used battery in data centers around the world and for good reason. The energy density is hard to match, and they can be installed in virtually any physical orientation. VRLAs are found mostly in mission critical components such as UPS systems and used to start engine generators. Other very important uses include breaker tripping and process controls like maintaining power and logic signals to Programmable Logic Controller (PLC) in the event utility voltage fails.
These batteries can fail the day after they pass testing so consistent maintenance at regular intervals is extremely important. Manufacturers market this type of battery as a maintenance free 10 year battery, but nothing could be further from the truth. They require constant attention. In over 18 years, including thousands of inspections, discharge tests, and replacements, I can count on one hand how many battery systems survived the harsh reality. The main reason is batteries operate in the real world and do not reside in the perfect environment of a test lab. Typical life cycle of the VRLA battery in a UPS system for example is 3-5 years based on my experience and Engine Generators (EG) start batteries should be replaced every 3 years.
The Institute of Electrical and Electronics Engineers (IEEE) states that the useful life of a battery ends when it can no longer supply 80% of its rated current in amp hours. This is difficult to determine without a live discharge test and this type of testing, albeit the most accurate form, can be considerably expensive and time consuming. So what can you do instead? There is an alternative method of testing batteries called ohmic, impedance, or conductance testing that is not only cheaper but can be done with much less time and is relatively accurate due to the correlation between battery capacity and internal resistance. This type of testing is normally done on a quarterly or monthly basis. It relies on measuring the internal resistance of the battery and comparing it to a known baseline of a brand new battery of the same type. A 30-40% departure from this baseline is a solid indicator that a closer look is warranted. Analyzing these results can be tricky, but the purpose of this testing goes beyond immediate results. Collecting quarterly and monthly data allows facilities to make well informed judgements about the battery system condition and more accurately predict service life. But if you stop here you could be rolling the dice. The combination of physical battery system maintenance and a 24/7 battery monitoring system is always recommended as batteries can fail at any time. A monitoring system provides the advantage of situational awareness. Knowing the plant conditions 4 times a year is just not good enough in this industry. Detecting problems as early as possible is one of the best ways to prevent premature battery system failures.
Most premature failures can be prevented and that is a fact. Although battery plates degrade over time there are some simple things we can look at to ensure reliability such as daily physical inspections, proper charge voltage (typically 13.5-13.8VDC @77F) and proper ambient temperature environment (typically 77F). Heat is the #1 enemy of the VRLA battery! Other things to consider are AC ripple voltage and the number of battery cycles. Yes, I said AC voltage! Commercial power is AC and has to be rectified to DC to charge batteries. Without a properly operating filter in place which usually consists of DC capacitors, rectifier diodes, and a voltage regulation circuit, AC voltage can pass through the circuit resulting in the presence of AC current and consequently increased internal heating of the batteries. A good rule of thumb is no more than 5 amps per 100AH of battery capacity.
A high battery cycle count could be the result of an improperly calibrated UPS system or it could be as simple as an unreliable utility source. Very little you can do about the latter; however, the UPS input tolerance windows are normally adjustable and should be adjusted based on your average year round utility source voltage, not the nominal source voltage!
Batteries will always consume a small current with a constant voltage source, but how much current is acceptable is beyond the scope of this article. Systems consuming high current could be another indicator of a problem because, as mentioned earlier, there is a correlation between capacity and resistance of a battery. Ohms law states that I=E/R or current equal’s voltage divided by resistance. If voltage remains constant and current goes up, what changed? The answer is simple, the resistance of the battery.
As you can see the simplest technology that we rely on so heavily is not so simple after all, and we barely scratched the surface. There are no guarantees your batteries will perform when called upon even with the most expensive test equipment and monitoring systems. That’s why battery maintenance is one item that should always be taken seriously and one of the last items you should consider cutting out of the budget. When you consider the cost of down time, which in some cases can be tens of thousands of dollars per second, it is easy to see why battery maintenance is so critical and why you can rest assured that data centers, like New Continuum Data Centers, is on top of it.