If LANs and PCs are the unsung heroes of modern business, then the uninterruptible power supply (UPS) is the "forgotten child" of most LANsóat least until there is a power problem! Unlike most other components of a LAN, the majority of UPSs spend 99% of their time waiting to do their job. Even tape backup units get at least some attention on a daily basis. When was the last time you did anything with your UPS?
UPSs used to be so undervalued that in the early days of LANs, many servers were installed without a UPS. It normally did not take very long before the value and importance of a good UPS system became veryósometimes painfullyóclear. This chapter discusses why UPSs are essential to any LAN, and explains how to live with them.
All computers are electrical devices, which means that they need electrical power to operate. Fortunately there is plenty of electrical power availableóAC outlets are everywhereóso this seems not to present a serious problem. While all electrical outlets look the same, however, the electrical power coming out of them is not the same, and a computer's power needs are different from most other electrical items' power needs. Computers are much more complicated than light bulbs, appliances, and even audio/visual equipment; therefore, the kind of electrical power that would be sufficient for those devices might not be appropriate for computers. While a brief power outage might cause only a barely perceptible flicker of a light bulb or radio, it could cause a catastrophic change to the data in a computer that's reindexing a database.
Unfortunately, the "AC" in "AC power" stands for "Alternating Current" and not for "Always Constant." What comes out of a typical AC outlet is supposed to be a nominal 110 volts. "Nominal" means "like" or "near" and this happens most of the time, which can cause problems for a computer that needs to be running all of the time. In a perfect world, every AC outlet would provide a constant 110 volts of AC powerówhat actually happens is quite different. A study by IBM has shown that a typical computer is subject to more than 120 power problems per month. Bell Laboratories has determined that the frequency of disturbances is
Disturbance Type Frequency
Sags 87%
Spikes 7.4%
Blackouts 4.7%
Surges 0.7%
Deltec, another UPS vendor, states in their literature, "A recent power study showed that in a one-year period, the average data center incurred 36 spikes, 264 sags, 128 surges, and 15 blackouts."
The following sections explain what these unwanted electrical events are, what causes them, and what happens to a computer when they occur.
A sag or brownout is a temporary drop in the voltage reaching a computer. Sags are caused by a big motor (like an elevator, compressor, or large shop tool) starting up, or by the electrical company reducing the voltage supplied to particular locations during periods of high demand.
I had one LAN running happily in an office environment until it began having electrical problems every day from roughly 6:30 to 7:00 AM. It turned out that the space next door to the office had been vacated, and a work crew was starting a compressor every morning while refurbishing the space. All the workers had to do to solve our office's problem was to plug their compressor into a different circuit.
Another time, I moved a LAN into a new office and immediately noticed electrical problems each morning and afternoon. It turned out that the wiring for the LAN room was on the same circuit as the elevators, and when all the elevators were heavily used (mornings and afternoons) the power dropped on that circuit. Again, switching circuits solved the problem easily.
In both situations, the UPS alarm systems were the main indicator that something was wrong with the electrical system. At the same time, however, the incidence of system hangs, frozen keyboards, and other problems increased significantly. Without UPS alarms telling us about the electrical problems, who knows how long I might have spent checking out configurations, memory chips, keyboards, and the like before I figured out that it was a sag problem?
A spike is an instantaneous high-voltage situation. These conditions typically occur when there is a lightning strike nearby or some kind of accident at a power pole or power station. A car crashing into a power pole can cause headaches for others besides the driver!
One of the LANs I took care of was in the Atlanta area, an area of the country subject to frequent lightning storms. The UPSs on the file servers were protecting that equipment, but we were losing motherboards and monitors on the workstations at a horrific rate. We finally solved the problem by putting a UPS on each workstation. The $150 cost per workstation was more than offset by reduced maintenance costs and worker downtime.
A blackout is a total loss of power. It can last anywhere from a split second to many days. A blackout can be caused by nature (fire, earthquake, lightning, and so on) man (backhoe, auto accident involving power pole, and so on) or equipment failure (blown circuit breaker, power station failure, power grid overload, and so on).
Most PCs can handle a voltage loss of 1/20th of a second (50 milliseconds), but anything longer than that and they usually reboot themselvesóno matter what they were doing at the time. Anyone who has watched the lights go out and then heard anguished cries of "My database!" will understand how heart-wrenching a blackout can be. Anyone who has been responsible for rebuilding a corporate database after the file server has irretrievably scrambled the data will instantly be a convert to the religion of reliable tape backups.
A surge or overvoltage is a high voltage situation that lasts for more than 1/120th of a second. These kinds of situations are typically caused when a large power-consuming device is turned off, spreading the excess voltage momentarily to other devices attached to the circuit. While not as instantly damaging as a spike, a surge eventually takes as much a toll on the equipment.
The "alternating" in "alternating current" means that the voltage alternates from one level to another. It does this in a smooth, sweeping motion called a sine wave. Computer equipment expects this nice, smooth transition and anything that disrupts the sine wave of the power can disrupt the operations of the computer. There are two primary kinds of noise that can alter the smooth sine wave, electromagnetic interference (EMI) and radio frequency interference (RFI). Since power cables travel long distances, they act like antennas and can pick up noise from a variety of sources: fluorescent lighting fixtures, generators, lightning, radio transmitters, and the like.
The results of noise on the line are varied and unpredictable, since the shape of the sine wave gets changed in unpredictable ways. Noise is comparable to having mini-spikes and mini-brownouts occurring at the same time.
By now it should be apparent that depending solely upon consistent power from your common AC outlet is foolhardy. Even if you have a dedicated circuit (one with no other equipment on it), a car might crash into a pole outside your building and ruin everything for you. Wouldn't it be nice if you could avoid all these problems and have dependable AC power all the timeóor at least enough time to shut down the system properly when there is a power problem, to avoid the catastrophic damage of a system crash? If you take the proper safety precautions, it's more possible than you think.
The first thing to do is to make sure you have proper wiring in your building. Then you need to determine how "clean" your power is. Perfectly clean power would be a constant 110 volts without ANY fluctuations. Since this is impossible in the real world, the question you need to answer is, "Just how 'dirty' is my power?." By doing thisóand documenting everything you find outóyou will have a known baseline from which to diagnose future problems.
Much goes into proper AC wiring, just as much goes into a good cable plant. Using the wrong type of wire and connectors or wiring up outlets improperly are just a couple of the things that can go wrong and cause power problems. (There are only three wires to a typical 120-volt AC circuit, but that means there are nine ways to wire it up, eight of which are wrong!) Since your goal is to maintain as trouble-free a LAN as possible, it makes sense to ensure that the site is wired properly before any computers are plugged in.
In a pinch you can look at the wiring yourself and use a pocket tester (obtained from a local hardware store) to test your circuits. The wiring is so important and fundamental to the operations of everything on the LAN, however, that I recommend hiring an electrical contractor to inspect and test your circuits. While doing this, they can also hook up an analyzer to tell you about the power you are getting from the power company.
Just because a room has 14 outlets in it doesn't mean that you can plug in 28 power strips and start running 50 computers and 10 laser printers! You need to know which outlets are on which circuits, and what the rated load for each circuit is. Clearly documenting these on a floor plan is about the only way to keep track. I've found that using highlighters to color-code different circuits by load capacity and location works well.
For example, a typical circuit is rated for 20 amps. This means that there is a 20-amp circuit breaker for all the outlets on this wiring, and if you try to run more than 20 amps of load on it, the circuit breaker pops, turning everything off instantly. To calculate the number of computers you can plug into a 20-amp circuit, simply determine the amp rating for each piece of equipment and then add these values together. A computer with a local hard drive and monitor typically draws anywhere from 1.5 to 4 amps, so you could run 5ñ13 of these computers on a 20-amp circuit before it would pop. Don't forget, though, to allow for any lights, radios, h uitóthis is where good documentation really pays off.
Having proper wiring is half the battle. The other half depends on the quality of power your power company is providing you. Does it run at a constant voltage, or does it occasionally sag or surge? Does an analyzer detect any spikes in the circuit? How about blackouts? No power source is perfect, so don't be surprised if yours turns up some of these anomalies.
There are line voltage analyzers that can be attached to your incoming power circuits. They will report on the quality of the power being provided to you by your power company. You can rent these units but without proper training, they can be quite perplexing so I prefer to hire an electrician to provide a final report. Remember, the report will only tell you what happened while the unit was attached, so try to leave it attached for at least one week in order to get as complete a picture as possible.
A dedicated circuit, as mentioned earlier, simply means that the circuit is dedicated to a particular outlet. Therefore nothing else exists on the circuit to cause a problem with whatever is plugged into the dedicated circuit's outlet. I prefer to have several dedicated circuits available for my LAN room equipment because I like to prevent problems rather than fix them.
Even after you have made sure that all your wiring is proper, and that the power company is providing you pretty clean power, you know that you are still going to be hit by sags, spikes, blackouts, and surges. The simplest way to prevent these electrical mishaps from disrupting your business and destroying your equipment is to install an uninterruptible power supply (UPS). This device is designed to eliminate sags, spikes, and surges while providing a temporary alternate source of power during a blackout. The following sections discuss several different ways that UPSs can be used.
A UPS does two different jobs: provide clean power (eliminate voltage sags, surges, and noise) whenever there is power from the outlet, and provide clean power for a while when the power goes off. There are a variety of topologies that aim for the same resultsówe will discuss each one so that you can determine which UPS system is best for you.
There are some arguments about the differences between UPS systems and SPS systems; some people insist that only online systems should be called UPSs, while standby systems should be called SPSs (standby power supply). There are arguments made on both sides that make sense. The central issue is discovering what type of system will reliably and cost-effectively protect your equipment. The decision to buy a UPS system should be based on effectiveness and cost, not on what the system is called.
An online UPS runs everything off of a battery/inverter circuit at all times. A built-in battery charger keeps the batteries constantly charged. If the batteries ever fail, the equipment is switched over to a filtered AC circuit.
One advantage of an online system is that it never has to switch when there is a power outage. Since everything is already running off of the battery/inverter circuit, all the UPS has to do is to sound an alarm indicating an error on the input power side.
A disadvantage of an online system is that since everything is always running off of batteries, the built-in charger must be capable of providing as much charge to the batteries as the equipment is draining offóotherwise, the batteries run down quickly. The cost of keeping the batteries charged makes an online system is more expensive than other topologies. This ongoing cost should be kept in mind when purchasing a unit.
A standby UPS runs everything off of a filtered AC circuit until it detects a power outage. Then it switches over to a battery/inverter circuit.
Since a standby system only has to keep the batteries "topped off" for an emergency, it can get by with a much smaller battery charger. This also means that a standby system generates a lot less heat, since it is doing less charging.
One of the interesting arguments people use when evaluating online versus standby systems is that since there are virtually no standby systems for large minicomputers and mainframes, online systems must be better. The truth of the matter is that at the higher power levels required for these larger computersótypically 2000VA and aboveóthe switches needed are too difficult and expensive to build, so standby systems are not feasible.
Another common concern about standby systems is that the time to transfer from AC to battery power might be too long. Current standby systems typically detect and switch over to battery power within 2ñ4ms, well under the 8.3ms standard for Computer Voltage Tolerance Envelope set by the Computer Business Equipment Manufacturers Association. Actual testing of IBM, Compaq, and other popular computers has shown that they continue to operate properly through transfer times of over 100ms.
Take an online UPS and remove the filtered AC circuit side, and you get an online without bypass UPS. The problem with this type of UPS is that if the battery charger, batteries, or inverter ever fail, there is no provision for switching over to normal power, so the whole system fails.
A standard online hybrid UPS takes the input AC, rectifies it to DC, runs it through a combiner, and then inverts it back to AC for normal operations. If a power outage is sensed, the standby DC/DC converter on the batteries kicks in and sends power to the combiner, then out through the inverter to provide AC power. With this type of UPS system, power doesn't really run through the batteries in a true online sense. Also, if the inverter fails, the entire unit fails. This is called a single point of failure situation and is not desirable for a protection system of any kind.
Take a standby system, replace the filtered AC circuit side with a ferro-resonant transformer, and you get a standby-ferro UPS. One of the nice things about such a system is that the transformer truly isolates the output power from any glitches on the input side. Unfortunately, the transformer itself creates its own distortions and transients in the output power. Also, the transformer creates quite a bit of heat because of inherent inefficiencies.
A line-interactive system looks too simple to work, yet when properly designed and engineered, can be very functional. The secret to this standby type of system is that all the power is always running through an inverter. The inverter provides protection from sags and also keeps the battery charged when input voltages are normal. When the input voltage drops, the power flow is from the battery back out through the inverter. Because all power is passing through the inverter always, it provides for additional filtering and reduced switching transients when compared to the basic standby topology. If the inverter is properly designed, it can fail but input AC power still gets routed to the output side. This is a very simple and efficient design that works well where there is very poor power.
After you complete the daunting task of choosing the best topology, you also need to compare the various features and programs offered by different UPS manufacturers.
A UPS is no longer just a box with an On/Off switch, power cord, and some outlets. A modern UPS has lights, buzzers, software, and can do things such as paging you when there is a problem! The following sections describe some of the more interesting and valuable features (by no means have we included all the features available on current systems), so that you are aware of some of your options when you decide to purchase, replace, or upgrade a UPS system.
Since batteries don't last forever, wouldn't it be nice if there were some way to tell if a battery is still good (without pulling the plug and seeing how long things continue to work)?
Some models don't have any battery-level indicators at all, and some have a light that only comes on when the battery needs replacing. I prefer models that have a series of LEDs that indicate present battery charge as a percentage of total battery capacity. Of course, nothing is a substitute for a real test. On a daily basis, however, it's nice to have some idea of how battery levels are changing before disaster strikes.
You can also find systems that provide software to monitor and log battery-level information so that you can keep in touch without looking at your UPS all day long.
This might not seem like much of a feature, but the method of battery replacement has a huge long-term impact. UPSs have batteries that wear out, so you expect to periodically replace the batteries. There isn't any way around this fact. Most UPSs require you to shut down the UPS, disconnect all devices, pull out the old batteries, insert the new batteries, reconnect all devices, and then restart the UPS. While this can be a real scheduling nightmare, it has long been an accepted inconvenience of LAN management. Someone has to come in at night or on the weekend and shut down the system while replacing the batteries.
Some manufacturers, however, have come out with hot-swappable batteries. You can swap out these batteries whenever you want, without having to bring down the system. This eases the burden of UPS managementóand facilitates keeping good batteries in a UPSóso much that I consider hot-swappable batteries an essential feature of a UPS system. I'm also impressed with those companies which allow you to return old batteries to them for recycling.
Another point to look into about replacing batteries is how they are shipped. Some batteries require special shipping and handling, while others can be shipped by UPS or other major carriers.
As your system grows, your need for additional UPS power also grows. This could mean replacing your entire UPS system, or buying more UPSs. With some systems, you can simply plug in more battery modules when you need more UPS power. This means you don't have to purchase any more control modulesójust batteries that plug into the control modules. Besides making things easier, this usually makes things cheaper.
How's your input power looking? Are you getting a lot of sags, blackouts, spikes, or surges? The equipment attached to the UPS is protected, but what about all of your PCs and other devices that aren't on a UPS? Some systems provide software that allows you to track this important information.
How hard are you pushing a UPS? Perhaps you have figured that it's only running at half capacity, but what does it think it's doing? Some systems provide software that allows you track load status information, which is really handy when, let's say, your assistant decides to clean things up by rearranging all the power cables or by plugging something into an open UPS outlet.
Battery life is directly related to temperature. Some manufacturers provide the ability to track the temperature of their UPS systems. I usually check the temperature to see if I should be worried about my file server or not. We ended up putting additional ventilation in one LAN room because the UPS temperature monitor showed that things became very warm in there on summer evenings after everyone went home and the air conditioning shut off.
What happens if there is a problem? Is it enough that the UPS takes care of the problem, or do you want to know when there has been a sag, surge, blackout, or spike? If so, how do you want to be notified?
Audible
Just about every UPS has a very annoying buzzer alarm that sounds when an error condition occurs. You might not benefit from audible alarms, however, if you don't spend all your time in the LAN room.
Visual
For those of us who do live in the LAN room, frequent sags or spikes can keep a UPS buzzing all day long. Sometimes its nice to be able to turn off the buzzer, and have LEDs flash to indicate alarms.
Send or E-Mail a Message
If your UPS detects a power error while you are in another room, wouldn't it be nice if it could e-mail you a message? Some manufacturers' software lets you do just that.
Pager
In the middle of the night at the end of the month, when an all-night database merge is being processed, you want to know if your UPS system detects a power lossóeven though you're at home. Some manufacturer's software will dial your pager if alarms occur, giving you a chance to stop critical processes before the system shuts down.
Log Errors to a File
Of course, you want all the errors to be logged to a file so that you can monitor and track unfortunate trends and events. After all, good documentation is one of the keys to success in LAN management.
How do you know that your UPS system will really work? You think the battery is still good, you think you haven't overloaded it, and you think your SHUTDOWN TIMEOUT value is shorter than the expected battery time, but how do you know for sure? You have to test it!
A pull-the-plug test might not tell you if your UPS will really work during a power outage. This is because pulling the plug sends a "nobody else is on the circuit" type of signal to the UPS, while in a real power outage other devices in your building can still be connected to the circuit your UPS is on. When the power goes out, they can show up as a very large load to the UPS, confusing or slowing down the UPS's response.
Doing a pull-the-plug test tells you how long the batteries really last, but a self-test switch is also very important; pressing this type of switch tests exactly how the UPS will act when the power really goes out. Doing both of these tests gives you a sense of security that the UPS will not hiccup or die when the power goes out, and will keep running long enough for your shutdown routines to execute properly.
Probably the most important computer to be plugged into a UPS is the file server. There's no reason to spend time and money to set up a file server and load it with data, just to lose it all to a power failure. I feel so strongly about this that I will not work on a system that does not have a UPS for the file server. To me, running a file server without a UPS is like driving a car without a steering wheel. Sure, you might not run into anything today, but eventually you're going to crash. To paraphrase an old boater's saying: "There are two kinds of LANsóthose experiencing a power failure and those that are going to experience a power failure."
If all the UPS does is postpone the power outage for a while, then what's the point? Of course, if you're nearby when the power outage occurs, you can run over and bring down the server properly before the UPS batteries are drained. What if, however, the power outage happens when no one is around, at night or on the weekend?
Almost all UPSs big enough to handle a file server include some kind of communications port so the UPS can talk to the file server. Whenever the UPS detects a power outage it notifies the file server and the server shuts itself down, preventing loss of data.
Some UPS systems require that a special card be installed in your file server for it to talk with the UPS. Other systems can talk directly to any free serial port on the file server. Depending upon your file server's configuration, this difference could be extremely important; operationally, though, the results are the same.
How do you stay in touch with the status of your UPS? One organization I worked with took the "If it ain't broke, don't fix it" logic to an extreme. They installed UPSs on every file server, then forgot about them. The only time any maintenance was done on the UPSs was when one failed. Sometimes they discovered that the only thing needing to be done was to reboot the server and replace the batteries. In some cases, data had been lost. In several cases, the batteries were so old that their cases had cracked, allowing fluids to leak out and destroy the UPS. One UPS's batteries had swollen so much that no one could open the unit to determine if anything was salvageable inside!
Obviously every piece of LAN equipment needs to be monitored. The more easily and completely you can monitor your equipment, the better off your LAN will be. Modern UPSs provide for monitoring in a multitude of ways. Choose the method that fits in best with your existing LAN monitoring policies and procedures.
Network Interface Port
Some UPS manufacturers provide their own software to expand on the services available through standard UPS-to-server communications links. Some functions available are: automatic battery testing, minimum and maximum power line voltages experienced, temperature, humidity, estimated battery runtime, incoming voltage, and current load on the UPS.
This software can be made accessible from any workstation on your network, so you do not even have to leave your desk to check every UPS on your system.
Network Interface Card/SNMP
Some UPSs allow you to install a network interface cardótypically Ethernetóso you can monitor and manage your UPSs via Simple Network Management Protocol (SNMP). If you are using NMS, this allows you to integrate your UPSs into your system without loading or learning any new software.
Modem
If you can't hear, see, or access your UPS over your LAN, you might still be able to monitor it via a modem. If you have UPSs spread around the country that are all networked together, this can make keeping track of their status much easier. It also means that a system failure does not keep you from staying in touch with your UPSs.
Not only do modern UPSs do more things, but you also can do more things to them. Because they include such a wide array of featuresódialing in across the country to check your UPS system's status, configuring when alarms are triggered, choosing what will happen when an alarm is triggered, and moreómodern UPSs are much more than dumb boxes sitting on the floor. Using the manufacturer's software, you can configure your UPS system to your specifications.
If you have several UPSs, you can assign unique ID codes to each one in order to keep track of your different units.
The low transfer point is the voltage level at which your UPS begins to operate when there is insufficient voltage. You might want to raise this level if you have equipment that is particularly sensitive to low-voltage conditions. By raising the low transfer point, you tell the UPS to protect your equipment sooner than the UPS would if it were set at a lower low transfer point. On the other hand, you might want to lower the low transfer point a bit if your power company keeps hitting you with brownouts that are triggering alarms, but are not affecting the operation of your equipment. This might help extend the life of your UPS by not using its batteries as often.
The high transfer point is the voltage level at which your UPS begins to operate when there is too much voltage. If you have equipment that is sensitive to slight voltage overages, you might want to lower the high transfer point. On the other hand, if you are getting frequent overvoltage situations that you know your equipment can handle, you might want to raise the high transfer point in order to minimize wear and tear on your UPS system.
Besides low and high voltage situations, UPSs can begin to operate if too much noise occurs on the incoming line. Fuel-powered generators typically create more line noise than your power company does, but not enough to damage your equipment. If your UPS is connected to a diesel generator, you might want to decrease the line noise sensitivity setting in order to minimize wear and tear on your UPS system.
Not only can modern UPSs test themselves, but some of them do it automatically on a regular basis! Using the manufacturers software, you might be able to set how often this testing occurs.
If you have frequent power problems that your UPS system is handling properly, it can be quite annoying to hear alarms all the time. Some systems allow you to turn off the alarm, or have it sound only during long-lasting events.
If your system takes a long time to shut down, you might want to increase the low battery point to a higher level to give your server more time to finish its routines before the power is turned off by the UPS.
Unfortunately, power outages do not necessarily occur once and then wait a long time before occurring again. Imagine that your UPS system experiences a power outage. It runs as long as it can, but has to shut down the system sometime before the power comes back on. When the power is restored, the UPS tells the server to bring your system back up, then starts recharging its batteries. Suppose the power fails again. Since the batteries are completely drained, the UPS cannot provide backup power. Maybe you're lucky, and the system has stabilized with no activity in progress, so the crash doesn't damage any data. What if it happens, though, while your system is loading or when power has been on just long enough so that the system is up and some users have opened up files? Could be nasty.
To avoid such a situation, a good UPS system allows you to configure the UPS to not allow the system to come back up until the batteries have been recharged sufficiently to allow for a proper shutdown.
If you have several UPSs on the same circuit, it can cause a problem if they all turn on at once when the power resumes. By setting a different return delay for each UPS, you can avoid tripping a circuit breaker.
A UPS system can do more than just beep and notify the server to shut itself down when there is an alarm. Logging each incident to a disk file allows you to diagnose what went wrongóand when. Using special software on Novell systems, the servers can be configured to send messages to the system administrator or users in general to notify them that there is a power problem. Since some systems have more complex shutdown routines, the ability to run a custom command file can be particularly helpful. If you have proper equipment connected to your system, the server can even send e-mail messages or page you!
Some UPS manufacturers do more than just sell you a box. They appear to take into consideration all the possible issues surrounding your UPS system.
What do you do with those old batteries? Even if they are hot-swappable, old batteries have to be disposed of in some fashion; it isn't okay to just drop them in the trash. Some companies provide replacement batteries in a reusable container so that you can ship the old ones to a recycling centeróor even back to the manufacturer! Convenient battery disposal might be a tempting factor when you are shopping for your UPS.
It's one thing to say that a UPS will work on your system. It's another thing if the UPS vendor has gone to the time and trouble to become certified by the vendor of the network operating system you are using. Look for such certification as an indicator that the UPS truly will work with whatever operating system you are running.
How much confidence does the UPS manufacturer have in the product? 90 days worth? 1 year? 2 years? 10 years? Also, how does the manufacturer respond to a UPS failure? These are things best figured out before you have a failure.
One company, American Power Conversion (APC), offers a $25,000 Lifetime Equipment Protection Policy with their units. This covers not only the UPS, but up to $25,000 worth of your computers and other equipment plugged into the UPS, should that equipment be damaged by a voltage surgeóeven lightning! This is such a valuable "plus" that I can't imagine other UPS manufacturers will not quickly follow suit.
Once you have decided on a topology and a vendor, you still have to decide how many UPSs you need, and how big they should be. The following sections provide an overview of some common issues and questions you'll encounter. You'll want to refer to literature from various UPS manufacturers for full details.
Obviously the file server, its monitor, and any attached external disk subsystems need to have UPS protection. This is the minimum acceptable level of UPS protection for a LAN. Then come the judgment callsóshould you protect devices like the gateway to the mainframe, the routers and hubs, the communication servers, and the printers? A key determinant for this decision is whether or not your workstations have UPS protection.
The simplest, easiest, and cheapest solution is to just protect your file servers so they can shut down politely. As soon as you enter the arena of keeping one or more workstations plus the servers up and running during a power outage, things become complicated.
First, if you put a UPS on a workstation, then you have to provide UPS protection for any and all powered hubs between the workstation and the server. Next, there is the issue of maintaining a connection to the remote mainframe or other service while the local power has gone out. Not to mention printing and power requirements to drive a laser printer for even a few minutes. Or the inevitable demand by a user to be able to work for at least an hour.
Since most data only becomes corrupted if the server crashes, I argue that after the power has gone out, there is likely a very good reason to leave the building. Since the UPS cannot keep things running indefinitely, it is difficult to justify spending much money to keep things running for an additional 10 or 20 minutes.
Given my experience with local power fluctuations, however, I advocate including a small UPS with each workstation. The investment per PC (well under $200) is far outweighed by the benefits of less lost data, fewer user problems, and greater reliability from the PCs themselves.
UPSs are rated in volt-amps (VA), from 200VA to greater than 5000VA. The bigger the number, the more powerful the UPSóand the bigger the price tag. While a 200VA UPS retailed for just over $100 in late 1995, a 2000VA costs around $2000 and a 5000VA unit goes for $5000 or more. The general interpretation of the rating is that the UPS can supply that many Volt-Amps for five minutes. Therefore, a UPS with a 600VA rating can provide 600 Volt-Amps for five minutes. Interestingly, if you only loaded a 600VA UPS with a 300VA load, it would last for 20 minutes. You need to decide how many Volt-Amps your equipment needs, and how long you need that power after the regular power supply has ended. While it would be nice to keep things running for hours on end, the cost can be prohibitive. The generally recommended runtime after a power outage is 15 minutes. This allows enough time to have your shutdown procedures execute completely, even leaving some room for additional equipment, batteries getting weak, and so on.
You can use the following calculation to determine approximately how long you have before your UPS is exhausted:
(Max. amp draw x 120) is the total amps of all the peripherals (such as the monitor) you have connected to the UPS, and (Power supply (in watts) x 1.4) is the watts rating of your computer's power supply.
The general rule of thumb is
So, for example, if your Total VA is 350 and the UPS you select is rated at 700VA, you should have at least 15 minutes of run time.
What if you have several servers to protect? If you add up all the servers, disk subsystems, monitors, and other equipment, you could easily need 1500VA to 2000VA of power for 15 minutes. Should you purchase one big unit, then, or several smaller units? There are two primary factors in making this decision: price and redundancy. Let's use a three-server situation as an example, with each server and related equipment needing 500VA. The total VA requirement is 500+500+500=1500VA.
If we're willing to go with 14 minutes of runtime, we can get by with a 2000VA unit, but we'd probably be well advised to move up to a 3000VA unit in order to have 22 minutes of runtime. Retail price in late 1995 is roughly $2000 or $3000, respectively. If we use three 900VA units, we'll have 18 minutes of runtime at a retail cost of about $700 each, for a total of $2100.
The desire to avoid a single point of failure argues strongly for using three 900VA units instead of a single 2000VA or 3000VA unit. While the newest UPSs are miracles of modern technology and reliability, it's still preferable to have three chances at something staying running. On the other hand, if you had 15 or 20 servers in a room to protect, the logistics of managing a whole herd of small UPSs would start to argue for consolidating several servers onto a larger UPS.
Unless you're purchasing huge (3000VA or bigger) UPSs, you should be able to install UPSs yourself. Even the 2000VA unit that weighs 100 pounds is actually two units of 40 and 60 pounds, respectively, so weight is not a concern. Since the batteries should be sealed units, there's no reason to worry about accidentally tipping a unit and spilling acid. That doesn't mean, however, that you can just turn on the unit, shove it into a corner, and forget it.
While it is tempting to just tuck the UPS in an out-of-the-way spot, resist the temptation. You or someone else will have to change the batteries at some point, so don't wedge the unit into a corner, criss-crossed with power cables. You lose the benefit of hot-swappable batteries if you have to bring down most of your network in order to physically access the UPS.
Another problem with tucking the UPS into a corner is that doing so impedes ventilation. All UPSs create some heat as they charge their batteries, and put out about 10 times as much heat when running off of the batteries. Leave lots of air space around your UPSs.
UPSs seem big and heavy, but they are sensitive pieces of electronic equipment. Don't leave them out where they'll get kicked and banged around. You not only don't want to damage them, you want to make sure the ON/OFF switch doesn't get accidently tapped by someone's wing-tip!
Charging and testing the batteries should all be handled automatically by the system. There's no need with modern UPSs for you to unbox separate batteries, hook up the leads, and charge the system before bringing it online.
Remember to hook up your server-monitoring cable! It's hard for you to overlook this cable during a new installation because the software usually has to be installed and tested at the same time. It is easy, however, to neglect to reconnect this cable after your UPS is moved. The problem often is that you become so concerned about getting all the power cables connected properly that the monitor cable is left out of the scenarioóat least, until there is a problem that forces you to remember what that extra cable is for.
Some network operating systems (NOS) are already UPS-aware, so you don't have to do muchóif anythingóto install the software. You're done after some very minor configuration. This software, however, is usually pretty rudimentaryóthe features you really want are only available if you purchase and install the vendor's software, and that's probably what you'll end up doing.
Write everything down. It is shameful that most UPS systems are installed and forgotten without a lick of documentation. At least make a three-ring binder for your UPSs, where you can write down pertinent information. Much of the information you want to monitor will be recorded by the software, but things like the purchase date of the batteries or what gets plugged into what are not covered by the software.
Many people hate to do this, but it's important! Send in the warranty registration card for each new UPS. For your own reference I recommend you mark down when you installed the batteries and when you expect them to need replacing. It might not be for years, but the day eventually will arrive, and it's better to anticipate it than to be surprised by it.
You've gone to great lengths to figure out exactly what is supposed to be plugged into what, so write it down! If you're really cagey, make a chart that indicates what you've plugged in, how much it draws, and how much room you have left over. Then, when you need to change or add any equipment, you can easily determine how to protect it.
Labeling your cords and cables might seem like overkill, but unlike the different cables that plug into the back of a computer, all AC power cable ends look alike! The only things that plug into a UPS (aside from the monitor cable that you'll probably forget, anywayósee the earlier section "File Server Connection") are AC power cables. You can't tell the AC power cable end for a file server from the AC power cable end for a monitor. If you plug all your servers into one UPS and all your monitors into another, then when the power fails you have nothing but a handful of brightly illuminated monitors connected to dead servers.
Bundling and labeling all the cables that go to each UPS is a good way to ensure not only that the correct units are plugged into the correct UPSs, but also that the monitor cable doesn't get left in the bottom of a box!
Monitoring your system is not the same as responding to errors. The most important thing about monitoring is to be so in touch with your system that you see strange things going on before they turn into problems. In other words, you shouldn't be looking for errors, but rather indicators of future errors. If you plug everything in and it is running properly, then a failure cannot occur unless something changes. If you monitor enough factors about your system, you can see changes as they start to occur, and take necessary actions to avoid the changes escalating to a full-blown failure.
One of the essential aspects of monitoring is to compare the current status to a known, good status. This involves creating a baseline of values to compare against. Maybe your system runs on 117 volts most of the time with an occasional sag on Friday afternoons. If you know this, and then all of a sudden the voltage starts running at 114 volts with daily spikes, you have reason to investigate this situation. The situation is not abnormal yet, but something has changed and you need to know what it means before it becomes a real problem.
If you were monitoring only to make sure that operations fell within acceptable levels, you might never have noticed a shift from 117 volts to 114 volts, since both levels are normal. Monitoring for variations can give you early warnings about detrimental changes to your system.
Always worry about the quality of the incoming power. The worse it is, the more work your UPS has to do, and therefore the sooner your UPS fails. Monitoring the incoming power condition for problems gives you some idea of how much work your UPS is doing. It also tells you when something has changed with the power going to your systems that do not have UPS protection. If everything has been running fine but your workstations start to lock up all the time, you should check the overall voltage level and incidence of sags to see if there's a problem with the power.
If your system has the ability to monitor temperature, you should check it regularly. Unlike mini- and mainframe data centers, LAN server rooms are not always air-conditioned, raised floor environments. You can use the temperature sensor in the UPS to monitor the conditions in your LAN room. Maybe everything is okay during business hours, but are you aware how hot that room gets over 4th of July weekend when the air-conditioning has been turned off for 3 days?
Battery level needs to be monitored carefully, because you want to replace batteries before they die, not after. All results of self-tests and pull-the-plug tests should be analyzed to make sure that you have full coverage at all times.
With the advent of software to automatically log most of the important details, you no longer need to be checking things on a daily basis. There are not any hard and fast rules for how often to check things, but I usually get nervous if I'm more than a week out of touch with my systems. As important as how frequently you check is how regularly you check. By getting into a routine, you minimize your chances of forgetting to do itómoreover, if you always check at the same time, things should always look pretty much the same unless there's a problem.
Any time there is a significant error, increase your monitoringótrouble rarely travels alone.
Just about everything you need to monitor will be logged to a disk file by one program or another. Even a pull-the-plug test generates data points in the UPS's log file as well as the SYSTEM ERROR log for most servers. Analyzing and making sense out of the information is up to you. You may prefer sheets of numbers to graphs with different colors. Whatever works best for you is what you should do, because you are the one who needs to understand what is changing and what is staying the same.
There is little value to monitoring a system without writing down pertinent information and analyzing trends. If you're not watching for trends, then you are just looking for instantaneous errors, something the UPS is perfectly equipped to handle with its alarms and other capabilities.
Nothing stays the same forever, and UPS systems are no exception. At some point you'll be faced with upgrading your UPS system. The following sections present items to keep in mind when you consider upgrading.
Once you have your UPS system installed you had better start preparing to change it. The following sections discuss a few of the factors that could force you to change your system before the batteries have worn out.
Computer systems rarely get smaller, and with the advent of the Information Superhighway mentality, it stands to reason that computer systems are going to expand at an even more prolific rate. As your computer system grows, your UPS system needs to grow, also.
Another occasion to upgrade is when your current system doesn't have all the features you want and need, but they are available on newer UPS systems. Perhaps some new monitoring feature, or the ability to hot-swap batteries, is all the justification you need to replace your current system.
Some of the older batteries are so expensive that you can buy an entire new UPS for approximately the same cost as replacing the batteries in an old unit.
Some newer units allow you to keep adding batteries (up to 100) to the main unit for rapid expansion of the capacity of your existing UPS.
Before you heave all your old UPSs out the door and buy new ones, check with the manufacturer for a trade-in policy. Not every manufacturer has such a policy, but it will save you a good deal of money if yours does.
UPSs are designed to be low maintenance; this unfortunately makes them easy to forget or take for granted. Unlike your PC that you boot up and use every day, your UPS might not actually do anything for months at a time. If you can remember to perform routine maintenance, however, you'll extend the life of your UPS.
Everyone knows that batteries don't last forever. Yet most organizations have no plan, budget, or timeline for replacing UPS batteries. Think of these batteries as longer lastingóand slightly more expensiveóprinter toner cartridges. Put them in the budget and the schedule; the batteries shouldn't have to fail during a power outage to remind you to replace them.
If you don't have hot-swappable batteries, you have to schedule a time to bring down the system while you swap batteries. If you haven't been monitoring well, and you find all of a sudden that your batteries are completely deadónot just weakóyou have to hope and pray that the new batteries get to you in time and that you can schedule replacement before you have a power outage. That's too nerve-wracking for me.
Being able to hot-swap dead or defective batteries is such a valuable feature that I won't purchase a system without this characteristic. Just make sure that you've left plenty of room to open the case without having to move any cables, and battery replacement will be a snap.
Some older UPS systems required you to open up the unit to tighten and clean the terminal connections to the batteries. Failure to do so resulted in improper charging and utilization of the batteries. Most newer systems do not require this often ignored bit of maintenance.
Just because the power LED is on doesn't mean that the unit is working. Even if the monitoring software says the unit is working, don't believe it is until you've tested it yourself. If the unit is continually handling sags, surges, and spikes, the question of what will happen when the power goes out still remainsóyou don't know how the unit will react, how long the batteries will run, and whether the shutdown software works properly.
Most units have a self-test that tests how well the unit would switch over during a power outage, and estimates how long the batteries would last. Pulling the plug is not a 100% accurate way to test how the unit would switch over during a power failure, but a pull-the-plug test does let you know exactly how long the batteries will keep running, and whether or not the software is configured and operating properly.
I use the self-test only to test how the UPS switches over during an actual power failure. I would like to believe that if the UPS passes the self-test, my batteries are fineóhowever, I don't have that kind of faith. Of course, if the unit fails the self-test, I feel confident that I truly have problems. The first problem is whatever's wrong with the unit, but a more important problem is why I didn't get warnings that something was going on earlierótime to check my monitoring policies and procedures.
One of the nicest things about the self-test is that is can be done anytime, since it does not affect the operations of the unit. In fact, some units actually self-test themselves periodically by default.
As mentioned earlier in this chapter, pull-the-plug tests don't create a true-to-life power failure. The power has failed, but since the UPS isn't plugged into the circuit along with everything else, the UPS gets different signals than it would during a real power failure. This is why I use the self-test to examine the system's switch-over capabilities.
A pull-the-plug test, however, does allow me to see exactly how long the batteries will run, and to make sure the software is configured and operating properly. It doesn't do much good to have the software configured to shut down the server after ten minutes if the batteries can only hold the server up for seven minutes! Moreover, if the monitoring cable is not connected, the software in the server never learns that the UPS is running on batteries, and is very surprised when the power goes out.
Even the best software in the world only works if it is loaded and running. If you don't add the correct software-loading commands to the automated start-up process, you could end up like I did once when I brought down a server to replace a defective NIC. After I brought the server back up, I noticed that the power protection software was not running because I had manually loaded the software when I had installed it. I immediately added the commands to my automated startup routine, and shuddered to think what could have happened because the software wasn't loaded had the system crashed during a power failure. The damage to the system would have been nothing compared to what my boss would have done to me.
The problem with pull-the-plug testing is that to do it right requires testing to make sure the shutdown procedures work. This means that you need to do it during a time when no one is on the system. In fact, since the system is going to be powered all the way off, be prepared for the possibility that it might not come back up right away. In the worst case scenario, one of your drives might die during the test, so make sure that you're aware of your tape backup situation. I like to do pull-the-plug tests on the weekendóoften on Saturday morningóas soon as possible after the Friday backup has been completed.
By scheduling, notifying the users, and doing the test on Saturday morning, I minimize downtime for the users. This also gives me a chance to make sure that I have a good backup in case something goes tremendously wrong. Lastly, it gives me all weekend to get the system back up and running in case something goes catastrophically wrong. Starting a pull-the-plug test an hour before the users arrive on Monday morning is an invitation for disaster.
Testing your system every month would be wonderful, but once per quarter is okay. You certainly should test it no less than twice per year.
All the monitoring and software in the world is not as important as the actual trend in runtime numbers for the pull-the-plug tests. If you're losing a couple of minutes every six months on the pull-the-plug tests, it is easy to calculate when you should have to replace the batteries. If you don't measure and document the battery duration every time you do a pull-the-plug test, you won't have this information.
What do you do when you know your UPS is defective? Maybe the batteries are dead, or maybe there is some kind of electronic malfunction. What can youóand should youódo? There isn't that much to do, but these are some of the things you can do.
Battery replacement should never be considered a repair to a UPS any more than replacing a toner cartridge should be considered a repair to a laser printer. Remember, though, that a battery or toner cartridge can be defective. Fortunately, replacing batteries in the new hot-swappable units is not much more complicated than replacing a toner cartridge.
I believe the surest way to invite a massive power failure is to attempt to run a file server without a UPS for a few days. To make sure you don't invite such disaster after your regular UPS malfunctions, you should include your UPSs in a maintenance contract. Most service organizations provide an equal or better UPS replacement while your unit is being repaired. If you don't participate in this kind of loaner program, then you need to borrow or rent a replacement unit, or else temporarily move a UPS off of a less critical piece of equipment.
If you don't have a functioning UPS that you can move in this type of situation, you should consider purchasing a UPS for a marginally-critical piece of equipment. In essence, you're buying yourself a hot spare, which you'll keep charged and ready at all times by using it on a device less important than the server. If your server UPS fails, you can start using a new unit immediately, get your system back up and running, and when the UPS comes back from repair, put it on something semi-important to prepare for the next disaster.
Appearances can be deceiving. UPSs are very heavy. But just because it looks like a brick doesn't mean you can or should treat like a brick.
Unlike PC cases, which are mostly air, a UPS case is mostly lead (from the battery), so be careful when you pick it up. They can weigh over 50 poundsóand usually sit on the flooróso watch your back and lift them carefully.
Remember that there is some very fancy electronic equipment in there along with the batteries! Don't drop, bang, or otherwise beat up the unit. If you treat it like a heavy computer, everything should be fine.
If your UPS doesn't use sealed batteries, you need to be very careful that the unit doesn't tip over and leak battery acid. Battery acid is extremely corrosive. Please review the vendor's recommendations before you move the unit to be sure there's no chance of spillage.
As long as you're moving the UPS, this might be a good time to replace the batteries. You're already going to be doing 90% of the work required.
If you've determined that your current UPS needs to be replaced, it really becomes a question of moving the old one out when the new one gets installed. You have to carefully plan what to do with the old one.
Since UPSs are not machine-specific, just about anything that plugs into an AC outlet is a candidate for benefiting from a functioning UPS. Think about which devices deserve one of your older UPSs that otherwise would just sit around. Maybe there's a fax machine, critical workstation, emergency printer, or other device that doesn't justify a UPS purchase, but which would be valuable to have operatingóeven temporarilyóduring a power outage.
Keep in mind that most UPSs provide excellent noise, spike, surge, and sometimes sag protection, even when the batteries are shot and they can't provide backup power.
Don't forget the option of donating an old UPS to a school or college so that the next generation of engineers can learn about the technologies they will face when they enter the job market. The educational experience your old UPS provides might save your data someday. Or donate to institutions such as senior citizens' centers or homeless shelters, which have less critical equipment needs but whose need to keep operating is every bit as crucial.
It is critically important to include a properly-configured UPS in any disaster-planning scenario because whatever causes the first disruption (earthquake after-shocks, flooding, riots, or whatever) might have multiple occurrences. You also have to take into account that the new site will not be as familiar to you as the original site, so mistakes can happen there. How embarrassing it would be to survive an earthquake, but then lose data due to a kicked-out plug or blown circuit breaker!
Your UPSs need to be as secure as the power switches to your equipment. This usually is not a problem, since the UPS typically is in the immediate vicinity of the equipment it is protecting.
But unlike the power switches to your equipment, modern UPSs can be turned on and off by software. Controlling access to the programs actually running on the server is typically handled by limiting physical access to the server, as well as limiting who is allowed to remotely control the server.
A situation that is often overlooked is guarding the software run on a workstation that communicates with and controls the software running on the server. This software is meant to be used by the UPS administrator from his or her PC. The software runs on that PC, and has to find and communicate with software running on the server. If your software does not require a specialized installation routine that utilizes a key disk, password, serial number, or other security instrument, you might have a security problem.
If anyone can install the client software on their own workstation, and have it successfully find and communicate with software running on your server, then someone could deliberately or accidentally bring down your system by reconfiguring the controlling software.
You might think that since the UPS just sits there, no training for using it is possible. There are many things, though, that you should train people to do properly: the hardware, the software, monitoring policies and procedures, and testing policies and proceduresójust to name a few important things. Also, since documentation seems to be a lost art, if you don't train someone else to know what you know about the system, disaster could strike sometime when you are gone. Worse yet, disaster could strike sometime when you have forgotten how to reload the software, check the batteries, or calculate the capacity of the units.
Monitoring and testing are the two most overlooked yet important things to do with a UPS system. You need to have people trained to respond properly to alarms and error conditions, but it is equally important to make sure you train people on the policies and procedures for monitoring and testing the UPS system. Of course, this assumes you have policies and procedures for monitoring, testing, and responding to alarms.
All too often, a UPS is installed and someone configures it, but that's the last attention paid to the UPS. One day, the UPS starts beeping or sending out warning messagesóthe only person who knows what should be done is at home, on vacation, at class, at a different site, or not with the company anymore. Getting into a situation like this is like buying a gun to defend your fort, but having only one person trained how to use it. It isn't much good if an attack comes when that person isn't available.
The following sections explain in more detail the types of UPS-related training you might want to do.
There's not much to tell anyone about the hardware itself. The most important thing is to make sure that everyone who has access to the UPS system understands its importance, and knows that turning it off crashes the system. Another important point is to make it clear to all involved that just because there is an open outlet on the back of a UPS does not mean that you can plug anything at all into it. If you've labeled all the cables that are supposed to be connected, you reduce the chances of improper plugging.
I've yet to find software that is so intuitive that no training is needed to know how to properly install, configure, and operate it. If there is a problem with the system while you are on vacation, who gets in troubleóthe person you never trained to run the system, or you? It's easy to overlook the need for training, or even tempting to avoid training others because you want to feel indispensable. The mark of excellent managers, however, is that their systems run as well when they are gone as when they are there.
Many decisions have to be made concerning all the possible error conditions a UPS can respond to. These decisions need to be formalized and proceduralized to prevent ignoring potentially catastrophic events, but also to eliminate overreacting to nuisance or expected errors.
I once had a conversation with a LAN administrator, during which his beeper went off. He glanced down at it, commented, "Oh, server XYZ switched over to battery power," and continued our discussion. My heart started racing a mile a minute, and I had to ask him why he wasn't doing anything about his apparent power problem. It turned out that he had been monitoring sags very closely, and had turned the low transfer point up very high so that the batteries would kick in at almost the slightest power sag. Since the batteries had kicked in, the system beeped him to let him know that a sag had occurred. It was a temporary situation he had created to closely monitor these events, and he knew that his system was in no danger. If you'd handed his beeper to me, however, I would have entered five-alarm fire mode. Training makes all the difference.
In order for monitoring to be effective, it needs to be constant and consistent. I can think of no way to do that without training people to do it. Train people to monitor regularly, and teach them what to look out for. The more eyes you have watching, the less chance there is that a surprise can sneak through and cause you trouble.
Testing, like monitoring, is only effective if is constant and consistent. Doing tests at different times and in different ways makes it much more difficult to detect any changes in the system. If everyone is trained to do tests the same wayóand at the same timeóthere is not only consistency, but also less chance that the testing will be forgotten or postponed.
One of the most interesting developments in power protection is the creation of power protection devices for NICs, modems, and other communication devices. It is often overlooked that the electrical circuits of network wiring, phone wiring, printer wiring, and other similar items are subject to many of the same problems faced by AC wiring.
According to APC, the IEEE 802.3 Ethernet standard lists four electrical hazards to which networks are susceptible during use:
In other words, power problems not only occur over power lines but also network cables, printer cables and modem cables due to static buildup, grounding potential differences and proximity to high voltage lines. For instance, a laptop traveler might be running off of internal batteries, but what happens when she plugs a modem into the data jack in her hotel room? A voltage spike could be created on her phone line by bad wiring or static buildup, and it could fry her modem and her computer. As computer components become smaller and more sensitive, we may find "data line surge suppressers" a requirement, not an option.
If you are having problems with your UPS system, you can use the following checklist to help pinpoint what is causing the trouble:
UPSs are needed to provide constant power in an inconstant world. By providing constant power we not only get constant use from the equipment, we get longer and better use because the UPS also protects the equipment from damage that can be caused by "bad" or "dirty" power. Modern UPSs can test themselves, warn us when their batteries are getting worn out, and page us when they detect a power problem. But they still need to be checked from time to time and no battery lasts forever.
UPSs are used primarily on file servers and other critical devices. With the decrease in the cost of UPSs and the increase in the reliance upon our PCs, more and more PCs are getting their own UPSs.
Proper power protection has its costs, but so does equipment failure and loss of data and worker productivity.
