If the Lights Go Out, Is Your Business Prepared?
As I left the office tonight, an intense band of thunder storms was moving quickly across Northern Mass/ Southern NH. The handful of fat but lazy raindrops that were falling as I crossed the parking lot had, by the time I reached the highway, turned into sheets of rain which were punctuated periodically by the flashing of lightning. To make matters worse, I had to stop and run an errand. Getting soaked in the process, seemed inevitable.
The rain hadn’t slowed as I stepped out of my car. Upon entering the store, while I made my way towards the back of the store, there was a sudden brown out. Power was never fully lost, but the lights dimmed and flickered for 2 or 3 seconds. Upon making my selections and heading to the counter, it became very clear that I wasn’t going anywhere.
As with most retail shops the store point of sale system was driven by 3 personal computers, each with an attached credit card processing unit and a printer, all of which had been rendered non-functional by the brief but deep brown out. This store has heavy traffic, and lines 2 or 3 deep had already formed at each register. The staff was forced into immediate “triage mode”, and one gentleman, clearly the manager, began issuing orders – “Folks who have cash go to register one”, “Bill, I need you writing down everything that is getting sold with the price so we can tie out inventory at the end of the night”, “Bob, get into the back room and reboot the router and the credit card system”, “Everyone who has to use a credit card is going to have to wait 5 or 10 minutes until we get ourselves back online”.
I must say the guy handled the situation like a seasoned retail pro, and while I was mildly inconvenienced by the extra 10 minutes added to the errand, I actually appreciated the experience because it reminded me of a few facts that have been consistent for as long as I have been in IT and lived in New England.
- Sudden strong thunderstorms happen in the summer months, and they can and do knock out power.
- Failure to plan for power outages can and will impact normal business operations.
- Computer systems really do not like to be shut off without warning. I mean they really, really don’t like it.
This article is offered as a reminder, just as we put out in the past when Thrive’s Tech Blog was the Tech Brief – the summer months are a fantastic time to take stock of your preparedness for power events, and to test the systems you have invested in previously to ensure they are still in good working order. You may be thinking that the situation I encountered earlier this evening can’t happen to you. After all, you don’t have a point of sale credit card machine in your office. Maybe you even know that just last year you had two brand-new uninterruptable power supply (UPS) units installed in your server room. The gent who installed them seemed very nice and professional at the time, so what could go wrong?
The answer is a lot. For one thing, batteries of all kinds – big, small, UPS, or automotive – degrade over time. From the moment you put them into service, they are becoming less and less effective. UPS devices need to be properly installed and tested on a regular basis to ensure that they work as expected when the situation requires them to.
Another important factor to consider is runtime – in other words, “how long do you want to have power supplied in the event of an outage?” For servers or other critical data systems, you need to have enough runtime to at least allow for the safe power down, In line with the proper installation point made earlier, for this to happen off hours or when a person cannot walk over and shut them down, you need to be able to have a signal sent from the UPS to the server. This requires proper configuration and regular testing.
Protection from power events doesn’t stop at the server or network infrastructure level either. Increasingly workstations are areas of heavy investment in the IT infrastructure of most small and medium sized organizations. Buying small UPS units for desktop protection can save big repair or replacement costs if a large surge from a close lightning strike affects your building. A sudden surge is very capable of destroying large numbers of computers in an instant. What would you rather throw away – 25 UPS devices at $75 each or 25 workstations at $900 each?
Summer Checklist for Your Small Business
So this summer, do you business a favor and use this checklist to evaluate your readiness for a power event:
- Are all of your business critical systems (servers, Firewalls, Routers, Switches, Backup devices, etc) plugged exclusively into one or multiple UPS devices?
- Do those UPS devices contain sufficient runtime to allow for safe and effective powering down of all connected devices? When did you last perform a runtime calibration on your UPS devices?
- Are your UPS batteries over 2 years old? Generally speaking if this is the case you should replace them with new ones and send back or properly dispose of the old batteries.
- Is the software, cables, adapter cards, or other hardware required to have the UPS send an automatic shutdown signal to the attached devices present in your infrastructure?
- Is the shutdown software configured properly? When was the last time you tested it by simulating a power failure and ensuring safe shutdown of network servers and devices?
- Are you currently monitoring your UPS devices such that power events will proactively send an alert notification by email, page or other mechanism to IT staff who can respond? Shameless plug – if you are a client of Thrive Networks, and are on our Thrive Full Protect package the answer is yes to this question!
- Do you have desktop grade UPS devices in place to protect workstation assets in your organization from power events? Is your standard procedure to purchase a desktop grade UPS device with every new workstation that you purchase?
- Do you have a Business Continuity Plan or other document to keep your business running in the event of a prolonged power outage? We often think of the power being out for days as inconceivable, but remember the ice storm of 2008. Several communities were out for a week or longer. It can happen.
A little preparation will make sure that you don’t find yourself with the summertime blues when the power is out.