Lightning Strike Damage

N

nowwhatnapster

Limp Gawd
Joined
Aug 9, 2009
Messages
406
Been working full time IT for about a year now and got to see first hand the devastation a lightning strike can have. Big storm rolled through the area struck near a building 3 clients of ours share. Thought I'd share the results.

  • 1(maybe more, wasn't keeping tabs on the ISP tech's, but there were aplenty) dead cable modem (replaced by ISP)
  • 3 dead switches, 8, 16, and 24 port. (replaced all)
  • 1 dead router, lights up but cant hit web interface (replaced)
  • 1 dead external interface on router, reprogramed to use different interface (to be replaced)
  • 5 dead NIC's
    • 3-workstations (installed add-in NIC's)
    • 1-laptop on a docking station (set to wifi only)
    • 1(maybe 2)-printers (set to wifi only)
  • 1 corrupt xerox copier (printer tech wiped HDD and reloaded)
  • 1 dead workstaion (no post, diagnostic lights indicate memory failure)
  • 1 dead battery backup
  • 1 untested router, proactively upgraded

Modems, routers and switches were on battery backups. Most of the battery backups are working fine surprisingly. It leads me to believe the surge traversed through the Cable> modems > routers > switches > NICs.

Any thoughts on prevention? I find inline surge protectors between stuff like the xerox and the switch generally cause more problems than they fix.
 
I'm actually curious about this myself. I have quite a lot of equipment at home and have a feeling I'm probably not adequately protected. Don't know if those cheapy power bars are enough.
 
Red Squirrel said:
I'm actually curious about this myself. I have quite a lot of equipment at home and have a feeling I'm probably not adequately protected. Don't know if those cheapy power bars are enough.
Click to expand...

Probably the best you can do for residential protection is a whole home surge suppressor on your main panel. That would hopefully shunt the surge to ground before affects any of the other circuits. The other common thing you can do is check that your ground connection is solid, i.e. tight connections, no corrosion, etc.

The problem with lightning is the shear amount of current it produces, on the order of 100,000 to 1,000,000 amps. When that much current is shunted to ground, the resistance of the earth actually causes the voltage of the ground to rise. We protect for this in substations, but even then we occasionally have equipment fried due to strikes.
 
Electrical Isolation is the best thing you can do.

Here are some steps:
  • Start with a whole panel surge protector
  • Anything that recieves a signal from "outside" copper (eg cable modem, dsl modem) should be placed on it's own UPS, seperate from any other equipment. Put it on it's own circuit if possible.
  • If the router is connected by ethernet to the WAN, it SHOULD NOT BE PLACED IN THE SAME RACK AS ACCESS NETWORK SWITCHES. Many devices ground the case and therefor the rack.
  • Since you have to do copper to router 99% of the time at a SMB's, be sure the router has it's own UPS also, and use the built in ethernet surge protector. Again, aim for it's own circuit.
  • Fiber from router to primary switches (removes copper electrical connection from WAN). All access network gear should have it's own UPS on a circuit separate from everything else.
  • Fiber from network switches to server switches. Again, make sure only server equipment are placed on the UPS's.

My minimum quote for a ground up SMB with a small server includes:
  • 4 electrical circuits
  • 4 UPS's
  • 2u vertical mounting shelf for router
  • rack for access network switch and cabling
  • whatever equipment is needed to mount server(s)+switch(es). If the SMB only has one server, i just recommend a nic with SFP's. Something like a synology nas I still recommend a switch to maintain electrical separation as much as possible.

I ALWAYS urge my customers to have separate equipment for FAX and printing. Never allow a FAX to be on the same surge protector/UPS as anything on the network. If the client won't budge then UPS with tel-line surge protector is your only hope. They don't work most of the time though. Make sure they know that not spending $300 on a nice fax could very likely could cost them the price of a switch and several workstations (plus your time to repair). E-FAX is their friend.
 
Last edited:
Subscribed.

Coincidentally we had a storm yesterday night with lightning. It affected our facilities in this region; we had a few things get knocked out.

Then I was also on the phone with an Outside Sales person who lives on the west coast and his router was acting funky and not working (said he'll have to replace it). They didn't have any thunderstorms. o_O

I also have a job at another company, and at first I thought that the FedEx computer and a Lexmark printer got their NIC ports fried but in recent discovery it turns out it is the 24-port switch ... the 24-port switch in server room is hooked up to a 2U-sized APC UPS, and the UPS is hooked into an APC 1200w voltage regulator. These are the only two network devices that are connected via wired Ethernet; the other workstations and printers are connected wirelessly.

At my second workplace where the 24-port switch took the damage (likely through Ethernet from the Lexmark and FedEx PC ... or just POOR BUILDING WIRING), we have a Netgear R7000 with DD-WRT; a Motorola SURFboard 6120 cable modem; a 4-port blue Netgear switch (plugged into router); a 24-port TRENDnet gigabit Desktop switch (plugged into router); an HP ProLiant DL380 G4 (network plugged into router); a 2U APC UPS that isn't even 2 years old (plugs into voltage regulator); and an APC 1200w voltage regulator (plugs into a Kill-A-Watt which then plugs into wall). All in the same rack except for the voltage regulator and Kill-A-Watt. Several weeks ago our ISP MediaCom ran a brand new coaxial run into our building -- this time going directly into the server room coming from the ceiling and plugging straight into the cable modem.

The cable modem use to be in the same closet as the telecom equipment on its own UPS (nothing else plugged into it), so I will grab that UPS tonight and move it into the server room to put the cable modem on it. With that, there was also an APC surge suppressor with coaxial surge protection as well. :) As far as the router goes ... well, I don't have another UPS. :(
 
Last edited:
Yeah, cable modems are bad for lightning strikes, often because incoming cables are not properly grounded to the main building ground and (at least in residential areas) they're not buried nearly as deep as electrical wiring, so are more susceptible to picking up discharge from nearby ground strikes. Hell, they're usually just laid down on the bare dirt with the sod put on top, ugh.

It's almost never a direct lightning strike that causes lightning damage, it's equipment picking up ground voltage differentials from nearby strikes, and incorrect grounding of equipment causing problems.

So say you get an nearby lightning ground strike, and this strike causes your incoming cable line to pick up a charge. If the shield on the cable isn't properly grounded to the building ground, bam, instant ground voltage differential between equipment and all the magic smoke gets let out as the voltage levels attempt to equalize through the equipment, instead of through the main building ground. :)

Cable modems are notorious for this, because they are rarely grounded with a 3 prong cable, so instead of the surge just taking out the modem and hopefully grounding through it, the surge travels on to the next thing in line, all your networking equipment. And in the case of your house, through your TVs as well. :)

Sorta related, if you're ever looking for a good way to convince a higher-up why you need that sweet incoming fibre connection instead of DSL, or why you should spend the bucks to run fibre between buildings, or why you need everything protected by a UPS, describe to them the cost of replacing every single piece of networking equipment when lightning strikes down the street.

Edit:
Also, if you're hit with a direct strike, you're probably fucked no matter what you do. But doing everything you can might at least limit the damage instead of having every single piece of equipment fried, and direct strikes are exceedingly rare. I can't even imagine what it would look like if a direct strike decided that the best path would be right through your server rack. I guess you could get a few bucks back when you take everything to the recyclers. :)
 
Last edited:
epimetheus said:
Probably the best you can do for residential protection is a whole home surge suppressor on your main panel. That would hopefully shunt the surge to ground before affects any of the other circuits. The other common thing you can do is check that your ground connection is solid, i.e. tight connections, no corrosion, etc.

The problem with lightning is the shear amount of current it produces, on the order of 100,000 to 1,000,000 amps. When that much current is shunted to ground, the resistance of the earth actually causes the voltage of the ground to rise. We protect for this in substations, but even then we occasionally have equipment fried due to strikes.
Click to expand...

Actually I always wondered about those surge protectors, as they're not even inline with the main, so how are they suppose to actually stop a surge without disconnecting both hots completely? Electricity does not take only the path of least resistance, it tends to spread out.

As for separate UPSes how is that handled in a situation where you have a single large UPS? Ex: my UPS has 400 amp hour of run time, not going to replace that with little 5 minute UPSes.

Though if I go with the rectifier-inverter method then technically that should stop surges right? That's my future goal. The rectifiers might take a hit but not the rest of the equipment. Still have to isolate ethernet though but if I ever have ethernet that has to go outside I'll probably use fibre.

For grounding is it also advisable to have each ground go to a main bus bar instead of just a string?
 
bds1904

Trying to understand the reasoning for separate battery backups. So if the surge initiates on a coax line potentially it will traverse Modem > BBU > devices attached to BBU > devices on same circuit, before it finds its way to the building ground? Whereas if it was on its own BBU and circuit it would hit the main panel (and hopefully building ground) before traversing to other circuits?

I have the same issue Red Squirrel has, many clients just have a single large UPS that runs all their equip. Do we bite the bullet and make them get a separate small BBU for modem?

Red Squirrel

My understanding is a surge strip will trip when a voltage exceeds a set amount and shunt the surge to the ground wire, effectively isolating the equipment on the other side from the surge. But it would still let some surge through initially until it trips. (feel free to correct me if im wrong)
 
nowwhatnapster said:
bds1904

Trying to understand the reasoning for separate battery backups. So if the surge initiates on a coax line potentially it will traverse Modem > BBU > devices attached to BBU > devices on same circuit, before it finds its way to the building ground? Whereas if it was on its own BBU and circuit it would hit the main panel (and hopefully building ground) before traversing to other circuits?

I have the same issue Red Squirrel has, many clients just have a single large UPS that runs all their equip. Do we bite the bullet and make them get a separate small BBU for modem?
Click to expand...

Bbu's arent very good at ground isolation between ports, just hot and nutral. Putting the modem on its own bbu will have a higher chance of shunting to ground, a seperate circuit is even better.
 
You must log in or register to reply here.
Back
Top