So two admins walk into a server room

B

Blue Knight

Gawd
Joined
Dec 2, 2000
Messages
699
And notice that there's a water sprinkler right above them. What do you do? :confused:

Hey guys, wanted to know what any of you have done, or would do in this situation. I just walked into my first real admin job (as remote hands at a branch office for HQ), and noticed this and it's bugging me. The heads are heat sensitive, so there'd have to be a fire in the room itself for it to go off (so I'm told), but yah....

A "tarp over the top of the rack" was suggested, and our APCs are supposed to have thermostats that will shut off the servers in case the room ever got too hot or had a fire.

Just trying to get an idea of my options. It's an average 50 person company, so not sure how well some of your suggestions will fit with a budget, but that's all I got for the moment.

Thanks in advance! :D
 
That's fine. If there's a fire in the server room, the servers are toast anyway.

Make sure you back your data up offsite, call it a day.
 
dont worry about it, they put it there for insurance reasons.. if it where to go off, it would drench/ruin the servers. and as it was required by insurance, it would be covered by insurance (might want to get an adviser in on this be be sure it is covered, as it should be) and it should cover down time and recovery ;)

on your end... make sure you have off site back ups... and thats about all you can/should do
 
XOR != OR said:
Make sure you back your data up offsite, call it a day.
Click to expand...

This.

Insurance covers equipment replacement. Good luck trying to recover data that was destroyed by fire that wasn't backed up offsite.
 
How many servers are we talking about here? 1, 2, 5? If that many yeah, just go with it and don't hit the head when moving big equipment. Make sure you have solid backups

Racks full of servers? Get a proper fire system for electronic equipment
 
Sure its they are not pre-action sprinklers? Our datacenter has both an INERGEN system but for any reason that fails or a fire starts after the tanks have fired we have sprinklers that kick in.

In the end the #1 goal is to save the building.

No fire suppression system should replace a good offsite backup.
 
Continue to stress to management how stupid that is, and suggest they get a FM200 system and deactivate + bleed out the system. Make sure you it's recorded somehow. Perhaps do it via email, make sure the manager replies to it. If he says "he'll get on it eventually" keep reminding him.

If something happens and those sprinklers go off, and there's no proof that you tried to do something about it, guess who gets blamed. In fact, where I work, we'd still get blamed, but at least from a legal perspective you are somewhat safe.
 
This is really simple. You from now on do offsite backup, and make sure your company doesnt miss an insurance payment. thats it. for the longest time we had a sprinkler system, and it didnt matter.

It has to get pretty dang hot in there to set off those heads. if your room gets that hot without a fire your a failed admin anyways. most of those sprinklers need to reach 8-900 degrees on the little red tube before they crack open..

unless your lab is already on fire, you shouldnt have an issue. if they leak, whatever, company will get some new servers :D

I honestly wouldnt worry about it. as an IT admin, you must pick your battles carefully or this industry can and will eat you alive.

From one IT Admin to another, I wish you the best of luck Mate.
 
I don't worry about heat setting off our sprinklers, I worry that they are only a few feet from the top of the racks and we run all our cables overhead. I'm not so sure insurance covers you accidentally tripping one...
 
Not too sure, if you have 6-8 racks full of equipment and had an AC failure. I could see a room getting pretty hot, not sure if it would make it to 135F but maybe. The equipment would be toast anyways if it didnt shutdown automatically.

I would say write the letter to MGMT and call it a day.

I would also call the vendor out to install sprinkler head cages/guards. This would help incase a cable got thrown onto the head or what not.
 
Haylon systems are good, they just emit gas which kills the fires and not the server equipment.

But yes, realistically, have an off-site backup.
 
Last edited:
Red Falcon said:
Haylon systems are good, they just emit CO2 gas which kills the fires and not the server equipment.

But yes, realistically, have an off-site backup.
Click to expand...

Halon and co2 are different and both have their pluses and minuses
 
It can, it is never recommended to stay IN the room when these systems have to be used :p
 
I have seen quite a few datacenters, big, small, or just average sized. This is the first time I've even heard of someone being concerned about the sprinkler system.

As others have said, don't worry about it. And offsite backups will save your life as an admin.
 
I've heard of sprinkler systems in data centers being a concern....and rightfully so.

It's not the damage of a real fire that is a concern...yes...we are all aware that a fire that takes down the building...will include the servers in its rampage. It's the legit concern of alllll the other days without a huge fire, that the sprinklers can cause something else bad to happen to the servers.

*Sometimes <gasp>....a head will fail..and it will leak/spray/drip....whatever is underneath...huh.....bummer.

*Sometimes some <event> other than an actual fire will trigger the sprinkler system. Hate to see a false alarm take down the servers for good.

*Many times a legit fire, yet a small one...will occur and actually be contained in a room..lets say, a small fire in the kitchen. Oh bummer....the servers got ruined again for no good reason!
 
wouldn't want to be stading on that wet flooe with a 1000000KVA pumping around the place!
 
We have the same issue in our "new" building. The built out a corner of the building for our server room (since that's where the telephone comes in) and on move in day we notice 2 sprinkler heads in the ceiling. We were told by management they had to be installed and we shouldn't worry about it. We've started backing up everything off site since then.

I try not to think about it but every time I'm in there I can't help but notice them. It's like they're waiting to one day flood my gear and make my life miserable.
 
c0rpt3ch said:
We have the same issue in our "new" building. The built out a corner of the building for our server room (since that's where the telephone comes in) and on move in day we notice 2 sprinkler heads in the ceiling. We were told by management they had to be installed and we shouldn't worry about it. We've started backing up everything off site since then.

I try not to think about it but every time I'm in there I can't help but notice them. It's like they're waiting to one day flood my gear and make my life miserable.
Click to expand...

job security ;)
 
Red Falcon said:
Haylon systems are good, they just emit gas which kills the fires and not the server equipment.

But yes, realistically, have an off-site backup.
Click to expand...

Halon is hazardous to humans, you really don't want to work in a Halon protected server room. You have to take all kinds of safety training and have emergency breathing apparatus in the server room. Luckily, there are other effective gases that don't kill you if you breathe them, you wouldn't want to stay in the room, but at least breathing it for a minute or 2 won't kill you.

Depending on the amount of equipment in the room, and the allowable down time that can take place before the business is no longer financially viable, from a cost perspective, it may make sense to use water based fire supression, but it is a good idea to have the heads at least offset from the racks, just in case of leaks.
 
Thanks for the reply guys. Good to know there's a crew of [H]ard admins who can back me up.

I guess I'll use a tarp or some sort of deflection in case of a leak. The head's right above the main rack of 5 servers.

They've been doing weekly offsite backups for a long time now so we should be good there. I'll probably test it to make sure it works though - :D. No sense in getting screwed on that side.

Peace!
 
Is it possible to move the rack? Like on a weekend? move it like ten feet away? Just curious!

I always loved weekends, get to move things around, keep others confused, and get paid double for it!
 
Or you could tape a beer bong over the sprinkler head and pipe it away lol.
 
LOTS of mis-information in this thread.

1) The sprinklers are most likely required by fire code to be in the room.
2) If there is a fire somewhere else in the building the heads on the other side do not trip, this is a Hollywood myth. Only the heads near enough to the fire to heat up go off.
3) The sprinklers are there to protect the building not the servers.
4) If the ambient air temp stays above 155F long enough to trip the sprinklers without a fire you have other problems.
5) Putting a tarp over your rack will not help, the amount of water released is immense. There is also no way to stop it until the pipe is empty.
6) Proper yearly maintenance will stop them from dripping, leaking, etc.

Just put a cage over it to stop accidentally hitting it with something and forget about it. I have never been in a sever room that did not have sprinkler heads. Stop worrying about it.
 
Well a proper server room should have an approved replacement to sprinklers, that will still abide by fire codes. I think when you have a few million dollars worth of equipment in that room not to mention the actual data and setup (aka labor to regain all that again) saving the building AND the equipment should also be a priority.

Even with backups, it would take weeks, maybe even months to order all new equipment, set it up, reconfigure it, then restore all the backups. It's worth investing money into a system that will not damage this equipment when you consider the man hours to rebuild if it is damaged from water.

The longer it takes, the longer everyone is down, which can have all sorts of other effects on the company's overall production.

Also if you want a safety hazard, you put sprinklers in a room that has tons of electrical capacity and lot of exposed electrical/electronic equipment. Remember this equipment is also UPS protected so that can make things even more interesting.

Lead acid and water is also not a good combination. (at least, at that volume).

Our PBX room is a joke, they have sprinklers and a huge battery bank in there. They made these custom plexy glass umbrellas over the equipment. That aint gonna do anything if those go off.

I would hate to be the PBX guys having to reconfigure every single extension. :eek:
 
You're way more likely to lose a compressor/head/cooling tower on your CRAC/HVAC or lose generator to your cooling and cook the room that way.

I know. It happened to me. More than once. Took 5 months to rebuild the worst time (right before I started). 10 ton compressors weigh 500lbs. Not like you can just unbolt it and swap in a new one easily.
 
dont worry about while im not a system admin, I run a small computer shop that deals with client machines. We have sprinklers in the building including over the bench, insurance ya know lol. My insurance covers up to 200,000 dollars just for client losses and another 500,000 on the actual shop, inventory, and equipment. Yes if my shop catch's fire we spray it with water, but my insurance covers that issue. You worry about your company, i worry about individual owners, right now i have 12 PC's from 12 diffrent people, and i can support up to 30 laptops and 20 desktops on my bench's, thats 50 machines on a bench, and I can store about 200 desktops and about 500 laptops in storage in the back that are waiting for repairs (never needed to use that space yet). Id say my risk is greater and im not worried, just do your offsite backups.
 
You must log in or register to reply here.
Back
Top