New Server Room Wishlist

P

Protoform-X

[H]ard|Gawd
Joined
Jan 30, 2002
Messages
1,203
The company I work for has just purchased a big lot up the road to build a new building since we're growing so rapidly. As the IT manager, I've been tasked with creating a wishlist for the new server room. They're going to try to finalize the plans pretty soon, so I need to get my ideas in quickly...

As it stands at the current building we have an APC 42U cabinet that's entirely full of equipment. I'd like to either keep the cabinet or do an open 4 post rack for servers and get a couple of open two post racks for network gear- I'm going to recommend two Cisco switch chassis (4500's perhaps?) and probably two ASA 5510's to replace our ASA 5505 since it's sitting at 70%+ CPU almost all the time now.

What should I ask for as far as the room itself goes? I believe it's approximately 10'x14' which is much larger than my current server room. The server room I have now has it's own AC, but other than that, it's just another "office." Some considerations:

-50 employees and growing fast.
-Downtime cost is very very high per hour per employee during business hours.
-The server room is on the second level. The building is two stories tall.
 
Multiple dedicated electrical circuits. If you have a full rack of stuff, then I'd assume yer well into the range of 220v UPS, so 220v circuits as well.
Dedicated ground to the server room.
 
Is all the cabling going in there as well? I'd ask for good conduit pathways into the room, with room for expansion.
 
Call up the nearest colocation provider and look at all the redundancy they provide. Now think about building that out in your own new environment. Can you do it? Is it cost effective?

Some things to think about.

Make sure that your AC for your room + UPS are also on a circuit protect by a generator or another source of electricity. All the redundant configuration won't do you any good if you can't cool the room.

Generators are awesome, as long as you make sure to test them every month. No point in spending all that money and only testing once a year.

You mentioned downtime costs are very high. It's one thing to provide a good redundant infrastructure for a datacenter. It's a whole other thing to extend that out to your office environment.
 
At least 3 spare 3 inch conduits from the first floor to the second floor.
Ask for a dedicated 3 phase panel into your room. At least 100 amps at 3phase.
If you can't get 3 phase then get a 300-400amp single phase panel. (Disconnect in the room is optional)

Dont waste money on raised floor.
Surround the room in ladder rack.
Setup redundant HVAC mini split units. 1 or 2 ton units depending on your estimated load.
Setup redundant UPS units. dual 6kva or 8kva's depending on estimated load.
Dont do 2 post racks unless your server room will have a dedicated lock or badge reader or some kind for physical security.
Make sure the door into your room is solid, not hollow.
Dont waste money on a floor. Just grind and polish the concrete. Will save lots of time later down the road on maintenance and upkeep.
 
What I'm currently running in to at work with our server room:

- Additional weatherproofing. Most flat-topped office buildings do have small leaks on the top floor even when they're brand new - look at additional internal coatings or other water protection that can be put over the server room.

- Proper Fire suppression system that DOES NOT USE WATER. Depending on where you are, this might be a code adherence nightmare to retrofit, or even to install in the first place, so make sure it's in the original plans.

- Room to grow - if you're at 50 employees now and "growing fast" you could need to double or triple your equipment footprint in the next couple years - chances are you're not going to get a new building every couple years. This includes physical space, cooling and power.

- Generator and UPS considerations as already brought up.

- Secure access.

- Depending on how secure you need to be, may require more than a couple sheets of drywall between the servers and hallway (literal concrete firewall).

- Cabling to offices - get 4x what you need run to endpoints, or have HUGE raceways and conduit that is easily accessible, IMO. Cable runs during construction should be absolutely dirt cheap compared to running them after the fact. We've more than doubled our CAT5e runs in our office over the past 4-5 years, with about the same number of employees. At $150-$300 per run, it gets expensive fast.
 
Everything that has been said. Also room for expansion, with everything. Power, cooling, patch panel room etc.... I cannot believe how many server rooms I've seen that have zero room to expand.
 
Very much appreciated, all! I've put in the request using everyone's advice- Lots of power, cooling, physical security, extra conduit, lots of Ethernet runs, ladder rack, non-water fire suppression, and nix to the raised floor. THANK YOU :cool:
 
don't just tell a contractor "I want lots of power".. they are going to laugh at you.
you need to define things for them.

Tell them, "I need at minimum 300 amps single phase on my own dedicated panel"
"I need at least 3 spare floor penatrations between floors"
"I need all sprinker heads capped"
"I need a fire extinguisher and proper signage on the outside of the door"
"make sure no building air handlers or drip pans are above my ceiling"
"make sure there are a minimum of condensate lines or other plumbing above my ceiling"
"make sure my door swings OUTWARD" (this gives you space inside your room)
"my room will most likely have positive air pressure, please plan accordingly"
"do not put any building air registers or returns into my room"
 
What all those guys said.

Also make sure the room is designed such that it cannot be used as a general storage/maintenance room. You do NOT want to have them move your office in to there when space gets tight, or use that area as general work space, even for repairing PCs.
 
jjeff1 said:
Also make sure the room is designed such that it cannot be used as a general storage/maintenance room. You do NOT want to have them move your office in to there when space gets tight, or use that area as general work space, even for repairing PCs.
Click to expand...

I would second this!..

I think it was already mentioned but proper AC (Mini-Split, Dedicated, whatever) would be high on the list.

Have they thought on how they are going to connect both buildings? Fiber, p2p wifi, something? Maybe talk about fiber between the data center and the closets and buildings. Were we are at we are small enough the city let us trench our own fiber under the street. So from our main building we have 4 runs of fiber spanning 4 different buildings
 
Make sure and get the network cabling done in CAT6, not CAT5e. You'll be thankful in the future for that.

I'd recommend investing a lot of time and effort into an initial physical inventory, cable management, established cable color coding, cable labeling on ALL cables, server and switch labeling with name and IP, and power distribution standards. These things are easy to set up initially, but very, very difficult to work in later.

However much cooling and space you get, you will probably need more before you move out of the building, so make sure they put it in with an eye on future expansion.
 
dgingeri said:
Make sure and get the network cabling done in CAT6, not CAT5e. You'll be thankful in the future for that.
Click to expand...
Not necessarily. In fact, depending on the current and/or future projected mobility of users, you might be better off saving money going Cat5e and taking that money saved and investing in wireless infrastructure.

Make sure to not only add drops in offices and such where people are, but also get drops added in the ceiling in the hall ways and common areas for future access points/other devices.

Most people never think to add drops in ceilings.
 
cyr0n_k0r said:
Not necessarily. In fact, depending on the current and/or future projected mobility of users, you might be better off saving money going Cat5e and taking that money saved and investing in wireless infrastructure.

Make sure to not only add drops in offices and such where people are, but also get drops added in the ceiling in the hall ways and common areas for future access points/other devices.

Most people never think to add drops in ceilings.
Click to expand...

GAH!!!! Wireless!! Sin!!

I absolutely HATE wireless networking. It is totally unreliable. Right now, sitting at my desk (remotely building 3 servers for another SW test) only ~10 feet from the wireless AP, my laptop says I have a 144Mb connection. Yet, I have horrible issues with slow responses, disconnection from the Exchange server, and file corruption.

I will oppose wireless technology vehemently until they get it stable and working right for at least 3 years. Even the newest 802.11n technology isn't any better than 802.11b for stability. I even ran a 50ft CAT6 cable around my apartment to get my roommate a stable connection for her Dell Zino. The wireless kept crashing her Pogo when it would disconnect, and she'd whine at me for hours at night because of it.

Add to that all the security issues with wireless. It is just not useful.

I honestly don't believe they'll ever get wireless networking reliably enough for real business use. Sure, people use it now, but it's such a pain that most of the people I know refuse to rely on it. The few that do also happen to be those "hell users" known so well to IT support. Wired is the only way to go for business.
 
dgingeri said:
GAH!!!! Wireless!! Sin!!

Add to that all the security issues with wireless. It is just not useful.

I honestly don't believe they'll ever get wireless networking reliably enough for real business use. Sure, people use it now, but it's such a pain that most of the people I know refuse to rely on it. The few that do also happen to be those "hell users" known so well to IT support. Wired is the only way to go for business.
Click to expand...
You do not belong in the enterprise. Let me know how burying your head in the sand works out for you.

We use enterprise level equipment, and we get enterprise level reliability from it. Cisco Aironet 1131AG's lightweight using redundant 5508 wireless lan controllers. Wireless is plug and play with us.

Especially since we use EAP/TLS and deploy the WPA2 keys via certificates using group policy. It couldn't be any easier for our users. If they are on our domain, wireless works, anywhere, everywhere. It auto joins, it "just works"

It sounds like you either
A) aren't using enterprise level equipment or
B) don't know how to use enterprise level equipment.
 
cyr0n_k0r said:
You do not belong in the enterprise. Let me know how burying your head in the sand works out for you.

We use enterprise level equipment, and we get enterprise level reliability from it. Cisco Aironet 1131AG's lightweight using redundant 5508 wireless lan controllers. Wireless is plug and play with us.

Especially since we use EAP/TLS and deploy the WPA2 keys via certificates using group policy. It couldn't be any easier for our users. If they are on our domain, wireless works, anywhere, everywhere. It auto joins, it "just works"

It sounds like you either
A) aren't using enterprise level equipment or
B) don't know how to use enterprise level equipment.
Click to expand...

Fortunately or unfortunately, I'm not on the IT team here. I don't even know what APs they're using. They're locked behind metal plates with the antennas sticking out.

However, I have had issues with wireless every single place I've worked.

I was desktop support for one company when we deployed Cisco APs, and had multiple problems. For the first 6 months, easily half of my support calls were for wireless. Then people slowly lost confidence in it and quit using it so much. the calls declined because people just figured it wouldn't work anyway.

The next place I worked, we tried deploying laptops locked onto metal carts with wireless connections in a hospital. Those lost connection constantly. Management recalled those repeatedly for rework and redeployment, and they finally gave up and started using CAT6 cables attached to the carts. Those were business class HP systems and Cisco APs.

now I'm at my current company, with many problems with wireless. Sure, some people work fine with it, but my laptop hates it.

On top of all that, I have to do support for my whole family (parents' house and 3 sisters' families' houses) and I have never been able to make it stable. All the settings are correct, it just up and disconnects periodically. I even have it at home, but I only use it for my Galaxy Tab. This is with every major brand, and even a couple unknown brand, routers. None of them work right.

That's my experience with it. I'm fine with new technologies. In fact, I love watching them work their way in. If wireless were stable and secure, I'd welcome it. At this point, my experiences with it have never been positive. If a technology fails, it's not worth using.
 
Wireless is great as long as you configure properly and realize its limitations. If you run 5 AP's on the same channel within a couple hundred feet of each other then you will have issues.
 
I guess the qualifier would be, "when setup correctly".

I use our wireless, 8 hours a day. 4402 controllers, 1242 AP's. 802.x authentication.

I guess you have had a bad streak of experiences, but to say wireless is completely unreliable is laughable at best.
 
Sorry I kind of hijacked the thread for a discussion on wireless. I didn't mean to.
 
Just do what most IT managers do, get some one else to do the work, hand it to you and you can take it to the boss and make it look like your own work. :D
 
Forget the chassis style cisco switches, huge PITA to cable with the blades being so close together. Also they are LOUD. If this room will also be your office you will regret it. But if you insist on a 4500 series chassis switch I have 6 I can sell you with blades. We are getting rid of them.

Get the 3750 model, use the stacking connector between them and you can have 3 or 4 of them and they act as a single switch.

Then you can arrange the rack in the order of 2u 48 port patch panel, 1u switch, 2u 48 port patch panel, 1u switch, etc etc. And just use 1ft patch cables to patch from the panel to the switch. As you add patch panels you can add a switch to go along with it.

MUCH MUCH cleaner and easier to add/move/change stuff than trying to remove a bunch of wire management covers, pull a wire through/out, etc etc etc.

Gigabit 3750s can be had refurb for under $3k.

Make sure that any racks you install have enough room to allow for equipment AND a person getting behind them so you are banging your head/shoulders/elbows on crap trying to get behind them. Don't put them too close to the wall or you will regret it.

To the poster who says you need eap-tls (btw certificates are not wpa2 keys) it's not neccesary. For a firm your size you can use peap and mschap and you will be absolutely fine, and then you only need a cert for the NPS/IAS server. No need to deploy certs to client machines.
 
As far as the network switches go, the OP would need to explain his environment before recommending for/against specific switches.

The 3750's are great in IDF use and in certain low -mid size core designs. But if you push line rate on some interfaces or have a mid-size SAN, you will absolutely starve the small buffers on the 3750's and lose packets left and right.
 
use a raised floor- let that place flood once, then explain why you didn't take the measure before hand.
 
ChRoNo16 said:
use a raised floor- let that place flood once, then explain why you didn't take the measure before hand.
Click to expand...
Read his post. The server room in the new building is on the second floor. Raised floor is a waste of money.
 
Lol. I didn't just tell my boss "LOTS OF POWER LULZ." I'm not retarded... Christ. I can't post an internal E-mail here due to our handbook rules; however, it's 9 paragraphs long. :rolleyes:

We'll be running a Cisco Wireless Controller with APs throughout the building. We provide wireless for employees, voice, and our guests on separate VLANs. I'm planning on having 4 Ethernet runs to each desk and many many more drops. Getting connected wont be a problem in this building.

I'll have my cabler run CAT6, as saving money on cabling is less of a concern than future-proofing within reason.
 
Protoform-X said:
I'm planning on having 4 Ethernet runs to each desk and many many more drops. Getting connected wont be a problem in this building.
Click to expand...

Do you really need 4 drops at each desk? I suppose I could see 4 drops at the end of a section like in one of the floor pillars, but doesn't 4 seem a tad bit over kill? With Wireless and pass through ports on VOIP Phones seems way over kill to me... I could see doing 2 per desk


Other than connectivity what is the logic?
 
calvinj said:
Do you really need 4 drops at each desk?
............
Other than connectivity what is the logic?
Click to expand...
We usually try and do (2) dual port drops. 1 on 2 walls within the office.
This way, you can configure an office several different ways depending on user preference without having to bring cables across a room.
 
cyr0n_k0r said:
We usually try and do (2) dual port drops. 1 on 2 walls within the office.
This way, you can configure an office several different ways depending on user preference without having to bring cables across a room.
Click to expand...

We do this too... maybe I misunderstood the final plan for the wiring. I took it as 4 ports in one wall. (for some reason I also envision a cube land type setup like we have too where 2 ports per cube is plenty)
 
You must log in or register to reply here.
Back
Top