Small datacenter build-out

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
Figured I would start a similar thread to The Spider outlining the buildout of our new datacenter.

We are renovating our current space (10th floor of an office building). The current datacenter will remain up until the new one is finished. Over one weekend we will move all the current gear and then demo the old room.

We are using this time to replace our aging 15kVA battery backup, 2-post network racks, and current AC units.

Hopefully you guys enjoy this build thread. For now I can leave you with just the current Visio drawing. Build-out is slated to start somewhere around 11/8-12/5.

I'm sure some will say the room is too small and the rack clearances aren't that great but I had very little space to work with. I think we did a pretty good job maximizing the space. We have an IT work area on our 4th floor that we will stage the gear before bringing it up.

Look forward to any questions.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
Floor Plan



Demolition has started...


Old computer room still running...
 
Last edited:

p3n

Gawd
Joined
Sep 29, 2002
Messages
631
Is that a window at the top? I presume servers etc will be 'facing' this? If so where is the hot/cold separation?
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
Correct, 1hr fire rated floor to ceiling window.

We have talked about doing a hot aisle containment but haven't really decided on it yet.

The original plan didn't include the Liebert units, those were just added. We had just planned vents behind the racks to just put the hot air into the plenum space. With the return air vents are on the front of the Liebert's we will need to come up with something. We may need to put a box on the right side of the air handlers that goes up to the ceiling for return air.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
Taking into consideration the return air. We are going to switch out the AC unit and just stock a few spare parts.
 

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
17,991
You should ditch the DS unit and go with two or three CRV's instead. A CRV produces 35KW (10Tons) in a line matching cabinet footprint. It will take up less floor room, provide better air management, more peak cooling capacity, more ideally designed for your senario... and the most important part is that you would have some sort of redundancy. The DS units are great, but they do break down, and when they do, you're going to regret it.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
I'm struggling with the change to the DS from the two Challenger units. We will be stocking spare parts but its still scary for me. Such a little room will heat up quick.

Spare parts we would stock are
microprocessor
fan motor
condenser motor
spare belts & filters
humidifier bulbs
 
Last edited:

Ockie

*** Self Proclaimed Storage King ***
Joined
Mar 2, 2005
Messages
17,991
I'm struggling with the change to the DS from the two Challenger units. We will be stocking spare parts but its still scary for me. Such a little room will heat up quick.

Spare parts we would stock are
microprocessor
fan motor
condenser motor
spare belts & filters
humidifier bulbs

No, not challenger units, but rather CRV's instead of the DS. With a single unit you can't account for ground out protection when your compressor blows, breaker failures, refrigerant leaks, compressor failures (although you should have two on a 12ton unit), main shaft failure, shaft bearing failures, blower fan becoming unbalanced, high head shutout due to dirty coils, and coil cleaning (fans needs to be off which knocks out your associated unit). Having parts on site isn't as much as a problem as much as having contractors available on that moment of time to do a quick swap (a compressor takes a good two days... a shaft replacement takes quite a bit more time)... and these failures usually happen on very hot days when contractor houses are slammed with calls.

Sounds like you already have some challengers... in that case, I would rather add more for capacity while still maintaining redundancy while keeping your costs down.
 
Last edited:

schnell

Gawd
Joined
Jul 22, 2005
Messages
763
I would agree on the CRV statement. You have the space to extend the rack row for 2 of them if you remove the DS unit. Otherwise if your building can supply chilled water you can go that route. I use APC in row chilled water units and they are great.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
We currently have nothing, we were talking originally about going with two 5ton challenger 3000's.

The AC units in our old computer room is are water cooled units but we want to move off the dependency of the building.

Its either a single DS or two 5 ton 3000's. I wasn't a fan of the CRV's but will look at them again. We have no raised floor so piping the units would be interested.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
Had a good talk with our AC vendor today, I got a little more insight into why he recommended the DS (mostly to save costs).

Hes making some modifications to the plans for the Challenger 3000's and I'm pretty sure that will be the route we go.

I appreciate everyones feedback.
 

SysOps

Weaksauce
Joined
Feb 22, 2010
Messages
72
i would go with the two systems, or have a emergency plan.

If you go with the one primary unit, like i had to, then make sure you have a back up plan. i have a contract with a local vendor to have two 5 ton mobile AC units at my site in under 3 hours. I also have two 1.5 tons mobile units for spot cooling and because they are handy..

In the design of my server room, i incorporated automated external air blowers / ductwork gates to permit the use of filtered outside air to save power and for emergency cooling. (at one point it was controlled by a arduino before i got a industrial controller)
During the winter, my primary AC unit usually only turns on for its its automated maintenance cycle.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
finally started the demolition/construction on our new computer room... added a few pictures above

Walls should be going up starting end of next week.
 

NTAuthourity

Limp Gawd
Joined
Jun 9, 2010
Messages
147
Love the time they are taking to make sure it is level. We had Emerson come in to install our Liebert UPS systems and they messed up on 2 Battery units next to the PDU and we cannot open the doors..
 

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211
At the hospital I used to work at they put a Powerware UPS too. Pretty nice systems. Never got to see the management side of it though, there was so much red tape in that place the whole install took over a year and there was a need for a jack to plug the management in and that was a whole can of worms that never got opened.

Are these UPSes dual conversion? I wonder at what point it's more cost effective to do a telco style setup where you have rectifiers, inverters, and a battery bank. I imagine these arn't cheap. I guess they are more turn key than a telco setup though.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
this one set us back around 30-35k, not too bad. Started at 20kva and can be changed to 30 and 40kva with a small payment and a tech onsite to install a license key.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
The day after the UPS turn-up we had someone hit a sprinkler head in an office outside of the room. No equipment was damaged but the flooring was. We are on hold till that gets taken care of. Should be back on track middle of next week.
 

Red Squirrel

[H]F Junkie
Joined
Nov 29, 2009
Messages
9,211
The day after the UPS turn-up we had someone hit a sprinkler head in an office outside of the room. No equipment was damaged but the flooring was. We are on hold till that gets taken care of. Should be back on track middle of next week.
Ouch, it's not a dual stage system? Where I work the system is air pressurized, if a sprinkler head goes it releases the air which triggers one stage. A smoke detector also has to go off before it puts water into the system.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
Computer room is pre-action and requires a smoke detector and a head to pop. Room also has a clean agent and only uses water as a last resort.

The rest of the floor is normal wet pipes
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
few more pictures, things are starting to come together. will post pictures of the racks and what not tomorrow. The fire rated glass was being installed when I left today.





 

cyr0n_k0r

Supreme [H]ardness
Joined
Mar 30, 2001
Messages
5,358
I've never understood why EMC can't put their crap in a normal rack.
Or let you choose a rack model you want and they will put it all together for you then ship it to you.
What if your entire datacenter uses APC racks, then you have this non standard junker rack at the end of a row.
 

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
You can order the gear without the rack...

on a VMAX the way the drive trays are mounted, i dont think a normal rack wouldn't work.
 
Last edited:

RiDDLeRThC

2[H]4U
Joined
Jun 13, 2002
Messages
3,894
65tb usable

Some more detail

19 200gb SLC EFD (Fast CACHE)
18 200gb eMLC EFD (Fast VP)
92 600gb 15k SAS
25 1000gb NL-SAS
 
Last edited:

VisBits

n00b
Joined
Nov 11, 2010
Messages
41
65tb usable

Some more detail

19 200gb SLC EFD (Fast CACHE)
18 200gb eMLC EFD (Fast VP)
92 600gb 15k SAS
25 1000gb NL-SAS
The $/Iop those units give you is terrible. Your into that what half a mill? Those 200gb drives are nearly $3500/ea from emc if not more.
 
Top