A small datacenter build and other projects

[Jay_2]

Hardly draws any power.

Are you 120v?

If so thats about 840w of power! not bad at all. I assume that PDU is maxed at 10Amps?

 

[The Spyder]

Sadly I am a Nikon guy myself. We have some very cool 1.2/1.8 lens combos, but they are no use to me, haha.

Jay, 240v and maxes at 24amps.

 

[adam30k]

My main server room has been awful cramped lately with the racks I ordered a few months back so I decided to replace our SAN's rack. I was also able to replace the PDU's with metered switched ones and install the new 6000va UPS.

Are you doing off-site backup? How?

 

[MrGuvernment]

some awesome gear... where can you get those metered power bars? we are growing here, and i wanted something reliable to see what power draw we have besides our massive UPS which i cant get the serial connection on to work.

 

[The Spyder]

Are you doing off-site backup? How?

Site to Site Metro-E fiber link- for right now we have no offsite backup of our main SAN. Most the data stored on it is purely old imagery, which we will be moving to LTO-6 for archival storage. Those two 140tb SAN's replicate each other every 30 minutes I believe (I need tear in to them one day.) One was meant to be moved offsite- and still may be. It depends on how much our cluster team pounds the 100mg fiber line. Ideally I will do a full sync here and then move it offsite.

 

[The Spyder]

some awesome gear... where can you get those metered power bars? we are growing here, and i wanted something reliable to see what power draw we have besides our massive UPS which i cant get the serial connection on to work.

Tripp Lite is the brand- Vertical 30amp 240v Switched PDU's. If you just want a 30 amp metered - non switched (L6-30P) I will give you a killer deal on a few we were accidentally shipped.

 

[MrGuvernment]

unfortunately for me i am way down in Costa Rica i do have shipping to Miami though... how big are they?? worried they would snap being shipped down or something...

would also have to check my cabinet see if they would fit!

 

[adam30k]

Site to Site Metro-E fiber link- for right now we have no offsite backup of our main SAN. Most the data stored on it is purely old imagery, which we will be moving to LTO-6 for archival storage. Those two 140tb SAN's replicate each other every 30 minutes I believe (I need tear in to them one day.) One was meant to be moved offsite- and still may be. It depends on how much our cluster team pounds the 100mg fiber line. Ideally I will do a full sync here and then move it offsite.

Very cool.. thanks for the write up.

 

[The Spyder]

Glorious words were spoken today by the HR/Operations manager- We want to give you a larger office so that when you hire your new assistant, they can be in the same office. (We are super cramped on space and I would rather have them in the same office with me then in the hall.) Wait, did she said I get a assistant? mgwtfbbq!: Sorry, nerd break down .

 

[marley1]

May be a little too personal...what's your pay like? What about the assistants?

 

[MrGuvernment]

Glorious words were spoken today by the HR/Operations manager- We want to give you a larger office so that when you hire your new assistant, they can be in the same office. (We are super cramped on space and I would rather have them in the same office with me then in the hall.) Wait, did she said I get a assistant? mgwtfbbq!: Sorry, nerd break down .

wow, that pretty awesome!

 

[The Spyder]

May be a little too personal...what's your pay like? What about the assistants?

Little more then I want to discuss publicly.

 

[The Spyder]

Teaser:

 

[The Spyder]

More updates:

I finally decided to finish color coding our network.

My LVE came out and gave me a hand finishing up the racks and some hard wiring.

Due to the short depth racks, we ghetto rigged external rear mounting for the vertical PDU's.

Got the 5406zl mounted.

All lit up.

A better view of the ghetto mounts.

A ground station came in. (20x Dual Hex Core/96gb ram/2x 256gb SSD/40gb Infiniband)

These two packages made my day. I have been waiting a month and a half for these two boxes.

They changed the colors...

Custom cisco board. Looks to be a xeon.

8gb DDR3 Micron.

The standard package.

I can not wait to install these. 2x 5515-x with HA and up to 500 SSL vpn connections.

 

[irrision]

You had any issues with those Tripplite stick PDUs? Used to use those and I'd say the failure rate was maybe 1 in 25. Switched to the APC units and haven't had a single one fail in the 3 years since I replaced them.

 

[The Spyder]

None so far. I will keep that in mind. Thanks.

 

[The Spyder]

This week has been... crazy. And it is only Tuesday night!

Anyways, been working on setting this up.

Hopefully I can get some input here on my design. After I get back from buying some fireworks, I will get some Visio drawings up.

 

[techtips]

Spyder, do you configure the HP switches using CLI? how would you compare them to some of the cisco hardware?

 

[TheFriskyPen]

Anything new Spyder?

 

[l3thal6]

Your posts are an inspiration!

I've been in IT professionally for about 4 years now and slowly moving up, I hope that I can someday work at an awesome place like this.

Spyder, do you configure the HP switches using CLI? how would you compare them to some of the cisco hardware?

I am interested as well, I've got a consulting contract where I may need to choose HP or Dell as the network vendor depending on client requirements.

 

[The Spyder]

Busy as always .

The HP switches are super easy to config. I have been pretty happy with them. The cmd line is very close to cisco.

I finally got a office. I actually liked working in the communal area, but this way I can shut the door and focus

A tough book. Neat little thing.

My LVE has been great... but common! Haha. Oh well, it matches upstairs too. (Patch from racks to network closet).

The former electricians did a terrible job grounding the racks, so I had it fixed.

Doing some performance testing on SSD's in my "loads" of spare time...

Keeps my monitors level

Some light reading showed up.

Got one of the engineering R+D labs racked finally.

I am going to need more plugs....

More power strips.

Back of the engineering lab. It is only 1/3rd complete.

Cable fail... Thanks QA!

Nice and neat bundle for the top of rack patches.

So this is what the insides of a JBOD looks like after it has been on a airbase in the desert for 6 months.

Stickers... Had my Senior EE try to plug in to one of our 240v c15 strips last week... Not good.

First rack in the warehouse, the ports go to patches at the top of the other racks.

Decided to punch this one down myself. Flipping it over is 10x easier then standing in the rack.

My finished job.

Vs the LVE.

Tool!

I had a 4000cfm 1hp 24" roof exhaust fan installed in the warehouse to suck out some of this heat. It's working great so far. (Louvers were installed in the roll up door as a entry to not create negative pressure.

My site to site fiber link is in the works. A crew came and pulled the first run. Not super happy with the quality of the work to be honest.

I would have put the hole on the other side...

Electrical tape is holding the fiber to the rack...

Again here. I made them come out and fix it. This was just plain shotty. The supervisor came out himself, took pictures and ripped his guys a new one... Can't say they did not deserve it.

Back of the termination.

 

[The Spyder]

I ended up a bit overloaded and brought in a outside consultant to setup the ASA5515x. One of the requirements is that he documents/goes over everything with me so I can manage them.

Small daily update: Got my two Supermicro 1u e3 boxes for a NVR at each location. Should be fun .

Some NeatPatches showed up too. Still need a few more.

 

[ToX]

awesome stuff Spyder. glad to see the workload hasnt killed you yet.

 

[MrGuvernment]

question, when your going from patch to switch, do you match numbers or just location

 

[The Spyder]

For the Office I do my best to port match the numbers to the switch. For my warehouse racks, it won't quite work that way, plus I am realizing how annoying a 48 port patch panels are when you really only have 44 ports after 4 are taken by the 4gb uplink.

 

[Shadowspawn]

Hold up, WTH? That looks like the old IBM that my dad would bring home on weekends, in the 80's - years before laptops were even dreamed of. And next to it is a box of 5.25" floppies?

Is that the assistants computer? LOL! It belongs in a museum!

 

[cymon]

For the Office I do my best to port match the numbers to the switch. For my warehouse racks, it won't quite work that way, plus I am realizing how annoying a 48 port patch panels are when you really only have 44 ports after 4 are taken by the 4gb uplink.

I just punch down the copper uplink ports as the last couple ports on the patch panel. However, I've only had the chance to use cheaper switches that are *exactly* 24 ports.

What you could try is mounting a surface mount box inside the rack, then adding four keystone jacks inside there.

 

[jeffmoss26]

Very nice work!!
What do you mean by 'LVE' in a few posts?
BTW, we use Dell switches that stack on the back of the units, so we can use all 48 ports in the MDF.

 

[The Spyder]

Hold up, WTH? That looks like the old IBM that my dad would bring home on weekends, in the 80's - years before laptops were even dreamed of. And next to it is a box of 5.25" floppies?

Is that the assistants computer? LOL! It belongs in a museum!

Compaq - I got it when I was 12? My school had a ton of used computers donated and this was the only one no one knew what to do with ( the rest were 386/486). It sat in storage until just recently. We had a "new" employee starting, who formerly worked for the company and came back. HR + Engineering wanted to prank him, soooo I dragged this in. I never thought they would actually set it up, but it worked... for about 15 minutes playing The Sims. Haha.
It let out the magic blue smoke though when they tried swapping to kings quest.

 

[The Spyder]

I just punch down the copper uplink ports as the last couple ports on the patch panel. However, I've only had the chance to use cheaper switches that are *exactly* 24 ports.

What you could try is mounting a surface mount box inside the rack, then adding four keystone jacks inside there.

The uplinks are on a separate panel, but I have 48 data ports. Thankfully we still have some analog phones, so 5 jacks weren't being used. It all worked out, but it is sure annoying.

 

[The Spyder]

Very nice work!!
What do you mean by 'LVE' in a few posts?
BTW, we use Dell switches that stack on the back of the units, so we can use all 48 ports in the MDF.

Low Voltage Electrician.

Sadly these are all tied back uplinks to the main switch, as I only need 1 switch (maybe two) in each telcom closet. Each closet has 48 ports of Data, so I am stuck either A) Leaving a few disconnected, or B) Installing a second switch for 4 ports... It is rather annoying. I guess I could use a stacking switch, uplink in to a smaller 24, then stack a 48- but that is a waste of a switch.

 

[Red Squirrel]

Awesome stuff, is it actually fun to do, or is it more stress because of short deadlines? I remember when I worked at the hospital and got to see some neat deployments (not this big mind you) but because of the tight deadlines I found it was more stress than actual fun. I like playing around with a product I'm new to before putting it live, but we were always forced to put it live right away without any testing. So I always found that tough.

And good idea on those 240v tags, We had one rack that they fed with 208 and nobody told us. We learned the hard way.

 

[The Spyder]

Sadly the stress of taking care of all of this is finally getting to me. Which is why I am on vacation for the next week . There are so many critical things that need to be done and between the warehouse datacenter build, rebuilding the entire network, replacing my ESXi servers, rebuilding all 4 of my NAS's (combined sum of 1/2 a petabyte), and doing daily support... My brain explodes some days...

Apparently we had the same thing happen. Software engineer did not check to see if his powersupply was auto switching and POOF.

 

[Brak710]

Sadly the stress of taking care of all of this is finally getting to me. Which is why I am on vacation for the next week . There are so many critical things that need to be done and between the warehouse datacenter build, rebuilding the entire network, replacing my ESXi servers, rebuilding all 4 of my NAS's (combined sum of 1/2 a petabyte), and doing daily support... My brain explodes some days...

Apparently we had the same thing happen. Software engineer did not check to see if his powersupply was auto switching and POOF.

I'd fill in for you for a week for free.

I wish I got to work on stuff like that, everything we do is "need a SAN? Just call EMC, order the best one, have them install it, and forget about it." No fun at all.

 

[Langly]

Hold up, WTH? That looks like the old IBM that my dad would bring home on weekends, in the 80's - years before laptops were even dreamed of. And next to it is a box of 5.25" floppies?

Is that the assistants computer? LOL! It belongs in a museum!

5.25 floppies are some of the most secure media you can put files on Cause no one has a drive to read it!

That computer on the desk may actually be worth something, Spyder you should look into that lol

 

[The Spyder]

Update:
Got my Supermicro e3 xeon 1u's in. 1230 with 16gb ram and 2x128gb SSD/2x3TB. Making one in to a basic NVR. The other in to a management box for my second site.

Some neat 14awg Y cables

Got a call yesterday at 3pm telling me the fiber install was moved to today. I had not had a chance to replace our telco rack in the termination closet yet. Guess who got to work till 10:30 last night.

Went to get some tools from my trunk and found a 2011 M3 and a 2010 M5 parked next to me... I can dream...

That went quicker then I thought....

This guy showed up downstairs today. E30 M3's are raaaare.

Conduit being pulled for the fiber...

Did I mention the door won't shut with the neat patches

And that is where they had to stop today. The tenant downstairs kicked out the construction guys despite having prior asked if they could be in there and had full approval from the building management. The business is weird, the owner is weirder. Because of this, I have no fiber and no idea when it is going to be installed. All for 3"ft of " conduit running through their ceiling... There is more to it, but I am ready to go home. Hopefully this will get resolved.

 

[jeffmoss26]

Your cleanup job looks a world better! Nice job...hope you can get your fiber soon!

 

[haunter]

nmmmm e30 M3

nice work too, you have some very sweet stuff!

 

[schnell]

Nice cleanup on that mini rack! But I hope you had nothing to do with the purchase of the Baracuda or the Netgear switches

I used to have an e30 (non m3) it was a sad day when I let her go.

 

[The Spyder]

Read back a bit, there is tons of new gear waiting to be installed .