Gem for those building a virtualization home lab

P

pjkenned

[H]ard|Gawd
Joined
Jan 8, 2010
Messages
1,971
Hey all. Usually hang out on the DC forums (go Team 33), but we recently found Dell XS23-TY3's on ebay very inexpensively. Basically they are 4 dual processor nodes in a 2U chassis, much like the Supermicro 2U Twin2. Cost is just under $1,100 shipped for four DP nodes w/ 24GB RAM each.

Lots of information links both in this forum post (details around drivers, spare parts and etc.) and the post here on the Dell C6100 XS23-TY3 (and why that model number is important.) Projects underway to tame the 77dba jet engines. Power consumption is not bad either.

This forum gave me the 2U Twin2 idea when I was looking for a virtualization cloud platform. Thought I would share while these are still very easy and inexpensive to get. Put three in colo (two live boxes and one cold spare chassis) and have one incoming for a home virtualization/ cloud lab. Here is the DC setup:
STH-Colo-10U-Rear-Picture.png


Currently testing Proxmox VE
 
Last edited:
I'll be keeping an eye on this thread to see if the user can get the noise levels reduced. When I had a server running full-bore in my office at my old apartment I had it in the closet, which reduced noise quite a bit. I think this would be an awesome home lab if you can dampen noise to levels.
 
benzino_86 said:
I'll be keeping an eye on this thread to see if the user can get the noise levels reduced. When I had a server running full-bore in my office at my old apartment I had it in the closet, which reduced noise quite a bit. I think this would be an awesome home lab if you can dampen noise to levels.
Click to expand...
Same. I had a SC942i running full tilt six feet from my head while I slept for several years. I'm done with anything louder than a whisper for home use. Maybe in a few years, I can build myself a sound deadened room dedicated to my server setup, but until then...
 
They are basically OK to stick behind a closed door when the fans are at lower speed. At higher speed not so good. Hoping that project works.
 
pjkenned said:
They are basically OK to stick behind a closed door when the fans are at lower speed. At higher speed not so good. Hoping that project works.
Click to expand...
Me too... I'm going to be out of footroom under my desk before I even get all my hardware. Converting to a rack would be wonderful. Plus, blinky light syndrome.

By the way, is that STH site yours? If so, well done! I've been reading it for a couple months now. I'm a fan of the content, and even the forums are full of useful information.
 
TeeJayHoward said:
Me too... I'm going to be out of footroom under my desk before I even get all my hardware. Converting to a rack would be wonderful. Plus, blinky light syndrome.

By the way, is that STH site yours? If so, well done! I've been reading it for a couple months now. I'm a fan of the content, and even the forums are full of useful information.
Click to expand...

Totally agree on the rack stuff. Thanks for the kind words. It is a small niche but trying to get a community resource on the segment.
 
I bought the 2 blade (4 CPU 96GB RAM) unit for $699.

Protip for those who want the 4 blade model of this for like $1,399.... Get two of the $699s, otherwise you end up with still only one PSU. The PSUs seem to sell for $199, but I've seen way lower on a few rare occasions.

I worry these are going to get all bought up by shared webhosting providers. I don't know how you could even consider building your own servers or other models compared to these.
 
Brak710 said:
...I worry these are going to get all bought up by shared webhosting providers. I don't know how you could even consider building your own servers or other models compared to these.
Click to expand...

Ummm - mostly its the shared webhosting providers selling them. Or, more specifically, the shared web hosting providers generally lease them and these are units that are off-lease. When the leases are renewed they are upgrading to Sandy-bridge (soon Ivy-bridge) units to gain capacity and save power/HVAC - and in the process spewing these out the back end into the resale channel.
 
That is awesome! Going to have to see if I can find some, been debating on building a new server but I find hardware is so expensive now. It's gone way up in the past years, probably due to lack of demand. Even a white box server will cost me a good 2-3k. Noise is never an issue for me, I've just gotten used to walking in my house and it sounds like a small telco central office. :D Only thing missing is the distinct sound of the DMS fans.
 
So tempting, hopefully i will be picking one of these up in the next couple of weeks. Right now i am running ESXi on (2) PE2950 Gen2. Would really like to mess around with one of these.
 
holden j caufield said:
very interesting, seems like it quite costly to make this thing quiet enough for my home office use.
Click to expand...

Less than $100 and 4 hours the first time you do one...four EverCool fans and the labor to mod the cables. Doesn't seem all that costly to me - about 10% of the purchase cost to tame it.
 
Yea we have a bunch of information on it on the thread linked above. Also have searches to a lot of the spare parts/ mezz controllers.

Here is PigLover's post on taming the Dell C6100 (noise wise)

I ordered supplies to do the same.
 
Got mine in today, strained a muscle in my left arm trying to get the top cover off haha.
 
What's the deal with addin cards to add additional network ports etc on these? If they have at least one per modular server..i'm all over it.:)
 
Vader said:
What's the deal with addin cards to add additional network ports etc on these? If they have at least one per modular server..i'm all over it.:)
Click to expand...
It looks to me like they support a single LP card. (Right above the VGA port)

edit: Top-right of this pic:
$(KGrHqN,!isFC73tFvJKBQ28Wdm32Q~~60_57.JPG


Better pic of the rear:
$T2eC16NHJF8E9nnC6IONBQ+K-3u5Jg~~60_57.JPG


Both images stolen from the eBay listing above.
 
Omfg...you guys are going to lead me to divorce..I am in the middle of an upgrade to my lab and now this. I don't like the fact that I can't have my standard 8 nics but I can live with it, i'm sure.;)

I may pull the trigger tonight..damn you..damn you aaaaaaaaaaalllllllll!!:eek:
 
Vader said:
I may pull the trigger tonight..damn you..damn you aaaaaaaaaaalllllllll!!:eek:
Click to expand...
Tell me about it. I am just about to put the finishing touches on my E3-based lab (which is supposed to last me another 3-4 years), and now I'm wondering if I mis-spent my monies.
 
Vader said:
Omfg...you guys are going to lead me to divorce..I am in the middle of an upgrade to my lab and now this. I don't like the fact that I can't have my standard 8 nics but I can live with it, i'm sure.;)

I may pull the trigger tonight..damn you..damn you aaaaaaaaaaalllllllll!!:eek:
Click to expand...

Sorry :) We have a bunch of folks in the STH community with these now. I was a bit worried prior to my first but I'm on #4 and a few folks have more than I do now. We had no idea before the first one was delivered but now they seem like a gem.

I do really like the Xeon E3 but for now just using these as inexpensive nodes. The E3 hits higher clock speeds which is nice. At sub <$1000 for 4x systems where one just needs to add drives makes this not too bad in terms of cost.
 
For me it'll be even cheaper, my ESXi box has been diskless since forever. iSCSI/FC boot ftw :D
Next upgrade would be adding the QDR mezz cards and pci-e dual nics to servers and one dual QDR card to NAS/SAN (which is also going bit of a monster)...
MSA70 and Dell MD1000 hooked into single supermico 2U box, yummy :D
 
I almost pulled the trigger.. then I noticed the shipping cost. $421 seriously? That's like half the price of the unit. I'd also hate to see the custom's bill on that. I guess at the end of it all it would still end up costing as much as if I just ordered myself a Supermicro Twin off a retail site. Damn these look like nice servers though. Are the nodes also hot swap? Like can you pull out out without affecting the others?
 
Red Squirrel said:
I almost pulled the trigger.. then I noticed the shipping cost. $421 seriously?
Click to expand...

Would have been over 800USD for me if I used UPS so I had my unit shipped over to Oregon to my friend who'll weight the package so it's under 70lbs as he's also shipping that MSA70 to me and evens out the weight. USPS does international shipping for around 180usd which is not that steep anymore.
 
Depending on who you are looking at, ive been quoted anywhere from 150-180 to Canada, not the 455 it says on the listing..




Red Squirrel said:
I almost pulled the trigger.. then I noticed the shipping cost. $421 seriously? That's like half the price of the unit. I'd also hate to see the custom's bill on that. I guess at the end of it all it would still end up costing as much as if I just ordered myself a Supermicro Twin off a retail site. Damn these look like nice servers though. Are the nodes also hot swap? Like can you pull out out without affecting the others?
Click to expand...
 
PigLover said:
Ummm - mostly its the shared webhosting providers selling them. Or, more specifically, the shared web hosting providers generally lease them and these are units that are off-lease. When the leases are renewed they are upgrading to Sandy-bridge (soon Ivy-bridge) units to gain capacity and save power/HVAC - and in the process spewing these out the back end into the resale channel.
Click to expand...

I disagree to an extent. If you saw some of the hardware used by some providers, you'd see how big of an upgrade these would be in power and efficiency.

The density and efficiency is good enough this make starting a hosting venture affordable to invest in. The market is already cut throat as it is, infrastructure based on these would be a huge help.
 
I just emailed them no issues, Peidnam or whatever sid i think 180 and mr computer depot or something of the sort said 150 i believe.. Mr Rackables was also reasnable... I think its quoted useing UPS world saver which is air i think hence the high price...

I'm still debating..

Also whats the deal with the Bios upgrade? I read on anonther forum about some not being upgradable.. is that actually an issue? As obviously due to shipping i wouldnt be shipping it back.
 
Angus said:
Also whats the deal with the Bios upgrade? I read on anonther forum about some not being upgradable.. is that actually an issue? As obviously due to shipping i wouldnt be shipping it back.
Click to expand...

AFAIK the bios chip might be fuse byted, ie not writeable or the write enable isn't pulled high etc. Should be flashable by force :D

Flash new chip in programmer -> old chip out -> new chip in. Done, soldering required ofc.
 
Having recently shipped an IBM AS/400 recently, I will tell you from experience the $421 shipping is about right considering the weight of the system, and the amount of packing material required to keep those negligent people in whatever shipping company from destroying it en route... I had to file a claim for the shipment and it was a nightmare... I even had them pack it themselves to their "server shipping specifications" ...
 
Well i guess be that as it may.. if they are willing to ship for 150 why should i say no? Or should i ask to pay more?
 
Brak710 said:
The density and efficiency is good enough this make starting a hosting venture affordable to invest in. The market is already cut throat as it is, infrastructure based on these would be a huge help.
Click to expand...

Yes... have quite a few readers doing exactly this. I know one was purchased by someone at the datacenter when I installed the servers after he saw them.

Got my 24x 2.5" chassis today... no PSUs/ heatsinks though. (They are apparently being shipped to me though.) Going to make a nice home lab machine.
 
Woah, that's awesome. I guess it's kinda useless without having a gig switch though. That's where remortgaging the house may be required. :p
 
TeeJayHoward said:
You can get the 40Gbps NICs for about $100 each, too, if you shop around. Infiniband is dirt cheap these days. Heck, I even saw a pair of 40Gbps switches for about a grand each recently!
Click to expand...

I've seen a lot of 40Gbps IB cards go for $125/piece but i've not seen a 40Gbps NIC yet to be in reasonable range. Yeah also noticed that 40Gbps IB switch, would be bit overkill in home lab, direct h2h connection would be enough :D
 
You must log in or register to reply here.
Back
Top