Compact, silent and powerful home lab. Possible?

Rivera

Limp Gawd
Joined
May 17, 2011
Messages
154
Hello. Still building my home lab, so here is my thoughts. Maybe someone can recommend better/cheaper/alternative parts...
So my point is to build pretty powerful (because i will do nesting virtualization and emulate cisco ios which is damn CPU intensive) and _silent_ testlab. Not completely silent, of course...

Main software - vmware ESXi. HA, FT, etc. Second "level" - MS Hyper-V. Same purposes. It should run inside ESXi without any problems. There will be some loss of speed, but i do not care since that's for education.

(There will be two of that servers)
Case: Antec ISK 300-150
Pretty compact and should be silent. Oh, and one low-profile PCI-E! Isn't that great? Bad part: custom PSU (like in notebooks). Very bad part...
OR
Case: Supermicro SC510L-200
"Compact" - will fit great on shelves. But 8500 RPM fan... uh. Maybe someone can recommend more silent chassis? CPU TDP will be around 70W. Also i does not know if 200W is enough for my config. By my calculations on full load it will require 198W, but maybe i'm wrong.

Motherboard: Intel S1200KP
Mini-itx, C206, server-grade, good warranity, two gigabit ports. Sadly this thing does not have IPMI like in supermicro :( I does not see any alternatives anyway.
OR:
If i find silent compact 1u chassis, i can install normal-sized mobo.

CPU: http://ark.intel.com/products/52271/Xeon e3-1230
I does not see any alternatives. Quad-core, 3.6 Ghz in TB, max 32 GB of RAM (will not fit on mATX mobo anyway :D), 80 W TDP. For 200$. Oh, and this is server-grade CPU. Just awesome.
UPD: Oh, there is v2 version of this cpu, which is slightly better at same cost. But then i need Ivy Bridge? :(

RAM: Samsung DDR3 1333 Registered ECC DIMM 16Gb

HDD: SATA DOM module for esxi, NAS (in my sig) for VM storage.

Did i missed something?

So here it is. Maybe you have some suggestions?

Oh, and sorry for my bad english, i'm not a native speaker and learned mostly by translating various documentation :D
 
Last edited:
Main software - vmware ESXi. HA, FT, etc. Second "level" - MS Hyper-V. Same purposes. It should run inside ESXi without any problems. There will be some loss of speed, but i do not care since that's for education.

Huh? You can't do nested virtualization. As in one inside the other.

I'm running Xen with passthrough on an mATX chassis with 5 HDDs, all my PCI slots are filled so I'd personally recommend going at least mATX as you'll outgrow the supertiny chassis quickly but maybe that's just me.
 
Why would you even test with nested virutalization? It makes no sense and adds a ton of overhead for the VMs you are going to try to run. Since you are already trying to set this up on a small silent system, that overhead is going to hurt you even more. I would pick 1 hypervisor you want to learn, and use that.
 
Also bear in mind that only one of the NICs on that board are supported by ESXi out of the box and the add on driver I personally haven't had any success with.
 
You can do everything in one box. I have a virtual san and virtrual ESXi servers w/nested VMs in multiple clusters. It's great for testing, especially if you have to do it in production later and want to make sure you know what you're doing. I just had to do a 9-host vDS migration with 500+ VMs and my home lab was key in letting me test each step of the process so I could get my plan documented just right and have it approved for production.
 
Case: Supermicro SC510L-200
"Compact" - will fit great on shelves. But 8500 RPM fan... uh. Maybe someone can recommend more silent chassis? CPU TDP will be around 70W. Also i does not know if 200W is enough for my config. By my calculations on full load it will require 198W, but maybe i'm wrong.

Supermicro's cases are really only designed to work with their own motherboards. They don't use a standard IO shield, instead having knockouts based on a standardized Supermicro layout. Also, this particular case is really only designed to work with very low-power CPUs, such as Atoms or the E3-1220L/E3-1260L. You'd be better off with something like these:

Case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811152087
MB: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182262
 
Supermicro's cases are really only designed to work with their own motherboards. They don't use a standard IO shield, instead having knockouts based on a standardized Supermicro layout. Also, this particular case is really only designed to work with very low-power CPUs, such as Atoms or the E3-1220L/E3-1260L. You'd be better off with something like these:

Case: http://www.newegg.com/Product/Product.aspx?Item=N82E16811152087
MB: http://www.newegg.com/Product/Product.aspx?Item=N82E16813182262

Thanks! I dealed with a lot of supermicro servers at previous work but haven't noticed that o_O
Regarding your advice, the chassis is a bit big, but whole config looks good. But since i decided to go with v2 gen of Xeon e3, i will wait for Ivy Bridge board from supermicro or intel
 
Also bear in mind that only one of the NICs on that board are supported by ESXi out of the box and the add on driver I personally haven't had any success with.

Well i installed a lot of "unsupported" drivers for ESXi at work (usually customers with unsupported RAIDs/NICs), and in most of cases it works.
 
You can do everything in one box. I have a virtual san and virtrual ESXi servers w/nested VMs in multiple clusters. It's great for testing, especially if you have to do it in production later and want to make sure you know what you're doing. I just had to do a 9-host vDS migration with 500+ VMs and my home lab was key in letting me test each step of the process so I could get my plan documented just right and have it approved for production.

Uh. All in one box? Can i see your config? Your test machine should be a beast :D
 
I'm running Xen with passthrough on an mATX chassis with 5 HDDs, all my PCI slots are filled so I'd personally recommend going at least mATX as you'll outgrow the supertiny chassis quickly but maybe that's just me.

As already stated, nested VMs works. Regarding outgrow... Personally i does not know how i can "outgrow" my config. Since i use external NAS i do not need HDD slots at all. PCIe? Maybe for additional NIC. Any other option?
 
Thanks! I dealed with a lot of supermicro servers at previous work but haven't noticed that o_O
Regarding your advice, the chassis is a bit big, but whole config looks good. But since i decided to go with v2 gen of Xeon e3, i will wait for Ivy Bridge board from supermicro or intel

The 512 is less than 3" deeper than the 510 (11.3" vs. 14"). And the 512's power supply gives you much more headroom.
 
Uh. All in one box? Can i see your config? Your test machine should be a beast :D

It's easily doable, I run all sorts of crap on my MBP. If all you're looking is to get the motions down then you can do a lot with a semi-decent machine and workstation or fusion.
 
Well i installed a lot of "unsupported" drivers for ESXi at work (usually customers with unsupported RAIDs/NICs), and in most of cases it works.
Likewise, but Chilly's present additional driver for the 82579 interface will cause issues with the 82574L interface, its fine when the 579 is the only card present but not when mixed with others.

Check the thread on here for details.
 
So, bringing this back. After playing with MiniITX appliances at work (we develop DLP software) i think it's not really great idea to build homelab in m350-like chassis and RAM is limited to 16GB just because of two slots on mobo :(

So, my ATM plans:

Supermicro 1U 512L-200B chassis. Looks like it's pretty silent? Oh, and it do have PCI slot, which is awesome. But i still do not know how noisy will be PSU.

X9SCL-F mobo. There is a lot of problems around that 82574L/LM interfaces with ESXi, but looks like community almost solved them. Oh, i can go with cheaper mobo without IPMI because i just got 4-port ip-kvm for a great price...

Xeon E3-1230v2.

32 GB RAM. I found this on ebay: http://www.ebay.com/itm/Komputerbay...5752554?pt=US_Memory_RAM_&hash=item20cc40cdea - yes, it does not have ECC, but i hope it will work without any problems?

SATA DOM or USB Flash for ESXi, still do not know. 120GB SSD for persistent VMs (mostly stuff for my internal network), all test stuff will be booted from my NAS.

Whole price will be around 600USD/node, which is great for me.

I plan to build two of that machines, because sometimes i test really CPU/RAM/both intense apps.

Any suggestions? Maybe removing sink and replacing it with low-profile CPU fan to lower blower RPMs?
 
As an eBay Associate, HardForum may earn from qualifying purchases.

I read whole thread. But in fact: i have some of that supermicro chassises at work, with 65W i5 CPU. Works! But fan is a bit dirty so it needs to be replaced...
Also i posted additional questions, like, for example, RAM.
 
So, bringing this back. After playing with MiniITX appliances at work (we develop DLP software) i think it's not really great idea to build homelab in m350-like chassis and RAM is limited to 16GB just because of two slots on mobo :(

So, my ATM plans:

Supermicro 1U 512L-200B chassis. Looks like it's pretty silent? Oh, and it do have PCI slot, which is awesome. But i still do not know how noisy will be PSU.

X9SCL-F mobo. There is a lot of problems around that 82574L/LM interfaces with ESXi, but looks like community almost solved them. Oh, i can go with cheaper mobo without IPMI because i just got 4-port ip-kvm for a great price...

Xeon E3-1230v2.

32 GB RAM. I found this on ebay: http://www.ebay.com/itm/Komputerbay...5752554?pt=US_Memory_RAM_&hash=item20cc40cdea - yes, it does not have ECC, but i hope it will work without any problems?

SATA DOM or USB Flash for ESXi, still do not know. 120GB SSD for persistent VMs (mostly stuff for my internal network), all test stuff will be booted from my NAS.

Whole price will be around 600USD/node, which is great for me.

I plan to build two of that machines, because sometimes i test really CPU/RAM/both intense apps.

Any suggestions? Maybe removing sink and replacing it with low-profile CPU fan to lower blower RPMs?

Using Non-ECC memory on a Xeon Processor usually doesnt work...
my first comment was going to be in Post1 with that Intel board the bios in it doesnt support VT-X VT-D cause they allow it in the bios (could be added they just didnt want to)

Cheap memory is one of those things i frown upon just because i like good memory thats warranty capable...
 
As an eBay Associate, HardForum may earn from qualifying purchases.
Supermicro 1U 512L-200B chassis. Looks like it's pretty silent? Oh, and it do have PCI slot, which is awesome. But i still do not know how noisy will be PSU.

You you'll probably be OK, but I'd still recommend getting the 512L-260B for a little more breathing room in the power supply. Also, having a PSU that won't get so close to it's ceiling means the fan is less likely to rev up to full speed, so it'll be quieter.


X9SCL-F mobo. There is a lot of problems around that 82574L/LM interfaces with ESXi, but looks like community almost solved them. Oh, i can go with cheaper mobo without IPMI because i just got 4-port ip-kvm for a great price...

No, the 82574 is the unit supported natively in ESXi. The 82579 is the one that requires non-standard drivers to be patched in. Get the X9SCL+-F or X9SCM-iiF.


Xeon E3-1230v2.

32 GB RAM. I found this on ebay: http://www.ebay.com/itm/Komputerbay...5752554?pt=US_Memory_RAM_&hash=item20cc40cdea - yes, it does not have ECC, but i hope it will work without any problems?

Unknown. These SuperMicro boards only officially support ECC RAM. Non-ECC might be problematic. Also, who is KomputerBay? Get RAM from a known-good manufacturer (e.g., Corsair, Crucial, Kingston, etc.).


SATA DOM or USB Flash for ESXi, still do not know. 120GB SSD for persistent VMs (mostly stuff for my internal network), all test stuff will be booted from my NAS.

Just put ESXi on a USB drive. putting it on a SATA drive would be wasteful.


Whole price will be around 600USD/node, which is great for me.

I plan to build two of that machines, because sometimes i test really CPU/RAM/both intense apps.

Any suggestions? Maybe removing sink and replacing it with low-profile CPU fan to lower blower RPMs?

Stick with the SuperMicro heatsink.
 
As an eBay Associate, HardForum may earn from qualifying purchases.
Thanks, very helpful! Ordered ECC RAM and still waiting for seller to respond.
 
I hope you ordered unbuffered ECC

Sure, i know the difference. However, i never dealt with building servers myself (except my NAS in signature, that was 2007 or 2008, got chassis full of HDDs and PSUs but without any other parts) - that's my third workplace, and always same shit - financial dept asks me for N of cpus/cores i need, amount of ram and etc and they order complete server, usually supermicro or dell. Maybe someone find that good, but i not since i can only replace parts of server and not build one myself :(

Also, special thanks for @BlueLineSwinger for recommending me X9SCL+-F. 30$ higher, but still a great deal and complete supported by ESXi. I plan to finish my homelab in about 4-5 month, since i live in Russia and shipping with russian post... well... a bit slow.
 
So my final build that got shipped:
Intel Xeon E3-1230V2
16 GB of unbuff ECC RAM
X9SCL+-F mobo
120 GB Crucial SSD (this is a "controller" node, it will hold persistent VMs, like vCenter and some other networking stuff like webproxy/freeradius/etc, currently i'm using Atom D525 for that purposes which is dying from heavy load)
512L-200B - finally replaced broken blower fan on one of these chassises at work and hey! They're quiet!
 
Good storage gets loud, as do good switches. I leave that shit at work.
 
got it today. Everything great except one thing: PSU is noisy.
If anyone interested, i can provide short video with server working.
 
Well... i need your help again.
PSU of that server constantly making high-freq noise. My friend said it is "choke", or "throttle", or "reactor" - i do not know how to say it in english :( Any ideas how i can fix that? Server itself is quiet (comparable to PS3 in terms of fan noise), but PSU is LOUD. And it's only 200W! It's actually overlapping any sound in my room - i can hear that beeeeeeeeeeeee everywhere, no matter if i listening to music or watching film...
 
RMA PSU if they'll let you, not sure SM's policy on replacing PSUs for being too noisy as they're intended for areas like server rooms not bedrooms, but on the off chance they won't you'd have to replace the PSU yourself as I doubt you want to be tearing it appart to desolder then resolder in new coils.
 
RMA PSU if they'll let you, not sure SM's policy on replacing PSUs for being too noisy as they're intended for areas like server rooms not bedrooms, but on the off chance they won't you'd have to replace the PSU yourself as I doubt you want to be tearing it appart to desolder then resolder in new coils.
So this is HW, uh, "bug"? RMA is not an option - ordered that in USA with shipping to RU, probably i will just replace it.
 
Well... i need your help again.
PSU of that server constantly making high-freq noise. My friend said it is "choke", or "throttle", or "reactor" - i do not know how to say it in english :( Any ideas how i can fix that? Server itself is quiet (comparable to PS3 in terms of fan noise), but PSU is LOUD. And it's only 200W! It's actually overlapping any sound in my room - i can hear that beeeeeeeeeeeee everywhere, no matter if i listening to music or watching film...

as I understand. the PSU fan make that high/pitch noise when running not fully speed. this is common since the ball bearing PSU fan is "loud" compared with Desktop fan.

you can replace the PSU fan with non ball bearing fan, for example, fluid dynamic, 1 ball, or even an extreme(not expensive compared with others) way with a good sleeve fan:).
you need 40mm fan. buy a regular 40mm fan with 4000-5000RPM, it would reduce the PSU noise/loud.

speaking on SM PSU:
mine is 1U SM 512... something. It has 260W, Had to replace PSU fan with a good non server fan(replace with 40mm Sunon brand, 1 ball fan ~4500RPM as I remember).
watch you PSU after replacing the fan.
Make sure the heat is coming uut from the fan and the PSU external case is not really hot aka touchable
Since mine pulling around 100W-120W on full power. I do not worry that 260W PSU would be overheated since drawn less 50% than advertised.
 
as I understand. the PSU fan make that high/pitch noise when running not fully speed. this is common since the ball bearing PSU fan is "loud" compared with Desktop fan.

you can replace the PSU fan with non ball bearing fan, for example, fluid dynamic, 1 ball, or even an extreme(not expensive compared with others) way with a good sleeve fan:).
you need 40mm fan. buy a regular 40mm fan with 4000-5000RPM, it would reduce the PSU noise/loud.

speaking on SM PSU:
mine is 1U SM 512... something. It has 260W, Had to replace PSU fan with a good non server fan(replace with 40mm Sunon brand, 1 ball fan ~4500RPM as I remember).
watch you PSU after replacing the fan.
Make sure the heat is coming uut from the fan and the PSU external case is not really hot aka touchable
Since mine pulling around 100W-120W on full power. I do not worry that 260W PSU would be overheated since drawn less 50% than advertised.
What connectors are on PSU fan? same as others? Found this one. Watched for PSU under some load and air flow coming from PSU is pretty cold. Probably i have same SM512-260.
 
If it's a high pitched brain peircing squeel that reminds you of nails on chalkboard it's much more likely to be coil noise than the fan. Easiest way to test this is turn it on and jam a small screwdriver into one of the fan holes about a quarter inch to stop the fan (or power it up with a screwdriver already in there if the fan is more than 1500rpm or so which it probably is to avoid damage) if it squeels with the fan stopped, it's a coil.

edit: Oh and not sure about your SM PSU but my SM 40mm PSUs have the fans running all the time whether the system is powered up or not (they run slower but still noticably when the system is off), so if it's a problem with the fan running to slow you'd probably hear it when it's off as well.
 
Last edited:
What connectors are on PSU fan? same as others? Found this one. Watched for PSU under some load and air flow coming from PSU is pretty cold. Probably i have same SM512-260.

It looks like -> http://www.newegg.com/Product/Product.aspx?Item=N82E16835119050
or
http://www.newegg.com/Product/Product.aspx?Item=N82E16835706027

I replaced with my available Sunon 40mm, not bought.
~5000-7000rpm should adequate when the system does consume less 50% than advertised on psu

please remember to test your PSU after replacing and load with full power of you mobo/system. since the "top" psu cover is aluminum is the real heatsink, you know by touching or measure temperature on that "top" cover.
 
If it's a high pitched brain peircing squeel that reminds you of nails on chalkboard it's much more likely to be coil noise than the fan. Easiest way to test this is turn it on and jam a small screwdriver into one of the fan holes about a quarter inch to stop the fan (or power it up with a screwdriver already in there if the fan is more than 1500rpm or so which it probably is to avoid damage) if it squeels with the fan stopped, it's a coil.

edit: Oh and not sure about your SM PSU but my SM 40mm PSUs have the fans running all the time whether the system is powered up or not (they run slower but still noticably when the system is off), so if it's a problem with the fan running to slow you'd probably hear it when it's off as well.

when off, the fan does not spin on mine (SM 1U 260W PSU).
my correction on my SM 1U:
the fan always on full speed ("pitch noise"), and gone when replace with lower non-server RPM fan.
 
More fail: it's both coil noise and shitty fan. Ordered Seasonic SS-300M1U - a bit pricey, but i had alot of good experience with desktop seasonics. Should get it within 1-2 days. More to come.
 
Now this is strange. Got SS-350M1U, and even if it mentions EPS12V there is no 8-pin connector... any ideas?
 
All the pictures and documentation on that MB show it only uses a 4pin anyway so what's the issue?
 
All the pictures and documentation on that MB show it only uses a 4pin anyway so what's the issue?

I tried plugging it (was in hurry so haven't tried all variants :D) with no luck. It have 24+4 pin, isn't that ATX12V?
BTW, two downsides of that PSU:
Will not fit in that small supermicro by default, requires additional 1-2mm on chasiss hole for PSU (done with rasp). After that fits perfectly.
There is much cables (and they are long!) and i doubt that there will be space for second hdd (near PSU).
Anyway, as testing i plugged it into one of our testing work servers. It does not start fan when on <50% load (typical scenario on my home server, huh), and even when it pypasses 50% of load, since there is PWM, it's quiet. If anyone thinking about having quiet 1U server at home with <350W load, well, this is PSU you're looking for :)
 
Back
Top