$1,500-$1,600 budget for 2-node home VM setup

jimphreak

jimphreak

[H]ard|Gawd
Joined
Nov 27, 2012
Messages
1,714
Ok so I'm trying to to put together a build list for a 2-node VM server setup at home. I will be running 2 production VM's (a media server that does 3-4 simultaneous transcodes daily) and a pfsense VM. All other VM's will be for testing/learning. The main reason for 2 nodes is to keep my media server as HA as possible but also for testing other HAVM's.

I already have 32GB of ECC RAM for one of the servers and I have two Micro-ATX cube cases for each as well (space is an issue otherwise I would have gone for rack mountable hardware).

So that leaves about $1,500-$1,600 for the rest of the parts. I've tried to pick the Mobo first or the CPUs first but I'm going around in circles trying to determine the best setup for my needs. It's kind of a mixed bag I know since I'm running a few production VMs but the rest is for testing.

I will have a dedicated NUS device (parts are already in I just have to configure it) where my media and backups will be stored. Contemplating using it to store HAVM's but I'm not sure the performance will be good enough. I was thinking of using 1-2 SSD's in each physical node for local VM storage and replication.

I've been leaning towards going with a SuperMicro X9/X10 board and Xeon E3's but I can't figure out if that'll be sufficient enough for me. Since my form factor is Micro-ATX that pretty much means max 32GB of RAM per server as well.
 
It is cheaper, by half, to go with (2) Dell C1100 or (2)HP G6s loaded with 72GB of ram. The power draw is minimal, heat and noise are also acceptable. However a limitation of the C1100 is that it only has 4 hot-swap bays (this may or may not affect you).
 
I looked at the C1100s and wish I could go with them but I don't have the space for rack mountable servers right now. I don't even have the space for full PC towers. Plus these servers will be in my bedroom so noise is a factor. All I can fit right now are these two Micro-ATX cases stacked on top of each other which is why I've settled on them.
 
jimphreak said:
I looked at the C1100s and wish I could go with them but I don't have the space for rack mountable servers right now. I don't even have the space for full PC towers. Plus these servers will be in my bedroom so noise is a factor. All I can fit right now are these two Micro-ATX cases stacked on top of each other which is why I've settled on them.
Click to expand...

Fair enough. I wish I could assist you with picking out mATX stuff but I don't know what is good and what isn't for VM in that field.
 
klank said:
Fair enough. I wish I could assist you with picking out mATX stuff but I don't know what is good and what isn't for VM in that field.
Click to expand...

What would be your suggestion for a non micro-atx build (still not rack mount hardware though) just so I have some food for thought?
 
You could always go this route. Not really a rack..but serves the purpose.

That's three ESXi hosts, one for Infrastructure that also houses a SSD/Hybrid SATA/SSD SAN serving NFS to my two cloud hosts in shelves 2 and 3. The 4th is a power conditioner soon to be changed to a UPS. Network switch is in the back.

I would certainly still look at SKT1366, you can get hexa-core and ram on ebay fairly cheap. The two ESXi hosts that serve my vCloud workloads are both dual proc 5620's w/48GB RAM, Local SSD Cache 10Gbe CNA's, and 1Gbe Nics. They have been the core of lab for 2 years and they run very strong.

 
Last edited:
Nice setup. I considered an open air rack like that but even that I have no room for. I basically have the room for either a mid-tower case or multiple cube cases that can sit on a shelving unit.
 
Vader said:
Check out Chris Wahl's setup here..http://wahlnetwork.com/my-home-lab/

Small footprint...
Click to expand...

Thanks, yea I've seen Chris' setup and it has certainly influenced my decisions. However I don't believe any of Chris' VM's require the kind of CPU power that my main (media server) VM will require so I've been unsure if his setup would be sufficient for my needs.
 
Chris and I have a similar host setup. Your biggest problem with media transcoding will be cores. Hard to find anything reasonable larger than 4 cores..and that's going to hurt you. For your budget it'll be tough to go bigger than what Chris and I use... X9SCM-iif-o and E3-1230v2s. Add RAM. Times two..and you're getting close to budget. You don't need a case so that helps and you can get a faster CPU..but still.
 
NetJunkie said:
Chris and I have a similar host setup. Your biggest problem with media transcoding will be cores. Hard to find anything reasonable larger than 4 cores..and that's going to hurt you. For your budget it'll be tough to go bigger than what Chris and I use... X9SCM-iif-o and E3-1230v2s. Add RAM. Times two..and you're getting close to budget. You don't need a case so that helps and you can get a faster CPU..but still.
Click to expand...

What if I were to just combine into one (2 socket maybe?) server and use my main PC (i7 3770k + 16GB of RAM) to run VMware Workstation 9 (which I have a license for) for the purposes of testing failover. I could always bring my media VM up on my PC if I need to take my main VM server down for maintenance or if it was to fail.
 
If you can make the numbers work....

How many cores you want for the media VM? Keep in mind you can't throw all cores at it....it'll run worse than giving it 2 or 3 out of 4. Scheduler needs to be able to do work plus other VMs.
 
NetJunkie said:
If you can make the numbers work....

How many cores you want for the media VM? Keep in mind you can't throw all cores at it....it'll run worse than giving it 2 or 3 out of 4. Scheduler needs to be able to do work plus other VMs.
Click to expand...

Right now my media server is running on a Q6600 and it handles it ok but it works hard to transcode 3+ files at once. I'm not sure how many cores/threads of new hardware I'd need to be honest with you.
 
jimphreak said:
Right now my media server is running on a Q6600 and it handles it ok but it works hard to transcode 3+ files at once. I'm not sure how many cores/threads of new hardware I'd need to be honest with you.
Click to expand...

They are all going to work at 100%...the question is how fast do you want it done? That Q6600 is old old. You may be surprised on new gear.
 
NetJunkie said:
They are all going to work at 100%...the question is how fast do you want it done? That Q6600 is old old. You may be surprised on new gear.
Click to expand...

Man I'm going around in circles. I just don't want to put my build together (whichever way I go) and regret it a year later. I want some amount of scalability to expand (run more VM's) in the future if I need to.
 
Then add more hosts. The size of one hosts only limits the size of your VMs. If you run out of capacity you add more. When this thing isn't crunching media I bet your boxes will be mostly idle.
 
NetJunkie said:
Then add more hosts. The size of one hosts only limits the size of your VMs. If you run out of capacity you add more. When this thing isn't crunching media I bet your boxes will be mostly idle.
Click to expand...

Yea I'm sure they will be. So you think a single Xeon E3 host with 32GB of RAM will suffice for now and to just build another if I need room for more VMs? Would certainly save me some money for the time being since I already have the RAM and case for one host.
 
Last edited:
jimphreak said:
Yea I'm sure they will be. So you think a single Xeon E3 host with 32GB of RAM will suffice for now and to just build another if I need room for more VMs? Would certainly save me some money for the time being since I already have the RAM and case for one host.
Click to expand...

I think that's your best option. There just isn't an economical way to build a home lab box that's bigger right now. Maybe find an off-lease hexacore if you want to deal with the noise and rack server...but to build one yourself...nothing much out there. I'd love to have a single 6-core box in my lab but so far no-go.
 
NetJunkie said:
I think that's your best option. There just isn't an economical way to build a home lab box that's bigger right now. Maybe find an off-lease hexacore if you want to deal with the noise and rack server...but to build one yourself...nothing much out there. I'd love to have a single 6-core box in my lab but so far no-go.
Click to expand...

Would a six core E5 suffer performance wise transcoding in comparison to an E3 running 1Ghz faster?
 
I think you are splitting hairs of sorts. Speed vs. Cores. To help give us an idea how does transcoding run on you i7 when running 3 files at once.
 
peanuthead said:
I think you are splitting hairs of sorts. Speed vs. Cores. To help give us an idea how does transcoding run on you i7 when running 3 files at once.
Click to expand...

It handles it with no hiccups. But then again, my i7 is running at 4.8Ghz and even at stock speeds it probably has more juice for things like transcoding than a Xeon would.
 
What if you load a VM on that pc and try transcoding with only say 2 cores. What I am ultimately hoping that will come out of this is that you will have some form of metrics to make a better decision. I personally think and E3 should be enough, possibly E5 if you want more life before upgrading. I have often found though that most will upgrade long before they have too. I have a 2500K @ stock speed for checking email and loading the occasional web page. Overkill, yes. Have to upgrade any time soon, heck no.
 
Looks to me like it would be worth taking another look at the use case.

Media server doing 3-4 simultaneous transcodes daily running in HA. TBH, it's not really clear to me why the media sever has to run in HA. So what if some hardware fails. Just start over. No lives or profits are lost, or are there profits lost?

HA isn't FT, so your transcodes will be interrupted and will likely require manual interaction anyway. You may even end up in a situation where you will have to restart from the beginning of the transcode regardless of HA.

Similarly you ought to think about whether your current transcoding habits will fade away because you lose interest and whether your setup will thus quickly turn into being idle 99% of the time.
 
peanuthead said:
What if you load a VM on that pc and try transcoding with only say 2 cores. What I am ultimately hoping that will come out of this is that you will have some form of metrics to make a better decision. I personally think and E3 should be enough, possibly E5 if you want more life before upgrading. I have often found though that most will upgrade long before they have too. I have a 2500K @ stock speed for checking email and loading the occasional web page. Overkill, yes. Have to upgrade any time soon, heck no.
Click to expand...

How exactly do I go about transcoding with only 2 cores on my i7? Is that something I can turn off in software or I'd have to turn the cores off in the BIOS?


Thuleman said:
Looks to me like it would be worth taking another look at the use case.

Media server doing 3-4 simultaneous transcodes daily running in HA. TBH, it's not really clear to me why the media sever has to run in HA. So what if some hardware fails. Just start over. No lives or profits are lost, or are there profits lost?

HA isn't FT, so your transcodes will be interrupted and will likely require manual interaction anyway. You may even end up in a situation where you will have to restart from the beginning of the transcode regardless of HA.

Similarly you ought to think about whether your current transcoding habits will fade away because you lose interest and whether your setup will thus quickly turn into being idle 99% of the time.
Click to expand...

I'm not sure what you mean but the use case.

As for having my media server being HA you're probably right, it's not mission critical I just kind of liked the thought of it. I do want to test out HAVM's though just for the experience and I always feel like unless I find a way to implement something that I'm going to use or benefit from myself than I just won't dedicate the time.

As for my transcoding habits, if anything they will increase. Keep in mind that it's not me personally transcoding 3-4 videos at once myself. My parents, two sisters, and a few close friends stream to their Roku's/iPads/Smart TV's and some of them (my younger sister) rely it on completely for TV/movies as she doesn't have TV service. I have a huge library of movies and TV shows and it's only going to grow so I don't see my media streaming needs lessening in the future at all.
 
If your media server needs to be HA, it shouldn't be sharing the same hardware as your test boxes.

HA home media server... right :)

I think my setup was slightly more with the Synology NAS (details in sig), but it works great.
 
REDYOUCH said:
If your media server needs to be HA, it shouldn't be sharing the same hardware as your test boxes.

HA home media server... right :)

I think my setup was slightly more with the Synology NAS (details in sig), but it works great.
Click to expand...

What do you run on your two node setup? Is it all for testing?
 
Is the transcoding needing to be done in real time (stream to iPads) or just batched for local sync (to like the Plex client on an iOS device)?
 
NetJunkie said:
Is the transcoding needing to be done in real time (stream to iPads) or just batched for local sync (to like the Plex client on an iOS device)?
Click to expand...

Real time. I often have multiple people streaming videos from my server over the internet to their Rokus, iPads, Smart TVs, etc and the transcoding has to be done on the fly for their to be no slowdowns or pauses in that video playback. This is in addition to whatever I may be streaming locally myself.
 
Considered two nodes with a media system on each? One for local, one for remote?
 
NetJunkie said:
Considered two nodes with a media system on each? One for local, one for remote?
Click to expand...

I guess that's an option but having to maintain two separate Plex servers is not exactly efficient and especially when half the time it's idle (during the work day). If it was being used 24/7 that might be a more reasonable option in my mind.
 
Then figure out what your peak load is. Not much else to say. Buy a host and test.
 
NetJunkie said:
Then figure out what your peak load is. Not much else to say. Buy a host and test.
Click to expand...

Yea that's the plan at this point. Gonna probably build one E3 host with 32GB and if it can't handle my needs I'll build a second one.
 
jimphreak said:
I looked at the C1100s and wish I could go with them but I don't have the space for rack mountable servers right now. I don't even have the space for full PC towers. Plus these servers will be in my bedroom so noise is a factor. All I can fit right now are these two Micro-ATX cases stacked on top of each other which is why I've settled on them.
Click to expand...

I was all about SFF and I would draw designs in my spare time but I kept coming back to the fact that the "pizza box" was really the most efficient use of space. A short 2U chassis is only ~15 litters.

As to racking you says you gotta? I just built a shelf and stuck a rack rail on the leading edge:
MHnVqPb.jpg

Not the best pic but you get the idea.
 
In answer to how do you test the transcoding on 2 cores, load up a vm with X resources and test. You need to do your own testing to come up with the results you find acceptable, then make a final decision from there.
 
unholythree said:
I was all about SFF and I would draw designs in my spare time but I kept coming back to the fact that the "pizza box" was really the most efficient use of space. A short 2U chassis is only ~15 litters.

As to racking you says you gotta? I just built a shelf and stuck a rack rail on the leading edge:
Not the best pic but you get the idea.
Click to expand...

I'm not all about SFF in fact I'd much prefer to use rack mountable hardware. When I first started looking into this project that's what I was looking to do but then I started actually specing out where I'm going to have to put these servers and I just don't have the room right now for rack mountable hardware. I honestly have room for like a few cube cases stacked or one large tower case, that's it. If I had room for rack mountable hardware I'd just pickup a pair of C1100s with dual Xeon's and 72GB of RAM for $400 each and be done with it.


peanuthead said:
In answer to how do you test the transcoding on 2 cores, load up a vm with X resources and test. You need to do your own testing to come up with the results you find acceptable, then make a final decision from there.
Click to expand...

That's a good idea I will test that out and see what kind of results I get. I'll make sure I put my CPU back at stock too because right now it's running at 4.8Ghz.
 
REDYOUCH said:
Just for testing, it's powered off most of the time.
Click to expand...

Yea I don't have to kind of mind laying around to put strictly towards a testing environment that will be off most of the time. I need to put it to work as well haha.
 
My home lab consists of two Intel NUC i5's with a Synology DS412+.

I've been very happy with the performance and one of the VM's it does run is a PLEX media server, I can watch shows on my front room and bedroom TV's without issue.
 
WoodiE said:
My home lab consists of two Intel NUC i5's with a Synology DS412+.

I've been very happy with the performance and one of the VM's it does run is a PLEX media server, I can watch shows on my front room and bedroom TV's without issue.
Click to expand...

What kind of files (mp4, mkv, etc.) are you streaming and what do you use to play them on your TV (DLNA or Plex client)?
 
jimpbreak,

Most of the files are MKV but do have some mp4 files, all of which play just fine. My front room TV is a LG LM8600 and uses a DLNA player to play the files. In the bed room we use a Roku 2 XS. Both the Roku and LG are connected via Cat5 ran throughout the house.
 
You must log in or register to reply here.
Back
Top