• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Home Lab Build

H

Hoota504

n00b
Joined
Jul 29, 2015
Messages
7
Hi everyone,

I am looking to build a home lab and could use some guidance. I will use this lab to study for different certifications, and learn about different technologies. VMWare certifications are my first goal, though I will also need the ability to experiment with Microsoft's SCCM 2012 R2 and further down the line SCVMM and Hyper-V.

Currently, all I have is a 24U rack and a Dell PowerEdge R900. I do not plan to use the R900 as it is power hungry/loud and the CPUs in it do not support VT-x with EPT. My only requirements from a hardware perspective is trying to draw as little power as possible, and be as quiet as possible. Here is what I have planned thus far.

Two hosts running ESXi, what I cant decide on is whether to build or buy used off of ebay? I think building the servers myself would allow for less noise and lower power consumption, but overall would be more expensive as you can get really good deals on used equipment.

For the network I was thinking about building a pfsense box, and buying some used Cisco switches off of ebay perhaps some 3750Gs as I have some experience with those through work. I am open to other suggestions though.

Storage is probably the area in which I am the most confused. Currently I am thinking of using an OmniOS VM along with Napp-IT and using VT-d to pass through the storage. I played around with this a bit last weekend using the R900 and it was very simple to setup, and I like the interface. My other options I would consider would be VMWare's vSAN which would require another host, or something like StarWind or XPEnology.

Thank you all in advance for any help.
 
What VMware certs are you going for? Is there a budget defined for the lab? Are you wanting to keep it as cheap as possible, have robustness and ability for the lab to grow out for years to come, or a blend of both?
 
upcoming version of vmware virtual san would be pretty much like their old vsa

data stored on two nodes and third maybe running vcenter is data-less witness only

so not exactly two like referenced starwind does but definitely better compared to one we have today :D

we use starwind virtual san free version to build veeam fault tolerant backup datastores

everything works well

never touched xpenology to share anything useful :p

Hoota504 said:
[ ... ]

Storage is probably the area in which I am the most confused. Currently I am thinking of using an OmniOS VM along with Napp-IT and using VT-d to pass through the storage. I played around with this a bit last weekend using the R900 and it was very simple to setup, and I like the interface. My other options I would consider would be VMWare's vSAN which would require another host, or something like StarWind or XPEnology.

Thank you all in advance for any help.
Click to expand...
 
Im looking to start with the vcp-dcv cert, and move right into vcap for data center virtualization. I am on a budget and for the time being would like to keep it around 2000 - 2500 dollars. I would like the lab to have the ability to expand in the future.
 
Its hard to make a good recommendation without asking millions of questions but here are some ideas that I am using.

VM Storage in no particular order:
Option 1: Synology (Pick your flavor, I have a DS1511+ and DS1512+) Populated with SSD's
Option 2: HP Micro server with omni os + napp it
Option 3: HP Micro server with XPENOLOGY

Host:
Rack mounted case that supports ATX PSU
x79 motherboard that supports 64gb ram (Asus P9X79 Pro)
3930k or 4930k processor

Building my own host has allowed me to keep noise down and the price/performance ratio of these hex cores are better than Xeons.

The biggest feature I miss is IPMI with a supermicro board, But with esxi installed i rarely have to do anything on the host itself.

This is basically a rough draft of my Lab. I have a full rack with 5 host and 5 different storage setups.
 
Thanks for all the information!

Monkizzle, what version of ESXi do you run on those Asus boards? Did you have to add any drivers to the ISO?

I will probably end up building a separate box and use Omni OS + napp-it.
 
I am up 5.5 U2 now, for the P9X79 boards I dont think I had to but dont quote me on that, I also dont use the onboard NICS, I have Intel Quad NIC's in each server.

Just google whatever board you are thinking about using and see what others have done to get it work. Its not hard to slipstream those nic drivers onto the esxi install image.

I prefer having my VM storage on a separate box because it it removed the dependency of that host to a dedicated storage box, so when you tinker with things on host you dont have to worry as much.
 
This is some good info. Now I have a 4 Bay Synology I just use for storage. I setup 2 partitions with one being multimedia and junk and another doing anything. Would building the workstation be ok and then tie in that storage to setup VM's?
 
After some more reading of the Napp-it site, I feel that running OmniOS + Napp-it as a VM, and passing the storage through to it is the preferred setup for non-enterprise use cases.

So now one of my hosts will use the SuperMicro X10SL7-F motherboard with a LSI9211-8i card (both onboard and PCIe LSI cards will be flashed to IT mode) to support the 16 drives I plan on using. I was considering an Intel SAS expander instead of a second HBA, but the Napp-it manual states that may cause problems with SATA drives. Has anyone experienced this? Using the SAS expander would give me the capability to easily upgrade to 24 drives in the future.

I will also purchase an Intel PRO 1000 PT Quad NIC for network connectivity.
 
If this would be a production setup, you will not get commercial support for an expander with Sata solution. While it works, there are reports that a failed disk may block the whole expander, makes it impossible to find the problem disk in a short time.

As dedicated HBAs are faster, without such possible problems and similar in price with HBAs like an IBM 1015 (can be flashed to LSI 9211-IT) or use the newer and faster LSI 9207 (IT mode per default).

Regarding quad-nic
Have you thought about 10 GbE?
There is no premium with some Supermicro boards with onboard 10 GbE. Intel 10Gb boards (X540, 520) are not cheap but sometimes affordable.
 
Thanks for the confirmation on the SAS expander. Although this is just for a lab, I want to make this process as smoothe as possible so Ill pick up one of the HBAs you mentioned.

As far as 10GbE goes I hadnt considered it at all. As it stands now I still need to build a pfsense box and buy a switch or two. Originally was just planning on getting a used 3750g off of ebay.

For my use case of a lab to learn the technology along with serving as a second back up location for a few machines in the house, do you think 10GbE is needed?
 
A quad nic (4 x 1 Gb) is not 4 Gb/s on an average setup.
It allows four clients to connect with 1 Gb/s simultaniously.

10 Gb/s gives you
- the ability to move/ copy/ backup large amounts of data
without a copy of a 4 TB disk can last days

- the ability to access storage over network with the performance of a fast SSD
(SSD performance is 300 MB/s up to 1 GB/s, 1 GB/s = about 10 Gb/s)

With a 10 Gb Link to a vlan capable switch connected to ESXi, you can use vlans
to separate networks, each with 10 Gb performance.

Only problem
Nics and switches are more expensive compared to 1Gb
 
If noise is an issue for you then another thing to consider with 10GE switches is they are typically pretty loud. I have yet to find a decent lab switch that would be tolerable for home use (assuming you need more than 4 10GE interfaces). I decided on going with Quad 1Gbps NIC's in my OmniOS/NappIT build until 10GE switches are more available. I have about 30 VM's running and the 4 1Gbps ports have been plenty sufficient using MPIO. If noise isn't a concern, I believe the best cost effective option today for 10GE is the XS708E from Netgear.
 
I dont think I will have that many VMs at first. Probably around 10 - 15 at first. When I start experimenting with SCCM though I will certainly have more. Maybe at that time I will go 10GbE.

I have been looking on the Internet at 10GbE equipment and although the NICs arent bad the switches get pricey rather quickly. I saw that Netgear though and perhaps I wont need more than 8 ports with them having that much bandwidth.

Did the rest of you purchase all of your equipment at once or was it start small and grew over time as the need arose? Im trying to not spend money in areas where it isnt needed right now. I know I can always use storage so it isnt a concern, but I wonder if I would be taking full advantage of 10GbE.
 
Is there a preference when it comes to SSDs and LSI HBAs? I thought I read somewhere that Samsung SSDs don't always play nice with LSI, is that true? Is it safest to go with Intel SSDs?
 
In the end I really don't think it matters. Just stick with a name brand that has good reviews. You really are not able to go wrong with Crucial, Intel or most Samsung drives. (The 840 EVO, etc. being the bad apple.)
 
You must log in or register to reply here.
Back
Top