Opinion on new ESXi build

G

GigaPie

n00b
Joined
Mar 17, 2012
Messages
28
Been reading here for some time on trying to figure out how I would put together a new ESXi build and think I finally found a solution that will work. Would like to hear people's thoughts on this build.

MSI Z68A-GD65 (G3) or Intel BOXDQ67SWB3
i7-2600
Intel EXPI9402PTBLK Two Gigabit
2x G.SKILL Ripjaws X Series 16GB
Fractal Design Define R3
Seasonic SS-560KM
LSI SATA/SAS 9211-8i

My plan is too use the Intel RAID card for RAID 1 2x1.5TB for the Datastores (if needed in the future 2 more 1.5TB).
Direct access 4x2TB to a FreeNAS or OpenFiler VM via the on-board Sata. Likely will do 2 RAID 1 for my WHS 11 server VM.

This is something I would like to use for a good 4-5 years. The price seems very reasonable for everything (I have some of the HDD) and I may add an SSD for swap down the road.

Edit: Updates some items.
 
Last edited:
"Direct access 4x2TB to a FreeNAS or OpenFiler VM via the on-board Sata"

on board sata works with ESXi ?
 
danswartz said:
Looks ok, assuming you've verified the HW raid works with esxi.
Click to expand...

Thanks, I checked the reviews on the HW Raid and folks are using them with ESXi 5.

dbwillis said:
"Direct access 4x2TB to a FreeNAS or OpenFiler VM via the on-board Sata"

on board sata works with ESXi ?
Click to expand...

They do on my current ESXi build. Never thought about on-board Sata ports not working.

I know a lot of folks recommend having shared storage, but that would require another PC. I'm not so sure I want another box running.
 
That RAM won't work with that motherboard/CPU: It's Registered ECC RAM. The motherboard/CPU only supports Un-buffered ECC RAM.
 
Last edited:
Danny Bui said:
That RAM won't work with that motherboard/CPU: It's Registered ECC RAM. The motherboard/CPU only supports Un-buffered ECC RAM.
Click to expand...

Ah dang it. I switched my choices in MB once already due to that. I thought this one can take just ECC RAM. Doesn't seem any of the LGA1155 Server MB's take anything but unbuffered. Looks like I will have to use a regular desktop MB as I really want 32GB of RAM. :( Really too bad as I did want to have a MB with IPMI....
I did find 2x8GB for ~$350 and that's just too expensive for 32GB.

dbwillis said:
Id check to see if the onboard sata will work with esxi
Click to expand...

I'll double check, though that may be a moot point for this board now.
 
Heh, I was checking the cpu/mobo (a popular combo.) I assumed you would pick ram that worked :)
 
GigaPie said:
Thanks, I checked the reviews on the HW Raid and folks are using them with ESXi 5.



They do on my current ESXi build. Never thought about on-board Sata ports not working.

I know a lot of folks recommend having shared storage, but that would require another PC. I'm not so sure I want another box running.
Click to expand...

Does ESXi see them as individual drives or as RAID set?
 
Ok, some updated items for my build.

i7-2600
ASRock Z68 EXTREME4 GEN3
2x G.SKILL Ripjaws X Series 16GB
LSI SATA/SAS 9211-8i

This will give me the 32GB of RAM I'm looking for.
The RAID card here should be a lot more compatible and is more powerful then the previous one I had listed as well.

I may have to go with a second RAID card as well I'm thinking. If I want to pass the LSI directly to a NAS VM I will need to have another RAID card for a RAID 1 array datastore just for the NAS VM, though I guess if hard drive fails it's easy enough to replace and restore it.
 
Netwerkz101 said:
Does ESXi see them as individual drives or as RAID set?
Click to expand...
That I'm not sure about. However I decided to switch the build to an LSI card.

Martian said:
Check out the Intel BOXDQ67SWB3 board. It's not as fancy as the Supermicros but it's a quality workstation board that supports remote management via vPRO and should meet your needs. It can handle 32 GB of DDR3.

Here is the thread with my build:

http://hardforum.com/showthread.php?t=1677498
Click to expand...

Thanks for the tip. I have read through that thread a few times before. Great information!
 
I cheaped out on getting a second raid card for the NAS OS and instead passed a VMDK of the same size as the original only on a second disk to my storage OS and added it to the rpool. Seems to work for but I'm sure there is some reason why it's not a good idea.

At least it give me a copy of the data if one drive dies.
 
Martian said:
I cheaped out on getting a second raid card for the NAS OS and instead passed a VMDK of the same size as the original only on a second disk to my storage OS and added it to the rpool. Seems to work for but I'm sure there is some reason why it's not a good idea.

At least it give me a copy of the data if one drive dies.
Click to expand...

Yeah, I'm not sure yet how I'm going to tackle that yet as I really don't want to buy 2 RAID cards. Wouldn't be too hard to back up the NAS VM once it's configured and throw it on a new datastore in case it fails.

Danny Bui said:
A bit hesistent in recommending that mobo for a few reasons:
1) The HardOCP review of it wasn't that great:
http://www.hardocp.com/article/2011/11/09/asrock_z68_extreme4_gen3_motherboard_review/

2) Has a short 1 to 2 year warranty

3) Overly expensive for what it offers.
Click to expand...

Thanks for the info in this, I will pass on this MB.
This board is looking really good in it's place. Only concern is I can't find any indication if VT-D is supported, but I have read that all MSI/ASRock Z68 boards do.
MSI Z68A-GD65 (G3)

This has been confirmed working with VT-D, but I'm not liking the layout as much on it.
MSI Z68A-GD80 (B3)

Edit: Was able to confirm that the MSI Z68A-GD65 (G3) supports VT-D from here. Looks like the board I will go with, unless there are good reasons not too.
 
Well it's not available from Newegg so you're gonna have to find it elsewhere.

Considering your need for VT-D and that the other mobos that support VT-D are AsRock, all the problems I have with that MSI are out the window as I'd rather you have a MSI mobo than an AsRock mobo.
 
Danny Bui said:
Well it's not available from Newegg so you're gonna have to find it elsewhere.

Considering your need for VT-D and that the other mobos that support VT-D are AsRock, all the problems I have with that MSI are out the window as I'd rather you have a MSI mobo than an AsRock mobo.
Click to expand...

Thanks for all the information. I'm likely not buying it for a couple of months, so if it's not available by then I will get it somewhere else. Will post pics as well.
 
What features are you looking for in a motherboard anyway besides VT-D support?

GigaPie said:
Thanks for all the information. I'm likely not buying it for a couple of months, so if it's not available by then I will get it somewhere else. Will post pics as well.
Click to expand...

Then you're planning your build a tad too early then considering that Intel's new Ivy Bridge CPUs will be out around that timeframe. As such, you may actually end up buying a slightly outdated setup. Or find that the parts you chose have either become overpriced or out of stock or replaced by a faster/better part.
 
Danny Bui said:
What features are you looking for in a motherboard anyway besides VT-D support?



Then you're planning your build a tad too early then considering that Intel's new Ivy Bridge CPUs will be out around that timeframe. As such, you may actually end up buying a slightly outdated setup. Or find that the parts you chose have either become overpriced or out of stock or replaced by a faster/better part.
Click to expand...

Very true. I may have to make some adjustments if it comes to that.
 
Is there a reason you are not considering the Intel BOXDQ67SWB3? You don't appear to need to extra ports the more expensive boards provide. The Q67 chipset fully supports VT-d and always will and you get vPro remote management. It's also a workstation class board and therefore in theory has better build and support.

Granted I'm biased since this is the board I bought...
 
Martian said:
Is there a reason you are not considering the Intel BOXDQ67SWB3? You don't appear to need to extra ports the more expensive boards provide. The Q67 chipset fully supports VT-d and always will and you get vPro remote management. It's also a workstation class board and therefore in theory has better build and support.

Granted I'm biased since this is the board I bought...
Click to expand...

The reason I had, as you pointed out are now invalid. Originally I wanted a board with a lot of SATA ports, but with the RAID card I have now decided on, it is now not necessary.
Originally I also thought the board didn't support the i7 integrated graphics and I didn't want to get a separate dedicated GPU.

The only other concern I have with that board is the lack of PCI-e slots.
One PCI-e x1 would be used by the Intel Dual GB and the PCI-e x16 would need to be used by the 9211-8i. Doesn't leave any room for expansion if I wanted to get a second RAID card, etc. Though I may never need one.
 
Don't forget there are two e-sata ports on the back in addition to the 4 internal ports. Also, by the time you are ready to add a 2nd HBA you may be willing to mod that x4 slot into an open ended slot :D. I know that's what I will be doing if I need to add an x8 card.

Oh, and the i7 2600 onboard video works great!
 
Martian said:
Don't forget there are two e-sata ports on the back in addition to the 4 internal ports. Also, by the time you are ready to add a 2nd HBA you may be willing to mod that x4 slot into an open ended slot :D. I know that's what I will be doing if I need to add an x8 card.

Oh, and the i7 2600 onboard video works great!
Click to expand...

I saw the e-sata ports, maybe they will come in handy down the road.
What do you mean by modding the x4 slot? Can't say I've ever modded a pci-e slot.

Thanks for confirming the onboard video, looks like this is the board I will likely go with.
 
PCI-e uses lanes. x8 = eight lanes, x4 = four lanes. You can operate an x8 card in an x4 slot, it will just have reduced bandwidth. This issue is making it physically fit in the slot. Some boards have open ended slots which basically means the card hangs out the back of the slot.

If you have the kuhunas and a dremel or soldering iron you can open the end of a slot. Obviously this is risky and voids your warranty but a couple years down the road it might be worth the risk. :)
 
Also, note that some boards need the hot-plug Presence pins on the pci-e port jumpered to register a larger card in a smaller port (i.e. 16x in a 8x/4x/1x). There's a bunch of info on this from the bitcoin miners as they would pack tons of pci-e vga's into 1x slots for GPGPU. I had one board I had to do this on, and another (a SM X9SCMF) that would not register it regardless of what I did, so who knows.

As far as taking a dremel or iron to the board, a less destructive means is to use a flexible riser that either fits the card you're trying to adapt to, or worse comes to worse you at least can dremel out the $5 riser instead of your mb.
 
Martian said:
PCI-e uses lanes. x8 = eight lanes, x4 = four lanes. You can operate an x8 card in an x4 slot, it will just have reduced bandwidth. This issue is making it physically fit in the slot. Some boards have open ended slots which basically means the card hangs out the back of the slot.

If you have the kuhunas and a dremel or soldering iron you can open the end of a slot. Obviously this is risky and voids your warranty but a couple years down the road it might be worth the risk. :)
Click to expand...

Tiporaro said:
Also, note that some boards need the hot-plug Presence pins on the pci-e port jumpered to register a larger card in a smaller port (i.e. 16x in a 8x/4x/1x). There's a bunch of info on this from the bitcoin miners as they would pack tons of pci-e vga's into 1x slots for GPGPU. I had one board I had to do this on, and another (a SM X9SCMF) that would not register it regardless of what I did, so who knows.

As far as taking a dremel or iron to the board, a less destructive means is to use a flexible riser that either fits the card you're trying to adapt to, or worse comes to worse you at least can dremel out the $5 riser instead of your mb.
Click to expand...

Ok, I understand now.
 
I found you setup really interesting
I like to know if you were able to make everything work FT , VT-D.
I want to buy the same setup as your. (2 like this).

thanks
 
bcorneau said:
I found you setup really interesting
I like to know if you were able to make everything work FT , VT-D.
I want to buy the same setup as your. (2 like this).

thanks
Click to expand...

I have not yet pulled the trigger yet on a new server.
I've done some more thinking and will likely not go with an all-in-one setup like I currently have. As such by build will be a bit different from what I have listed on the front page.

I'll post the setup once I have everything planned out.
 
GigaPie said:
I have not yet pulled the trigger yet on a new server.
I've done some more thinking and will likely not go with an all-in-one setup like I currently have. As such by build will be a bit different from what I have listed on the front page.

I'll post the setup once I have everything planned out.
Click to expand...

And pictures :)
 
I'm not sure if lsi-2008 (the chip I think 9211 have) is raid compatible with esxi. I'm sure it will recognize the card the the supported drives. but I don't know if software-depended raid can work with esxi.

Anyway, if not, you can upgrade to 9260 for a bit more and it should work.
 
You must log in or register to reply here.
Back
Top