Motherboard supporting i5 2400 for home virtualization lab

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
I am thinking of picking up an i5 2400 from Microcenter for $129.

It is about 50% less than a comparable 1155 Xeon and it supports Vt-d, Vt-x and other goodies except for Multi-Threading. Intel Specs

Which motherboard would you suggest to run a home virtual machine lab on this CPU?

->It should support Vt-d etc,
->Be compatible with the common free Virtualization packages

Are the Q67 based motherboards the only option? What else do I need to worry about in terms of support for virtualization?
 
Last edited:

satterth

n00b
Joined
Jul 16, 2007
Messages
32
If you really need VT-d support then Q67, X58, X79, C202, C204, C206, 5500, 5500 and most likely a few others are the choices.

If you don't need "passthrough" then the options really open up.
 

lopoetve

Fully [H]
Joined
Oct 11, 2001
Messages
30,397
If you don't need passthrough, anythign works almost ;)

I've got D630's running ESX5... yes, cheap dell laptops. They're cute ;)
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
Intel DP67BG. I'm passing through an IBM M1015 to a VM. I've written about 75GB in Bonnie++ benchmarks (using _Gea's Napp-It setup) across 6 drives in 3 different arrays (2 mirror, and one "RAID10" stripe/mirror) and have now written about 40GB in data to one of my ZFS mirrors. Those figures can essentially be doubled if you're looking for total bandwidth through the passed-through controller with how ZFS mirrors (and not the controller) the data before sending it to the controller.

Early in this board's life, Intel claimed VT-d support and then removed it when they made it official that Q67 was going to be the only LGA1155 VT-d-capable chipset.

It still works. The option hasn't even been removed from the BIOS.

Only downsides to this board:

1. It is P67 (unable to use the CPU IGP.. I'm using an old Radeon X300SE that I modified to work in a 1x PCIE slot.. saved 3w idle doing this)
2. It wouldn't flash my IBM M1015, and it doesn't have an EFI shell.
3. No IPMI/AMT (Q67 boards should have AMT I believe)
4. Onboard Intel gigabit NIC isn't supported by ESXi out of the box, but Chilly made a beautiful driver for it a few threads down from this one.


If Intel made a full ATX Q67 board, I'd have probably bought that initially, but I am definitely happy with this so far.
 

jin0

n00b
Joined
Sep 26, 2010
Messages
58
i have this paired with the i7-2600 and it's working fine. mATX and only 130 bucks.

Intel BOXDQ67SWB3 LGA 1155 Intel Q67 SATA 6Gb/s USB 3.0 Micro ATX Intel Motherboard

http://www.newegg.com/Product/Product.aspx?Item=N82E16813121515

 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
Thanks Guys. For my immediate needs I would have liked a board with at least 6 sata ports but the Intel Q67 boards have only 4 internal but two external eSata ports.

Comparison of Q67 boards on Newegg indicates that the Asus P8Q67 board will probably work for me but it is not available new.

I have some Hightpoint HBA cards (2xSata=> 1x PCIe) but OpenIndiana does not support them, though they work OTB on W7, Linux and FreeBSD. So if I buy one of these, I will have to get a HBAs too.

With the no return policies on the boards I do not want to gamble with the IntelP67BOX.

The whole point of this exercise is to set something up which I can expand and play around with in the future. Initially I will be just using it as a NappIT ZFS server running 5 disks (3 data+1 parity+1 hot-spare) so need at least 6 ports (1 for the boot drive).
 

jin0

n00b
Joined
Sep 26, 2010
Messages
58
the asus has vt-d but it's not working as expected: http://www.amazon.com/ASUS-LGA-1155-Corporate-Motherboards/dp/B004QOCIG0

So if you wanna gamble and wait until Asus release a bios that will support that, then it's up to you :D

As for the return policy? I dont get what you mean by this, they all have return policy?, if it does work, call newegg and tell them you wanna exchange it for something else :cool:

the IBM raid controller is around 50 bucks something if you wanna get that.
 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
jin0:
Yes I read those reviews and decided if it is Q67 it has to be Intel only.

While browzing the vmware help forums I found this link hosted by ASRock showing Vt-d support on H67 boards
http://www.vm-help.com/forum/viewtopic.php?f=27&t=3553 leads to
http://www.asrock.com/support/note/vt-d.pdf

http://www.vm-help.com/forum/viewtopic.php?f=27&t=3555 suggests it also works on Z68.

Again I have no idea what exactly this means. Could it be that though BIOS reports support, when it actually comes to real hardware action, it does not work?

TinkerGuy is likely to be more reliable and he also claims Vt-d working on certain Z68 boards from ASRock and MSI
http://tinkertry.com/files/Z68-Motherboard-VT-d-ESXiNIC-RAID-Test-Matrix.pdf

And then I have this comment about the Q67 board from Intel; I have absolutely no idea about the implications though.

http://siphon9.net/loune/2011/01/li...at-support-vt-d/comment-page-2/#comment-17276

There is however a caveat. The only PCI slot on the board shares intA with the network card, the video adapter, the serial port and the SATA ports. Virtualbox, at least, does not work with shared interrupts (yet). Oddly, there is nothing on intD! How lame is that? Put the only PCI card on a shared line and don’t use intD at all? In all fairness, it may be used by something that I have disabled in the BIOS. Intel does not give out the interrupt assignments (they claim I have to sign an NDA to get them???). In any case, simply using “lspci -vv” will list them all anyway.
 
Last edited:

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
I picked up two IBM BR10i cards from EBay. Hopefully one of them will work. So that take cares of the port problem; still have to buy the SAS-SATA cables though.

I am not sure what kind of cable do I need though. Reverse Breakout/Forward Breakout??
^^^^^^^^^^^^^^^^

Ordered 2 of these Discrete Forward Breakout Cables https://www.superbiiz.com/detail.php?name=CBLSFFCF05
 
Last edited:

jin0

n00b
Joined
Sep 26, 2010
Messages
58
there's a few asrock mobo and one MSI board that support vt-d last time i checked. They may have more now. I havent done much research since i bought 3 of those Intel. lol

What board ya buying?
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
http://www.mwave.com/mwave/SKUSearc... X9SCL-O LGA 1155 Intel C202 Chipset MicroATX

Found this motherboard that's c202 chipset: Supermicro X9SCL-O for 90$ refurb at mwave might fit the bill.

Wow, that's a good price. Too bad it isn't the X9SCL-F or I'd consider going that route for IPMI. Those require ECC though and can't be run with an i5.. something like Xeon or i3 only right?

I'd have bought the board jin0 mentinoed (or maybe the OW version instead of the SW.. I think it was lower-end but had a feature or two I liked better over the SW) if I liked their PCI/PCIe layout better.

Like I said above though, my Intel DP67BG is working fantastically well for exactly what you're looking to do. Vt-d works fine so far, has plenty of slots, fan headers are all %RPM controllable. Cheapest on eBay right now is ~$150, but I thought I saw some on there a week or two ago for ~$120. I paid ~$100 for mine used from a member on here a couple months ago.
 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
Thanks Guys.

DuronClocker: I hear you about the IntelP67. It is just that I am a first timer and it will be sometime before I actually get to do virtualization. I want to make sure that there are no future hassles where some software/BIOS update disables the features I need.

That is why I am reluctant to go with the MSI/Asrock Z68/H67 boards also.

I do not need the SATA 6GB/S ports so the C202 might just work for me. I almost ordered the Q67 board but Amazon is out of stock (I need to use a coupon)
 
Last edited:

jin0

n00b
Joined
Sep 26, 2010
Messages
58
that board is very nice. The 2 onboard NIC are supported by esxi. Make sure you update the bios to get VT-D support.
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
What processor are you buying now? I had looked at that board; I love the layout of the PCI/PCIe slots. I couldn't spend the extra $$ for the Xeon, and with how cheap 4GB sticks of non-ECC were at the time, I couldn't afford to spend money on ECC.
 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
DuronClocker:

That board pretty much supports every LGA-1155 CPU out there. It can support ECC and non-ECC memory.

I currently have a spare Pentium G620 which while being great for my needs does not support VT-d.
I also have a reservation for the i5-2400 for $130+tax at Microcenter. But for HyperThreading the i5-2400 supports all the Intel technologies.

There is a part of me which says that perhaps I should not go for the i5 but an equivalent Xeon. For the time being I will not be doing any virtualization, and I could keep a look out for an inexpensive SB Xeon. This thinking is coming from _Gea's suggestions to keep everything server grade to prevent compatibility issues.

But as an Valley resident, I find it very difficult to overpay for the same silicon simply because of market segmentation :).

I did my reading on the ASUS board and they seem to have gone through some growing pains but now seem to have ironed out everything (VT-d support, clarification on the ECC support (unregistered only)). The MB has no PCI-e switching chip which is another obstacle on VT-d, and has ample PCIe slots.

If both of the IBM HBAs work, I could create two independent servers one running ZFS on OI and another running WHS, each with a dedicated controller card all running on the same box. But I want all that to work seamlessly.

Thoughts?
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
Hm, very cool. I had seen the main specs page only listing i3 and Xeon, which is how I thought the SuperMicro boards were as well.. and actually one of the reasons I ended up going with the DP67BG (Vt-d + no i3/Xeon limitation). I'm not sure what my uprgade path will be, if any. I won't need any more CPU power for quite some time, though moving up to 24GB will be a nice upgrade in a few months. I wonder if at that point I should upgrade everything to workstation-class for ECC support. Bleh.

The price on that open box P8B WS is very attractive.

EDIT: Bleh, I just bought it. Figure I'll give it a shot with the current RAM/CPU and then upgrade down the line to Xeon/ECC. Should be able to sell the Intel board for about the same so I won't be out anything.
 
Last edited:

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
EDIT: Bleh, I just bought it. Figure I'll give it a shot with the current RAM/CPU and then upgrade down the line to Xeon/ECC. Should be able to sell the Intel board for about the same so I won't be out anything.

I had known it all along.....:D
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
I'm actually quite disappointed I'm giving up the second onboard SATA controller going this route because that means I'm going to have to let ESXi handle all 6 of the internal ports without being able to pass any through to my OI VM. I really wish the 3Gb/s and 6Gb/s controllers were separated in these chipsets. I'll be fine for the # of drives I have now (have 8 managed by OI), but I'd like to add at least another pair of 3TBs whenever drive prices come back down, which means I'll need another controller. On the plus side I no longer need to have this eSATA-SATA cable running from outside to inside the case for my 'local' datastore.

Also a bit bummed to be giving up a PCI slot as I've got an old dual 10/100 for my IPCop VM as well as an Intel PCI gigabit card.. maybe I'll sell the PCI one and get another PCIe one or I suppose realistically with the extra onboard one, I can now upgrade my gaming/HTPC to an Intel NIC.

I'm also gaining quite a bit though.. a second onboard NIC that shouldn't require any 3rd party driver (Chilly's is working great for me, but I'd prefer if I didn't need it), better spacing on the PCIe slots (for my uses), internal USB ports, "official" Vt-d support, ECC support, IGP graphics, and it looks like maybe iAMT which would be awesome. Hopefully that actually works.. it is hard to tell from searching.

Upgrade path now is to pick up a pair of 8GB ECC sticks in a couple months (ECC disabled of course), and then upgrade to an Ivy Bridge Xeon, which it looks like the latest BIOS update already added support for. I'll either go with the E3-1225 v2 or E3-1245 v2 (HT) once they are widely available. At that point I'll pick up another pair of ECC sticks (either 4GB or 8GB depending on what prices are like then) to be able to enable ECC.
 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
I got the board yesterday. It actually is a ASUS recertified board; clearly states so on the box. I do not know how that will affect the warranty; Newegg should not have sold it as an open box. It did not have the IO shield!

However I could not get the CPU from Microcenter. It is not a pleasant story, something I hope I do not have to share.
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
Mine showed up today. It is also a manufacturer refurbished/recertified board, but I got all of the accessories with it. I'm more comfortable with it being refurbished than open box. I swapped the board out on my lunch break, but I haven't finished getting everything hooked back up and cables re-run yet. I'll be doing that in the next hour or so, and I hope to have it up and running all the way before I go to bed in 2-3 hours.
 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
Mine showed up today. It is also a manufacturer refurbished/recertified board, but I got all of the accessories with it. I'm more comfortable with it being refurbished than open box. I swapped the board out on my lunch break, but I haven't finished getting everything hooked back up and cables re-run yet. I'll be doing that in the next hour or so, and I hope to have it up and running all the way before I go to bed in 2-3 hours.

Good to know that you also got the IO-shield. Is there a way you can trace the outline of the holes on a piece of paper and scan it? Do not bother if you are busy. That way I can try and fabricate it.

The funny thing which happened to me was that the ON-OFF switch which I used from this Bezel replacement kit for my lab conked out! For quite some time I was cursing NE for sending me a dud board. After almost giving up, I decided to switch the power and the reset buttons, and am now able to power up with one push :)

I think it is the Tech's Gods way of telling me to tinker less.
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
Hm, I wish I'd have thought to do that for you earlier before I assembled everything. What if I took a high-res pic of it installed? That might be a bit better than just looking at your board.

So far, so good. Had to remove/re-add the passed-through IBM M1015 because I imagine the mapping changed with the different board. Seems to be working so far.. ran a Bonnie++ bench w/ Napp-it (12GB test), and the sequential read speeds are actually improved a bit, but the random seeks are quite a bit poorer... I'm re-running the test. It is entirely possible that the score from the single run on the old system was a freak anomaly or something.

Also successfully passed both controllers of my old dual-port 10/100 server PCI card through to my IPCop VM without trouble. I set up the USB 3.0 controller for pass-through as well even though I don't have a VM set up to pass it to yet (will likely be a Windows VM so I can easily create an offline backup of all of my data in NTFS).

System looks cleaner with 2 less NICs (moved PCIe gigabit to gaming/HTPC, removed PCI gigabit NIC), no discrete graphics card, and better fan header locations (got rid of a 12" extension across the bottom.

The only exception is a currently-ugly routing for the 8-pin power connector (it was an inch closer to the middle of the board from where it was on the old board, making it JUST out of reach of my old perfect route).

PS/2 port is nice because I can use one of my old keyboards instead of having to swap my USB receiver back and forth for my Logitech K320. Internal USB ports are cool because I no longer have one sticking out the back of the case.

BIOS is very nice and easy to use. All of those reviews that say ASUS' UEFI implementation are top-notch aren't joking. Leaps and bounds better designed than my MSI UEFI BIOS (and the Intel DP67BG BIOS was basically just an old keyboard BIOS with mouse support added). Updated to the latest with no issues whatsoever.

Now to wait for my 8GB ECC sticks to show up and hope they are compatible :D

Enough about my setup, I'll create my own thread with pics this weekend if not sooner. Thanks for finding this board :)
 
Last edited:

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
Nice to see the Vt-d stuff work out. I have never done virtualization before. A good friend works for VMWare and we took a road-trip with them during the holidays; that was the incentive for me to catch up with the times :); and that the Gigabyte board I had allocated for this project had a Realtek LAN and no PCI slot for the Intel cards.

The rear ports are quiet densely packed so there are not gaping holes; but it still leaves it open to dust which is not your friend.

A picture would be great. I could do some simple image processing on it with some manual help to get the template, and then cut out a cardboard to keep the dust out. Do post a link here about your thread once you are done.
 

nicka

n00b
Joined
Dec 30, 2010
Messages
27
I got the board yesterday. It actually is a ASUS recertified board; clearly states so on the box. I do not know how that will affect the warranty; Newegg should not have sold it as an open box. It did not have the IO shield!

However I could not get the CPU from Microcenter. It is not a pleasant story, something I hope I do not have to share.

I just got this board from the open box deal on newegg yestersday.

I can't get it to boot with all 4 dimms in though, only 2x4 in the first two slots closest to the cpu socket. I get an error 45 on the little LED error code TPM gadget. Heres the ram I'm using:

http://www.newegg.com/Product/Product.aspx?Item=N82E16820231312

So I ordered this these last night, hopefully they work;

http://www.newegg.com/Product/Product.aspx?Item=N82E16820139077
 

aviat72

Limp Gawd
Joined
Dec 19, 2010
Messages
505
Hmm.
What to say.
After I saw nicka's message above, I tried to fiddle with the setup. It has been running with just one stick.

I added the second stick and it did not boot. I tried the memOK a few times but it did not work.

I went to the BIOS and it was set at Auto everything but it was running CL7. The RAM is GSkill 1600@CL9.

So I set the Speed selection to 1333 from Auto and then it worked an booted to 8GB. I noticed that the speed it was running at was even lower 1066@CL7 according to CPUZ but I did not care since this is going to be a server and tighter timings are probably better than higher speeds.

I then added a second set of 4GB sticks to make it 16GB overall. Within a few seconds of switching it on, the blue CPU light stayed on. I shut it down. I felt the motherboard and it had become very hot.

I took out the two new sticks, but the blue CPU light will still not go and the motherboard felt hot again.

So not only is the board misbehaving with 4 RAM Sticks, it may have also done something to the CPU. Luckily this is a low cost Pentium G620 which technically supports up to 32GB of RAM via its memory controller.

But the memory problems may have more to do with the board than anything else.
========================================================

Update 1: It booted back up :). This time it has 8GB and it is running the RAM at 1066@CL8. I think 1066 is the official supported frequency for this CPU.
Update 2: Installed all the 4 DIMMs, and it booted up. So running 16GB, 1066@CL8. I think I should be able to tighten the timings even more.
Incidentally the memory I am using is the G.Skill Ripjaw X F3-12800CL9D-8GBXL . I have two sets of 2x4GB each.

For me the moral of the story is:
(1) Make sure everything is seated properly. The RAM sticks with these large heat spreaders are not the most convenient when it comes to pressing things down. On a new motherboard, the RAM slots are still tight since they have never held a stick. In my second attempt at 16GB I made sure that they were firmly pushed in.
(2) Make sure you are not using an Auto settings which force it to run above spec. In my case the firs stick was running at 1333@CL7 (added; more likely 1066@CL7) and when I put the second stick in it would not run at that tight timings.
 
Last edited:

TType85

[H]ard|Gawd
Joined
Jul 8, 2001
Messages
1,463
The ASRock Z68 Extreme 4 Gen 3 board supports vt-d, but the onboard nic wouldn't pull an IP address (I added a intel nic).. I am not sure about any of their cheaper boards though.

I have my 8 port LSI SAS controller passed through to my ZFS server.
 

DuronClocker

[H]ard|Gawd
Joined
Mar 12, 2004
Messages
1,063
I'm running 4x4GB G.Skill (not RipJaws, these don't have heatspreaders) 1333MHz sticks without a problem. I did think the memory slots in this board were weirdly designed.. not sure if that is common with all newer ASUS boards or what. I haven't owned an ASUS board since the Socket 7 days (SP97-V).

Haven't had a problem with it so far though. Set the sticks to auto and everything worked without a hitch. I'm running a i5 though, so you may have something there with the G620 only supporting 1066MHz.. not sure.
 

nicka

n00b
Joined
Dec 30, 2010
Messages
27
Whats the model number on that ram? Which bios version?

I'm running 4x4GB G.Skill (not RipJaws, these don't have heatspreaders) 1333MHz sticks without a problem. I did think the memory slots in this board were weirdly designed.. not sure if that is common with all newer ASUS boards or what. I haven't owned an ASUS board since the Socket 7 days (SP97-V).

Haven't had a problem with it so far though. Set the sticks to auto and everything worked without a hitch. I'm running a i5 though, so you may have something there with the G620 only supporting 1066MHz.. not sure.
 

pitne

Weaksauce
Joined
Aug 3, 2010
Messages
77
New ESXi 5 I just built

Ram 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16820231486 (32gb total). had to return 1 package because was getting memtest errors. replacement works good. also had to clock it at 1066 because 'auto' was causing weird issues during memtest. The display would tweak out near the end of the test. also matched the timings recomended by the manufacturer because the 'auto' timings at 1066 are increased-- I may not have had to do this but I did it anyway cause I was tired of messing with it.

cpu http://www.newegg.com/Product/Product.aspx?Item=N82E16819115086

nic 2x http://www.newegg.com/Product/Product.aspx?Item=N82E16833106015

mobo http://www.newegg.com/Product/Product.aspx?Item=N82E16813131725&Tpk=p8b ws

everythings been working great
 

nicka

n00b
Joined
Dec 30, 2010
Messages
27
RMA'd the board, Asus was convinced 2 DIMM slots are toasted. Went with a MBD-X9SCL+-F..
 
Top