Motherboard recommendation for ZFS file server

C

Child of Wonder

2[H]4U
Joined
May 22, 2006
Messages
3,270
Would anyone have a recommendation for a motherboard to use on a ZFS file server? After over a month of struggling, I'm finding the ASUS M4A89GTD PRO/USB3 is not the solution. I'm using an IBM BR10i flashed with IT firmware, a cheap Syba card just to give me 4 more SATA ports, an Intel quad port PCI-E NIC, and an Intel Gb PCI NIC but this is apparently too much for the board to handle and I frequently get "insufficient runtime space for MPS data" errors which I've found essentially means I have too many add-on cards.

If I remove some of the cards and boot only with the BR10i and Intel quad port NIC, I can make it past the BIOS splash screen but then the system simply reboots itself. Memory tests OK and I've tried two different Corsair PSUs (HX620 and AX850) with the same results.

I'd prefer AMD but would switch to Intel for the right board. I need something that will take the IBM BR10i card, Intel quad port NIC, and Syba controller. It needs an eSATA port and onboard video so I'm not wasting a PCI-E slot on a GPU. I'll be installing Nexenta Community Edition or Openindiana + Napp-It.
 
I would have started with a server class board for this, but before you give up on that ASUS board completely: the error message you are getting indicates that there is insufficient option-ram to load the boot bios of your add-on cards. To stabilize it, try disabling every on-board peripheral that you don't actually use in the BIOS. Also, if the BIOS has an option to "load add-on cards before on-board" try that. Two raid cards and two extra NICs shouldn't put it over the top. And there is no promise that another board will work better. Also check to see if there is a way to disable to boot-rom on either the BR10i or Syba card - I don't have either one, but many HBAs have a jumper to disable loading the boot code and I'd guess you are not planning to use either one as a boot device.

If you can't get the board working, +1 on Supermicro, but I'd probably look at the X9SCL+-F. For your application you don't need the extras that the C204 chipset brings (like the two SATA-600 ports on board). And both NICs on the X9SCL+-F are the Intel 82574L, which seem to have better support, especially in ESXi if you ever choose to virtualize.
 
IBM BR10i is PCI-e x8 ver 1.0 card and it needs at pci-e x8 slot or x16 working at x8 (or x16) to provide full speed. That said motherboard with a single x16 slot might not like to have there anything else but video card, so "not wasting x16 slot" might not be an option for you.
ZFS needs TONS of ram for cache, so 16gb would only be ok, 32gb much better,
It doesn't have to be ultra fast dimm, cheap value dimm of right capacity will do
Since AMD have very limited 32gb memory support on older platform, it basically limits you to brand new Liano platform board like this asus:
http://www.newegg.com/Product/Product.aspx?Item=N82E16813131784

It SHOULD support the add-on cards you need
 
PigLover said:
I would have started with a server class board for this, but before you give up on that ASUS board completely: the error message you are getting indicates that there is insufficient option-ram to load the boot bios of your add-on cards. To stabilize it, try disabling every on-board peripheral that you don't actually use in the BIOS. Also, if the BIOS has an option to "load add-on cards before on-board" try that. Two raid cards and two extra NICs shouldn't put it over the top. And there is no promise that another board will work better. Also check to see if there is a way to disable to boot-rom on either the BR10i or Syba card - I don't have either one, but many HBAs have a jumper to disable loading the boot code and I'd guess you are not planning to use either one as a boot device.

If you can't get the board working, +1 on Supermicro, but I'd probably look at the X9SCL+-F. For your application you don't need the extras that the C204 chipset brings (like the two SATA-600 ports on board). And both NICs on the X9SCL+-F are the Intel 82574L, which seem to have better support, especially in ESXi if you ever choose to virtualize.
Click to expand...

The Br10i, at least when flashed with IT firmware can be set to not load the boot code. I have it set like that in my system.
 
davros123 said:
+1. great board. 4 pcie slots, IPMI, KVM over LAN.
Click to expand...

+1 as well; I have this board. The IPMI has been very handy.

Only caveat (I think this applies for all P202/4 boards) is that while all 4 slots are x8 form factor, 2 of the PCIe slots are x4 and 2 are x8. Most cards will negotiate down properly, but if you go with something like a LSI 9211-16i you probably want that on one of the true x8 slots. This is pretty minor but you just need to keep it in mind for your I/O planning.
 
davros123 said:
+1. great board. 4 pcie slots, IPMI, KVM over LAN.
Click to expand...

I have this board as well. Seems great so far, however is there anyway to update drivers for Solaris 11?

Display is very laggy and very low res, like 1280x1024. Hoping to get a little higher for my 24" display..
 
The KVM over IP is only for basic maintenance tasks like OS install/repair tasks. Everything else should be done over OS-integrated remote solutions like RDP or VNC.
 
we need an X9SC* appreciation society :)

Thanks Supermicro for making such a sweet ass board.
 
It will be sweet until the e7 series motherboards start to come out. I'm hoping I can afford one. Looking at the expected channel pricing for the e7s though, it'll be tough...
 
I really want an intel file server but I don't want to buy another Supermicro board and another Xeon for ECC support. I'm thinking abou the Phenom II 840 + AM3 mobo Microcenter deal. I could still use ECC with that combo.
 
shiznit said:
I really want an intel file server but I don't want to buy another Supermicro board and another Xeon for ECC support. I'm thinking abou the Phenom II 840 + AM3 mobo Microcenter deal. I could still use ECC with that combo.
Click to expand...


you loose upgrade options and you mostly get poor Realtec nics.
I would buy a SuperMicro Board, optionally with a smaller or dual CPU (even non-Xeon like i3)
 
_Gea said:
you loose upgrade options and you mostly get poor Realtec nics.
I would buy a SuperMicro Board, optionally with a smaller or dual CPU (even non-Xeon like i3)
Click to expand...

That's why I got a PCI-E Intel quad port Gb NIC off ebay for my file server. :D The onboard Realtek port is just used for management traffic.
 
_Gea said:
you loose upgrade options and you mostly get poor Realtec nics.
I would buy a SuperMicro Board, optionally with a smaller or dual CPU (even non-Xeon like i3)
Click to expand...
I already have a quad port Intel nic, and the i3s don't support ECC.
 
shiznit said:
I already have a quad port Intel nic, and the i3s don't support ECC.
Click to expand...

Actually the I3s do support ECC when run on a 3400 series chipset motherboard. The I5s don't. It is odd, but I have E-mails from Intel support confirming this.
 
PigLover said:
Actually the I3s do support ECC when run on a 3400 series chipset motherboard. The I5s don't. It is odd, but I have E-mails from Intel support confirming this.
Click to expand...
The memory controller in the i3 does not support ECC, it doesn't matter what motherboard chipset you use. That part has been disabled by Intel. 3400 motherboards allow for ECC "support" (misleading statement because the chipset has nothing to do with it really) when used with a compatible Xeon processor.
 
OP: I'm looking at the Supermicro H8DCL for my fileserver one of these days. It comes to $730 with a 4170, 2*8GB of ECC registered memory, and a heatsink, and you can easily expand to 64GB with 8gb dimms and 12 cores down the road.

PigLover said:
Actually the I3s do support ECC when run on a 3400 series chipset motherboard. The I5s don't. It is odd, but I have E-mails from Intel support confirming this.
Click to expand...

"Support ECC" can mean "accept memory that has multiple of 9 chips on it, then ignore the ECC capability... but it boots!".
 
I would love to see an email from Intel saying the i3's memory controller runs in ECC mode when used with a 3400 series chipsed. The fact that it accepts ECC memory is meaningless.
 
The i3+ECC issue does show up every few months or so. The i3 datasheet states ECC support in conjunction with server chipsets.

I also found http://www.nexenta.org/boards/1/topics/1433 before I bought my server (but since then I upgraded to a Xeon for VT-d support). Somewhat down on this page you can read:

I also asked Intel and their response:

The Intel® 5 series chipset supports non-ECC memory only when using any of the Intel® Core™ i5 or i3 processors. However, using those processors on a workstation motherboard based on 3400 series chipsets it will support ECC and non-ECC memory ... that means that will fully support ECC technology. That means that it will correct single bit memory errors with ECC memory.​
Click to expand...

Of course not everything from Clarkdale may apply to Sandybridge, but this is good enough for me. Further, linux does load the EDAC driver even with an i3.
 
Last edited:
I just found this:

http://www.intel.com/support/processors/corei3/sb/CS-031175.htm#10
Does the Intel Core i3 desktop processor support Error Correction Code (ECC) memory?

The Intel Core i3 desktop processor does not support ECC memory. Typically ECC memory is used on servers and workstations, rather than on desktop platforms. This is due to the price premium and extremely low likelihood of a data error occurring even on memory not utilizing ECC.
Click to expand...


Microcenter has the Phenom II 840 for $50 but Newegg has 20% server motherboards (Intel), does anyone know for sure?
 
You must log in or register to reply here.
Back
Top