Cheap & fast ZFS server?

E

ediazc

n00b
Joined
Aug 24, 2013
Messages
7
Hi,

I am looking how to make a cheap, fast & dense ZFS server. My research so far has reached to HP DL180 G6 machines, with 14 drive trays, 2 proc (Xeon 55xx), 48 GB RAM.

14 drives should be enough to my project, even with the drives lost due to several RAIDZ groups (4x3drive RAIDz and 2 for ssd).

I've read that the P410 raid card that comes with the system is mostly useless in this case, so I plan to take it out and use the PCIe x16 bus for a 10 GbEth card. If I am not mistaken, the system board has a ICH-10R controller, which will work in a JBOD mode.

I will connect the sas backplane to the system board using a 8087 reverse breakout cable. The sas backplane has got an expander chip so it controls all the 14 drives. I will have 2 extra SATA headers that I can use if I find how to fix two extra drives inside the case.

If everything is right, the system would be quite cheap to build, as there are plenty of this machines in eBay.

My concerns are that the ICH-10R controller may not provide enough speed. I only have one PCIe slot, and it will be used for the 10 GbE card. If I need to use special controllers I will loose the 2 rear trays and the price of the machine will easily double.

For the OS part, I will try ZFS on Linux or Nexenta.

Does anyone have experiences with this machine and drive performance? I've looked at older threads but found none using the reverse breakout cable.
 
I'm pretty certain you can't use the ICH-10R with an expander. It will only function as a discreet connection. You will need a real SAS card to use any sort of expander.
 
Further search in the forum says the same as you. At http://hardforum.com/showthread.php?t=1631959 they say that SATA controllers can't talk to SAS expanders.

Unfortunately, as the DL180G6 has only one PCIe slot when configured with 14 external drives, I'll have to either go with 12 drives or give up using the 10 GbE card. I think I will stick to the 12 drive setup and try to find some extra room inside the case for the SSD drives.

Any recommendation for cheap systems that have better density (> 6 drives per U)?
 
There isn't really a current deal option of 14 LFF 2u chassis better than the DL180 G6.

With SATA drives the P410 really isn't that bad a performer as a dumb HBA, which is all you need for ZFS. Agree that if you want SAS drives or hardware raid the P410 is not a star.

The single PCIe slot is a limit if you need high speed networking (10gbe or IB), I did a ghetto mod to access the 2nd PCIe slot using a PCIe 2.0 flex riser and sitting the HBA behind the rear drive cage. It's not really good for a production environment but for home or lab use it does work.

There is always the 12 bay DL180 option. You lose two drives but gain 3 PCIe x8 slots. And a 12 drive file server in 2u is still pretty useful - especially at current price points.
 
Thanks for the advice. PigLover, can you post a couple of pictures of your mod? Do you have any performance figures?

IceDigger, no problem about the noise. The rest of the machines in the rack are loud enough :D
 
The SmartArray cards like the P410 will not pass through drives for ZFS. You want something different.

Also, in my experience, ZFS performance with SATA drives behind expanders leaves a lot to be desired. Cards with discrete ports are bound to be faster.
 
I finally picked one of the servers mentioned in this thread http://hardforum.com/showthread.php?t=1770557

If I am correct I will be able to put 24 external drives plus a couple of internal ones, which will yield the same density as the dl180 g6 at least. I know that processors and controllers are one generation behind, but the price justifies the difference (to me, at least)
 
You must log in or register to reply here.
Back
Top