• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

zfs ssd only pool recommendations

R

reco

n00b
Joined
Dec 7, 2012
Messages
2
hi there,

i want to build a 8TB high IO ssd only pool for photo video work flow.
the data is accessed for 3-5 workstations via 10g ethernet afp.

i was thinking of getting samsung 960GB SM843T and put 16 of them in a 2way mirror pool.
they will be connected to a super micro server. i don't want to mess with SATA on the SAS expanders so i am thinking of getting 2x lsi 9206/16e and connect them directly to the controllers.

as an alternative i would love to look into pcie storage but cannot find any compatible ones.

what do you guys think?
 
I just ordered quite a similar storage for some 10G MacPros with FinalCut, Premiere, AE, Cinema etc.

I took
- http://www.supermicro.nl/products/chassis/2U/216/SC216BA-R920LP.cfm
- http://www.supermicro.nl/products/motherboard/Xeon/C600/X9SRH-7TF.cfm
onboard 10 GbE, LSI 2308 (similar 9207) flashed to IT mode
- 6-Core Xeon with 64 GB RAM opt upgrade to 128GB
- 2 x LSI 9207 8i
- 2 x small SSDs as mirrorred bootdisk (rear side of the case)
- OmniOS (AFP, NFS, SMB, iSCSI - will first try NFS)

- Netgear XS 708/ XS 712 10G switches


For SSD I chose Sandisk Extreme2-960 (mainly for price reasons as they should
offer quite a similar performance under load as the Samsungs with 10y guarantee ).
I start with a 10 x SSD Raid Z2 pool and will add a second 10 SSD vdev later (plus hotspare)

I do not use mirrors as SSD offer enough iops especially with SSDs overprovisioned to 960GB -
eventually I will add another 10% overprovisioning with host protected areas.
 
Just remember to add an UPS if you use a large amount of SSDs without power-loss protection.
 
Just FYI I seem to be severely CPU bound on my ZFS system and 10GBE using samba.

Specs:
FreeBSD 10
Celeron G530 (sandy bridge @ 2.4 Ghz)
8x 3TB 7200RPM @ RaidZ2
Mellanox ConnectX-2 10GBE Direct connection to two clients (no switch).

I only get ~250 MB/s over samba, even with just large files. I realize my CPU is quite low end, but Samba doesn't multithread so a high end Xeon will apparently only help with its higher frequency, and maybe the cache slightly, but not core count. Thus even if I get a chip that is twice as fast as a G530 (a theoretical 4.5 GHz + Sandy Bridge), it would still be a CPU bottleneck. I can imagine a high random I/O Environment being even more CPU strenuous. That being said I thought photo editing would be mostly sequential transfers of moderately sized files, so I don't know why SSDs are required.

I only set it up recently so I need to see what kind of performance NFS or iSCSI can give. I really don't to use either of those since it is a multi-client environment and mostly windows, but I will check them for the sake of comparison.
 
hi gea,

nice box. thanks for your post.

do you trust the Sandisk Extreme2-960 for long term endurance?
i see you use sata. my jbods all have expanders.
is there a nice solution to connect 16 drives in an external enclosure with sata connections? no sas/sata mess?

reco
 
about the Sandisk
I have very good experiences with the Sandisk Pro 480
so I expect no problems (ok the 960 is new, so we will see).

But I usually replace SSDs in servers after 3-4 years
(despite the 10 years warranty) and use them then in desktops
or uncritical systems.

I would also avoid Sata disks over SAS expanders, especially
in external enclosures (max cable length of Sata = 1m) therefor
I use the 26 x 2,5" cases from SuperMicro with 3 x LSI HBA controller.
 
I would like to add that if you use RAIDZ or RAIDZ2 with ashift=12 or even 13, you will have to use an appropriate zvol blocksize should you decide to go with iSCSI.
Otherwise you will be down to the space efficiency of mirroring while not getting the read performance advantage of mirrors.
 
Cable length should not be an issue, when using expanders, as the disk -> expander cable length is normally in the .5m or less range. The hba -> expander cable length will be limited to normal sas lengths

I think all this has been mostly sorted out, and works noe, but, like gea, Perfer to go with something known to work, then debugging something unknown, if it can be avoided easily.
 
You must log in or register to reply here.
Back
Top