• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZFS config

M

miercoles13

n00b
Joined
Jul 25, 2012
Messages
10
I'm sorry if this is a duplicate question, I seen similar post but under many threads that I cant seem to find at the moment.

Well I recently started to build a NAS system played around with different brews ended up going with NAS4Free. I proceeded to go with the 20 bay NORCO server case and already have 12 drives currently allocated as:

2x 30Gig SSD cache drives (internal non hot swappable)
6x 750gig + 4x 1tb drives in a RAIDZ2 (hot swappable) reports about 6.8Tbs with about 2.47Tbs used. (still testing)

I been doing some reading on how to best organize my drives and I seem to understand I would get better performance and redundancy if I created smaller drive groups(vdevs) as the following example:

4x 1tb RaidZ1 3tb usable (already owned)
4x 1tb RaidZ1 3tb usable (to be purchased)
4x 1tb RaidZ1 3tb usable (to be purchased)
4x 1tb RaidZ1 3tb usable (to be purchased)
4x 750gig Raidz1 2.2tb usable (already owned)
Estimated total = 14.2Tb

By doing this I completely remove two of my 750gigs to be used somewhere else and I could easily upgrade the remaining 4x 750 drives to 1 or 2 tb drives in the future.

I would wonder if this would grant me better performance over a giant RAIDZ2 vdev?
Also would SSD ZIL be recommended if I plan on running 10+ VMs ranging from DCs to servers running medium sized databases?

Just for reference

Norco RPC-4226 4U http://www.newegg.com/Product/Product.aspx?Item=N82E16811219033
NC360T - PCI EXP Dual Port Gigabit Server Adapter http://www.serversupply.com/NETWORKING/NETWORK INTERFACE CARD (NIC)/2 PORT/HP/NC360T.htm
2x Samsung DDR3-133 8GB ECC M391B1G73BH0-CH9 http://www.superbiiz.com/detail.php?name=D38GE133S2
Supermicro MDB-X9SCM-F-O LGA 1155 Intel C204 Micro ATX http://www.newegg.com/Product/Product.aspx?Item=N82E16813182253
Intel Core i3-2100 Sandy Bridge 3.1Ghz http://www.newegg.com/Product/Product.aspx?Item=N82E16819115078
3x IBM M1015 PCI-E X8 SAS/SATA Raid Controller http://www.serversupply.com/CONTROLLERS/SAS-SATA/SERVERAID M1015/IBM/46M0861.htm
2x Corsair Nova Series 2 CSSD-V30GB 2.5 SSD Drives http://www.newegg.com/Product/Product.aspx?Item=N82E16820233266
 
Why so many 1TB drives? I would just get a few 2 or 3TB drives and use raidz2.
 
I currently have the WD black RE3 1tb drives and have experience extremely good performances for the past 2 years. I want to continue using WD RE drives and currently the 1tb drives are definitely within range.I love to jump onto 2tb drives with RE4, but they are just so expensive at the moment.
 
I would go for raidz2 instead of raidz1. Eight disk raidz2 is much safer than two four disk raidz1, right? In both cases you have two disks for redundancy,, but in one case loss of two disks will make you loose the whole pool.

Instead, buy eight disks and make a new zpool of it. Move all data from the old pool to the new pool and reuse the old four disks when you add a new eight disk raidz2 vdev.
 
I see, I would picture this as something like the following:

Code: 
NAME        STATE     READ WRITE CKSUM
   pool1       ONLINE       0     0     0
     raidz2-0  ONLINE      0     0     0
       da0     ONLINE       0     0     0
       da1     ONLINE       0     0     0
       da2     ONLINE       0     0     0
       da3     ONLINE       0     0     0
       da4     ONLINE       0     0     0
       da5     ONLINE       0     0     0
       da6     ONLINE       0     0     0
       da7     ONLINE       0     0     0
     raidz2-1  ONLINE     0     0     0
       da8     ONLINE      0     0     0
       da9     ONLINE      0     0     0
       da10     ONLINE     0     0     0
       da11     ONLINE     0     0     0
       da12     ONLINE     0     0     0
       da13     ONLINE     0     0     0
       da14     ONLINE     0     0     0
       da15     ONLINE     0     0     0

It looks like a simplified way of creating the vdevs using 8x2 drives. I can think of two possible drawbacks if my understanding of ZFS is correct, for future upgrade/expanding I would need to upgrade all of the drives in each 8 drive vdev to gain extra storage. Like if time passed and I decided to switch the 1tb drives with 2 tb drives it would have to be all 8 drives.

Not only that but this would leave me with 4 hot swappable bays left forcing me to make a 4 drive vdev. Not a major issue if my understanding of ZFS pools they can continue to be expanded, but I read sometime ago it was recommended to maintain vdev consistency.
 
I just received a newegg email for Western Digital Caviar Black WD2002FAEX 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive -Bare Drive at 169$ that seems like a pretty dam good deal for the performance and 5 year warranty. Anyone know if the RE are preferred over the black caviar? All i heard was the RE did not spin down like regular desktop drives.
 
I'm not sure if this is the right place to ask this, but how bad is it to create a raidz2 with five 3TB disks?
I've been told that this is an issue due to the small stripesize.
Is this a huge problem? Or is there just a small performance hit I wouldn't notice over my Gbit network connection anyway.

Thanks!
 
2x Corsair Nova Series 2 CSSD-V30GB 2.5 SSD Drives
Click to expand...
For ? ... ZIL ??!!?? oO
- 2x SSD mirrored ZIL, according to your budget, by performance :
OWC Mercury Entreprise Pro 100 Go (ref OWCSSDMEP6G100 | OWC SSD MEP 6G 100 , 480 € / 7 years of waranty ) < Micron Real SSD P300 < Hitachi Ultrastar SSD400M Enterprise </= Toshiba MKx001GRZB < STEC ZeusRAM
=> OWC Mercury Entreprise Pro 100 Go seems to be the most reliable price / performance rate !

cf : http://www.storagereview.com/owc_mercury_enterprise_pro_6g_ssd_review
+ http://www.storagereview.com/hitachi_ultrastar_ssd400m_enterprise_ssd_review

owc_mercury_enterprise_pro_6g_200gb_hba_4k_writesteady_qd32.png

miercoles13 said:
I just received a newegg email for Western Digital Caviar Black WD2002FAEX 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive -Bare Drive at 169$ that seems like a pretty dam good deal for the performance and 5 year warranty. Anyone know if the RE are preferred over the black caviar? All i heard was the RE did not spin down like regular desktop drives.
Click to expand...
For BIG array, WD RE or Seagate Constallation ES2 are preferred.
For customer Raid use, WD Black is ok.
You can check my benchmark with 5x 1 To WD Black in ZFS RaidZ
=> http://hardforum.com/showpost.php?p=1038981342&postcount=3628

I'm not sure if this is the right place to ask this, but how bad is it to create a raidz2 with five 3TB disks?
I've been told that this is an issue due to the small stripesize.
Is this a huge problem? Or is there just a small performance hit I wouldn't notice over my Gbit network connection anyway
Click to expand...
... have a look : http://hardforum.com/showthread.php?t=1546137

Cheers.

St3F.
 
Last edited:
ST3F said:
For ? ... ZIL ??!!?? oO
- 2x SSD mirrored ZIL, according to your budget, by performance :
Click to expand...
These i originally bought for cache, wasnt even thinking of ZIL. I been reading about ZIL and have considered the Intel 330 Series Maple Crest SSDSC2CT060A3K5 drives.

http://www.newegg.com/Product/Product.aspx?Item=N82E16820167120

As for some of the drives you mentioned, they seem a bit high on the budget at the moment.

ST3F said:
For BIG array, WD RE or Seagate Constallation ES2 are preferred.
For customer Raid use, WD Black is ok.
You can check my benchmark with 5x 1 To WD Black in ZFS RaidZ
=> http://hardforum.com/showpost.php?p=1038981342&postcount=3628
... have a look : http://hardforum.com/showthread.php?t=1546137
Click to expand...

Those are some neat numbers, thanks for sharing.
 
miercoles13 said:
These i originally bought for cache, wasnt even thinking of ZIL. I been reading about ZIL and have considered the Intel 330 Series Maple Crest SSDSC2CT060A3K5 drives.

http://www.newegg.com/Product/Product.aspx?Item=N82E16820167120

As for some of the drives you mentioned, they seem a bit high on the budget at the moment.
Click to expand...
Here, a test of SSD for ZIL
=> http://forums.freebsd.org/showthread.php?t=23566

NFS Mirrored ZIL

Code: 
Ordinary HW
Intel 320    40GB  = 30MB/s
OCZ Vertex 2 60GB  = 32MB/s
OCZ Vertex 2 120GB = 36MB/s
Intel 320    120GB = 52MB/s
Zeus IOPS    16GB  = 55MB/s
Intel X25-E  32GB  = 60MB/s
OCZ Deneva 2 200GB = 67MB/s
OCZ Vertex 3 240GB = 70MB/s

As you know, SSD SLC or SSD MLC with supercap / eMLC are preferred for ZIL.

miercoles13 said:
Those are some neat numbers, thanks for sharing.
Click to expand...
You're welcome.
If you want to have a look with 2 differents vDev
=> 5x 1 WD Black + 5x 2 To Hitachi 7k3000 : http://hardforum.com/showpost.php?p=1038983445&postcount=3629

Have a look at the thread of devilmouse with 6x 2TB Samsung F4s
=> http://hardforum.com/showpost.php?p=1036642118&postcount=156
Performance a similar to me in sequential : RaidZ1 with 5 drives or RaidZ2 with 6 drives, for sequential transfer it seems only the rate of 4 disks is reached.

Cheers.

St3F
 
Unfortunately for me it comes down how much $ i can afford to throw at this server. While I'm keeping a close watch on prices for WD drives with 5yr warranty I'm also looking at SSD options for cache and or ZIL.

ST3F said:
Here, a test of SSD for ZIL
=> http://forums.freebsd.org/showthread.php?t=23566

NFS Mirrored ZIL

Code: 
Ordinary HW
Intel 320    40GB  = 30MB/s
OCZ Vertex 2 60GB  = 32MB/s
OCZ Vertex 2 120GB = 36MB/s
Intel 320    120GB = 52MB/s
Zeus IOPS    16GB  = 55MB/s
Intel X25-E  32GB  = 60MB/s
OCZ Deneva 2 200GB = 67MB/s
OCZ Vertex 3 240GB = 70MB/s

As you know, SSD SLC or SSD MLC with supercap / eMLC are preferred for ZIL.
Click to expand...

These are some sweet numbers, I wonder what the OCZ Vertex 4 can put out on similar setups.

On a side question, after ZFS 28 if the ZIL drive takes a dive the data is no longer completely gone and can run in degraded mode? If so I could reduce the cost by not using a mirror for ZIL.

Anyhow, at the moment my immediate target is to double the RAM to 32Gigs since it seems at the simplest and cheapest way to increase performance. Secondly I would vest some money on some storage drives and finally obtain some fast SSDs preferably SATAIII for cache and finally at the end determine if ZIL drives will be needed or can be afforded :eek:

For example:

http://www.newegg.com/Product/Product.aspx?Item=N82E16820139917
 
You must log in or register to reply here.
Back
Top