• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZFS Data Storage Question

E

extide

2[H]4U
Joined
Dec 19, 2008
Messages
3,494
I currently have a server at home with 4x1.5TB disks in two mirrors. I just obtained 4x2TB disks.

I plan on rebuilding my old LGA1366 system into a Linux F@H system / ZFS server, most likely using Ubuntu (unless there are serious objections to this.)

I would like to know the optimal way to use these 8 disks in ZFS.

Should I create 2 RAIDZ1 pools, each using four disks (4x2TB + 4x1.5TB)?

Or is there a way I could create one large pool with all 8 drives and use a RAIDZ2 implementation, but that doesnt waste the extra 500GB on each of the 2TB drives?

Both solutions would give me 2-disk failure, but the latter solution would be better as I could lose any two disks, where as the former I would only have one redundant disk per group of 4.

Note that I would also like to migrate the data from the 4x1.5TB disks into the new pool(s). They are currently on a windows 2003 server that will remain in existence. They are using windows software raid mirrors. I plan on initially making a 4x2TB pool with the 4 new disks in a RAIDZ config, then migrating the data (I plan to use samba and transfer the files using windows file sharing) THEN taking the 4x1.5TB disks out of the old w2003 server and moving then into the new linux server, and either adding them to the existing pool, OR making a new pool just for them.

I would like to maximize available storage and have 2 disk fail redundancy if possible. The array will mostly store large files (mp3, iso, movies, etc) that will consist of mostly sequential reads and writes.


TIA!
 
So, after doing some research on this, and optimal pool sizes, I decided to get some more drives.

I am going to do a 6x2TB pool with RaidZ2, and have 2 extra matching disks. I may end up using one as a hot spare but we'll see how it goes as far as getting them all in the case, heh. I like this plan a lot better as those 1.5TB drives are really old and it would be best if they were just retired.
 
If you're going to use Ubuntu I would read up on the native ZFS kernel available for some Linux distro's, otherwise you'll have to run ZFS on FUSE which offers slow performance. A 6x 2TB RAIDZ2 pool is a good setup, I'd add one as a hot spare and leave the other ready to replace a drive in case of failure. If you want to add the 1.5TB disks you'll want to make a new pool, mixing drive sizes will truncate the largest drives to match those of the smaller drives.
 
Yeah, I went ahead and tried this out on a VM, using ubuntu, and the native ZFS stuff, as well as getting samba to work properly from it. Got it all running just fine. I wanted to test it out in a VM forst so when I do it on the real hardware I will know wtf I am doing.

I am going to go with that config, 6x2TB in raidz2, with 1 hotspare and 1 coldspare.
 
You'd probably get a little more flexibility using OpenIndiana + NappIt over Ubuntu for ZFS plus much better performance with Windows file sharing... Just a thought.
 
@tdg - you would not need to create a new pool to use the 1.5TB drives, just a new VDEV. You could use 6x 2TB drives in a raidz2 VDEV and 6x 1.5TB drives in a raidz2 VDEV all within the same pool and the pool size would not be limited to the effective capacity of two 6x1.5TB raidz2 VDEVs. Drive capacity is only truncated if you use disks of varying capacity within the same VDEV, such as a raidz or mirror.
This is my understanding of ZFS anyhow.
 
Dalfo002 said:
@tdg - you would not need to create a new pool to use the 1.5TB drives, just a new VDEV. You could use 6x 2TB drives in a raidz2 VDEV and 6x 1.5TB drives in a raidz2 VDEV all within the same pool and the pool size would not be limited to the effective capacity of two 6x1.5TB raidz2 VDEVs. Drive capacity is only truncated if you use disks of varying capacity within the same VDEV, such as a raidz or mirror.
This is my understanding of ZFS anyhow.
Click to expand...

Yep, that's pretty much how it works. I have 6x3tb and 4x2tb raidz2 vdev's in the same pool, the 6x3tb vdev is not being limited in any way.
 
coolrunnings said:
You'd probably get a little more flexibility using OpenIndiana + NappIt over Ubuntu for ZFS plus much better performance with Windows file sharing... Just a thought.
Click to expand...

I will look into this, but I need to be able to fold on this box as well.

What are the benefits of OpenIndiana?
 
It's an opensolaris derivative, so the ZFS support is more up to date than freebsd and more stable (more exposure in the field) than linux.
 
I would add:
the Solaris Kernel based SMB Server is mostly faster than SAMBA based solutions
and is more Windows compatible regarding ACL and you have access to ZFS snaps
via Windows "previous version"

Not to mention goodies like dtrace, Comstar, Crossbow and service mangement
 
Interesting. Faster SMB performance would definitely be a plus, but I really do like the flexibility of using Linux as the host os. I will definitely play around with OpenIndiana in a vm, at the very least.
 
Incase anyone is wondering, I decided to keep Ubuntu, and make it be an iSCSI target. Using my win2003 box as the iSCSI initiator, and then sharing out out over smb.

Using SAMBA on the Ubuntu box and sharing out the ZFS volume that way was really slow, ~2-3MB a sec max, where as I can see WAY better performance using iSCSI.

The win2003 box and ubuntu box each are connected to the lan by gigE, and they additionally have a fiber gig link directly between them. Once I migrate more of my storage out of the win2003 box and into the Ubuntu box (and thus free up a pci slot in the w2003 box) I will add another pair of fiber gig card and another dedicated fiber link between them.

Using MPIO / round robin with the two interfaces currently, and getting the following performance between them. I am pretty satisfied with this (this is as observed from the win2003 box).

-----------------------------------------------------------------------
CrystalDiskMark 3.0 (C) 2007-2010 hiyohiyo
Crystal Dew World : http://crystalmark.info/
-----------------------------------------------------------------------
* MB/s = 1,000,000 byte/s [SATA/300 = 300,000,000 byte/s]

Sequential Read : 60.882 MB/s
Sequential Write : 102.570 MB/s
Random Read 512KB : 62.706 MB/s
Random Write 512KB : 79.385 MB/s
Random Read 4KB (QD=1) : 1.526 MB/s [ 372.4 IOPS]
Random Write 4KB (QD=1) : 1.527 MB/s [ 372.9 IOPS]
Random Read 4KB (QD=32) : 62.681 MB/s [ 15302.9 IOPS]
Random Write 4KB (QD=32) : 52.046 MB/s [ 12706.7 IOPS]

Test : 1000 MB [H: 20.2% (1430.8/7076.9 GB)] (x1)
Date : 2012/04/09 20:15:50
OS : Windows Server 2003 SP2 [5.2 Build 3790] (x86)
 
Last edited:
You must log in or register to reply here.
Back
Top