• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

ZFS, adding more disks into zpool

M

mephistopoa

Weaksauce
Joined
Dec 11, 2010
Messages
122
Hi gents,

I'm still evaluating my backup server solution and I'm in doubt about the expansion of a zpool by adding more disks. I know some earlier versions of ZFS especially on freebsd are quite old comparing to Solaris (currently on version 33 I think) and it was not possible to extend a existing zpool, lets say a raidz pool with more disks.

My plan is to begin with a small number of 2 or 3TB drives and as I manage to get more clients on board we slowly add more disks and expand the existing array. My idea is to share the storage box via iSCSI to a Windows server that will manage all backup jobs (VM backups using Veeam).

It is not clear for me even is Solaris and ZFS version 33 can do this, so I'm also wondering if open indiana or any other good distribution has support for this as it will make a massive difference on my decision about the hardware.

At the moment I'm planning to get a Dell R515 with 12x 3.5 hot swap bays + 2 internal 2.5 inches bays (probably will be used with 2 small drives for OS mirrored)

Could you guys clarify this for me please? Or where I could find related documentation because so far I've got tons of articles from different dates talking about this as a not supported feature yet.
 
Zpool is build by vdevs-one or more,you can add vdev to the pool all of the time.You cannot add drive to existing raidz based vdev except spare.You can add drive to existing mirrored vdev [ i.e. triple mirror ].
You can add drive to existing single drive vdev and it's become mirror vdev.

The slices/partitions can participate in vdev too,but it's rare cases.
 
dedobot said:
Zpool is build by vdevs-one or more,you can add vdev to the pool all of the time.You cannot add drive to existing raidz based vdev except spare.You can add drive to existing mirrored vdev [ i.e. triple mirror ].
You can add drive to existing single drive vdev and it's become mirror vdev.

The slices/partitions can participate in vdev too,but it's rare cases.
Click to expand...

Does this apply for a striped mirror as well?

for example, I have 2 striped mirrored pool of 2 disks each, I want to add other 2 disks to extend the stripe/mirror, not possible?
 
The zpool is always a stripe of vdevs if they are [vdevs] more of one.
My understanding is that you have two mirror vdevs in one pool. If this is right- you can add another vdev[ mirrored or what you want] to the pool.
 
please be clear with names especially with pool and vdev:

You have a ZFS pool that is build from vdevs. You can increase poolsize by adding new vdevs
You can add spares or Cache drives to a pool. A pool is always striped over all vdevs.

A vdev is a single disk, a mirror or a Raid-Z.
Beside mirrors you cannot add disks to vdevs

If you have a pool build from two vdevs now (each a mirror = Raid 10), you can
- add more vdevs to the pool (mirror or others) to increase pool performance and space
- add disks to a current mirror to have a 3way mirror (increase security and read-performance)
 
will ZFS then make the striped data across the last pair of disks like some sort of resilvering?

My idea is, there is data on 2 mirrored striped pairs, I add a 3rd pair of disks. Will be the existing striped data across the 2 pairs be spread evenly with the addition of the new drive?
 
mephistopoa said:
will ZFS then make the striped data across the last pair of disks like some sort of resilvering?

My idea is, there is data on 2 mirrored striped pairs, I add a 3rd pair of disks. Will be the existing striped data across the 2 pairs be spread evenly with the addition of the new drive?
Click to expand...

No,the pool will grow in size,but the data will sits physically where is it. To benefit of the bandwidth gained with adding more vdev ,you need re-copy all existing data to the pool,of course all new written data to the pool will get the benefit.
Keep in mind- adding more vdevs to the pool will NOT help for redundancy/reliability in any way,but you gain more space and speed.
 
I'm very confused with this vdev and vpool thing now. In my understanding vdev was nothing more than a virtual representation of a disk, so in case I have 8 raw disks, I have 8 vdevs. Then zpool would be the raid logical container of a group of disks. Why then I issue a zpool command to create a pool? Why simply not use a "vdev command" to create the pool of disks then?

I must admit that this got me completely lost. I read some stuff here that doesn't make much sense to me:

http://constantin.glez.de/blog/2010/06/closer-look-zfs-vdevs-and-performance

Vdev examples

The following typical command creates a new pool out of one vdev which is a mirror of two disks:

zpool create tank1 mirror c0t0d0 c0t1d


What is the difference between a vdev and a zpool? Why he sayas create a mirror from one vdev consisting of 2 disks if for me it simply looks as 2 single drives?
 
A vdev is a virtual device. It can be a single disk, an n-way mirror, a collection of physical disks in some flavor of raidz*. You can then concatenate (stripe) one or more such vdevs into a pool. So, no, your 8 physical disks are not 8 vdevs (unless you set up a pool with 8 disks in a raid0 - highly not recommended.) In the case you pasted, you could do something like:

zpool create tank1 mirror c0t0d0 c0t1d0 mirror c1t0d0 c1t1d0 and have two mirrors (each a vdev) striped into a pool. Make more sense?
 
danswartz said:
A vdev is a virtual device. It can be a single disk, an n-way mirror, a collection of physical disks in some flavor of raidz*. You can then concatenate (stripe) one or more such vdevs into a pool. So, no, your 8 physical disks are not 8 vdevs (unless you set up a pool with 8 disks in a raid0 - highly not recommended.) In the case you pasted, you could do something like:

zpool create tank1 mirror c0t0d0 c0t1d0 mirror c1t0d0 c1t1d0 and have two mirrors (each a vdev) striped into a pool. Make more sense?
Click to expand...

So, I can have a vdev of several disks in raidz, what is the need of the pool command? just to group diferent vdevs into a single container?
 
Raw disk/s -> vdev [single disk,stripe,mirror or raid5,6,7]->zpool [one or more vdevs,always stripe/raid0]
 
Consider yourselves given a pint each!:cool:

much appreciated your patience guys, when I get skilled on this thing I'll pass the knowledge forward
 
Ow, now I understand exactly why to zpool 2 raidz vdevs... makes so much more sense now!

Now, back to the drawing board :D
 
Hey hey, I want to add a spare to a pool - in this example I'll use the syspool, but the problem is the same when I try to add 'whole d0 disk / LUN' to other pools.

Below, example - the pool exists, the disk shows up in format, I try to add it as a spare, but it's not in /dev/dsk, but it shows in cfgadm.

Problem is the d0 devices are missing. How do I get them back?
I've tried reboot -- -rv and format & fdisk.

root@zappy:/export/home/admin# zpool status syspool
pool: syspool
state: ONLINE
scan: scrub repaired 0 in 0h4m with 0 errors on Fri Aug 10 21:51:36 2012
config:
NAME STATE READ WRITE CKSUM
syspool ONLINE 0 0 0
c1d1s0 ONLINE 0 0 0
errors: No known data errors

root@zappy:/export/home/admin# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1d1 <DEFAULT cyl 19454 alt 2 hd 255 sec 63>
/pci@0,0/pci-ide@14,1/ide@0/cmdk@1,0
1. c2t0d0 <ATA-WDC WD20EARS-00M-AB51-1.82TB>
/pci@0,0/pci103c,1609@11/disk@0,0
2. c2t1d0 <ATA-WDC WD20EARS-00M-AB51-1.82TB>
/pci@0,0/pci103c,1609@11/disk@1,0
3. c2t2d0 <ATA-WDC WD20EARS-00M-AB51-1.82TB>
/pci@0,0/pci103c,1609@11/disk@2,0
4. c2t3d0 <ATA-WDC WD20EARS-00M-AB51-1.82TB>
/pci@0,0/pci103c,1609@11/disk@3,0
5. c3t0d0 <DEFAULT cyl 30398 alt 2 hd 255 sec 63>
/pci@0,0/pci1022,9604@4/pci1095,7132@0/disk@0,0
6. c3t1d0 <ATA-WDC WD20EARS-00M-AB51-1.82TB>
/pci@0,0/pci1022,9604@4/pci1095,7132@0/disk@1,0
7. c4t50014EE25C2D192Dd0 <DEFAULT cyl 60798 alt 2 hd 255 sec 252>
/scsi_vhci/disk@g50014ee25c2d192d
8. c4t50014EE25C37BA64d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 252>
/scsi_vhci/disk@g50014ee25c37ba64
9. c4t50014EE6006C36EFd0 <ATA-WDC WD20EARS-00M-AB51-1.82TB>
/scsi_vhci/disk@g50014ee6006c36ef
Specify disk (enter its number): ^C
root@zappy:/export/home/admin#


root@zappy:/export/home/admin# zpool add syspool c3t0d0 spare
invalid vdev specification: spare requires at least 1 devices


root@zappy:/export/home/admin# ls /dev/dsk/*d0
/dev/dsk/c2t0d0 /dev/dsk/c3t1d0 /dev/dsk/c4t50014EE6AB16CBC1d0
/dev/dsk/c2t1d0 /dev/dsk/c4t50014EE25B9793EEd0 /dev/dsk/c4t50014EE6AB16CC1Fd0
/dev/dsk/c2t2d0 /dev/dsk/c4t50014EE6006C1926d0 /dev/dsk/c4t50014EE6AB16D061d0
/dev/dsk/c2t3d0 /dev/dsk/c4t50014EE6006C36EFd0


SO d0 devices don't exist for c3t0d0, c4t50014EE25C2D192Dd0, and c4t50014EE6006C36EFd0 the other wwn devices are synonyms for the c#t#d0 devices.


root@zappy:/export/home/admin# cfgadm -al | grep 'disk\|scsi'
c9 scsi-sas connected configured unknown
c9::w50014ee6006c36ef,0 disk-path connected configured unknown
c10 scsi-sas connected configured unknown
c10::w50014ee25c37ba64,0 disk-path connected configured unknown
c11 scsi-sas connected configured unknown
c11::w50014ee25c2d192d,0 disk-path connected configured unknown
c12 scsi-sas connected unconfigured unknown
sata0/0::dsk/c2t0d0 disk connected configured ok
sata0/1::dsk/c2t1d0 disk connected configured ok
sata0/2::dsk/c2t2d0 disk connected configured ok
sata0/3::dsk/c2t3d0 disk connected configured ok
sata1/0::dsk/c3t0d0 disk connected configured ok
sata1/1::dsk/c3t1d0 disk connected configured ok
root@zappy:/export/home/admin#
root@zappy:/export/home/admin#

How do I get the d0 devices so I can add the missing devices to the pools. I would like to add c3t0d0 to syspool and the other two to another pool - all as spares - and then do some replaces.

thanks,
joseph
 
three are missing from /dev/dsk - they show up in format.
d0 devices don't exist for c3t0d0, c4t50014EE25C2D192Dd0, and c4t50014EE6006C36EFd0.
the other wwn devices are synonyms for the c#t#d0 devices.

root@zappy:/export/home/admin# ls /dev/dsk/*d0
/dev/dsk/c2t0d0
/dev/dsk/c3t1d0
/dev/dsk/c4t50014EE6AB16CBC1d0
/dev/dsk/c2t1d0
/dev/dsk/c4t50014EE25B9793EEd0
/dev/dsk/c4t50014EE6AB16CC1Fd0
/dev/dsk/c2t2d0
/dev/dsk/c4t50014EE6006C1926d0
/dev/dsk/c4t50014EE6AB16D061d0
/dev/dsk/c2t3d0
/dev/dsk/c4t50014EE6006C36EFd0

see what I mean, those three are not there. is there a way to create the d0 devices in /dev/dsk. Actually the p0, & s0 etc devices are there for each of the three, but the d0 is not.
thanks,
joseph
 
here's what I mean - /dev/dsk/c3t0d0 is missing. c3t1d0 is there. similarly c4t5//xxx192Dd0, and c4t5//xxx36EFd0 are also missing. How do I get these 'whole disk / LUN / d0' devices to be there?

root@zappy:/export/home/admin# ls /dev/dsk/c3t*
/dev/dsk/c3t0d0p0 /dev/dsk/c3t0d0s14 /dev/dsk/c3t1d0p0 /dev/dsk/c3t1d0s14
/dev/dsk/c3t0d0p1 /dev/dsk/c3t0d0s15 /dev/dsk/c3t1d0p1 /dev/dsk/c3t1d0s15
/dev/dsk/c3t0d0p2 /dev/dsk/c3t0d0s2 /dev/dsk/c3t1d0p2 /dev/dsk/c3t1d0s2
/dev/dsk/c3t0d0p3 /dev/dsk/c3t0d0s3 /dev/dsk/c3t1d0p3 /dev/dsk/c3t1d0s3
/dev/dsk/c3t0d0p4 /dev/dsk/c3t0d0s4 /dev/dsk/c3t1d0p4 /dev/dsk/c3t1d0s4
/dev/dsk/c3t0d0s0 /dev/dsk/c3t0d0s5 /dev/dsk/c3t1d0s0 /dev/dsk/c3t1d0s5
/dev/dsk/c3t0d0s1 /dev/dsk/c3t0d0s6 /dev/dsk/c3t1d0s1 /dev/dsk/c3t1d0s6
/dev/dsk/c3t0d0s10 /dev/dsk/c3t0d0s7 /dev/dsk/c3t1d0s10 /dev/dsk/c3t1d0s8
/dev/dsk/c3t0d0s11 /dev/dsk/c3t0d0s8 /dev/dsk/c3t1d0s11 /dev/dsk/c3t1d0s9
/dev/dsk/c3t0d0s12 /dev/dsk/c3t0d0s9 /dev/dsk/c3t1d0s12
/dev/dsk/c3t0d0s13 /dev/dsk/c3t1d0 /dev/dsk/c3t1d0s13
root@zappy:/export/home/admin#
 
You must log in or register to reply here.
Back
Top