HELP: no space left on device ZFS, and pool setup help

[DataMine]

Hello I need help and my google searches have done nothing to help me in the last 2 hours.

I have a pool media03 that had 20 GB of free space. I know its not idea to let the pool fill up this much but I was going to move some data to anouther pool (media02) once I got enough drives to fully upgrade it. How ever a backgroup backup process from my laptop completely filled the pool up last night. I tried deleting/moving files. no luck I keep getting "no space left on device" error. I tried rm -f command. It looks like the only way get back in is to expand the pool. but I know you cant add a single drive to a vdev. (unless its a mirror which its not) so how do I add a single drive vdev to a zraid3 pool? (I know you cant remove a vdev, I am planning to tear down the pool latter once all the data has been moved).


Also I am thinking of changing my pool configs which is better?

I am planning on taking down the three pools that I have (media01,media02,media03) which use the following config
Media01 : 19x2TB Raidz3
Media02 : 5x1TB Raidz1
Media03 : 11x3TB Raidz3

I am thinking of 2 different configs
Media01
vdev1 -11x3TB Raidz3
vdev2 -11x2TB Raidz3
vdev3 -11x3/2TB Raidz3 (plan on upgrading all disk to 3 TB)
how every each set of disk is a mix of 5400-7200 sata drives (most of which are in the current pool expect for the ones that are make checksum errors or read errors)

my other config is just one pool (Media01) with mirror sets of 2,3,4 TB hard drives with vdevs added or upgraded as more space is needed. Adding space or upgrading should be easy since I only need to change 2 drives at a time with it. But I lose 1/2 of my total space. Also are 3/4/5/6Tb hard drives safe with only 1 mirror?

Once set I plan on using the configuration for 3-4 years before upgrading again. I dont want to have the current mess like I do with 3 different pools to manage/backup and scrub weekly/biweekly

 

[Shockey]

Have you tried adding single drive vdev to the pool? Also what OS and version of ZFS are you running?

 

[danswartz]

You can add a single drive to get out of this, but you will certainly want to copy everything off, remove the added drive and rebuild the pool.

 

[DataMine]

Ok I was able to add a single disk vdev and expand the array, I am now moving data off of it.

 

[DataMine]

anyone have any ideas on the pool configuration I should go with. I am leaning towards the mirrored drive pool

Media01
vdev1 2x3TB
vdev2 2x3TB
vdev3 2x3TB
vdev4 2x3TB
vdev5 2x3TB
vdev6 2x3TB
vdev7 2x3TB
vdev8 2x3TB
vdev9 2x2TB
vdev10 2x2TB
vdev11 2x2TB
ect..

But I am worried when I upgrade to larger drives like 4/5/6/7 TB drives if a single mirror will be enough for data protection, how ever uprading drives should be easy since I only need to upgrade 2 drives at a time. I just have to make sure that the batches are different. I also can pair the drives with the same RPM in the same vdev, unlike now were they are all mixed.

vs

Media01
vdev1 -11x3TB Raidz3 (mixed RPM, mixed manufactures)
vdev2 -11x3/2TB Raidz3 (mixed RPM, plan on upgrading all disk to 3 TB)
vdev3 -11x2TB Raidz3 (mixed RPM, mixed manufactures) (uprade to 4/5Tb drives?)

 

[danswartz]

I would stick with the raidz3 approach.

 

[zrav]

Can't you delete a snapshot to make space or zfs send a smaller dataset to somewhere then delete the dataset?

 

[DataMine]

This array had no snap shots