• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Under-sizing ZFS for future replacement

T

tarnar

n00b
Joined
Feb 23, 2014
Messages
16
One of the quirks I've read about ZFS during my research / build of my current storage server is that replacing disks absolutely requires the new disk to have the same number of sectors or more.

For example, if you replace a 2TB drive with a 2TB drive that's even a few sectors smaller then you are screwed, the pool won't accept it.

So what's the best way to solve this problem?

I've read that in certain configs there are reasons not to partition drives when using ZFS (related to disk cache?). I don't know if those are still relevant today.

Are there ways to create a pool / vdev and "hard-code" the sizes to be purposely under-sized?

Is there a recommended amount to under-size by?
 
I have heard about this too about ZFS. Hard disks vary in size from vendor to vendor, some are 10,000 sectors big, others are 10,000+1 sectors big, or 10,000-2 sectors. etc. This has bugged me some, and I checked it up.

ZFS solves this problem like this. I dont remember all the details, but something like this: ZFS allocates 9,990 sectors, so it under sizes always the disks. Problem solved.
 
Hard disks vary in size from vendor to vendor
Click to expand...

I have not seen that in a long time. I mean the following lists are from SATA II and SATA III drives from all of the 5 different manufacturers (Hitachi, Seagate, WDC, Toshiba and Samsung):

Code: 
# fdisk -l /dev/sd? 2> /dev/null | grep 2000
Disk /dev/sda: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sde: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdf: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdg: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdh: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdi: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdj: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdk: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors


datastore4 ~ # fdisk -l /dev/sd? 2> /dev/null | grep 2000
Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sde: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdf: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdg: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdh: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdi: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdj: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdk: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdl: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdm: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdn: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdo: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors

jmd0 ~ # fdisk -l /dev/sd? 2> /dev/null | grep 2000
Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdf: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdg: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdh: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Disk /dev/sdi: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
 
Last edited:
Oh, thanks for telling me! I did not know. Anyway, ZFS solves this problem by automatically under size the disks. This was a concern some 5 years ago or so, and I checked it up back then.
 
brutalizer said:
Oh, thanks for telling me! I did not know. Anyway, ZFS solves this problem by automatically under size the disks. This was a concern some 5 years ago or so, and I checked it up back then.
Click to expand...

Got any links for this? This seems contrary to the POLA.

If you want to be sure that your pool is "undersized", create it with one component being a sparse-allocated file of the desired size, then replace that component with a real disk. Make sure the pool's autoexpand property is off.
 
TCM2 said:
Got any links for this? This seems contrary to the POLA.

If you want to be sure that your pool is "undersized", create it with one component being a sparse-allocated file of the desired size, then replace that component with a real disk. Make sure the pool's autoexpand property is off.
Click to expand...
What is POLA?

No, I have no links. This was something like 5 years. But I mailed the ZFS mail list with this, and among others, the zfs developers from Sun Microsystems answered (I think). Dont remember the details, but it is not a problem.
 
POLA = principle of least astonishment. Software shouldn't do things that are unexpected and surprising.

When I tell ZFS to use a block device, I expect it to use it fully and not just some part of it.
 
ZFS will spare some blocks that will be unused so you dont get a surprise when you add a new disk. ;)
 
brutalizer said:
ZFS will spare some blocks that will be unused so you dont get a surprise when you add a new disk. ;)
Click to expand...

The surprise would still be that it does this.

"Astonishment" refers to the informed administrator, not a clueless user. We all know that clueless noobs are astonished by so many things. Software should first and foremost be logical in itself and not cater to the ignorant.

BTW, how far does that reservation go? Can I add smaller and smaller disks without hitting problems? No? That's a hint that it's a questionable feature and shouldn't be there in the first place - if it's even there at all.
 
I dont remember. You should check this up, and please post your findings here when you do.

The Sun Microsystem zfs mail list is deleted, if not, I could have looked for that thread (that I started there). Or, if someone knows how to get a copy of the mail list, we can look there.
 
Nope, not that what I asked when I posted my question to the ZFS mail list.

Regarding his problem, that he could not delete any files when he hit 100% full, I think I have read on the ZFS mail list that the way to solve the problem is not to delete files, but to concatenate them shorter, something similar to this:
# echo "" > largefile
which will always work, no matter how full the zpool is. Just shorten the file, and it is gone.

Can anyone confirm?
 
His problem were snapshots. He snapshotted every dataset so deleting doesn't really free space but takes up even more space. And somehow he couldn't delete snapshots either.

But anyway, it's completely unrelated to this thread.
 
Greetings

brutalizer said:
I have heard about this too about ZFS. Hard disks vary in size from vendor to vendor, some are 10,000 sectors big, others are 10,000+1 sectors big, or 10,000-2 sectors. etc. This has bugged me some, and I checked it up.

ZFS solves this problem like this. I dont remember all the details, but something like this: ZFS allocates 9,990 sectors, so it under sizes always the disks. Problem solved.
Click to expand...

The easiest way to manually do this would be (assuming you have a number of identical 2TB drives intended for a Raid-Z2 array) is to take one of the drives and then using the manufacturer supplied tools create a Host protected area on one of the drives of say 10MB, when you then create the array ZFS will create the largest array it can which will be determined by the "smaller" drive, that way if you replace with another branded drive that is even a thousand sectors smaller it will still work.

Same applies if you are going to replace all the drives with identical 3TB ones with autoexpand being on, you take the first 3TB drive and create another 10MB host protected area on that as well, finally replace all the other drives with the rest of the 3TB drives and your new array will be (3TB - 10MB) in size per drive.

As far as any OS is concerned any drive with a HPA on it just looks like a regular drive (although smaller than it otherwise should be). This way is easier as you can just give ZFS the whole drive as you normally would do and you don't have to muck around with fake dummy partitions or anything else like that. This method should work with any other raids out there including hardware raid.

Cheers
 
Yeah, but my understanding is that ZFS does this automatically, so you dont have to think about it.
 
drescherjm said:
I have not seen that in a long time. I mean the following lists are from SATA II and SATA III drives from all of the 5 different manufacturers (Hitachi, Seagate, WDC, Toshiba and Samsung):
Click to expand...

Though IDEMA sizes are very common, not all the manufactures adhere to it across all their product lines yet so their is still risk, though probably less likely if you buy drives intended for RAID/NAS/Enterprisey things.

IDEMA sector formula is:
sectors = 97696368 + ( 1953504 * ( ( # of GB ) - 50) )

Thus follows for your 2TB disks
3907029168 = 97696368 + (1953504 * ((2000) - 50))

On my own ZFS pool I partitioned the disks manually and short stroked them by 100 MiB. 100 MiB is probably excessive, but probably not going to be missed.
 
You must log in or register to reply here.
Back
Top