ZFS Space Accounting

[Yagobg]

Hello All,

I have a question regarding space allocation when creating a zpool with one vdev.
I'm using OI( OpenIndiana Development oi_151.1.5 X86 (powered by illumos)) and napp-it (running version : 0.8h nightly May.03.2012).

I have five 3T in raidz configuration.

zpool status storage
pool: storage
state: ONLINE
scan: none requested
config:

NAME STATE READ WRITE CKSUM
storage ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c5t5000CCA228C44639d0 ONLINE 0 0 0
c5t5000CCA228C46203d0 ONLINE 0 0 0
c5t5000CCA228C46269d0 ONLINE 0 0 0
c5t5000CCA228C4645Ad0 ONLINE 0 0 0
c5t5000CCA228C4670Bd0 ONLINE 0 0 0

errors: No known data errors

My understanding is that I will get about 12T of usable space due to 5 x 3TB = 15TB - 3T(raidz1) = 12TB.
Bellow you can see that the storage pool has 13.6T which great but when I create a Dataset under storage ZFS does not allocate the full size!!! It only allocates about 8TB. There are about 5TB left behind

# zpool list storage
NAME SIZE ALLOC FREE EXPANDSZ CAP DEDUP HEALTH ALTROOT
storage 13.6T 8.48T 5.15T - 62% 1.00x ONLINE -

Even if I create more Datasets under "storage" it will not use the free space in the "storage"pool.

# zfs list|grep storage
storage 7.89T 3.90T 59.1K /storage
storage/dds_del 6.82T 2.83T 6.82T /storage/dds_del
storage/test1 46.3K 2.83T 46.3K /storage/test1

I tried using "zpool set allocated=12T storage" but it comes back with an error saying that allocated is read only.

So is there a way to use the full space of the "storage" pool into a dataset ? Does ZFS reserve some space for snapshots or should I recreate the dataset with some special flag?( I really don't want to recreate it, have few TB of data on it). What em I missing ???

Thank you for the help

 

[patrickdk]

well, 12tb of raw data, is really only going give you 10.9tb of diskspace.

So that is 13.6tb for your pool, that matchs storage pool at 13.6t just fine, you have used up 8.48tb so far

Your storage fs has 7.89tb of diskspace used, that would blow up to 9.86tb with parity info, guess you have some compression or something goin on, as 9.86tb is > 8.48tb

now why one says 3.9tb free and the others 2.83tb, probably did you create this with nappit? and it set a 10% reserve on the zfs?

 

[madrebel]

So is there a way to use the full space of the "storage" pool into a dataset ?

by default, unless you set a quota, the data set will use space until it can't.

zfs get all -o space -r

will give you another view.

 

[danswartz]

This syntax seems illegal?

 

[madrebel]

hmm, might be -o used -r

 

[Yagobg]

Thanks for the explanation guys.
After some more testing it is more clear to me.
I was thinking that the ALLOC value under the pool is fixed and that the dataset would not grow pass that value.

NAME SIZE ALLOC FREE EXPANDSZ CAP DEDUP HEALTH ALTROOT
storage 13.6T 11.1T 2.57T - 81% 1.00x ONLINE -
root@DGA-NAS05:/# zpool list storage
NAME SIZE ALLOC FREE EXPANDSZ CAP DEDUP HEALTH ALTROOT
storage 13.6T 12.0T 1.66T - 87% 1.00x ONLINE -

Thanks again.