• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

RaidZ3 much smaller available disk space than expected

M

MarcusXP

Gawd
Joined
Sep 23, 2007
Messages
926
Hello,

I've recently set-up a ZFS box using 23 x 4TB Hitachi drives in RaidZ3
According to this calculator:
http://www.servethehome.com/raid-calculator/
the expected size of the pool should be about 72.8TB (taking in consideration the 3 drives lost for parity and the conversion between 1000 and 1024 - TB/TiB)
However, I am only getting 65.7TB...
This is the listing showing the available space (removed some unrelated information):

root@openindiana:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
Pool-Storage1 65.7T 2.85G 460K /Pool-Storage1
Pool-Storage1/Volume-Storage1 65.7T 65.7T 230K -
rpool 16.8G 2.78G 47K /rpool
Click to expand...


This is my pool with 23 x 4TB drives:

Pool-Storage1 5000 14237568027059623914 vdevs: 1
vdev 1: raidz3 12 92.02 TB 12843423673337650988
c10t5000CCA22EC008ADd0 11897191478853965569 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec008ad:a
id1,sd@n5000cca22ec008ad/a
PL1310LAG029NA
c10t5000CCA22EC00A7Cd0 4700561813297779659 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec00a7c:a
id1,sd@n5000cca22ec00a7c/a
PL1310LAG02TLA
c10t5000CCA22EC00F4Fd0 18078600867002041605 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec00f4f:a
id1,sd@n5000cca22ec00f4f/a
PL1310LAG042EA
c10t5000CCA22EC01A8Dd0 17529955265994712780 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec01a8d:a
id1,sd@n5000cca22ec01a8d/a
PL1310LAG0728A
c10t5000CCA22EC0210Ed0 521233862270374400 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec0210e:a
id1,sd@n5000cca22ec0210e/a
PL1310LAG08TZA
c10t5000CCA22EC021D9d0 16504663500950716529 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec021d9:a
id1,sd@n5000cca22ec021d9/a
PL1310LAG090JA
c10t5000CCA22EC02208d0 17286534736261518309 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec02208:a
id1,sd@n5000cca22ec02208/a
PL1310LAG0921A
c10t5000CCA22EC0221Bd0 11471414523034938154 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec0221b:a
id1,sd@n5000cca22ec0221b/a
PL1310LAG092NA
c10t5000CCA22EC0222Bd0 10137454426417349138 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec0222b:a
id1,sd@n5000cca22ec0222b/a
PL1310LAG0935A
c10t5000CCA22EC02234d0 9661598762332469726 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec02234:a
id1,sd@n5000cca22ec02234/a
PL1310LAG093GA
c10t5000CCA22EC02287d0 2707340434424007771 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec02287:a
id1,sd@n5000cca22ec02287/a
PL1310LAG0964A
c10t5000CCA22EC022ACd0 1409056393389450611 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec022ac:a
id1,sd@n5000cca22ec022ac/a
PL1310LAG097AA
c10t5000CCA22EC022E7d0 13341020487462083677 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec022e7:a
id1,sd@n5000cca22ec022e7/a
PL1310LAG0997A
c10t5000CCA22EC02D19d0 7239933575572325830 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec02d19:a
id1,sd@n5000cca22ec02d19/a
PL1310LAG0D0EA
c10t5000CCA22EC032E8d0 11341704217867462188 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec032e8:a
id1,sd@n5000cca22ec032e8/a
PL1310LAG0EKDA
c10t5000CCA22EC032F8d0 14003683354051041944 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec032f8:a
id1,sd@n5000cca22ec032f8/a
PL1310LAG0EKXA
c10t5000CCA22EC0330Bd0 15602634426552806117 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec0330b:a
id1,sd@n5000cca22ec0330b/a
PL1310LAG0ELJA
c10t5000CCA22EC0340Bd0 1306860508072491531 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec0340b:a
id1,sd@n5000cca22ec0340b/a
PL1310LAG0EVTA
c10t5000CCA22EC04C15d0 16571289578325581979 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec04c15:a
id1,sd@n5000cca22ec04c15/a
PL1310LAG0N89A
c10t5000CCA22EC04C26d0 14950040610680530891 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec04c26:a
id1,sd@n5000cca22ec04c26/a
PL1310LAG0N8VA
c10t5000CCA22EC04C35d0 6692930116809901728 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec04c35:a
id1,sd@n5000cca22ec04c35/a
PL1310LAG0N9AA
c10t5000CCA22EC04C48d0 10360558934820987983 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec04c48:a
id1,sd@n5000cca22ec04c48/a
PL1310LAG0N9YA
c10t5000CCA22EC04C87d0 5707996551731273821 Hitachi HDS5C404
/scsi_vhci/disk@g5000cca22ec04c87:a
id1,sd@n5000cca22ec04c87/a
PL1310LAG0NBZA
Click to expand...

Can anyone more experienced with ZFS tell me what is going on?
Where is the ~9TB lost space? Is it reserved for snapshots, or anything else? Can I recover it?

thanks a lot,
-Marcus
 
Well, you really made a very strange pool there.

with 20 data disks, a 128k block will get striped at 6.4k written per disk. That means on your 4k disks, it will use 8k, so 1.6k wasted per disk per write.

So over 20 disks, the 128k block will use 160k of diskspace, making about 20% of your diskspace wasted.
 
patrickdk said:
Well, you really made a very strange pool there.

with 20 data disks, a 128k block will get striped at 6.4k written per disk. That means on your 4k disks, it will use 8k, so 1.6k wasted per disk per write.

So over 20 disks, the 128k block will use 160k of diskspace, making about 20% of your diskspace wasted.
Click to expand...

Hey Patrick, please reply to my other thread, I opened this one as duplicate by mistake.. thanks a lot!

http://hardforum.com/showthread.php?p=1039865398
 
You must log in or register to reply here.
Back
Top