Switched filesystems and used space changed.

M

misterpat

Weaksauce
Joined
Dec 24, 2011
Messages
103
Hey all.

I recently switched from EXT4 to XFS to allow me to resize above 16tb safely. When I was on EXT4, Webmin was showing I had 1.92TB of space used. Thats including the OS also.

Once I switched to XFS, Webmin now shows just over 1TB. All files are accounted for. I FTP'ed them off the old filesystem onto a backup server I have. All files went over fine, no errors. Once I had XFS set up, I FTP'ed them back over, again, no errors.

The only thing I did different was the chunk size. On EXT4, I set it to 64kb. On XFS, I set it to defaults, 4kb.

Any ideas?
 
Edit: just realized that you said 1.92tb vs 1tb used - that definitely is a big discrepancy. Did you actually check the files to make sure they're intact and all functioning properly?
 
Yes, it is a big discrepancy. Weird thing is, the files, so far, seem to be ok. Music plays fine and watched a few MKV video files and all seemed well.

Would the chuck size have anything to do with it?
 
What kind of files would be a big factor. All large media files or all small/very small files?
 
it's possible you had an absolute fuckton of files in lost+found on the ext setup. if you're using an insane amount of small files, it's possible the size difference matters.

my guess though? you're missing something. do a direct du -h comparison of a folder that you know is on the old drive vs the new drive. see if there is a file size difference and if there is, make sure the files match exactly.
 
Tiporaro said:
Edit: just realized that you said 1.92tb vs 1tb used - that definitely is a big discrepancy. Did you actually check the files to make sure they're intact and all functioning properly?
Click to expand...

Are you missing the reserve space that ext filesystems allocate?


Edit: I just woke up... Even a 5% reserve would not be enough..
 
Hm, just came back to this, but one other thought - decently compressible files and some sort of compression feature turned on in the new fs that wasn't there before?
 
By default neither xfs nor ext4 support compression. You can do this with fuse modules on top of the filesystem though I doubt that is the case in this thread..
 
drescherjm said:
Are you missing the reserve space that ext filesystems allocate?


Edit: I just woke up... Even a 5% reserve would not be enough..
Click to expand...

You were right before you corrected yourself.

The 5% in reserved blocks is relative to the size of the file system not the amount of space used on the file system. 5% of 16TB is roughly 800GB. 0.92GB of reserved blocks would equate to a file system about 18.4TB in size.
 
thedge said:
What kind of files would be a big factor. All large media files or all small/very small files?
Click to expand...

Its a fair mix between the 2. About 75 mkv movies, 180 or so gigs worth of mp3's and the rest is smaller misc files.

gjs278 said:
it's possible you had an absolute fuckton of files in lost+found on the ext setup. if you're using an insane amount of small files, it's possible the size difference matters.

my guess though? you're missing something. do a direct du -h comparison of a folder that you know is on the old drive vs the new drive. see if there is a file size difference and if there is, make sure the files match exactly.
Click to expand...

Its too late for that. The files are back on the server. Watching the second movie and a few hours of mp3's and so far everything is ok.

My guess is the lost & found was loaded up. How or why, I have no idea.

Thanks for the ideas!
 
AFAIK ext* is a pretty ancient/odd file system and it still needs to write the data structure for the entire drive to disk in fixed locations when the disk is formatted. Needless to say a table of pointers to files that don't exist is going to waste a lot of space on a big file system...I'm not sure exactly how much, but it's got to be significant on huge volumes.
 
Like Volkin said, it sounds suspiciously like the 5% default reserved space that ext4 automatically allocates. Don't know if XFS has something similar, but maybe it doesn't so that would explain why it seems you're not using less space on XFS.
 
You must log in or register to reply here.
Back
Top