Random HDD Questions/File Size Inaccurate?

Discussion in 'SSDs & Data Storage' started by FenFox, Aug 11, 2018.

  1. FenFox

    FenFox Limp Gawd

    Messages:
    163
    Joined:
    Dec 20, 2016
    Ok, I've got 4 questions.

    1.) Can filling up a HDD to 100% capacity cause performance issues or shorten the lifespan of the hard drive? If so, what's the cutoff point before I'll notice performance degradation/shorten lifespan? Keeping in mind, I'm not running Windows/programs from this HDD, It's just used for storage/accessing media. Also, does the same rule apply to RAID?

    2.) I notice a few HDDs in my PC are only fastened down with 2 screws instead of 4 like the others in my computer. Due to the drive bay spacing of the screw holes, there's no way to use 4 screws. Since they're only held down by 2 screws I can easily lift up one end of the HDD with a single finger. I'm wondering, since they're not completely fastened with 4 screws, can this result is vibration issues over time?

    3.) I sort've asked this once before, but I ran some additional tests. I notice when I transfer files from one HDD to another, there can often be differences in bytes. Sometimes It's fairly dramatic and other times it's subtle. And yes, even with hidden files/folders selected and every other applicable folder option I can think of with just a single file in a folder, there can still be a difference. Even if I use TeraCopy to hash check via MD5 during and after the transfer, TeraCopy says the transferred file is 100% correct, if so, what would explain the reduced file size? I think someone said this may be a Windows bug.

    4.) If I right click a file > Properties, Windows says GB, but do they really mean GiB? Because I notice when I download files, websites will often say GiB, but when I check the file in Windows It'll say GB, but I think Windows really calculates in GiB, so why not list it as GiB? There's a difference.

    1 Gigabyte = 1000 Megabytes
    1 Gibibyte = 1073.74 Megabytes

    Also, kind've annoying, but if say--as an example--I download a file from a site that says: 26.68 GiB, then I right click > Properties in Windows, it'll just say 26.6x (missing the 8) which leaves me to calculate the bytes. WHY can't Windows display the full file size in GB/GiB? Is there a third party piece of software I can use that will make this calculation? Because I backup A LOT of data and It's very important for me to know if my transfers/backups are accurate. I need 100% identical files, but Windows 10 doesn't exactly instill confidence.


    TeraCopy & Missing 8.png
     
  2. SvenBent

    SvenBent 2[H]4U

    Messages:
    2,252
    Joined:
    Sep 13, 2008
    1: no not for the harddrive.
    But if you system expect free space and there is not funny thing can happen.

    2: na.h
    but for sounds issues 4 screws would be better

    3: You are doing something wrong then. or looking at the wrong numbers size on disk and filesize are not the same
    Filesize is how big the files are ( and on your picture its the same
    Size on disk is how much diskpace is reserved for containing the file

    You reserve diskspace in lumps at at ime called clusters. if the cluster size is different. files will take up different amount of diskspac.e however they contain the same amount of date. the waste is diffrent
    See its this way/ you have 2 ppl and buy them each a house. if you buy them both a huge house they take up more space. but ist still the same amount of ppl. just a lot of empty rooms)


    Also GiB i believe is not a standard. is just something ppl came up with to denote he difference.

    4: Because it only shows 3 digits. so once you go above 10 of something it will only show 1 number after the comma/dot
     
  3. AlienTech

    AlienTech Limp Gawd

    Messages:
    286
    Joined:
    Sep 30, 2004
    NTFS will write data inside its own cluster if its small enough to fit so you dont waste space so that migtht be the descrepency. If I create a cluster with like 64 or 128K then files amller than those wont take up extra space on the drive with the file name and attributes etc..

    And after 80% the drive does slow down and at over 90% it slows down drastically. I think this is because of the operating system trying to find best free space area since copy programs like terracopy etc wont slow down like that. This is not so bad with SSD's but it happens there too. Trying to fill 100% using just explorer etc is very slow after like 95%.
     
  4. Tiberian

    Tiberian DILLIGAFuck

    Messages:
    5,704
    Joined:
    Feb 12, 2012
    1) Not for the hard drive, no, but for the OS in use, yes. The general rule of thumb is never fill a physical hard drive more than roughly 85% full with content because the file systems in use today (all of them) do require some "breathing room" so to speak to manage files and file allocation tables/journal tables/etc. When you fill a hard drive past that 85% full point the drive controller sees things as "starved" to some degrees and the drive itself - the physical aspects meaning the spinning platters and the moving heads - spend more time shifting around than actually finding and reading/writing data because it's "starved" for someplace to store data.

    There was time in the past where, with OSX machines, if you filled a hard drive more than 85% full the older versions of HPFS (not HPFS+) would actually become corrupt and people would lose a whole bunch of data in the process but then Apple spent time resolving it and HPFS+ was created to address those issues and others. Regardless, the advice still stands: if you're using physical hard drives, do your best to never fill them more than 85% full unless you have absolutely no other choice.

    Having said that, if the hard drive is going to be used for nothing more than pure raw storage, meaning the file(s) are written to it one time and then they'll stay that way and only be accessed in read operations, I would say it's OK to fill a bit more than 85%, maybe even to about 92-95% if you absolutely must do it but realize if you need to write anything more than that the drive is going to suffer performance wise in the write operations. Obviously I get that some people have large drive arrays with tons of movie files - 'cause there is NOTHING ELSE out there that requires that kind of raw storage space but video files - and all they do is write them once and read 'em as required.

    The point is: if you're using the drive regularly as an operational storage device, reading and writing to and from it constantly, more than 85% filled will make it suffer performance losses, sometimes dramatic ones where your system gets really lagged as the drive is desperately trying to cope with the lack of available free space. If that's the usage, then no, don't cross that 85% point. If it's just raw pure storage like for movie files, etc, and it's a one-time write operation for the future, sure, go for 90-95% but be warned there could be some negatives to that, I'd say no more than 92% ever as my personal max.

    Say you have a hard drive with 85% capacity filled, that means if you want to write new data to the remaining 15% of the drive it's going to have to spend a great deal of time for the head assembly to track all the way to the inside of the drive (the area next to the spindle) which is the slowest part of a physical hard drive, and the more time the drive spends in that kind of state, the more performance suffers. It's just one particular reason for the decreased performance but it's the biggest one of all.

    2) 2 screws, 3 screws, 4 screws, in my own experience it's not relevant at all. The only time I've worried about just 2 screws on one side was using very old cases long ago that for whatever reason didn't even allow me to gain access to the far side of the drive's mounting housing to screw in the other 2 screws. It was baffling why a case would be designed that way - and no it didn't have removable side panels and it wasn't possible to gain access to that side of the drive mount unless I removed the entire chassis from inside the case itself and I wasn't interested in doing that at all.

    Biggest reason for 4 screws complete? Vibration control more than anything else since just fastening 2 screws on one side could lead to less vibration dampening overall.

    3) The file size reduction thing, not sure about that but I'm suspecting it's related to file size allocation more than anything else. What that means is the file system the file existed on could be something with 4K clusters (standard NTFS cluster size) and if you're copying it to say a USB stick that's formatted FAT32 but over 8GB in size the cluster size will be 8K - if it's 8GB or smaller the traditional 4K cluster size will be in effect. If you have a small text file, say it's got 1024 characters of text in it, that's 1KB in technical size but to store it on the platters of a hard drive (based on the cluster size) using NTFS that means it'll take up 4KB of space, which means literally 3KB of wasted space commonly referred to as 'slack' space. If you had a 1TB hard drive with 100 million such text files on it they'd technically be 102,400,000,000 aka about 103GB size but because of the slack space they'd consume ~410GB.

    It's weird, yes, but when you say a file is one size on one drive and another size on another drive my first thought it the two file systems could be using different cluster sizes if the files are actually identical, even by using an MD5 checksum to verify it.

    4) I'm old school and I use Megabits and bytes (Mb/MB), Gigabits and bytes (Gb/GB), Terabits and bytes (Tb/TB), and so on and I will continue to do so regardless of whatever newfangled terminology the modern tech society deems worthy of their consideration and use. I find it ridiculous to keep inventing new names for the same old thing and yes I understand the intention with the new nomenclature, I just don't give a fuck about it personally. :)

    As for the final thing, you're worrying way way too much. If modern file systems didn't work, none of this - meaning even this text appearing on my screen to be posted in this thread for someone else to read - wouldn't be possible. NTFS has checksum verification built into it to various degrees and if there was an issue copying/moving files you'd know it. Teracopy is useful for the checksum verification but in the long run it's not nearly as fast as they'd like people to believe, and I had it crash so often over the years I gave up using it since it's still using the Windows copy/move routines that already get the same damned job done. I use a tool for creating checksums (SHA1) and then creating a checksum file (*.sha1) that I can then do a verify on if I really seriously believe there could be any issues.

    The stuff you're doing is working fine, really. Teracopy already proved to that with a checksum verification, so you're good. I know you've said this is a learning process but it's working as expected from all the info you've presented. Windows isn't "missing" any bytes just because it's not reporting that "8" for you as a 4th digit, it's just not relevant to the big picture. The actual difference between 26.6GB and 26.68GB might be 80MB technically but really, it's 80MB, come on. :D
     
    Last edited: Aug 16, 2018