Defragging a mechanical HDD - How much free space do you need?

F

FenFox

Limp Gawd
Joined
Dec 20, 2016
Messages
296
1.) So, some people, such as the person in this article:

https://www.howtogeek.com/324956/how-much-free-space-should-you-leave-on-your-windows-pc/

Are suggesting 15% free space for a mechanical hard drive defrag.

However, I believe I've heard others say this practice has been irrelevant since Windows Visa, and depending on who I talk to, a little over 200 MB to a few GB is enough to defrag a mechanical hard drive. My OS isn't installed on this HDD, just self-created media files ranging from 4 GB to 80 GB. It's a high-use 10 TB HDD.

So 15% of that would be 1.5 terabytes. That's an insane amount of storage space I'd be giving up if this were true.

But some are saying if It's not 15-20% free, the HDD will work a lot harder, be slower and the risk of failure will go up.

And I guess Microsoft's own documentation says 15%
https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/defrag

But still, giving up 1.5 terabytes seems kinda insane. So what should I do?

2.) If I recall correctly, MS automatically defrags internal HDDs via the "Optimize Drives" application? My "Scheduled Optimization" is "On" so if I have less than 15% free space, that could cause a problem?
 
OK, the basic gist of it is this: if you've actively using a fucking shitload of small files on that drive, I mean actively like daily writing, reading, erasing, writing new files, reading new files, deleting old ones, constantly thrashing it to high degrees with files that have content that actively changes then sure, a massive defrag operation can improve performance, absolutely, because the small files will fragment like crazy.

If all you've got on the drive are large media files and all you're really doing with it is playback primarily (after it's stored on the drive of course) then defragging may not do much for you unless you've got like 75% fragmentation or some insane amount of it. Audio and video are basically the ONLY such things that require massive amounts of space, perhaps some audio files in 24/96 or even higher bit/sample rates, and of course video sucks up space like a sponge in water, and with 4K becoming a standard and now 8K moving in soon enough.

Playing a video file is simple, and it actually doesn't require a lot of bandwidth so in that respect performance on a fragmented drive suffers the biggest penalty when such large video files are written, not read. During the write process to a heavily fragmented drive it means the heads are twitching all over the platters looking for someplace to store the data as you're feeding it to the drive so that's where you'd have an issue if one exists. Reading files will take a hit but not as much as the write process does so that's a positive I suppose.

This is one reason why having those "media hard drives" at 5400 rpm or whatever are just fine for most intents and purposes for HTPCs and big SANs that do nothing but serve up video (and audio) libraries - it's not a big thing.

I'd say check the drive for fragmentation and if it's not > 50% then don't worry about it unless you really think it needs to be defragged. I mean, the idea is just how fast does a video file need to play? Even 4K content in a typically compressed format (h.264 files with high bitrates or even h.265) don't need a huge amount of bandwidth, anywhere from 4-10 Megabytes per second (remember to divide the bitrate of the video stream by 8 to get the byte count) and ANY hard drive made in the past 30 years can do ~10MB/s sustained reads, that's a cakewalk really.

If you're using one 10TB drive to read and write a lot of video content from a multitude of machines on a network in real-time then defragging may help in some respects there as well so things are a bit more efficient, but if it's just a media server drive with one or two machines accessing it at any given moment I'd say do that fragmentation check and if it's not severe aka over 40% then it's not a concern till it gets that high or higher and even then if it's just audio/video content and nothing else I'd say the hell with it unless it gets into the 75% range for fragmentation. Yes, I'm completely serious with that 75% level too. :p

As for how much free space, some file systems can cause or exhibit corruption if you fill the given partition to more than 85% capacity which literally starves the file system of available space to work with aka "breathing room." I never go past that point, it's never been a big issue for me, but yes with 1.5TB of space in play not being actively used I'd say you can go higher than that since it's a far far larger drive, but you'll have to decide for yourself how much higher.

If you're looking for a ballpark amount of space to consider, look at maybe the largest files on the drive, you say some of them reach up to 80GB in size so, if you have several of those I'd say use something like no less than about 300GB of free space which is basically 1/5th of that 1.5TB figure and gives you a bit more room to work with for storage. Since defragging a hard drive means it's going to work on one single file at a time (even though it can seem like it's doing more than one at one time) then you're going to need space for at least the largest file on the drive if it's got fragmentation, so 300GB gives the defrag process the breathing room it requires too.
 
Goes like this:

-The HDD is mostly reads.
-I only do maybe 5-15 writes per day on it.
-I don't expect to ever delete more than 1/4 of the files.
-Mostly files 4-80 GB.
-Sometimes I store .txt files documenting the specifics of the content for larger projects. So not a huge amount of smaller files overall.
-Pretty much once It's full, It's full (whatever "full" will be).

What's the best way to defrag a HDD these days? I don't think I've bothered with that since XP.

The Defragment and Optimize Drives Desktop app?
Says it has a "Schedule Optimization" Frequency: Weekly. So isn't Windows defragging my drive already?
 
With so little infrequent writes I wouldn't worry that much about fragmentation. The fact that you plan to rarely delete files means fragmentation isn't really going to happen to high degrees at all. Text files are so tiny as to be effectively irrelevant unless we're talking about log files gigabytes in size and even then, again, it's a low bandwidth item meaning it takes almost no effort for a hard drive to read the file into memory.

As for what's the best way to defrag, the funny thing is all Windows defrag applications use the same Windows routines so they only differ in how they prioritize some file types over others based on the Superfetch and prefetch information that Windows gathers based on loading patterns (stuff that loads at boot time should be at the outer edges of the platters aka the fastest part) and so on. But since it's a data storage drive and not a boot volume none of that will matter at all so any defrag app you choose will basically end up doing the same basic thing: plain old simple file defrag.

But again I'll say it: media files, even something 80GB in size (pretty big files, but some Blu-ray content from BDR100 media could be that large, sure, especially UHD content) don't necessarily require defragging at all, it's effectively pointless unless that 80GB file has been broken down at 95% fragmentation which would be fucking crazy severe.

For defragging physical hard drives I still use the very simple but useful Auslogics Disk Defrag - it's free, it's relatively quick to get things done (with its internal priority routines), and it works just fine. There's no reason to pay some crazy fees or prices for something like Diskeeper (which has never really been all that fantastic) or O&O (decent but again it uses the same damned Windows disk management routines) or anything else. Years ago such software was somewhat specialized, sure, but that was 2 decades ago and nowadays something free like the Auslogics tool does all that and it's totally free of any actual monetary cost.

Or use whatever other application you want, they all end up doing the same thing when it's all said and done.

As for Windows defragging a drive, it's designed to do that based on whatever schedule is in place, you can always look at that with the built-in Disk Defrag or go load up Task Scheduler and find that info as well. You'd have to delete and write 1,000 of those 80GB files each day to do any real fragmentation damage - your setup is probably just fine as it is and will be for a long long time to come.

Grab the Auslogics tool, do an analyze pass, see what it says for fragmentation and then go from there. I'd be highly surprised if it's more than 10% based on what you said is on the drive and how often you write to it.
 
Last edited by a moderator:
FenFox said:
The Defragment and Optimize Drives Desktop app?
Says it has a "Schedule Optimization" Frequency: Weekly. So isn't Windows defragging my drive already?
Click to expand...

Yes, Windows will do that for you automatically. Both 7 and 10. When you open Disk Defragmentator (Win 7) or Optimize Drives (Win 10) there should be a % sign next to each drive. If it's running automatically, it should say 0% for each drive (or very low number).
 
Just a note about Windows Server (2016). It doesn't seem to follow Win10 in many areas. So it couldn't be put under "Windows" in every aspect.
I observe (on all machines I run it on) its Defrag & Optimize drives "component"/app runs once on boot and then it doesn't run or can run once in a while (don't remember exactly). Win2016 doesn't seem to run Maintenance tasks (Defrag & Optimize Drives is such task) regularly, only once at boot time.
I created a scheduled task to at least run this task regularly (once bi-weekly) to at least perform the RETRIM on SSDs.
 
From Windows vista onwards it automatically schedules a defrag a hdd, I think it has to be beyond a certain percentage before it will run defrag after automatic analyse I believe

Windows 10 (and possibly 8) is aware what a ssd is and will trim automatically on a maintenance task runs (it won't defrag a ssd as a ssd at NAND level is fragmented intentionally for Speed reasons, defragmenting an SSD will still be fragmented any way)

I would not bother defraging a hdd unless it's not running it on its own (the defrag application says when it was last run)


Even doing the trim defrag is not completely necessary because as long as trim command is enabled in Windows all parts of free space should already be trimmed ( but I guess it could happen but some parts of free space have not been erased)
 
only as much space for your largest file on the drive + 10%. If that file needs to move, you have white space to move it to.
 
I turn off defragging, it's a waste of time. A little fragmentation is harmless.
 
It won't defrag automatically on its own unless it more then 5-10% fragmented (it used to be documented but can't find it) so no need to Turn off defrag
 
As to defraging mechanical HDDs, I should mention that at least with my usage patterns I've never had any of my disks go beyond that threshold of the built-in defrag tool (I think it was 6 or 8%) even after many years. Defrag is not a very trivial and safe operation so I always have been turning off scheduled defrags.
Once in a dozen blue moons I may check manually the state of the disks but....
And when I created my own scheduled task to perform retrim on SSDs, I only specified SSDs as targets for the command. HDDs now have become secondary storage only and used more and more rarely. All current work is done on SSDs. So these things are less and less of an importance to care about.
 
You must log in or register to reply here.
Back
Top