Need faster data transfer speeds

SultanGris

Limp Gawd
Joined
May 14, 2006
Messages
330
Ok, not sure what the deal is but i have multiple thumb drives and a network drive and a couple usb terabyte drives and i get just shit transfer speeds usually. Doesnt matter what drives i use to or from. have ssd in the pc. I have an older pc running windows 7, only usb 2 technology, but ill get up to 35mbps a sec transfers sometimes for a couple minutes but if transferring large amounts of data that speed will always drop to sub 1 meg and or often freeze entirely and fail if trying to transfer over a few gigs of data at a time, why does this happen and what can i do to fix it, and how can i get decent transfer times all the time? USB 2 claims 60mbps transfer rates and ive never seen anything even remotely close to that. thanks.
 
60mbps is not the same as 60MB/s
60mbps is 7.5MB/s

large files usually transfer much faster than a bunch of small files, due to Windows verifying every file after it transfers.

this is what I was getting from a my WD My Book USB2.0
usb2.jpg
 
Last edited:
ok, i was confused about the wording of the rates i guess, so when it says 35MB/s its just straight up fucking lying to me then? lol, its running around 1MB/s give or take .3 after it levels out, about 1/7th of advertised speeds. any way to improve? Best to transfer smaller file amounts in multiple times then? Im trying to tranfer 4 terrabytes to a new drive, gonna take 3 years at this rate, ha!
 
I use TeraCopy, as it has less overhead when copying.
 
trying that teracopy, its giving me slower speeds than windows was previously, but not by much. its also jumping all over the place from 145KB to 1.1MB
 
also gave that crystaldisk a try, my usb drive is about the same as your speeds in the screenshot, about 1MB faster on both up and down, lol, but my network drive doesnt even show up so i cant test it.
 
also gave that crystaldisk a try, my usb drive is about the same as your speeds in the screenshot, about 1MB faster on both up and down, lol, but my network drive doesnt even show up so i cant test it.

You have to map the drive first. Also if these USB flash drives are connected to a router it's going to be slow regardless of the drive being used.
 
The faster speeds in the beginning is due to caching. The older usb2 thumb drives pretty much suck. The new usb3 ones are much better. Also if they are lots of little files, old thumb drives are bad at writing those fast too.

Also make sure you're only doing one transfer at a time, those thumb drives can't handle much and multiple threads would bring it to a crawl.

If you get horrible speeds between your system ssd and the usb terabyte disk drives then there's a separate issue at play.
 
Last edited:
You have to map the drive first. Also if these USB flash drives are connected to a router it's going to be slow regardless of the drive being used.
Not sure what you mean by map the drive. windows file explorer can see it, but cmd.exe cannot.

my main concern right now is from a 1 terabyte network drive to a 4 terabyte usb drive, usb drives arent through a router or hub, just plugged into the motherboard usb ports, or the extra case usb ports(seems to make no difference for speeds which port it used), but the network drive is obviously through a router. My various usb thumb drives range from 4 gig to 128 gig and all vary in speeds but none are anywhere close to the 7.5MB/s that usb2 claims to be capable to handle.
 
The faster speeds in the beginning is due to caching. The older usb2 thumb drives pretty much suck. The new usb3 ones are much better. Also if they are lots of little files, old thumb drives are bad at writing those fast too.

Also make sure you're only doing one transfer at a time, those thumb drives can't handle much and multiple threads would bring it to a crawl.

If you get horrible speeds between your system ssd and the usb terabyte disk drives then there's a separate issue at play.
yes ive noticed that multiple transfers at the same time kills performance exponentially. seems like more total gigabyte transfer sizes fail far more often than smaller transfer sizes also, if that makes sense.
 
The hardware is generally fine we need a better file system that can handle the trend of tens of thousands of small microfiles.
 
well shes been working on a 39Gig chunk of files for about 20 hours now, 107 files most are 233meg or so, a few smaller files than that and its only 32.1 percent completed. gotta be a better faster way than this, its dropped to around 650KB/s max and dips down to mid 100's and bounces around. infuriatingly slow. If i take my network and usb drives apart and physically put them in the machine would it be faster you think? they are both spinning drives im sure, not ssds unfortunately.
 
Last edited:
What is the type of data? 2.0 only supports BOT and not UASP so suffers tremendously when transferring large amounts of small files. Typically it will only max out with large contiguous files. Painintheworlds suggestion is the only workaround (archiving to a single file then copying to the drive).
 
What is the type of data? 2.0 only supports BOT and not UASP so suffers tremendously when transferring large amounts of small files. Typically it will only max out with large contiguous files. Painintheworlds suggestion is the only workaround (archiving to a single file then copying to the drive).
video and pictures from a drone mostly. Ill definitely try zipping a bunch and see how it works, this batch is at 50 percent and im not stopping it now, lol
 
xcopy or zip a large chunk of files before transferring.
well it zips about the same slow ass rate it transfers at, so even if it transfers faster im not going to gain anything, actually unless it transfers instantly its gonna take longer to zip them all and then transfer and unzip, lol
 
The file sizes you are transfering are plenty large enough not to be an issue, zipping won't really help the transfer much in this case. In the future though when you zip, the best speed is using the storage method and do it on a SSD or at least set the destination path to the SSD or a different HDD than what the files are on. Storage method is best for media files since the majority can't be compressed through zip anyway. Data files are better used with the compression methods for shrinkage. Storage method zip is limited by disk speed, compressed is more CPU.

For better help post the all the system hardware involved. Is it still just as slow transferring to a HDD as the thumb drive? What kind of drive are all the files on? Size of drive and how full is the drive? Also run Crystaldiskinfo on the drives involved of these transfers, are there any errors? What kind of AV software is running?

Without further info and just judging off this, it sounds like one of the drives involved could be damaged or the thumb drive is dying.
 
Last edited:
The file sizes you are transfering are plenty large enough not to be an issue, zipping won't really help the transfer much in this case. In the future though when you zip, the best speed is using the storage method and do it on a SSD or at least set the destination path to the SSD or a different HDD than what the files are on. Storage method is best for media files since the majority can't be compressed through zip anyway. Data files are better used with the compression methods for shrinkage. Storage method zip is limited by disk speed, compressed is more CPU.

For better help post the all the system hardware involved. Is it still just as slow transferring to a HDD as the thumb drive? What kind of drive are all the files on? Size of drive and how full is the drive? Also run Crystaldiskinfo on the drives involved of these transfers, are there any errors? What kind of AV software is running?

Without further info and just judging off this, it sounds like one of the drives involved could be damaged or the thumb drive is dying.

right now im going from a 1 terabyte network drive to a 4 terabyte usb drive. crystal disk cannot see the network drive but the usb drive tested about the same as the guys screenshot above for the first set, sequential, then i canceled the test cause it took forever and i was transfering files at the same time so it may not have been completely accurate, and i dont know what that other stuff is anyway. thumb drives vary, but i thought they were slow as well, however i was confused on the mbps and MB/s so it might not be as slow as i thought, usb 2 claims 60mbps which i mistakenly thought was 60MB/s, havent used one recently so I forget exactly what speeds i get with them.

Av is avast, computer is a asus p6t board with an intel 920 i7 chip, 16 gig ddr3 ram, 500gig ssd samsung, radeon 6970 card
 
crystaldiskinfo is not the same as the post above, it's for checking drive health.

If drives are full and heavily fragmented that will cause significant slow downs.

But usb2.0 in general averages 30-40MB/s at it's best even going from SSD to SSD without any caching utilities involved.

Some features of AV software can cause performance issues as well.
 
crystaldiskinfo is not the same as the post above, it's for checking drive health.

If drives are full and heavily fragmented that will cause significant slow downs.

But usb2.0 in general averages 30-40MB/s at it's best even going from SSD to SSD without any caching utilities involved.

Some features of AV software can cause performance issues as well.
well its pretty full, only 80 gig free out of a terabyte, but ive never really removed anything from it so it shouldnt be too fragmented, havent seen a defrag utility in about 6 years, didnt know you still had to even worry about that, thought windows handled it these days.

I just tried some tests and my 4terabyte drive was running about 29 MB/s from internal ssd, a 64gig thumb drive held 13MB/s so i think the network drive is the weakest link, I might disassemble it and physically put it in my machine to speed things up if i get feeling ambitious later, unless you have any better ideas at sub 1 MB/s its gonna take forever to get my terabyte of data transferred, lol
 
Large contiguous aka single-chunk files like video files, large picture files (think RAW images 25MB or larger), and other such "solid" files transfer faster and are able to be written faster in terms of throughput on most any storage medium - small non-contiguous stuff, small pictures, small data files, etc, under 4KB in size but up to make 1MB as well - cause the transfer throughput to drop through the floor and bottom out because devices have so much trouble with that kind of small file writing even on the best SSDs with NVMe interfaces.

Big files = fast(er) transfers
Small files = slow(er) transfers

regardless of the medium, this is how storage works, there's nothing anyone can really do about it so, you just have to deal with it. If you're shifting a lot of small stuff around it's going to be a much slower process than transferring a few big huge files.

If you have a shitload of small files to transfer to such a device in terms of pure storage and archiving/backup purposes, Zip 'em up or even TAR them as one solid archive and get 'er done.
 
She didn't wanna cooperate but she's gonna be an internal drive from now on.
 

Attachments

  • 20180605_140927.jpg
    20180605_140927.jpg
    402.2 KB · Views: 0
poop on a stick, that didnt work, device manager see it but windows explorer cant see it. Is there any way to get it to show up other than formatting and partitioning it? Just cant win, I hate to have to put it back together and wait 3 years to transfer everything off at 1MB/s. grr!
 
well i put it back together and im getting 10mb/s now so maybe something was loose, still pretty slow and crappy, but ten time better than it was before!
 
Ok so how is this possible, different usb 2.0 1 terabyte external drive. Lol
 

Attachments

  • 20180617_163444.jpg
    20180617_163444.jpg
    931 KB · Views: 0
passports have been usb3 for a while. you sure it isn't plugged into a usb3 port?
 
same port my newer 4 terabyte drive was plugged into earlier getting only 20-30 I think it was, forget exactly. I thought my board only had usb 2, its an older asus p6t.
 
A couple of things--first your usb devices could be slowing down due to internal heat problems. It depends on what type of memory chips or drives are in the enclosure and how much heat builds up inside the enclosure as large amounts of data are transferred to them. The same applies to the ssds you're reading the source material from, but to a lesser extent depending on the type of ssd (enclosed or m.2) and how its mounted, especially m.2. I have one that's mounted on the underside of my ASUS H97i motherboard in the limited space between the motherboard and case bottom. I can use AIDA64 to monitor all my internal drive temperatures and see how much higher the unventilated m.2 temperature is compared to my standard ssd and a 4tb hdd. Unfortunately I don't know of any utility to monitor the temperature inside a usb drive enclosure and I don't think any manufacturers address heat problems since they sort of assume that you're only transferring small amounts of data.

Second if you want to live dangerously, you could try editing your registry and set a higher maximum transfer length (don't try this if you're not familiar with registry editing). Find the registry key

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\usbstor\054C00C1

Edit/Add a New DWORD (32-bit) Value named MaximumTransferLength and enter the maximum value of 2097120 as a decimal value. That allows transfers in 2mb blocks.

That may speed up your transfer speed depending on what type of chips you have on your motherboard controlling usb transfer speed. Your results may vary. No implied guarantees or warrantees. Void where prohibited.
 
its getting pulled off an older internal sata spinner in this case, not ssd, but those were larger files in the previous pic doing 90MB/s, shes slowed to 30MB/s same drives doing smaller files in a new transfer now.

I got my network drive files all transferred and ive pulled it apart and made it an internal drive now, just cleaning up some other smaller drives and consolidating and sorting all my data, got lots of duplicate files due to so many smaller older drives.
 
actually its slowed to 20MB/s now, but it is a little warm, been transferring for a couple hours.

Edit: I pulled it out of the case and after a minute it jumped back to 90 after i posted this and now its jumping around from 20-90, so it must be related to file size, cant imagine it cooled enough to make a difference in that short amount of time, its not excessively hot, but warm to the touch.
 
Smaller files means a higher rate of movement of the actuator arm mechanism that moves the read/write heads in and out over the disk surface, so heat buildup with such higher activity rates can be a problem. And just because the outside of the drive case isn't that hot doesn't mean a an individual component inside the case isn't hotter. That's one reason why there are different types of hdds with some specifically designed for high read/write activity scenarios. Most consumer drives generally assume a lighter level of usage.
 
Smaller files means a higher rate of movement of the actuator arm mechanism that moves the read/write heads in and out over the disk surface, so heat buildup with such higher activity rates can be a problem. And just because the outside of the drive case isn't that hot doesn't mean a an individual component inside the case isn't hotter. That's one reason why there are different types of hdds with some specifically designed for high read/write activity scenarios. Most consumer drives generally assume a lighter level of usage.
Right on, thanks for info. Makes sense. I checked it again and its cooled down to the point its not even warm, I set it on top of my PC case which has a huge exhaust fan on top. The drive was in a zippered cause before, sitting in the same spot but it was warm when i removed it from that case.
 
Back
Top