Putting 2GB of data on hundreds of usb thumb drives

W

WhatTheSchmidt

Limp Gawd
Joined
Jun 4, 2007
Messages
324
I'm wondering what the fastest way of doing this would be. I'm assuming this will quickly overload the southbridge(this is what controls the usb right?) and slow down at doing a few transfers at once.

I have 12 total USB ports on my computer, 6 on the front. Right now I'm uploading to 6 on the front and 2 on the back(on the 3.0 not that these usb drives are 3.0 capable though) since mouse and keyboard are on back as well.

Is there some trick way of telling the computer to send the same data to all of the usb thumb drives at once? a program?

Or is doing it this way the best I got? takes about 3 minutes to do 8 drives at once
 
well it takes 4.5 minutes whether I do 1 drive or 10 at once

so I guess I can't go faster than this, I do wonder if there is a better setup (besides expansion cards to have a lot more usb ports
 
Some manufacturers (or resellers for example http://www.ideareplication.com/flash_drive_duplication.shtml) when there are hundreds or thousands of thumb drives involved will package, label/print and duplicate them for you. I am expecting that this is a commercial venture of some kind, so you might want to think of going that route. Another option is purchasing a mass USB duplicator, kind of like one of those CD duplicator towers except for USB devices. For example, http://www.premiumusb.com/product/accutower-flash-vx-standalone-usb-duplicator_401_23.htm I don't know how many/how often you will need to do this, but it can get expensive.
 
yea I've seen the cd/dvd duplication before

One of those mass usb duplicators won't be necessary, I was mainly just wondering if there was software to speed it up but using all the usb ports didn't slow anything down anyway. So 10 every 4 minutes isn't that bad.
 
Can't you use some USB hubs? Should probably able to support more than 10 at full speed.
 
s0rce said:
Can't you use some USB hubs? Should probably able to support more than 10 at full speed.
Click to expand...

we actually got a couple and it made it go a LOT slower

like 2 hours for 7 on a hub vs 5 min each (this was on a laptop)

my computer can do 11-12 in 4 minutes
 
Are you using USB 1.1 or 2.0?

Remember, 2.0 caps at 60MB/s total.
This is a bus, meaning the bandwidth is split between the devices.

So if you have 100+ flash drives, divide that number by 60 and you will have your data transfer rate per flash drive.

If this is 1.1, I'm LOLing at you. :D
 
If you have access to a machine running Linux you can easily setup a script that will image the drives when they are plugged in. If the drives have an LED that would be an easy way to know when they are done writing.
 
Make sure your hubs are Multi-TT hubs - that was probably your problem with the slowdown.
 
I've seen some hubs do well on this kind of thing, and others that choked horribly. That said, an auto start script like the one mentioned above would probably be your biggest speed gain at this point.
 
ChrisBenn said:
Make sure your hubs are Multi-TT hubs - that was probably your problem with the slowdown.
Click to expand...

+1

This would definitely help in your situation.
Do a google search on Multi-TT hubs, there is definitely a difference in performance.
 
s0rce said:
Can't you use some USB hubs?
Click to expand...
This is a good idea. I connected 4 flash drives (1 actual flash drive and 3 SD readers) to a single cheap ($2) 4-port hub. I started 4 simultaneous read transfer tests; the combined throughput was 42 MB/s (real megabytes). Individually, the devices did 16.2, 16.2, 16.8 & 14.1 MB/s. Simulataneously, they did 12.2, 10.1, 9.9 & 9.9 MB/s (total=42+).
Should probably able to support more than 10 at full speed.
Click to expand...
full speed ?? (In the context of USB,) I think we are talking about high speed:).

Further improvement by writing RAW/cloned images (probably best done under *nix). Especially good if you are essentially using the full 2GB capacity. This way you bypass the entire filesystem layer and all file/folder creation.

--UhClem
 
Well, a USB2 ports maximum throughput is 480mbits a second, so 42MB a second is actually very good. I'm not sure I've ever seen a USB2 port actually puke that much data.
 
dustNbone said:
Well, a USB2 ports maximum throughput is 480mbits a second, so 42MB a second is actually very good. I'm not sure I've ever seen a USB2 port actually puke that much data.
Click to expand...

You will never get that from a single device though. About 30MB/s is the max you will ever see from a single device. Not to mention those flash drives will probably only be capable of writing at about 8MB/s at best, depending on the flash drive brand and model.
 
dustNbone said:
Well, a USB2 ports maximum throughput is 480mbits a second, so 42MB a second is actually very good. I'm not sure I've ever seen a USB2 port actually puke that much data.
Click to expand...
Actually, it scarfed the data (a read test) [first you scarf, then you puke:)).

Yeah, I was pleased (and surprised) to see the 42 M/s total; that is why I specified that they were real MBs (2^20). And that was on an old NForce4 mobo running XP64; tests were done under CygWin, and reliable.

--UhClem
 
Red Falcon said:
You will never get that from a single device though. About 30MB/s is the max you will ever see from a single device.
Click to expand...

Speak for yourself; maybe you just have blah devices (or a very broad definition of "About"). I have a flash drive that gets 33.6 MB/s (Patriot XT Boost***) and an Antec MX-1 enclosure that gets 37+ (again, real MBs).

By the way, the whole point of the thread was multiple devices, wasn't it? Also, the technically perceptive can now surmise that the USB controllers in most devices (including USB2<=>SATA enclosures) are sub-optimal.

Not to mention those flash drives will probably only be capable of writing at about 8MB/s at best, depending on the flash drive brand and model.
Click to expand...
So what! The point is that OP can quadruple his throughput.

(And, there are flash drives (USB2) that do get 20+ MB/s write rate.)

*** This is a 8 month-old sample; more recent samples get 27-28.

--UhClem
 
UhClem said:
Speak for yourself; maybe you just have blah devices (or a very broad definition of "About"). I have a flash drive that gets 33.6 MB/s (Patriot XT Boost***) and an Antec MX-1 enclosure that gets 37+ (again, real MBs).

By the way, the whole point of the thread was multiple devices, wasn't it? Also, the technically perceptive can now surmise that the USB controllers in most devices (including USB2<=>SATA enclosures) are sub-optimal.


So what! The point is that OP can quadruple his throughput.

(And, there are flash drives (USB2) that do get 20+ MB/s write rate.)

*** This is a 8 month-old sample; more recent samples get 27-28.

--UhClem
Click to expand...

:rolleyes:
Nuff said.
 
Quick question though. Is the main issue you have

a) I don't want to spend a lot of time doing it.
b) I need it done fast.

Put it this way. My current FPGA build takes ~1 hour of machine time but it take me 30 seconds to get it going. Now, I could get a much faster machine and cut the time to 30 minutes..but I can definately fill up the hour doing other tasks.
 
You must log in or register to reply here.
Back
Top