external hard drive over Gb - bottleneck

[GLSauron]

In *real world* usage, with an older HDD, whats going to be the bottleneck?
HDD transfer rates: figure 30MB/s
USB2.0 transfer rates: 480MB/s
or Gb network connection: 1000Mb/s = 128MB/s (right?)

When acting as a server (2-3 clients each on 100Mb? Gb?)

So itd take 2x100Mb connections would saturate the drive and 1 Gb? theoretically, at least. How about in the real world?

[GLSauron]

anybodies?

[UltimaParadox]

Its hard to say I really have not seen much testing, but in real world performance I think its obvious that USB 2.0 is not capable of delivering the performance you are looking for..

From what I have seen USB 2.0 has connection rates upwards to actual 20MB/s which is even slower than your typical internal hard drive. A lot of reviews show 11MB/s which is suppose ro be the therotical for USB 1.1 as the actual transfer rates for USB 2.0.

That being said over a 10/100 connection its more around 10 than 100 in actual throughput, but I figure more than one person hammering the external hard drive, would pretty much be overkill for your standard one.

[Bullitt]

USB 2.0 is going to be your bottleneck.

Case and point, External enclosures for DVD burners. USB 2.0 simply cannot sustain the data needed to prevent buffer underruns @ 16x.

[Shmuckety]

To me it is unclear what you are asking here. How many ethernet connections would it take to make full use of the drive or USB vs ethernet?

http://www.hardfolding.com/ftag1.php/mem/8510.png (http://www.hardfolding.com?go=38&id=8510)

[GLSauron]

based on that 20MB/s HDD speed, whats going to be the bottleneck? HDD, USB, or ethernet?

[UltimaParadox]

based on that 20MB/s HDD speed, whats going to be the bottleneck? HDD, USB, or ethernet?

I would have to say USB at this point, because I am pretty sure most people actually only see throughputs of 11MB/s, as long as we are comparing it to GigE

[r00k]

HDD transfer rates: figure 30MB/s
USB2.0 transfer rates: 480MB/s


USB 2.0 = 480Mbps, not MB/s

e.g. 480 / 8 = 60 MB/s

[Biest]

USB 2.0 is your bottleneck here, since (from my experience) the transfer rate fluctuates greatly esp. when you are transfering large amount of data (190 gig took me 3 days straight).

that is why i use Firewire 800 for my external hdd.

[enginurd]

0.75MB/s for USB2.0? Either its late and my math is wrong, or that is one slow enclosure.

In my experience, it takes me about 2 hours to transfer the entire contents of my external 200GB ATA100 HDD... roughly 25MB/s. My internal 80GB ATA100 gets about 36.5MB/s, on average. So in my case, the USB is the bottleneck.

Based on 20MB/s HDD and my USB2.0 transfer rate, the HDD would be the bottleneck, lol.

[Madwand]

based on that 20MB/s HDD speed, whats going to be the bottleneck? HDD, USB, or ethernet?

I think reality generally isn't this simple -- that for example if you have a component that measures 20 MB/s in one interface and another interface that measures 30 MB/s with another component, you can't just say that the 20 MB/s component is "the bottleneck" and you'd get 20 MB/s transfer rates. I think that in reality, the complexities of the protocol are going to be such that both affect and reduce the effective transfer rate, to a degree dependent on how close one's rate is to the other's -- i.e. that you'd get less than 20 MB/s when switching to a lower-speed interface, though that interface is faster than 20 MB/s.

To answer this question in practice, I'd simply measure the HD again using the USB2 interface instead of a direct connection -- now you have a composite figure made using the correct interface. It could be 19.5 MB/s, it could be 15 MB/s or lower; I couldn't just guess; the implementation details matter.

The mult-user use case also introduces additional variables -- even when the HD can sustain 20 MB/s for STR, it can't do that for 2 independent request -- each is likely to be less than 10 MB/s. The proportion of loss will depend on a lot of details; details of the data access, and the implementation. If 2 requests request the same data, then, to a degree limited by size and time, the 2nd request could get a very nice benefit from the cache.

Regarding the ethernet; at these speeds, gigabit implementation details are not likely to have a negative impact. 100 Mb/s links are usually going to have an impact, and sometimes you won't even get 100 Mb/s (i.e. 10 MB/s typical), because the HD's performance is saturated by multiple concurrent requests.

Have I helped?

[drizzt81]

USB2.0 transfer rates: 480MB/s


USB2 = 480Mbit/s and that is only valid for isochronous transfers, which do not guarantee delivery or data integrity, i.e. are not used for mass storage devices, but rather for webcams and the like. Real-world USB 2.0 xfer rates for a mass-storage device is ~20-25 MB/s

[drizzt81]

0.75MB/s for USB2.0? Either its late and my math is wrong, or that is one slow enclosure.

In my experience, it takes me about 2 hours to transfer the entire contents of my external 200GB ATA100 HDD... roughly 25MB/s. My internal 80GB ATA100 gets about 36.5MB/s, on average. So in my case, the USB is the bottleneck.

Based on 20MB/s HDD and my USB2.0 transfer rate, the HDD would be the bottleneck, lol.
iirc Beast had/ has some awkward problem with his USB2 enclosure which may be related to how ntbackup writes/ compresses data.I don't think that his results are representative of USB performance. In fact, I build a USB device that transferred data at a higher rate, based on a Cypress chipset.

[Mega2]

esata 150MB~ to 300MB~ >usb2.0 20MB~ to 25MB~

lan 100mb = 10MB~
lan 1gb= 100MB~

[GLSauron]

/me looks into esata enclosures... (which is just a sata drive in an external enclosure connected over some "esata" connection (which sounds like just a normal sata, signalling wise). Would be nice to be able to bring *most* (if not all) of my files with me on the go.

[MrGuvernment]

....(190 gig took me 3 days straight).
.

something wrong with your setup then.

[paradoxblue]

something wrong with your setup then.

i agree. wow. ive done usb in larger volumes then 200. way less speed then that

take into account it takes me 6 mins ot burn a 4.7 gig dvd over usb 2.0 ( im not 100% sur eon exact numbers heh.

i dun see 190 taking u 3 days.. on anyrate unless your hard drive hadnt been defraged since winblows was invented :|