How simultaneous reading and writing affects RAID performance

C

CC_Pirate

Limp Gawd
Joined
Aug 1, 2005
Messages
463
I am wondering how writing to a RAID 0 array of two 500GB drives affects the read speeds. or vice versa.
Take this scenario: Capturing video (SD) to the drives, and simultaneously reading from the drives to a server (over Gigabit LAN). Will I be able to sustain write speeds (most important due to potential dropping of frames) while reading from the drives? What is the worst case scenario, that I get speeds similar to a single drive?

Thanks!
 
If your raid card is decent, you should get about 1/2 to 1/3 the max write throughput versus just a single write operation.

For example, I have a 7 drive raid6 that can perform about 300-350MB/sec read/writes.
If I copy to and from the raid6, I generally get about 120-150MB/sec thoroughput, although the beginning of the copy is a bit faster due to caching effects. Interleaved read/writes to the same logical drive isnt very fast.

If your operations are mostly sequential and deal with larger files, I wouldn't expect more performance degradation than that. It also depends how fast that ethernet connection is going, too, but in my experience they max out around 60-80MB/sec if your setup is well optimized. It should be plenty for HD streaming - but maybe not RAW HD.

From what I understand, SAS drives actually let you issue simultaneous read/write commands (full duplex) and moreover you can use dual porting to increase performance or connection reliability, but I haven't seen any benchmarking as to how significant these affects are. The 1TB SAS drive from seagate is about $220ish, which is quite a significant difference from 1.5TB SATA drives that can be had for $130ish now.
 
To put it bluntly:

Even in a RAID situation such as the one you spoke of, since each hard drive is going to be in operation at the same time when it comes to a read operation or a write operation, you're still going to encounter the biggest performance destroyer of all which is the fact that hard drives can't read and write at the same time.

One hard drive can perform a read operation or a write operation at any given moment, but not both. If you have two separate drives configured as individual drives, then you'd encounter a situation where one could be read from or written to while the other performed the same or exactly the opposite function.

But, since you're saying these two drives are in RAID 0 that effectively hamstrings them by making them perform as one logical drive which means you're still crucified by the read or write limitation.

Sure, it'll be faster (not quite twice as fast precisely, of course), but you will still only be able to read a file off the array or write a file to the array at any given moment of time and not both operations simultaneously. I know a lot of people that think because of the performance improvements they're getting more from the array than not, but it's simply not the case. You're still only able to read from the array or write to it - only one thing at time, even so.

Not even 1080p content will choke most modern drives in such situations unless you're transmitting it totally uncompressed (entirely possible but unlikely). You could not do what your example suggests which is:

Capturing video (SD) to the drives, and simultaneously reading from the drives to a server (over Gigabit LAN).
Click to expand...

You can't write to the array (as you're capturing) and read from it at the same moment, doesn't work that way. I'd say nothing would allow you to do this simultaneously... you'll end up capturing to a target location and then doing a transfer after the capture over the LAN. You'd be taxing that system considerably trying to do all of it simultaneously with any storage solution.

Hope this helps...

(after posting this I read the one above; alamone offers some more exotic solutions, but I'm betting the OP simply has 2 500GB drives and he's using the onboard "RAID" capabilities of the chipset on his motherboard so, my answers and suggestions were all based on that assumption...)

One last note: most current hard drive technology (within the past 2 years or so) should easily be able to handle SD content bandwidth without issues using a single drive...
 
Thanks for the input guys, much appreciated. So how does this approach sound:
We have two separate drives, A and B, no RAID.
At any time, any drive is either the write drive or the read drive. So it would work like this:
Capture video to drive A. When finished, transfer from drive A to server. While transferring from drive A, capture to drive B. When capture operation is done, read from drive B to server, and capture to drive A again. This loop will go over and over.

It will limit me to the performance of a single drive, but I would at least get the maximum write/read performance of any drive at any time, correct?

Thanks!
 
Depending on the amount of data I'd buffer in memory and from there stream it to the server.
 
Yeah, assuming network throughput and memory is sufficient. I guess what I'm really asking is, is this a viable solution?
 
CC_Pirate said:
Yeah, assuming network throughput and memory is sufficient. I guess what I'm really asking is, is this a viable solution?
Click to expand...

Writing to HDD and then immediately reading, sending it off and deleting it? Can't see a practical reason to do it this way. If you've got a few GB of RAM in the capturing system it should be more than enough coupled with a 100 MBit LAN or better.
 
This might be a good use for a huge RAM drive? Get 16 mb on your systems, then set up an 8 or 12 gig ram drive. Write to that, then upload from there to the servers.
 
The performance will always be affected by simple disk drive dynamics that they cannot read and write at the same time, as Joe Average described in such detail. :)
 
I write up to 6 OTA HD streams and watch 2 HD streams at a time - from one disk.

You should not have a problem doing what you want.

On the other hand copying 2 files onto a single disk has a great deal of problem in that each copy operation has a low priority and wants to wait for the other. Classic problem that leads to a lot of task switching - a lot of time wasting.
 
You must log in or register to reply here.
Back
Top