A few RAID/ file system questions

[Zero1]

Heya, just ordered a couple of WD SATA drives 2x120GB WD1200JD and a Highpoint Rocket RAID 1540.

It's my first time with RAID, and have been tempted to do it for ages, I have a few questions though and would be very grateful if anyone can help

1.) How would I determine the best stripe size? I heard you can choose between good write performance on a sliding scale depending on the stripe size. If possible, id like a balance between the two like below.
Read ----------|---------- Write

2.) How does the stripe size relate to the cluster size?

3.) FAT32's file size limit is 4GB, if I had 2 drives on RAID 0, would the max filesize be 8GB (is it 4GB limit per disk or 4GB limit overall?)

4.) How do you go on about formatting, partitioning and defragging a RAID 0 set? Is it just the same as you would with 1 drive?

5.) I think the defauly cluster size for NTFS in XP Home is 4kb, would the stripe size be best as a multiple of 4?

6.) If the cluster size if 4kb and the stripe was set to 2kb for example, would I waste a ton of space? (I noticed that smaller cluster size is more efficient storage, probably because of part filled clusters)

7.) Is RAID really unreliable? I hear a few stories that put me off a bit, but it's something I want to try for myself.



Thanks for your time!

 

[Ice Czar]

1.) How would I determine the best stripe size? I heard you can choose between good write performance on a sliding scale depending on the stripe size. If possible, id like a balance between the two like below.
Read ----------|---------- Write
Stripe Size

Decreasing Stripe Size: As stripe size is decreased, files are broken into smaller and smaller pieces. This increases the number of drives that an average file will use to hold all the blocks containing the data of that file, theoretically increasing transfer performance, but decreasing positioning performance.

Increasing Stripe Size: Increasing the stripe size of the array does the opposite of decreasing it, of course. Fewer drives are required to store files of a given size, so transfer performance decreases. However, if the controller is optimized to allow it, the requirement for fewer drives allows the drives not needed for a particular access to be used for another one, improving positioning performance

the read vs write balance isnt controlled by the stripe size per se, that involves caches, the controller, the size of the access, and the stripe.

2.) How does the stripe size relate to the cluster size?
same link as above

There are many "rules of thumb" that are thrown around to tell people how they should choose stripe size, but unfortunately they are all, at best, oversimplified. For example, some say to match the stripe size to the cluster size of FAT file system logical volumes. The theory is that by doing this you can fit an entire cluster in one stripe. Nice theory, but there's no practical way to ensure that each stripe contains exactly one cluster. Even if you could, this optimization only makes sense if you value positioning performance over transfer performance; many people do striping specifically for transfer performance.

3.) FAT32's file size limit is 4GB, if I had 2 drives on RAID 0, would the max filesize be 8GB (is it 4GB limit per disk or 4GB limit overall?)

No, the OS see the array as a single HDD, and its a filesystem limit anyway (you should be using NTFS without some other truely compelling reason anyway.)

4.) How do you go on about formatting, partitioning and defragging a RAID 0 set? Is it just the same as you would with 1 drive?

Yes, however making an array bootable is more complex, typically you have to set it bootable in the RAID's BIOS and when installing the OS you will be prompted to load additional SCSI drivers by pressing F6, and then you load the RAID driver (which is SCSI)

5.) I think the defauly cluster size for NTFS in XP Home is 4kb, would the stripe size be best as a multiple of 4?

The default cluster size is dependent on the size of the partition you can manually set a cluster size when you partition

6.) If the cluster size if 4kb and the stripe was set to 2kb for example, would I waste a ton of space? (I noticed that smaller cluster size is more efficient storage, probably because of part filled clusters)
refer to above

7.) Is RAID really unreliable? I hear a few stories that put me off a bit, but it's something I want to try for myself.

Small problems can become big, the potential to loose data more than doubles. MTBF HDD1 + MTBF HDD2 + MTBF of the controller (which could be replaced probably) double the suceptibility to impact ,power event, ect. loss of a single drive means the whole array is gone

It should be transient performance space not long term storage, I run HDDs in other redundant arrays or as singles for several months, so that any issues that may have occured in handling or shipping have time to develop before I trust it to a RAID 0, if they havent developed by then with care they shouldnt (good environment, proper handling)
If you do employ a RAID0 as storage, get very serious about backup, also, its not so easy to image RAID arrays, there is no official support for doing it with Ghost or Drive Image, sometimes it does work however

In short the size of the stripe greatly depends on your typical access pattern, a large stripe is ideal for working with say large video files, smaller stripe sizes will benchamrk better

 

[Zero1]

Thanks Ice Czar, that's a sweet in-depth reply, I owe you one

I think I'll choose a 4 or 16kb stripe size, what do you think?

 

[Ice Czar]

dont you mean KB? (Kilobyte?) not kb (Kilobit?)

whats it for? if your dealing with larger accesses the bigger youd want, for the increased transfer performance, if on the other hand its lots of small accesses, where positional performance would be the primary concern Id say the 16KB looks good (like a database access pattern)

32KB is what I have my RAID 5 set to sort of middle of the road
though it will go as high as 128KB

 

[Zero1]

--------
edit
Yeah, I did mean KB, I guess I was lazy with the shift button, but that reminds me to be more careful, thanks
--------

Ideally I'd like to improve general Windows usage, and specifically things like boot times, and loading drivers, & programs such as Media Player 9, Internet Explorer etc, you know, just to speed up what's used most.

I know that's pretty vague lol, please bear with me.

I also spend a lot of time in Dreamweaver, Photoshop and Windows Movie Maker.

Photoshop seems to take a hell of a long time to load, it seems to be a case of loading lots of small files, rather than one big file.

The video's in movie maker could sometimes be a single 4 - 5GB file (converted to DivX from DVD, so it's a similar data rate). So that's probably somewhere in the region of 800KB/s and I usually only have 1 or 2 files being read at once, so thats like a max of 1600KB/s, not a huge amount at all, again it doesn't seem to be a problem that there is too much data trying to be read at once, but more a problem of reading multiple files at once.



I'm struggling to word this lol, but my aim is to speed up the access of multiple files (such as drivers at boot, or programs in Windows), rather than to increase the data rate of a single large file (like trying to watch an uncompressed avi file).




However, sometimes I may need to extract large compressed files with little or no compression. I'd appreciate an increase in the time it takes to do this, becuase my hard drive is bottlenecking the process, but obviously I appreciate that its one or the other lol.

If I chose to optimise the access of multiple small files at the expense of the access of single large files, is it possible that the large file access could be worse than a single disk, or would it be better, but not as good as it could be?

Thanks for you're help, I really appreciate this... God I feel like a newb lol.

 

[drizzt81]

just a quicky: RAID-0 ing your drives is not going to improve your performance much, if at all. If you set your drives up in a 'smart' way, you will probably have better performance than using RAID-0. For example, having your page file on two seperate drives will reduce the performance hit you get when windows needs to access this and still work on another HDD transfer.

 

[Ice Czar]

some more stripe size info
http://www.storagereview.com/php/tiki/tiki-pagehistory.php?page=StripeSize&diff2=2

"Smaller stripe sizes improve STR of smaller files at the expense of service times (similar to "access time") for 2 or more randomly positioned/accessed files. The larger the stripe size, the greater the probability that any two files will be on opposite drives in their unstriped form, thus facilitating concurrent access to them. This improves multitasking ability and service times of multiple random I/O requests (even at lower I/O queue depths) at the expense of small file average STR. "

STR= Sustained Transfer Rate

as drizzt81 mentions multiple pagefiles have their use, but even better is more RAM

you have a choice, to optimize for general usage or graphics (large files), the graphics can actually benefit friom the RAID quite a bit. How much you benefit from the RAID in general usage is often fairly subjective.

you should also read the "As the disc spins" series, linked to at the top of the board (Sticky thread)