Questions on Expanding My RAID Array

E

ethraax

Limp Gawd
Joined
Apr 11, 2008
Messages
174
I currently have a P55 motherboard. It has 6 SATA II slots, raid compatible. Right now, the slots are being used as:

Slot 0: CD/DVD drive
Slot 1: 500 GB HDD (array 1)
Slot 2: 500 GB HDD (array 1)
Slot 3: 1.5 TB HDD (arrays 2,3)
Slot 4: 1.5 TB HDD (arrays 2,3)
Slot 5: Empty

Array 1: RAID 1 (contains my OS)
Array 2: RAID 1 (50 GB)
Array 3: RAID 0 (whatever's left, around 2.6 TB)

Now, I want to get a bit more storage. I have bays left in my case so that's not an issue. However, I only have one empty slot, and I have a RAID 0+1 array that I'm starting to regret. Preferably, I'd like to set up a big RAID 5 array.

One possibility is to buy 3 more 1.5 TB drives (roughly $300 total) and replace the whole mess with a big RAID 5 array.

I'd like to do this without losing any data though. I have enough space on the big RAID 0 array to fit the contents of the array in slots 1 and 2.

My plan is to copy the contents of array 1 onto array 3, and then remove the 500 GB drives from my computer. Then, I could set up a RAID 5 array with 3 hard drives (slots 1, 2, 5) and install my OS. Then my two arrays would have roughly the same amount of storage space, so I could copy the files from the old RAID 0 array onto the new RAID 5 array. Finally, I could disband the RAID arrays in slots 3 and 4 and add those hard drives to my RAID 5 array, so I end up with a big RAID 5 array with my OS, data, and 6 TB of advertised space (about 5.4 TB actual space).

I could also swap out the CD/DVD drive (get one that doesn't use SATA) and get a sixth hard drive for another $100. Then I'd have 7.5 TB of advertised space, which should be plenty for now.

Is this possible? Can I simply add extra hard drives to a RAID 5 setup without losing any data? It seems completely reasonable - I can't think of a good reason why not - but I want to double check before I spend money on it.

Thanks for the help!

-- Ethan

PS: The RAID controller reads: ICH8R/ICH9R/ICH10R/DO/PCH SATA RAID Controller. If that helps with anything.
 
It's generally not recommended to keep your OS on the RAID array. As such it might be a good idea to get a seperate OS drive.

You can expand a RAID 5 array without losing the data as long as the RAID controller supports online capacity expasnion. Oh and do remember that you'll see constant CPU usage ranging 15% to 50% depending on the size of the array.
 
Is the CPU usage significantly higher than with my RAID 1 arrays? Because it basically renders my computer unusable while they're verifying (after a power outage, for example) but other than that their effect is negligible.

Also, why is it not recommended to keep my OS on my RAID array? I'm not doubting you, I'm just curious - I always figured redundancy would be good for an OS.
 
ethraax said:
Is the CPU usage significantly higher than with my RAID 1 arrays? Because it basically renders my computer unusable while they're verifying (after a power outage, for example) but other than that their effect is negligible.

Also, why is it not recommended to keep my OS on my RAID array? I'm not doubting you, I'm just curious - I always figured redundancy would be good for an OS.
Click to expand...

RAID 1 and RAID 0 hardly use that much CPU during general usage. Yes RAID 5 does have significantly higher CPU usage than RAID 1 or RAID 0 arrays. Like I said, it ranges from 15% to 50% constant CPU usage depending on the size of the array and the CPU.

Well the OS drive should be seperate from a RAID 5 array to make sure that should you ever need to reformat, fix, or replace the OS or OS drive, it will not affect the RAID 5 array. So in other words, if anything happens to the OS, it will not affect the RAID 5 array. In addition, should you ever need to rebuild, repair, restore, upgrade, or remove the RAID 5 array, the OS will not be affected and therefore you'll still have a functioning PC, just maybe without a RAID 5 array.
 
Like I said, it ranges from 15% to 50% constant CPU usage depending on the size of the array and the CPU.
Click to expand...

This is under windows using software raid or fakeraid. With a hardware raid card under windows or software raid on most other operating systems the cpu load under raid 5 is minimal. For example my i7 920 hits a 1% load on one of the 8 threads while writing to the mdadm software raid 5. 5 year old systems at work hit 7% load on a single core during raid5/6 writes.
 
drescherjm said:
This is under windows using software raid or fakeraid. With a hardware raid card under windows or software raid on most other operating systems the cpu load under raid 5 is minimal. For example my i7 920 hits a 1% load on one of the 8 threads while writing to the mdadm software raid 5. 5 year old systems at work hit 7% load on a single core during raid5/6 writes.
Click to expand...

Yeah I should have made that clearer: The high CPU usage is with Windows using software RAID.
 
Just to double check: that's 15-50% load while writing, correct? So reading data wouldn't put so much strain on my CPU (i5 @ 3.4 GHz, under Windows 7 64-bit). If so, then that's certainly tolerable, considering most of my data manipulation is likely to be on the OS drive. If I'm migrating a large amount of data, I could always do it overnight.
 
ethraax said:
Just to double check: that's 15-50% load while writing, correct? So reading data wouldn't put so much strain on my CPU (i5 @ 3.4 GHz, under Windows 7 64-bit). If so, then that's certainly tolerable, considering most of my data manipulation is likely to be on the OS drive. If I'm migrating a large amount of data, I could always do it overnight.
Click to expand...

From what I've seen, thats 15% to 50% CPU usage at all times even when there is no writing or reading involved.
 
Danny Bui said:
From what I've seen, thats 15% to 50% CPU usage at all times even when there is no writing or reading involved.
Click to expand...

are you sure thats not while you are building the array or some such

i cant see what the CPU would be doing if the array was not writing
 
@OP

OK, let me get this straight. You have:

a) 2x 500GB drives for your OS, in a RAID1 configuration

b) 2x 1.5TB drives. Each is partitioned so that 50GB of the 3.0TB total is in RAID 1, and 2.6TB of the 3.0TB total is in RAID 0. Is this right? I ask, because there's 350GB missing...

Anyway, here's what I'd do:

First, replace your CD/DVD drive with an external USB unit - that'll free up a port immediately, and to be honest you won't lose much performance anyway.

Second (and this is the fun part), remove one of your 500GB OS drives (port 2). This frees up a second port, which will give you a total of three free ports. You will still be able to boot from your remaining OS drive.

Third, go buy your three 1.5TB hard disks and create your RAID 5 array (ports 0, 2, and 5). Migrate the data from your current arrays (arrays 2 and 3) to the new array. Now break down arrays 2 and 3 - you will now have two 1.5TB drives which can be added to your new RAID 5 array.

Of course, if you go and buy a decent hardware RAID card, life will be much simpler - simply create the array on the new card with the three new drives, migrate the data across from arrays 2 and 3, then add the drives from arrays 2 and 3 to the RAID card so that you can expand the array, while leaving your DVD drive and OS drives untouched.

As for putting your OS on a RAID 5 array, yes it can be done but unless the RAID is controlled by a hardware RAID controller, it is ill-advised (performance reasons). An OS on RAID 0, RAID 1 or RAID 10 is the preferred set up (at least for me).

Forgot to add: I'm not sure if the ICH9R/10R support online capacity expansion. If they do, all's well. If not then it'll have to be a hardware RAID controller.
 
Butcher9_9 said:
are you sure thats not while you are building the array or some such

i cant see what the CPU would be doing if the array was not writing
Click to expand...

Parity calculations. Since you're planning on using the motherboard's onboard RAID, there is no dedicated XOR engine that takes care of the parity calculations. As such, the PC's CPU is used to do all of the parity calculation for a RAID 5 array. Therefore, constant CPU usage.
 
Danny Bui said:
Parity calculations. Since you're planning on using the motherboard's onboard RAID, there is no dedicated XOR engine that takes care of the parity calculations. As such, the PC's CPU is used to do all of the parity calculation for a RAID 5 array. Therefore, constant CPU usage.
Click to expand...

Why would there be a need to calculate parity when no data is to be written on the disk ? (aside from error checking)
 
Parity is not the reason RAID5 is slow. Parity = XOR and that is super-simple for your CPU. Even slow CPUs can do 2GB/s+ with XOR; you can see the benchmarks for your CPU when you boot Ubuntu which uses the md-driver to test which algoritm performs best on your CPU. 20GB/s+ should be possible with a high-end setup; it basically scales with memory bandwidth.

So if parity isn't making it slow, what is? Well in most simple RAID5 drivers like on XP and most onboard RAID5 except Intel, the CPU usage is actually very low because it is a 'dumb' driver; it will let the disks seek and cause low write performance, but cost very little CPU cycles (1-5%).

More advanced drivers use write-back and write-combining techniques to give RAID5 high sequential write performance. To do this, it has to keep writes in memory, wait until multiple ones stack up and then write in the optimal block size, the "full block stripe" calculated with (number_of_disks - 1) * stripesize. So with an 8-disk RAID5 with 128KiB stripe the full block stripe would be 7*128 = 896KiB. That magic number would be very fast when writing in RAID5; as fast as RAID0 with one disk less.

However, all these combines and splits of chunks of writes cause alot of memory bandwidth; and thus this is what more than 50% of the CPU time of an advanced RAID5 driver goes to. One example of such a driver is geom_raid5 on FreeBSD; though not included in the base install. It can write faster than my Areca HW RAID can in RAID5.

The cost, however, is increased CPU usage; which can max a single core. But what would you prefer; everything in your PC waiting for disk and your CPU idling because it cannot do any work; or let your CPU work and do at least 1000% faster I/O? The overall performance would be much higher.

So, you actually want the CPU being used, by a smart driver; than let a dumb driver let the disks seek in eternity.
 
sub.mesa said:
Parity is not the reason RAID5 is slow. Parity = XOR and that is super-simple for your CPU. Even slow CPUs can do 2GB/s+ with XOR; you can see the benchmarks for your CPU when you boot Ubuntu which uses the md-driver to test which algoritm performs best on your CPU. 20GB/s+ should be possible with a high-end setup; it basically scales with memory bandwidth.

So if parity isn't making it slow, what is? Well in most simple RAID5 drivers like on XP and most onboard RAID5 except Intel, the CPU usage is actually very low because it is a 'dumb' driver; it will let the disks seek and cause low write performance, but cost very little CPU cycles (1-5%).

More advanced drivers use write-back and write-combining techniques to give RAID5 high sequential write performance. To do this, it has to keep writes in memory, wait until multiple ones stack up and then write in the optimal block size, the "full block stripe" calculated with (number_of_disks - 1) * stripesize. So with an 8-disk RAID5 with 128KiB stripe the full block stripe would be 7*128 = 896KiB. That magic number would be very fast when writing in RAID5; as fast as RAID0 with one disk less.

However, all these combines and splits of chunks of writes cause alot of memory bandwidth; and thus this is what more than 50% of the CPU time of an advanced RAID5 driver goes to. One example of such a driver is geom_raid5 on FreeBSD; though not included in the base install. It can write faster than my Areca HW RAID can in RAID5.

The cost, however, is increased CPU usage; which can max a single core. But what would you prefer; everything in your PC waiting for disk and your CPU idling because it cannot do any work; or let your CPU work and do at least 1000% faster I/O? The overall performance would be much higher.

So, you actually want the CPU being used, by a smart driver; than let a dumb driver let the disks seek in eternity.
Click to expand...

So...don't hardware RAID5/6 controllers have to address these issues also?
 
Yes, the Intel IOP is specifically designed to handle XOR and does it reasonably well. But nothing beats your system CPU and it is no bottleneck in any way in all but the most extreme cases.

The real bottleneck is how smart the RAID engine is (in hardware or software). An intelligent RAID5 driver would be much more complex than a dumb simple one that let the disks seek more but have a much simpler design. I have not seen a perfect RAID5 driver yet; though geom_raid5 on FreeBSD got me quite high sequential write speeds (400MB/s+) and scales about 90%. Its not fully multithreaded, though.
 
ICH10R supports capacity expansion. Don't count on it though...
 
You must log in or register to reply here.
Back
Top