Building 1st SSD RAID

[labcoat]

I have 24 200GB SATA MLC SSD drives in a brand new Dell server. I need this server and these drives to last the 3 years warrantied and then some. We typically use our servers long past 3 years.

Questions:
1. Is correlated failure a real concern I should have? I.e., when all SSDs in the same array fail at practically the same time? I gleaned info about this here and here. How best to mitigate?

2. I'm familiar with RAID setups, Percs, RAID5 & 10 mostly and hot spares. How will the controller handle 12 hot spares ready to replace 12 SSDs in the working VD?

3. What will prices on current SSD SATA MLCs be like in 3 years (or less)? Will they be far cheaper or more expensive?

4. Best RAID for SSDs: 5? 10? 50?

5. Am I just being overly paranoid about SSD and RAID, and I should just treat it as any old plattered array?

I am at a fork wherein, I can provision all 24 of these drives into something over 3TB of storage, with maybe a half dozen hot spares (or something like this) and then deal with drive replacement later _or_ I can do 12 in an array/12 hot spares and feel that best DR for the duration has been achieved.

-ty

 

[labcoat]

/taps mic

is this thing on?

 

[hotcrandel]

I have 24 200GB SATA MLC SSD drives in a brand new Dell server. I need this server and these drives to last the 3 years warrantied and then some. We typically use our servers long past 3 years.

Most people that have used SSD's this long haven't had any problems. I've read ZFS installations using Intel G2 80GB drives as ARC space for years on some of the ZFS production blogs. Unless your write work load is insane (databases), I don't see why they wouldn't have anything beyond their normal failure % (1%? per year).

Questions:
1. Is correlated failure a real concern I should have? I.e., when all SSDs in the same array fail at practically the same time? I gleaned info about this here and here. How best to mitigate?

From what I know, I've not heard of anything like this. If you have a bunch of drives die at the same time, its a power supply, HBA or back plane issue. Usually replacing the faulty part will fix the problem.

2. I'm familiar with RAID setups, Percs, RAID5 & 10 mostly and hot spares. How will the controller handle 12 hot spares ready to replace 12 SSDs in the working VD?

I'm sure it will work fine, but it seems absurd. You shouldn't need more than one hot spare, and a "cold spare" for a deployment of this size.

You should use the right RAID for your workload, depending on card speed, reads and writes. If you're really worried about it I'd suggest RAID1. And as always, a good backup plan. A good backup plan mitigates issues like your aforementioned "Correlated Failure"

3. What will prices on current SSD SATA MLCs be like in 3 years (or less)? Will they be far cheaper or more expensive?

I'd suspect cheaper. There is so much demand in the massive server environment. A magnitude faster speeds, less heat/electricity use.

4. Best RAID for SSDs: 5? 10? 50?

Depends on workload. 10 sounds good. I almost never recommend RAID5, but it might be suitable in small deployments of SSD's. Keep an eye out for firmware updates that relate to your setup for the HBA and SSD drives.

5. Am I just being overly paranoid about SSD and RAID, and I should just treat it as any old plattered array?
-ty

Very much so. Assuming everything is on HCL's and supported by dell, I'd set it up and install it, and use it like any other server.

Maybe 8 drives in RAID6 x 3, three 1.2TB LUN's. Keep two spares on the shelf. Have a backup plan that you test and feel confident in.

Oh, one more thing.

Have a backup plan that you test and feel confident in.

 

[Old Hippie]

Am I just being overly paranoid about SSD and RAID, and I should just treat it as any old plattered array?

Yes.

 

[PigLover]

The one issue with SSD in an enterprise (heavy use) environment is the limited number of write cycles the NAND-flash can support. For most uses this is no longer a real issue as typical use won't hit the write limit for more years that the drive is expected to last anyway. With heavy enterprise uses, however, the situation can be different.

If you are going to run them in a raid you can help this by ensuring you have a large cache on the raid controller with battery backup so that you can use one of the many forms of a delayed write policy ('write-back', etc.). Newer raids from LSI use an SSD-based cache instead of battery backup, but the idea is the same. If you are using ZFS you want a large, fast ZIL. Using the cache/ZIL will help prevent small, frequent writes by allowing the controller (or ZFS) to delay writes and write in bigger chunks, lowering the write counts on the SSDs.

Otherwise, either current gen SSDs, just treat them like normal spinny disks for raid.

 

[labcoat]

Most people that have used SSD's this long haven't had any problems. I've read ZFS installations using Intel G2 80GB drives as ARC space for years on some of the ZFS production blogs. Unless your write work load is insane (databases), I don't see why they wouldn't have anything beyond their normal failure % (1%? per year).

It will be our 3rd VMware ESXi host and it will indeed house some DBs -- at the very least, one new SQL DB server that will do a lot of work. To date, I've not placed my heavy SQL DBs in a VM due to I/O. These SSDs should finally provide the speeds to do so. tl;dr: I will be putting DBs on this as well as the work a VMware host performs anyhow.

From what I know, I've not heard of anything like this. If you have a bunch of drives die at the same time, its a power supply, HBA or back plane issue. Usually replacing the faulty part will fix the problem.

I discovered "Correlated failure" and "Differential RAID" (which is only theory?) from these forums which provided the MS link on the issue. I suppose it is like googling symptoms to think you are dying when you just have a mild headache. Still, why is there even this discussion unless it can happen?

I'm sure it will work fine, but it seems absurd. You shouldn't need more than one hot spare, and a "cold spare" for a deployment of this size.

The _only_ reason I would do 12 hot spares is the same reason I might just pull 12 SSD drives out and shelve them at put bay covers over the spots instead -- in order to not need to purchase new drives in 3 years at all and/or give best possible DR for this specific server.

It is difficult for me to explain, but where I work, I am tasked with both providing the best possible performance from hardware, but also not given the funds to get said hardware. It is very frustrating and, again, a long story....

You should use the right RAID for your workload, depending on card speed, reads and writes. If you're really worried about it I'd suggest RAID1. And as always, a good backup plan. A good backup plan mitigates issues like your aforementioned "Correlated Failure"

I use Veeam B&R and it works great. I send all my VMs across my 2 existing hosts that are in the same building, and then to another sight miles away that has battery and generator. I have 3 total data centers with DC1 & DC3 having battery and generator. DC2 has only battery (trying to get generator, but again, $). This new server will go in DC2 and needs to do both production and DR for DC1 (DC3 is really only backup).

With VMware 4.x, the 1.6ish TB store limit made it tough over a couple of years, but I've managed to keep glut off my VMs and I have a half TB on each host and can clean up more.

This new server will be VMware 5.1 which sends store size upwards of 64TB I read/heard (we do not _yet_ have a SAN <-- no shared storage).

tl;dr: I need to provision all best/possible space in this new server now. It will have only one VD.

I'd suspect cheaper. There is so much demand in the massive server environment. A magnitude faster speeds, less heat/electricity use.

If I knew for sure that these drives will be cheaper (hopefully, far cheaper) then that would solve all of my problems. My biggest concern is to need some of these drives in a few years and have to tell mgt that they are the same price or more.

Depends on workload. 10 sounds good. I almost never recommend RAID5, but it might be suitable in small deployments of SSD's. Keep an eye out for firmware updates that relate to your setup for the HBA and SSD drives.

I do RAID10 when I have to and RAID5 when I can get away with it. I'm not asking for a debate between the two but honestly wondering if SSD changes anything with RAID. I have read some articles where people advocate 5 for SSD arrays since RAID10 is meaningless due to correlated failure, but I'm feeling the room is looking at me oddly over my paranoia about now concerning CF.... /digress

Very much so. Assuming everything is on HCL's and supported by dell, I'd set it up and install it, and use it like any other server.

One thing mgt has been good at is listening to me concerning best support contracts. I will keep a best-possible, 4 hr turn-around on this server as I have on the others.

Maybe 8 drives in RAID6 x 3, three 1.2TB LUN's. Keep two spares on the shelf. Have a backup plan that you test and feel confident in.

I've never done a RAID6. Dell is recommending RAID10. RAID5 would make me happiest (storage) <-- I honestly need this new server to run the 2 total TBs from the 1st 2 hosts, and then any new production he'll begin (which will then need to be shared btwn the 2 in the 1st DC.

Oh, one more thing.

Have a backup plan that you test and feel confident in.

Good advice. We use Veeam B&R as discussed, and then home-grown scripts via linux that work well + another aged backup software solution for flat files and other various methods. We sorta back everything up some where 2 and 3 times over....

 

[labcoat]

I need to emphasize that these are Dell SSD MLC SATAs with a warranty of 3 years _only_ and that Dell will not warranty past 3 years (they say this in big, bold letters).

^this more than reading about CF has affected my paranoia (and wearing empty kleenex boxes for shoes)....

 

[ToddW2]

1. Depending on usage but after 3 Years those SSD could be near EOL.
In my home systems after 3 years I've replaced all my SSD with better, faster newer models. In my parents system, and friends they are still happily chugging along with their old SSD (Intel G2s).

2. There's a LOT of good info on WebHostingTalk.com about SSD in RAID because so many web hosting providers are going this route.

2a. A lot of what you read on WHT is not on-par with what people at [H] experience due to the enterprise / heavy work-load of a webserver vs. workstation or at home user.

3. IIRC on WHT the Intel 520, and Samsung 830 are among the most reliable, and favorites among the providers. Obviously there are more expensive Intels MADE for enterprise but they cost a LOT more.

I`m not sure what your SSD Dell is a re-brand of if any?


Just some info I've learned in the past few months as I look to build out some webservers with SSD.

 

[hotcrandel]

It will be our 3rd VMware ESXi host and it will indeed house some DBs -- at the very least, one new SQL DB server that will do a lot of work. To date, I've not placed my heavy SQL DBs in a VM due to I/O. These SSDs should finally provide the speeds to do so. tl;dr: I will be putting DBs on this as well as the work a VMware host performs anyhow.

SSD's are amazing for database throughput, so, yes. How big is the database?, and do you happen to know how much read/write it does? You might consider isolating it on a few drives in its own LUN instead of spreading it out over many. There is likely less IOPS benefits doing this on a few SSD's instead of spreading it out over many spindles.

I discovered "Correlated failure" and "Differential RAID" (which is only theory?) from these forums which provided the MS link on the issue. I suppose it is like googling symptoms to think you are dying when you just have a mild headache. Still, why is there even this discussion unless it can happen?

Tornadoes and meteors, solar flares, gamma ray strikes, they all can happen too. Planning for a whole bank of SSD's to fail, you will want to move into a shared storage system running an active-active or active-passive configuration. I am running some HP systems that do this, and they were pretty reasonable, though not nearly as fast as your SSD's. I believe I got 8TB RAW nearline and 3.2TB 15k SAS raw, x2, 15k per pair? I'm really pleased with them and use them in ESX 5.0.

I've got the project for getting paid-for veeam on the docket this year, but as you said, "budget".

The _only_ reason I would do 12 hot spares is the same reason I might just pull 12 SSD drives out and shelve them at put bay covers over the spots instead -- in order to not need to purchase new drives in 3 years at all and/or give best possible DR for this specific server.

It seems like you're trying to prevent a problem with budget later by spending more now. I understand why you're doing it, but it doesn't make a lot of sense to me. I'd buy a server with 24 drives, and pull 12 of them out an inch, and leave them in the chassis. No need to have them run for 2 years not doing anything. Either way it still feels like overkill.

It is difficult for me to explain, but where I work, I am tasked with both providing the best possible performance from hardware, but also not given the funds to get said hardware. It is very frustrating and, again, a long story....

You're not the first, man. Trust us. Have you looked into white-box solutions? In particular solaris running ZFS, and then using ZFS replication to do your backups? ZFS allows you to have tiered storage.. It's pretty sexy and I'm learning more and more about it in the server environment by tinkering with it at home.

I use Veeam B&R and it works great. I send all my VMs across my 2 existing hosts that are in the same building, and then to another sight miles away that has battery and generator. I have 3 total data centers with DC1 & DC3 having battery and generator. DC2 has only battery (trying to get generator, but again, $). This new server will go in DC2 and needs to do both production and DR for DC1 (DC3 is really only backup).

So it sounds like uptime is pretty important =). Weird you get so much money for generators and have three sites..... but can't quite get enough to get something with shared storage for HA vmware.

With VMware 4.x, the 1.6ish TB store limit made it tough over a couple of years, but I've managed to keep glut off my VMs and I have a half TB on each host and can clean up more.

It feels really great getting into 5.0 and better. The VMFS limitations in 4.0 are pretty impractical.

This new server will be VMware 5.1 which sends store size upwards of 64TB I read/heard (we do not _yet_ have a SAN <-- no shared storage).

Correct, I've read that its best practice (maybe its only applicable to my environment) that individual vmdk files / VMFS file systems are no larger than 2TB. If you need more you want to use spanning or something. This is mostly due to backup/restore times of individual VMDK's

tl;dr: I need to provision all best/possible space in this new server now. It will have only one VD.

This seems like a bad idea, in my opinion. I suppose I'd consider a 12 disk RAID6, but not a RAID5, let alone something bigger. Even with SSD's.

If I knew for sure that these drives will be cheaper (hopefully, far cheaper) then that would solve all of my problems. My biggest concern is to need some of these drives in a few years and have to tell mgt that they are the same price or more.

After thinking about it a bit, and knowing dell, they will probably be exactly the same price three years from now, for the same model. They know they can "exploit" their customers in this manner. Look up how much an official dell 1TB SAS drive is, its probably near $200.

I do RAID10 when I have to and RAID5 when I can get away with it. I'm not asking for a debate between the two but honestly wondering if SSD changes anything with RAID. I have read some articles where people advocate 5 for SSD arrays since RAID10 is meaningless due to correlated failure, but I'm feeling the room is looking at me oddly over my paranoia about now concerning CF.... /digress

RAID10 requires much less (zero?) parity calculation. With such a big array with so much IOPS, I think you're going to hit the ceiling pretty quickly with RAID5 or RAID6, where as RAID10 or RAID1 there is much less computation.

One thing mgt has been good at is listening to me concerning best support contracts. I will keep a best-possible, 4 hr turn-around on this server as I have on the others.

Keep in mind how fast you can shift around a huge LUN or huge VMDK if your vm's are bigger than a TB or two. I'd rethink my backup and implementation strategy to keep my VM's from ballooning too much. I created a 12TB file server that became too much to manage over a few years in a reasonable amount of time. I have gotten a lot of benefit and more successful backups by breaking it into different 1TB smaller file servers. (Admittedly with much better storage IO in the new servers)

I've never done a RAID6. Dell is recommending RAID10. RAID5 would make me happiest (storage) <-- I honestly need this new server to run the 2 total TBs from the 1st 2 hosts, and then any new production he'll begin (which will then need to be shared btwn the 2 in the 1st DC.

There was some discussion a while ago about dell not recommending R5 anymore. It's reasoning is less applicable to smaller arrays, but again a 12 disk or even 6 disk R5, with what I've read about URE errors during rebuilds, seems like plenty enough good reason not to use it.

Good advice. We use Veeam B&R as discussed, and then home-grown scripts via linux that work well + another aged backup software solution for flat files and other various methods. We sorta back everything up some where 2 and 3 times over....

If thats the case, then you should have confidence in whatever you decide. Assuming all the hardware is on HCL's and things, you should be successful. Things will always eventually fail, so do the best plan you can for what you have.

 

[labcoat]

1. Depending on usage but after 3 Years those SSD could be near EOL.
In my home systems after 3 years I've replaced all my SSD with better, faster newer models. In my parents system, and friends they are still happily chugging along with their old SSD (Intel G2s).

^this is what I'm talking about. Dell states up front they will _not_ support these drives after 3 years. There has to be a reason for that ... for them to turn down good $.

2. There's a LOT of good info on WebHostingTalk.com about SSD in RAID because so many web hosting providers are going this route.

Thx for sending me to WHT. I'm reading thru now.

3. IIRC on WHT the Intel 520, and Samsung 830 are among the most reliable, and favorites among the providers. Obviously there are more expensive Intels MADE for enterprise but they cost a LOT more.

Of course, I in no way could get SLCs. What has happened is platters and SATA SSD MLCs simply got close enough, and we needed the I/O enough, that I pushed for them. I could have gotten 900GB SAS and had no want for space, but checking stats, even SATA SSDs have some 20x the throughput of the best SAS. My only concern is the 3Gbps controller bottleneck.

I`m not sure what your SSD Dell is a re-brand of if any?

These are the drives

Just some info I've learned in the past few months as I look to build out some webservers with SSD.

thx for your input much appreciated.

 

[labcoat]

The one issue with SSD in an enterprise (heavy use) environment is the limited number of write cycles the NAND-flash can support. For most uses this is no longer a real issue as typical use won't hit the write limit for more years that the drive is expected to last anyway. With heavy enterprise uses, however, the situation can be different.

That's just it. I really would like for these 24 drives to last beyond the 3 years warrantied. If not, I could have gotten away with half of them. I pushed for 24 due to the, "might not get this chance again" and it is solid DR anyhow (if not paranoid DR right?).

If you are going to run them in a raid you can help this by ensuring you have a large cache on the raid controller with battery backup so that you can use one of the many forms of a delayed write policy ('write-back', etc.).

The controller is a PERC H710P. Doc says 1GB cache memory (72-bit DDR3). It does have a battery. My "Write Policy" settings are "Write Through" and "Write Back" with "Write Back" as default. (Read being "Adaptive Read Ahead").

Newer raids from LSI use an SSD-based cache instead of battery backup, but the idea is the same. If you are using ZFS you want a large, fast ZIL. Using the cache/ZIL will help prevent small, frequent writes by allowing the controller (or ZFS) to delay writes and write in bigger chunks, lowering the write counts on the SSDs.

Otherwise, either current gen SSDs, just treat them like normal spinny disks for raid.

Since I'm doing RAID with the PERC, I do not have LVM or OS-side RAID (I don't think VMware can do RAID. So, my file system and RAID cannot do ZFS.

My RAID choices are:

RAID-0
RAID-1
RAID-5
RAID-6
RAID-10
RAID-50
RAID-60

 

[labcoat]

SSD's are amazing for database throughput, so, yes. How big is the database?

It will be approx. 50GB give/take, SQL DB. But the server will be a VMware host hosing who-knows-what VMs in the coming few years. (IT planning ... yea right).

and do you happen to know how much read/write it does? You might consider isolating it on a few drives in its own LUN instead of spreading it out over many. There is likely less IOPS benefits doing this on a few SSD's instead of spreading it out over many spindles.

Read/writes: the worst or the least or the best <-- it is up to the whims of the DB admins who, sometimes, write stuff that sends my servers to their knees for months. Then, magically, they fix and the I/O drops to nothing. In the mean time, I'm told, "it's the hardware!" ... says the DB admins. "The hw was just fine, it's your new calc that's killing it" ... says me. Magically, it is fixed. I try to build for the worst-case. I know I totally didn't answer your question, but such is my hell.

Tornadoes and meteors, solar flares, gamma ray strikes, they all can happen too. Planning for a whole bank of SSD's to fail,

The whole "NOT WARRANTIED, PERIOD, AFTER 3 YEARS" by Dell and the MS doc on CF and then the, "you'll be fine as long as no DBs" ... when I _will_ have DBs, and the whole, "you'll be good for a year or 2" but I need 3+ years, has made me look hard into planning.

I'll either do a RAID5 in 12 disks with 12 hot spares (crazy right)? Or a RAID10 with all 24 (or 22) with no or 2 hot spares. I would really like to do the RAID10 _if_ it'll buy me more life with these SSDs.

It seems like you're trying to prevent a problem with budget later by spending more now.

Them: "why can't we run these reports?"
Me: "You need a new report DB server."
Them: "why can't we have a new DB server?"
Me: "We are out of memory and getting close on storage on the 2 VMware hosts we have, since I was never able/allowed to purchase the 3rd. _and_ I cannot guarantee performance of the 3rd with traditional platters and the track record of our DB admins."
[Time goes by]
Them: "Ok, we want you to get what you say you need. How much?"
Me: "$Chaching!"
Them: "Why did this cost so much!?!"
Me: "Bcs if it takes as long to get the next server and/or upgrades as it did this one, I'll need this to get by...."

^something like that

I understand why you're doing it, but it doesn't make a lot of sense to me. I'd buy a server with 24 drives, and pull 12 of them out an inch, and leave them in the chassis. No need to have them run for 2 years not doing anything. Either way it still feels like overkill.

In one way it _is_ over kill. In another, it's preparing to be told "no" again for years. In congress, it's called "ear marks."

So it sounds like uptime is pretty important =). Weird you get so much money for generators and have three sites..... but can't quite get enough to get something with shared storage for HA vmware.

DC1's generator was a grant, no cost to us.
DC3's generator came with the co-lo, rolled into payments or, as the salesman so eloquently put it, "revenue neutral."
DC2's generator is my next challenge. I've already been told, "hrm, prolly not...." Of course, if we go down it'll be, "why didn't you!?!?"

It feels really great getting into 5.0 and better. The VMFS limitations in 4.0 are pretty impractical.

I grew up with VMware GSX (remember!) and VMware Server 1.x and 2.x on Linux. I used Ubuntu. Most folks don't know that with free VMS2.x, we had thin provisioning before the for-pay VMware admins had it. Also, I had unlimited access to the local store, so I built 2 7TB and 9TB "ghetto SANs" <-- I called them. It was the good ol' days.

I put these boxes in the different buildings, and on my own figured out rsync and scp would do the trick. I could failover. All I didn't have was Vmotion-style users-don't-know-it-movedness....

ff to today: VMware has officially killed VMS2.x <-- it will not install with the current Linux kernel without a heart transplant. I punted.

I'm actually looking at open source virtualization as a failover to the failover.

Veeam was nothing new to me, just a GUI wrapper, and oh, that renamed the command to "fastwhatever." I'm not knocking Veeam, but they did what companies have done for decades now in computing: put a GUI on it (If you really love it right!).

Correct, I've read that its best practice (maybe its only applicable to my environment) that individual vmdk files / VMFS file systems are no larger than 2TB. If you need more you want to use spanning or something. This is mostly due to backup/restore times of individual VMDK's

As just stated, VMS2.x had no trouble with lotsa storage, but I understand ESXx is a different animal, relying on VMware's file system. This is something to consider -- too big a store -- but I just assumed since 5.1 allows 64TB that it was ok to go bigger....

This seems like a bad idea, in my opinion. I suppose I'd consider a 12 disk RAID6, but not a RAID5, let alone something bigger. Even with SSD's.

Deciding on the RAID and the number of disks to use is driving me to the bottle. I have to decide soon....

After thinking about it a bit, and knowing dell, they will probably be exactly the same price three years from now, for the same model. They know they can "exploit" their customers in this manner. Look up how much an official dell 1TB SAS drive is, its probably near $200.

I've gone back and forth on Dell and the cost of their drives. Article 1 says, "they suck! over-priced!" Article 2 says, "no no, there's a reason, here's why."

RAID10 requires much less (zero?) parity calculation. With such a big array with so much IOPS, I think you're going to hit the ceiling pretty quickly with RAID5 or RAID6, where as RAID10 or RAID1 there is much less computation.
[/quote ]

^this is sounding like RAID10 might be the ticket. I'm thinking about doing all 24 in RAID10. This will give me something in the order of 2.2TB of store. A RAID5 with 12 is approx. 2.0, so I'll have another precious 200GB. Then ($ talk here), I can beg and plead for an SSD once a quarter or 6 months or come next budget time. Over a couple of years, I can have enough mebe... mebe.

Keep in mind how fast you can shift around a huge LUN or huge VMDK if your vm's are bigger than a TB or two.

Veeam has made nice work of moves, but I'm eager to see what VMware 5.1 can do. Since I replicate VMs to each host, I just do the perm failover replica option.

I'd rethink my backup and implementation strategy to keep my VM's from ballooning too much. I created a 12TB file server that became too much to manage over a few years in a reasonable amount of time. I have gotten a lot of benefit and more successful backups by breaking it into different 1TB smaller file servers. (Admittedly with much better storage IO in the new servers).

The biggest VM footprints I've allowed so far are approx. 400GB. I fuss a lot. There's been a few times others on the team dumped stuff they shouldn't've but I caught it and they removed. I've done thinning and documented the procedure (need to do more). The worst VMs I have so far are actually Exchange servers. I hate hosting email and keep telling them to off-load it to gmail or whatever, but there's something about voice and email that mgrs feel needs to be in house.

There was some discussion a while ago about dell not recommending R5 anymore. It's reasoning is less applicable to smaller arrays, but again a 12 disk or even 6 disk R5, with what I've read about URE errors during rebuilds, seems like plenty enough good reason not to use it.

I know that the magic number of disks in a RAID5 array has always been 3, and after that, the striping just gets too much, or something. I'm fried right now after other stuff today and need to stop thinking right about now.

If thats the case, then you should have confidence in whatever you decide. Assuming all the hardware is on HCL's and things, you should be successful. Things will always eventually fail, so do the best plan you can for what you have.

It'll be either a RAID5 in 12 disks, or a RAID10 with almost all or _all_ disks.

I'm leaning at RAID10 atm....