Media Server - Setup Advice (RAID6)

V

VenomousDesigns

Limp Gawd
Joined
Jan 4, 2009
Messages
173
Good afternoon all,

I've recently come to the realization that the only way to keep my current surplus of HDDs in order/have somewhat of a data retention (please note that I understand RAID cannot and shouldn't be used as a backup replacement).

I currently live down-under (Australia), so high hopes of getting certain items is alittle limited outside of importing which I will no doubt have to do for my selected RAID controller.

Details:

The use of the Server will be to serve as a HTPC/Media Storage as the majority of files are backups of my Blu-rays, DVDs and all my TV Shows to save me having to switch discs in and out nor concern myself with damage to the physical media.

Currently have 16xSamsung 1TB F3 Spinpoint, 2xWD Green 1Tb and a flurry of various smaller sized HDDs and external encloses. I have reached the limits of my personal Gaming desktop (8 SATA ports on its MB) and I figured I have to stop kidding myself and realize I need to buy/create a dedicated Server for these HDDs.

What I Know:

Have been doing a flurry of reading over the past week or so, on here and various other forums. ATM I am not to concerned with the internal specs of the Server as that can be done at a later date once I have my RAID solution sorted.

The case I've been eye'ing is the Lian Li PC-343B Modular Aluminium Cube Case:

700817-055_p001.jpg


From that I was hoping to run a RAID6 solution - I've heard of a range of different options to here:

  1. Splitting up my various HDDs into smaller RAID6 arrays to lessen the risk of drive failure
  2. Having one large RAID6 array
  3. Leaving them all as individual HDDs (Dislike this option as it provides no protection, and moving of data is a PAIN)

Once that question is answered, it then comes down to what I should be looking for in a RAID Card/Controller - and the possibility of future expansion/newer technologies. This is where I will be guided by you - I don't really have a budget as per-say, but would defiantly like to keep it under the US $2000 mark as I already have all the HDDs (minus the ones needed for parity).

The Server is not located in a bedroom so noise is a moot point, I would like the option of expanding in the future (a given with the increase of Media). I've heard of numerous companies thrown around; Adaptec, Highpoint etc - so I'll be guided by you on the best overall solution for my setup.

Questions:

As I'm new to the whole RAID thing I had a few quick questions;

1. Is it going to be fine that the HDDs I already have (not including the ones I will need for RAID parity) already have data on them? I obviously don't want to risk plugging all 18xHDDs in just to find out I've lost everything.

2. Future HDD expansion (a given), I've heard horror stories of RAID expansion times - is this another advantage in the column of splitting ones RAID array into separate smaller arrays?

3. Is it a good option to have a Server double as a HTPC? Or better to move to your desired device - aka PC connected to TV rather than hooking up the Server, Laptop or w/e? Does it affect what OS I put on it etc

Cheers,
VD
 
VenomousDesigns said:
Questions:
As I'm new to the whole RAID thing I had a few quick questions;

1. Is it going to be fine that the HDDs I already have (not including the ones I will need for RAID parity) already have data on them? I obviously don't want to risk plugging all 18xHDDs in just to find out I've lost everything.

2. Future HDD expansion (a given), I've heard horror stories of RAID expansion times - is this another advantage in the column of splitting ones RAID array into separate smaller arrays?

3. Is it a good option to have a Server double as a HTPC? Or better to move to your desired device - aka PC connected to TV rather than hooking up the Server, Laptop or w/e? Does it affect what OS I put on it etc
Click to expand...

1. Unfortuantely you will lose all that data if you add it to a RAID array. Most if not all RAID controllers will reformat the drives before use. So you're gonna have to A) buy more drives to setup the initial RAID array, copy data to that RAID array from one hard drive, then add that hard drive to the RAID array, and repeat until all the drives in a a RAID array or B) Find a way to backup that data

2. Yeah that's one advantage. However there's a possibility of the other RAID arrays being affected by the heavy toll a rebuild takes.

3. Not a good idea at all. It's a general consensus now that you should not combine the the server and HTPC:
http://hardforum.com/showthread.php?t=1419906

Hardware wise, it might be more cost effective to buy two Norco 4020 or 4220 cases than to fill up that Lian Li with 5in3 hot-swap bays. Connect the second 4020 or 4220 as a DAS to the server. Example of what I mean here:
http://www.servethehome.com/category/the-big-whs-30-drive-whs/

Think about it: It'll cost roughly $1100 for that Lian Li case + all the 5in3 hot-swap bays which will only net you 30 hot-swappable drives in return. It'll only cost roughly $600 to $700 to get two of the Norco 4020 or 4220 cases which will get you 40 hot-swappable bays.
 
Don't get the Lian Li case. As suggested above, the Norcos are much better for storage and cheaper (the Lian Li is cramped as hell and a pain to work with actually).
 
VenomousDesigns said:
As I'm new to the whole RAID thing I had a few quick questions;

1. Is it going to be fine that the HDDs I already have (not including the ones I will need for RAID parity) already have data on them? I obviously don't want to risk plugging all 18xHDDs in just to find out I've lost everything.
Click to expand...

Just so you know all options, with Flex-RAID you don't have to lose the current data on your drives to get them going on Flex. It's very "flex"-ible, hence the name.

It's at least worth it to consider the option.

http://hardforum.com/showpost.php?p=1035978811&postcount=277
http://hardforum.com/showthread.php?t=1512972

As you can see, Treadstone built a 100TB server, and is running Flex-RAID on it.

http://hardforum.com/showpost.php?p=1035975129&postcount=264
http://hardforum.com/showpost.php?p=1035963060&postcount=243
http://hardforum.com/showpost.php?p=1035963728&postcount=245

It's also good for lower power consumption. He knocked his 1100W server down to 200W with Flex.

(Where I live that would equate to almost a $72/m savings! You can buy an extra 1.5TB every month with that savings, and immediately add that drive every month to your Flex array to expand your storage!)
 
flex-raid or some unraid server is definately a good idea.

you could also look into WHS. V2 should be out early 2011, i would guess and it would allow you to add 1 drive at a time and add the data to the storage pool, format the drive and then add it to the storage pool and continue until all 18+ drives are done.

i love WHS, i am running Vail ATM but it is still in beta and do not recommend it if you do not have complete backups of all data...

the Norco 4020 or 4220 are nice cases... loud and large, but that comes with the territory. I own a 4020 with 2 Supermicro MV8 SAS cards and about 6TB of storage. the extra room to expand is nice to have, though you starting with 18 is a little crazy
 
Tolyngee said:
Just so you know all options, with Flex-RAID you don't have to lose the current data on your drives to get them going on Flex. It's very "flex"-ible, hence the name.

It's at least worth it to consider the option.

http://hardforum.com/showpost.php?p=1035978811&postcount=277
http://hardforum.com/showthread.php?t=1512972

As you can see, Treadstone built a 100TB server, and is running Flex-RAID on it.

http://hardforum.com/showpost.php?p=1035975129&postcount=264
http://hardforum.com/showpost.php?p=1035963060&postcount=243
http://hardforum.com/showpost.php?p=1035963728&postcount=245

It's also good for lower power consumption. He knocked his 1100W server down to 200W with Flex.

(Where I live that would equate to almost a $72/m savings! You can buy an extra 1.5TB every month with that savings, and immediately add that drive every month to your Flex array to expand your storage!)
Click to expand...

Is Flex-raid under active developement again yet? From what I've heard, the original author hasn't been heard from for a while. Also, treadstone found a problem with shares being seen on Windows 7 machines if I'm not mistaken. Has this been resolved yet? I would love to play with Flex-raid, but at this point I don't have a lot of comfidence in it.
 
epimetheus said:
Is Flex-raid under active developement again yet? From what I've heard, the original author hasn't been heard from for a while. Also, treadstone found a problem with shares being seen on Windows 7 machines if I'm not mistaken. Has this been resolved yet? I would love to play with Flex-raid, but at this point I don't have a lot of comfidence in it.
Click to expand...

Yeah, Brahim's sorta AWOL, but I imagine he'll return.

I believe the problem was with FlexRAID-View. You don't have to use it. I'm not planning to use View.

FWIW - Flex is great where it currently is right now. If Brahim would just even release a version without the time limitations, most would be happy. But those time limits are there to force people to upgrade and support the development of Flex.

I'll admit I walked away from Flex over two years ago due to Brahim. But he returned and developed it pretty well. It would really be an awful waste if he completely walked away from it and refused to release the source.

He'll either return, or he'll release the source.

But, again: it's still another option to consider. Even unRAID's dev has gone AWOL before, from what I've read, and that's not even a free product...
 
Tolyngee said:
Yeah, Brahim's sorta AWOL, but I imagine he'll return.

I believe the problem was with FlexRAID-View. You don't have to use it. I'm not planning to use View.

FWIW - Flex is great where it currently is right now. If Brahim would just even release a version without the time limitations, most would be happy. But those time limits are there to force people to upgrade and support the development of Flex.

I'll admit I walked away from Flex over two years ago due to Brahim. But he returned and developed it pretty well. It would really be an awful waste if he completely walked away from it and refused to release the source.

He'll either return, or he'll release the source.

But, again: it's still another option to consider. Even unRAID's dev has gone AWOL before, from what I've read, and that's not even a free product...
Click to expand...

I don't mean to totally hijack the OP's thread but I am not sure I could really recommend anyone go with FlexRaid. It's a really, really cool idea that I put serious consideration into when I was building my storage server. However, I just can't see putting my data redundancy in the hands of a product worked on by ONE developer. He's a good guy, clearly smart and a good developer but still. If it wasn't something as critical as data I'd consider it but I'd much rather be using a more rock-solid solution for data redundancy that has a company behind it or at least a lot of people using it that I can turn to if I really need support. Plus, without Flex-Raid Live out it seems like a bit of a pain in the ass. Honestly, I was hoping Microsoft would just buy out a product like Flex-Raid Live and incorporate it into Vail as an alternative to drive duplication. Anyhow, just my two cents for the OP if he considers Flex-Raid.
 
pirivan said:
Plus, without Flex-Raid Live out it seems like a bit of a pain in the ass. Honestly, I was hoping Microsoft would just buy out a product like Flex-Raid Live and incorporate it into Vail as an alternative to drive duplication. Anyhow, just my two cents for the OP if he considers Flex-Raid.
Click to expand...

Funny you mention Live, as I had suggested to Brahim before to not make Live mandatory, and he said he wouldn't. I suggested to him scenarios as well where I wouldn't want to be using Live to make my point. (Personally, I don't miss not having Live...)

But like RAID-Z, Flex can also detect corrupt data and repair it. Drive duplication still doesn't offer that.

Well, the current build of Flex runs out on Sep. 23rd. So, Brahim has four weeks to show up and release a new build, or Flex risks being damaged anyway...

FWIW - everything about Flex would be solved if it had just been open source like Brahim initially said it would be. When I accepted it wouldn't be, I left the project two years ago. But when I wanted to run a file server again, I noticed how far Flex has advanced since he returned.

Then, oddly enough, I come back, and he disappears again... :confused:

Hope I'm not the jinx! :eek::D
 
Thanks for all your comments guys - been away on a recent trip and been checking back daily on the phone (couldn't bring myself to type an entire reply over a tiny on-screen keyboard haha).

Been doing alot of reading into unRaid and FlexRaid but think I've come to the conclusion if I'm going to spend this money, I'm going to to it right (now I'm not saying that either of the above solutions are indeed wrong, but having read through the various specs of each I am concerned about a few points). My main concern was with the parity and the fact with FlexRaid that unlike Hardware Raid if I lose a HDD I am able to re-build it and not lose any data, I would in fact lose an entire HDD.

From what I understand - please correct me if I am wrong - FlexRaid is similar to the old school way of Software RAID. It gives the benefit of combining all my HDDs into a singular HDD, reduces the cost of expensive Hardware RAID and allows certain HDDs to power down when not in use; But doesn't provide the benefits of if a drive (or two using RAID6/60) dies it simply cannot 'rebuild/repair' and get the data back. I'm not quiet understanding the actually difference of FlexRaid is compared to say keeping my drives singular.

-----------------------------------------------------------------

Onto hardware; taken your suggestions to heart, and indeed the Nocro 4224 or 4220 look the way to go in the way of an alternative to the more expensive Lian Li PC-343B. It was suggested that I look at two of these Norco, but from what I understand I am able to fill it with 24xHot-Swapable HDDs and then a further 24xInternal HDDs bringing the total up to a whopping 48 HDDs which is more than enough for the meantime?

Looking into RAID cards it seems that the Areca 1680x (or soon to be released 1880 series) looks to be the way to go in regards to overall performance, and compatibility (in regards to HP SAS Expanders). Although I have heard some negative reviews in response to their multiple RAID array performance.

My question here is that if I was to use an Areca 1680x (or newer 1880 series) with a HP SAS Expander, is there any lose in performance on the drives in RAID? Is it preferred to use the on-board FIRST then move to an HP SAS Expander when out? Of the 8x on-board SAS ports on the 1860, I'm expected to be able to connect 32 HDDs correct?

Had a final question;

1. I remember reading a while back (god knows where the article is now) on the comparison of HDD spin-ups/downs versus an 'always-on' Raid system. It's my belief that the most wear on a HDD is during its startup, and if I was to use a FlexRaid solution/continue using singular HDDs (Hot-Swaping when I wanted a certain Movie/File) that the life of my discs would be shorter than having a Raid solution.
 
VenomousDesigns said:
Been doing alot of reading into unRaid and FlexRaid but think I've come to the conclusion if I'm going to spend this money, I'm going to to it right (now I'm not saying that either of the above solutions are indeed wrong, but having read through the various specs of each I am concerned about a few points). My main concern was with the parity and the fact with FlexRaid that unlike Hardware Raid if I lose a HDD I am able to re-build it and not lose any data, I would in fact lose an entire HDD.

From what I understand - please correct me if I am wrong - FlexRaid is similar to the old school way of Software RAID. It gives the benefit of combining all my HDDs into a singular HDD, reduces the cost of expensive Hardware RAID and allows certain HDDs to power down when not in use; But doesn't provide the benefits of if a drive (or two using RAID6/60) dies it simply cannot 'rebuild/repair' and get the data back. I'm not quiet understanding the actually difference of FlexRaid is compared to say keeping my drives singular.
Click to expand...

It keeps X parity for X number of failed drives.

If you keep one level of parity, it can survive one DRU of loss and still repair.

You need one level of PPU for every DRU of loss to repair.

Where you're getting confused on the drive loss issue is this:

If you only have one level of parity (PPU), and you lose two DRUs, it cannot repair, but you also only lose the data on those two DRUs. You do not lose any other data.

So, let's take this scenario:

Norco, 20 2TB HDs.

RAID-5, 19 data drives, one lost to parity. So, ~38TB drive. Lose any two drives, lose the entire array, lose all 38TB.

Flex-RAID: 19 2TB DRUs, one 2TB PPU. Protection just like Raid-5 (although Flex can also detect corrupt data, RAID-5 cannot), but all DRUs are their own separate drive. So, if you lose two drives, while you do lose 2TB (assuming one was the PPU) or 4TB of data (if both are DRUs), your max data loss is only 4TB.

You still have 34TB of data available, no problem. With the RAID-5, your array has failed. All data is gone.

Restoring 4TB (or maybe only 2TB) of data from back-ups is faster than restoring 38TB.

Flex-View (or even just junctions can make all drive look like one:

http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html
 
VenomousDesigns said:
Had a final question;

1. I remember reading a while back (god knows where the article is now) on the comparison of HDD spin-ups/downs versus an 'always-on' Raid system. It's my belief that the most wear on a HDD is during its startup, and if I was to use a FlexRaid solution/continue using singular HDDs (Hot-Swaping when I wanted a certain Movie/File) that the life of my discs would be shorter than having a Raid solution.
Click to expand...

I had a file server with about 20 HDs before, cost about $40/m in electricity to run the drives 24/7.

Cost of a 1.5TB drive? $68 right now. 2TB? $92 sometimes.

So, about every two months I saved enough to replace a drive.

Guess what? I doubt having a hard drive fail every-other month. You saved.
 
Tolyngee said:
It keeps X parity for X number of failed drives.

If you keep one level of parity, it can survive one DRU of loss and still repair.

You need one level of PPU for every DRU of loss to repair.

Where you're getting confused on the drive loss issue is this:

If you only have one level of parity (PPU), and you lose two DRUs, it cannot repair, but you also only lose the data on those two DRUs. You do not lose any other data.

So, let's take this scenario:

Norco, 20 2TB HDs.

RAID-5, 19 data drives, one lost to parity. So, ~38TB drive. Lose any two drives, lose the entire array, lose all 38TB.

Flex-RAID: 19 2TB DRUs, one 2TB PPU. Protection just like Raid-5 (although Flex can also detect corrupt data, RAID-5 cannot), but all DRUs are their own separate drive. So, if you lose two drives, while you do lose 2TB (assuming one was the PPU) or 4TB of data (if both are DRUs), your max data loss is only 4TB.

You still have 34TB of data available, no problem. With the RAID-5, your array has failed. All data is gone.

Restoring 4TB (or maybe only 2TB) of data from back-ups is faster than restoring 38TB.

Flex-View (or even just junctions can make all drive look like one:

http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html
Click to expand...

Brilliant - thank you so much for that. So if I was looking at comparing to RAID6/60 (as I am) - I could infact add two (2) HDDs for Parity under FlexRaid and resemble the Parity of RAID6 - in the fact that I can lose 2xHDDs (both DRUs) and still be able to repair?

How does the FlexRaid PPU know which Data to keep in the PPU? Aka; Nocro 20TB - 18xDRUs, 2xPPUs, Loose one of the 18 - the PPU can rebuild/repair that - hows it know that fella was about to die haha.

Other than the Parity side of things, what is the basis of FlexRaid - it is effectively just like having individual HDDs with data on each correct? Just with FlexRaid it allows me to:

1. Use Flex-View (to view all HDDs within FlexRaid as if they were indeed RAIDed)
2. Provide Parity, in the sense you explained above - allows me not lose an ENTIRE RAID array as opposed to just the DRUs/PPUs.
3. Powerdown/Spindown HDDs when not in use (However this can be down without using a RAID solution, and just Hot-Swapping drives when necessary).

Also as FlexRaid isn't like RAID6 - when I want to 'expand' the RAID it would obviously be a HELL of alot quicker correct? Am I able to 'remove' a HDD from the RAID as easily - say if I wanted to take on out to use as an External HDD.



Tolyngee said:
I had a file server with about 20 HDs before, cost about $40/m in electricity to run the drives 24/7.

Cost of a 1.5TB drive? $68 right now. 2TB? $92 sometimes.

So, about every two months I saved enough to replace a drive.

Guess what? I doubt having a hard drive fail every-other month. You saved.
Click to expand...

Wasn't quiet understanding you there (I'm to tired probably haha) - you're saying with the money you save in having a RAID5/6 running 24/7 I would be able to purchase additional HDDs to expand the RAID in 1-2 months?
 
VenomousDesigns said:
Wasn't quiet understanding you there (I'm to tired probably haha) - you're saying with the money you save in having a RAID5/6 running 24/7 I would be able to purchase additional HDDs to expand the RAID in 1-2 months?
Click to expand...

From Treadstone here:

http://hardforum.com/showpost.php?p=1035963728&postcount=245

treadstone said:
My current setup consists of 48 x 2TB drives as the storage pool, 1 x 2TB drive for music storage, 1 x 2TB drive as the parity drive and the X25-M as the OS boot drive. This gives me a total of 98 TB of actual storage space. one of the major reasons I moved to this setup vs. any of the hardware based RAID setups was power consumption. My current setup allows me to have only the drive that is streaming the video to be active and all other drives are in standby. It makes a HUGE difference in the power consumption. With a RAID 50 or 60 setup the system would consume in excess of 1100W! now it is down to below 350W and once I get around to modify the fan setup, I should be below 200W.
Click to expand...

He went from 1100W to 350W (and maybe even down to 200W). My electric costs me about a dime/Kwh. So, about $2.64/$0.84/$0.48 per day, or $79.20/$25.20/$14.40/month.

So, Treadstone was able to immediately save ~$54.00/month USD (assuming he lives where I live). With some work, he can even bring that down another ~$10/month.

$54/month in savings can offset the cost of replacing drives. If you're losing a drive every two months, I'd probably consider going with another model/manuf.

Also, keep in mind Treadstone's numbers are only the computer parts at the wall: it does not include the cost of cooling that computer. Having to run an A/C for my 20-drive (or whatever it was) configuration was part of the $40/month I was burning.

IMO Flex is something to consider.
 
pjkenned said:
Just to give you an idea with 20 drives here is a pretty simple RAID 5 v. RAID 6 v. RAID 50 v. RAID 60 failure chance comparison.

Bottom line, run like heck from single parity with 20 drives. RAID 6 is much better (even moreso with UBEs in the mix which are not in those graphs). Two independent RAID 6 arrays would be a best bet.
Click to expand...

The OP was comparing to RAID 6 or 60.

But, just to state, for Flex running two arrays of 8 DRUS each with 2 PPUs each is not the same as running one single array of 16 DRUS with 4 PPUs.

One single array of 16 DRUs with 4 PPUs allows any of your four DRUs to fail before you lose any data.

The above example is even better than the mentioned RAID-Z3 offered in ZFS that's mentioned in your link.

With RAID-Z3, you can lose any of your three disks. Even RAID-60 can't say this. Lose three disks on one of the individual RAID-6 arrays, and you've lost it.

But with 16 DRUS and 4 PPUs, you can lose 4 DRUs.

So, with Flex, you can lose the same number of disks to parity as you do with RAID-60 (16 data/DRU disks, four parity/PRU disks in a 20-disk config), but can survive the failure of any four drives, which neither RAID-Z3 nor RAID-60 can offer.

But, again, unlike that link, with Flex you do not lose the entire array ever, unless all DRUs have failed. But no level of RAID will help you there.

(Actually, that statement above isn't quite accurate. If you had equal or more PPUs as DRUs, you can lose all data DRUS disks and still not lose anything, as the PPUs can rebuild it all... so if you want to be paranoid and run 10 DRUs with 10 PPUs, you can have half of your disks fail, and it can be any of those 10 disks that fail, too. So, this is even better protection than a RAID-10, where any disk and its mirror fail, and the array now fails. A RAID-10 of 20 disks can lose up to 10 disks, but can also fail with as few as two disk failures. Flex would still need more than 10 disks to fail before there's any data loss.)
 
Tolyngee said:
But, just to state, for Flex running two arrays of 8 DRUS each with 2 PPUs each is not the same as running one single array of 16 DRUS with 4 PPUs.

One single array of 16 DRUs with 4 PPUs allows any of your four DRUs to fail before you lose any data.
Click to expand...

Are you saying the Flex RAID can run without data loss in an array that has had 4 drives fail? Something that ZFS would call RAID-Z4 if it existed?

If that is what you are saying, can you provide a reference? The reason I am skeptical is that any RAID level beyond RAID 5 actually requires a more complicated algorithm than just parity, even though it is sometimes informally called parity. In fact, RAID-6 uses a form of error correcting code (ECC), most commonly a Reed-Solomon code. RAID-Z3 also uses a Reed-Solomon code

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/vdev_raidz.c

but apparently the algorithm is difficult to extend beyond triple parity.
 
FlexRAID is not the same and isn't intended on replacing a hardware based RAID system. The Areca RAID cards do a block based RAID vs. FlexRAID is a file based RAID setup. Performance isn't bad. The major difference between normal RAID systems and FlexRAID is that on FlexRAID you either do a parity generation on request or you can schedule it. Regular RAID systems generate the parity on the fly while the data gets stored on the array. In my case, since I am basically just storing a backup copy of my movies, if I have a bad drive, I can replace the drive and put the data back on it (although it will take some time to re-rip the movies). Now with the help of FlexRAID, when I add a movie to the collection, I can run the parity generation afterward. In case a drive fails, I replace the drive and restore the data with the help of the FlexRAID generated parity data. If a drive fails before I had a chance to run the parity resynchronization, I only loose the data that was not included since the last parity resynchronization, which would mean the last movie I added is lost so I only need to re-rip that one movie. If the parity drive should fail, I can simply replace the parity drive and run the create parity task again.
Click to expand...

From what I'm gathering from the above - whenever one adds a new HDD to the FlexRaid array - makes changes or anything - he/she should run the Parity Generation tool. Now I'm still having a tough time understanding how FlexRaid works with PPUs - how does it REBUILD something that is not either on at the time of crash or sharing data across all DRUs...

FlexRaid 20TB (18x1TB HDDs [DRUs] + 2xPRUs) - 2 DRUs fail, I am able to rebuild due to my 2xPRU - how does it KNOW haha - I mean I'm sure it works, but just can't fathom how it works in a non-RAID5/6 environment.
 
john4200 said:
Are you saying the Flex RAID can run without data loss in an array that has had 4 drives fail? Something that ZFS would call RAID-Z4 if it existed?

If that is what you are saying, can you provide a reference? The reason I am skeptical is that any RAID level beyond RAID 5 actually requires a more complicated algorithm than just parity, even though it is sometimes informally called parity. In fact, RAID-6 uses a form of error correcting code (ECC), most commonly a Reed-Solomon code. RAID-Z3 also uses a Reed-Solomon code

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/vdev_raidz.c

but apparently the algorithm is difficult to extend beyond triple parity.
Click to expand...

http://hardforum.com/showthread.php?t=1376886&page=4

Brahim is on here as Spectrumbx.

He's calling it RAID∞.

The website's here:

http://www.openegg.org/FlexRAID.curi
 
VenomousDesigns said:
From what I'm gathering from the above - whenever one adds a new HDD to the FlexRaid array - makes changes or anything - he/she should run the Parity Generation tool. Now I'm still having a tough time understanding how FlexRaid works with PPUs - how does it REBUILD something that is not either on at the time of crash or sharing data across all DRUs...

FlexRaid 20TB (18x1TB HDDs [DRUs] + 2xPRUs) - 2 DRUs fail, I am able to rebuild due to my 2xPRU - how does it KNOW haha - I mean I'm sure it works, but just can't fathom how it works in a non-RAID5/6 environment.
Click to expand...

The concept works exactly the same.

If the data was all static on a RAID-5/6, and the data was not stripped across all of the drives, then the only drive that would need to be running is the drive that has the data you are accessing. Only during a re-sync/repair do the PPUs need to be on.

http://en.wikipedia.org/wiki/File:RAID_4.svg

Look at the RAID-4 example of a dedicated parity drive. That parity drive is only ever access either when there's a write and it has to re-calculate the parity, or if there's a rebuild and that parity is needed.

Otherwise, it's not accessed, and it actually could be shut off.

That's what Flex-RAID does. For reading, each disk is its own disk. Only the DRUs containing the data your are accessing is used. All others can be shut off if desired.

The PPUs aren't accessed at all unless there's a build/re-sync/repair operation.

Currently Flex is a snapshot RAID, but LIVE! will change that. But even live will only access drives as needed; it'll just automatically be doing the re-sync operation, just like a hardware RAID.

There are advantages though to using a snapshot RAID vs. a LIVE! RAID if you're paranoid and are doing a RAID expansion/contraction. IMO Flex-RAID Basic gives better control for paranoids. For this reason, I've asked Brahim to never drop Basic altogether, which last I knew he has no intention to ever complete drop.

FWIW - I'm actually not sure I'll ever use LIVE! or View. Basic's fine for me, as I plan to keep my data static on my file server. I just desire flexibility with redundancy, which Flex offers.

(I've actually asked Brahim back two years ago to actually include a Flex-RAID level ZERO... Where it doesn't create a PPU, but does still create the metadata about your data, thus it's not fault tolerant, just like a RAID-0, but CAN detect data corruption on your drives... So, if everything's backed-up (no RAID is a back-up), it can tell you what you need to restore. But I have my doubts we'll see a Flex-RAID-0.)
 
Tolyngee said:
http://hardforum.com/showthread.php?t=1376886&page=4

Brahim is on here as Spectrumbx.

He's calling it RAID∞.

The website's here:

http://www.openegg.org/FlexRAID.curi
Click to expand...

I read through all of that, and I find it quite vague whether he has actually implemented T3 or T4 or higher. The only levels I saw specifically mentioned as implemented were RAID 5 and RAID 6.

Have you actually tried the 4 "parity" level? Have you checksummed a file, simulated 4 failures, regenerated the data, and then seen if the file checksum is intact?
 
john4200 said:
I read through all of that, and I find it quite vague whether he has actually implemented T3 or T4 or higher. The only levels I saw specifically mentioned as implemented were RAID 5 and RAID 6.

Have you actually tried the 4 "parity" level? Have you checksummed a file, simulated 4 failures, regenerated the data, and then seen if the file checksum is intact?
Click to expand...

Apparently I might be ahead of myself and it's not implemented yet. (Never tried using it, don't need that level of redundancy myself.)

I figured the concept works as simple as Quickpar (or PAR2), and that can continue and go beyond even 100% parity of the data (where you basically are now calculating parity of the parity or something), just the author never saw a reason to put that in. You just continue calculating the parity against the original data and the previous parity?

(I'm not a programmer, but I thought the simplest way of doing this was just continuing the XORing calculations of it all... I do know PAR3 was going to do something well advanced of that though to make the disks the bottleneck and not the CPU anymore, but then the author sort of abandoned it. Notes on it can be found at Quickpar's website.)

I take it Brahim's attempting something more advanced than I am understanding? But if you've already created parity based on a series of data, why can't it just continue on doing parity calculations as Quickpar's PAR2 does?

That's my understanding of the concept Flex-RAID was going to use. Perhaps I'm way off.

Hmmm, again, not a programmer, but while I just did the math of how a RAID-5 works, I just noticed that I can't even get this method to work for RAID-6. But I do know PAR2 and Winrar's recovery records work, and with PAR2 all you need is 100% parity to rebuild all of your data.

Without being a programmer, I was under the impression RAID∞ works under the same concept as PAR2. Guess I'm way off.

Are you then suggesting something about Brahim's claim then? I do know the equivalent of RAIDs 5 & 6 are working, and that the engine for RAID∞ is at minimum in the works.

If you're suggesting this cannot work, then how can PAR2 re-create all of your data based on the "parity" data (whatever it may actually be), assuming you had 100% "parity." I do know PAR2's parity data is actually larger than your original data (so it's holding extra calculations), and Flex has metadata about your data, so it's keeping extra data as well. Without being a programmer, I thought the concept was similar.

You may want to jump on the Flex forums then. I am not aware anyone there's doubting Brahim.
 
I was not saying it could not be done, only that it is difficult. The1997 James Plank paper, on which these Reed-Solomon algorithms are based, had an error that was not corrected until 6 years later! That gives you an idea of the difficulties involved.

Adam Leventhal, who implemented RAID-Z3 in ZFS mentioned that he wanted to do an arbitrary N+M implementation, but that optimizing it would be tricky.

You are right that PAR2 uses a similar Reed-Solomon coding that sounds like it can be arbitrarily extended to N+M. But I wonder how fast it is. Maybe it would work for the snapshot parity in Flex Raid, but I wonder if it is fast enough for live parity?

But remember that we are using the word parity loosely here. It is not simple as you first assumed. Double or higher "parity" is a lot more complicated than doing a few XORs. Read up on Reed-Solomon coding if you want the details.

By the way, I am not sure what you mean by "100% parity". RAID-Z4, if implemented, would have N data blocks and M "parity" blocks, where M=4 for RAID-Z4. It would presumably need at least N+M=8, in which case there are equal number of data and parity blocks. But it would also work for N > M, such as 16 data blocks and 4 parity blocks.
 
Last edited:
You must log in or register to reply here.
Back
Top