![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
-
Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.
Raid 1 vs. Raid 5
- Thread starter SpongeBob
- Start date
Depends, I decided to go JBOD for my media server and forgo the raid solutions.
The Raid 1 solution while producing a great absolute backup solution for my media would of added a lot of costs too.
Raid 5 would of giving me great speed, speed I didn't need an up-time solution I didn't need either.
I have a Raid 6 solution here but not for my media. I decided to store certain files across multiple disc's instead for my backup solution for my media. I can always re-create most of my media from the disc's I have.
The Raid 1 solution while producing a great absolute backup solution for my media would of added a lot of costs too.
Raid 5 would of giving me great speed, speed I didn't need an up-time solution I didn't need either.
I have a Raid 6 solution here but not for my media. I decided to store certain files across multiple disc's instead for my backup solution for my media. I can always re-create most of my media from the disc's I have.
D
Deleted member 126051
Guest
First off, keep in mind that RAID is *not* a backup solution.
RAID is an uptime cheat (disk redundancy) and/or a performance cheat (striping of data across multiple spindles).
For RAID-1, depending on I/O load, random read performance can vary, but caps out at the combined I/O limits of all the drives in the array or the controller (whichever is lower).
For writes, RAID-1 performance caps out at the performance of a single disk in the array.
Also remember with RAID-1, your storage space is limited to the size of a single disk in the array.
With RAID-1, as long as one disk in the array remains uncorrupted, you can still access your data, albeit at a degraded rate.
For RAID-5, both read and write performance generally increase as you add more spindles to the array. Since the parity calculations are distributed across the array, you see less I/O bottlenecking than you would from an array that uses a dedicated parity disk.
Also, by striping, you wind up with more usable disk space than a RAID-1 solution (though you always "lose" the equivalent of one member disk's space for parity information).
As disk (and thus, array) sizes increase, rebuilding a failed disk is taking longer and longer. Additionally, the likelihood of encountering an unrecoverable disk error during the rebuild process increases. This is true for any RAID level where the array can be rebuilt.
The last rebuild I did was with 600GB 15K rpm disks. Rebuilding a RAID5 array took something in the neighborhood of 6-7 hours.
Now think about today's 6TB 7200 rpm drives, or worse, the drives with the unstated spindle speeds (which are generally closer to 5400 rpm). You willing to bet on a stable setup for a recovery spread over 6-10 days?
In short, if this is going to be a media SERVER, treat it as such. And do NOT use it as a backup solution. Then, depending on the importance of the content you're storing, decide whether or not you want to run with or without some sort of backup solution behind it.
RAID is an uptime cheat (disk redundancy) and/or a performance cheat (striping of data across multiple spindles).
For RAID-1, depending on I/O load, random read performance can vary, but caps out at the combined I/O limits of all the drives in the array or the controller (whichever is lower).
For writes, RAID-1 performance caps out at the performance of a single disk in the array.
Also remember with RAID-1, your storage space is limited to the size of a single disk in the array.
With RAID-1, as long as one disk in the array remains uncorrupted, you can still access your data, albeit at a degraded rate.
For RAID-5, both read and write performance generally increase as you add more spindles to the array. Since the parity calculations are distributed across the array, you see less I/O bottlenecking than you would from an array that uses a dedicated parity disk.
Also, by striping, you wind up with more usable disk space than a RAID-1 solution (though you always "lose" the equivalent of one member disk's space for parity information).
As disk (and thus, array) sizes increase, rebuilding a failed disk is taking longer and longer. Additionally, the likelihood of encountering an unrecoverable disk error during the rebuild process increases. This is true for any RAID level where the array can be rebuilt.
The last rebuild I did was with 600GB 15K rpm disks. Rebuilding a RAID5 array took something in the neighborhood of 6-7 hours.
Now think about today's 6TB 7200 rpm drives, or worse, the drives with the unstated spindle speeds (which are generally closer to 5400 rpm). You willing to bet on a stable setup for a recovery spread over 6-10 days?
In short, if this is going to be a media SERVER, treat it as such. And do NOT use it as a backup solution. Then, depending on the importance of the content you're storing, decide whether or not you want to run with or without some sort of backup solution behind it.
It does not take anywhere near that long on a modern RAID controller. Generally rebuilds go as fast as the disks as capable of performing. So, with a 6TB disk, you'd be looking at about 15 hours.
raid10
raid5 is slow with ntfs, zfs and xfs because of read-modify-write and unreliable because double faults during rebuild happen
raid6 fixes double fault issue but performance for random writes is even worse
in general
http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt
raid5 is slow with ntfs, zfs and xfs because of read-modify-write and unreliable because double faults during rebuild happen
raid6 fixes double fault issue but performance for random writes is even worse
in general
http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt
A 6tb disk is going take around 22hours, at max speed, they do not maintain top speed over 100% of the disk.
How is raid6 even worse than raid5? for performance? It's the same result, alittle extra cpu work, but that isn't going be very noticable over the delay of the seeking.
If you did zfs ontop of raid that is your own issue. You should have used raidz, and not have the issue of the read/modify/write cycles.
How is raid6 even worse than raid5? for performance? It's the same result, alittle extra cpu work, but that isn't going be very noticable over the delay of the seeking.
If you did zfs ontop of raid that is your own issue. You should have used raidz, and not have the issue of the read/modify/write cycles.
D
Deleted member 126051
Guest
I think you're looking at it as a problem of data involved divided by maximum data thruput of the disk.
Realistically, you're NOT going to see maximum thruput in more than fits and spurts.
D
Deleted member 126051
Guest
Exactly.
DeathFromBelow
Supreme [H]ardness
- Joined
- Jul 15, 2005
- Messages
- 7,315
I'm running a RAID10 array with 8x 3tb drives. Drives are cheap and I didn't want to deal with RAID5/6 performance overhead and rebuild times.
That's my desktop though. The drives are directly connected and I want maximum performance. If I were building a dedicated server I'd skip RAID altogether and go with ZFS for redundancy and data scrubbing. FreeNAS is pretty easy to set up if you want to try that.
That's my desktop though. The drives are directly connected and I want maximum performance. If I were building a dedicated server I'd skip RAID altogether and go with ZFS for redundancy and data scrubbing. FreeNAS is pretty easy to set up if you want to try that.
this is what i do as well
also i don't buy cheap dual disk synology, netgear and qnaps anymore
also i don't buy cheap dual disk synology, netgear and qnaps anymore
why raid6 is much slower then raid5? let's start with simple reads
4 disk raid5 setup (makes no sense but that's an example)
a, b, c, xor(a,b,c) = 25% of iops and bandwidth lost
4 disk raid6 setup
a, b, xor(a,b), poly(a,b) = 50% of iops and bandwidth lost
of course it gets better with more disks in a setup (raid group) but it's incorrect to say raid6 = raid5 in terms of read performance
now more complex which is write
it's not "little more cpu" it's actually atomic read and write operations standing on a "critical path" to write the data
also i have an impression you think dual parity is actually a second copy of xor of a user blocks and you can just go and update whole stripe covering all disks @ the same time
this is not not true...
write goes like this
read user data, read xor and read poly
calculate and check xor and poly to match or write triggers read recovery alert
update user block, update xor and update poly in memory
write user data
write xor
write poly <-- you don't do this step with raid5 as there's no poly and and only xor
you cannot stream user data and parity writes in parallel as incomplete writes would result no way to know what's broken and what's not so you write them sequentially with fua flag raised in scsi commands to avoid data cached by drive even if somebody had wce = on
so write penalty for raid5 is 3 to 4 (depends do you read user data and xor in parallel) and write penalty for raid6 is 4 to 6 (depends do you read user data, xor and poly in parallel)
4 disk raid5 setup (makes no sense but that's an example)
a, b, c, xor(a,b,c) = 25% of iops and bandwidth lost
4 disk raid6 setup
a, b, xor(a,b), poly(a,b) = 50% of iops and bandwidth lost
of course it gets better with more disks in a setup (raid group) but it's incorrect to say raid6 = raid5 in terms of read performance
now more complex which is write
it's not "little more cpu" it's actually atomic read and write operations standing on a "critical path" to write the data
also i have an impression you think dual parity is actually a second copy of xor of a user blocks and you can just go and update whole stripe covering all disks @ the same time
this is not not true...
write goes like this
read user data, read xor and read poly
calculate and check xor and poly to match or write triggers read recovery alert
update user block, update xor and update poly in memory
write user data
write xor
write poly <-- you don't do this step with raid5 as there's no poly and and only xor
you cannot stream user data and parity writes in parallel as incomplete writes would result no way to know what's broken and what's not so you write them sequentially with fua flag raised in scsi commands to avoid data cached by drive even if somebody had wce = on
so write penalty for raid5 is 3 to 4 (depends do you read user data and xor in parallel) and write penalty for raid6 is 4 to 6 (depends do you read user data, xor and poly in parallel)
in our case it was raidz2 actually but who cares it's still raid6 under the hood
zfs is not true log structured file system on top of a dual parity raid (think about wafl and raid4-dp and casl and raid6) so it still has read-modify-write issue and zil turned on helps to adsorb the load but not infinitely
zfs is not true log structured file system on top of a dual parity raid (think about wafl and raid4-dp and casl and raid6) so it still has read-modify-write issue and zil turned on helps to adsorb the load but not infinitely
That is invalid, this is raid5, NOT raid3.
You read reads from ALL disks, so no 25% loss of iops. (what you said would be valid for writes, but you clearly said reads)
Invalid again, this is raid6, not raid3, you get 100% read iops, not 50%
It's very simple, raid5 yes, is xor, and raid6 adds the poly.
But it isn't so many distict steps.
Calc xor, calc poly, write data + xor + poly
The reads are optional, assuming you need them, cache miss, or not writing a full stripe.
Also, you don't read xor/poly data, in raid5/6, unless you have a failed disk or two.
I don't know of anything that bothers to write data and parity on seperate operations.
As far as for how zfs and raid6 works, that is completely invalid.
zfs does not do it's log ontop of raid6, It does the raid6 stuff ontop of the log.
This is why raidz2 is completely different from raid6, in how it works, and how it reacts. It's stacked the other way around.
zfs NEVER does read-modify-write, it will only replace the entire block.
If you configure your recordsize to be huge, 128k and install a database ontop of it doing 8k records, that is your own fault. If you tell zfs your using 8k, zfs will be happy, and none of that nonsense will happen.
raid 3 does everything per byte so when you think raid with dedicated parity it's called raid 4
but it does not matter as in terms of one stripe context raid 4 and raid 5 are the same
let's take 3 disks for example
raid 4
a1 b1 xor(a1,b1)
a2 b2 xor(a2,b2)
a3 b3 xor(a3,b3)
raid 5
a1 b1 xor(a1,b2)
a2 xor(a2,c2) c2
xor(b3, c3) b3 c3
before doing small write you need to do full stripe read and with 3 disks 66% belongs to user data and 33% is xor
when doing 100% small random reads from many requestors all load split between all spindles equally and theres no degradation
streaming reads like sql log parsing, vm recovery, video playback do suffer from part of the bandwidth reserved for xor traffic
but it does not matter as in terms of one stripe context raid 4 and raid 5 are the same
let's take 3 disks for example
raid 4
a1 b1 xor(a1,b1)
a2 b2 xor(a2,b2)
a3 b3 xor(a3,b3)
raid 5
a1 b1 xor(a1,b2)
a2 xor(a2,c2) c2
xor(b3, c3) b3 c3
before doing small write you need to do full stripe read and with 3 disks 66% belongs to user data and 33% is xor
when doing 100% small random reads from many requestors all load split between all spindles equally and theres no degradation
streaming reads like sql log parsing, vm recovery, video playback do suffer from part of the bandwidth reserved for xor traffic
not different just a bit complex and code path or data flow pipeline is few steps longer
xor & poly reads maybe optional if you don't care much about tler
user data, xor and poly belong to the different luns so you cannot write them with one operation physically
no raid or file system writes data and metadata independently from each other everybody serialize for a good reason
xor & poly reads maybe optional if you don't care much about tler
user data, xor and poly belong to the different luns so you cannot write them with one operation physically
no raid or file system writes data and metadata independently from each other everybody serialize for a good reason
theres a confusion
please separate zfs on top of a hardware raid5/6 and zfs with raidz(2,3,100&)
raidz is an ordynary parity raid but it does not have fixed stripe and does dynamic striping - zfs is file system and a bit of a logical volume manager together
zfs with raidz = all writes have own xor part, no read-mofidy-write
zfs with hardware raid5/6 = writes have no xor (surprise!), read-modify-write done below file system level
please separate zfs on top of a hardware raid5/6 and zfs with raidz(2,3,100&)
raidz is an ordynary parity raid but it does not have fixed stripe and does dynamic striping - zfs is file system and a bit of a logical volume manager together
zfs with raidz = all writes have own xor part, no read-mofidy-write
zfs with hardware raid5/6 = writes have no xor (surprise!), read-modify-write done below file system level
brutalizer
[H]ard|Gawd
- Joined
- Oct 23, 2010
- Messages
- 1,603
You are wrong. ZFS does not do parity calcutions. Hardware raid do that. ZFS does checksum calculations (fletcher, SHA-256, etc). Please study ZFS more before you write stuff that are wrong.
I thought that was the point of Raid 1 to basically copy what's on one drive in case on fails. I think I'd opt to use the default windows software raid just in case the mobo craps out I can still access my files after a re-install. Or here's a scenario, if you guys want to make sure data on one drive is backed up on another what route are you guys going now? This is not an OS drive this is just a file drive. Are you guys opting for enclosures instead or what is your means of backing up?
Side note how are WD 3-4TB drives vs. Toshiba 3-4TB drives?
xor for simplicity
erasure coding
parity
polynom
...
whatever you want to call it
sha256 and fletcher2/4 are used for hash summing integrity purpose incl. in-memory dedupe tables
some private fork outs use blake32 aka sha-3 as it's much faster
erasure coding
parity
polynom
...
whatever you want to call it
sha256 and fletcher2/4 are used for hash summing integrity purpose incl. in-memory dedupe tables
some private fork outs use blake32 aka sha-3 as it's much faster
You get hit by crytolocker both drives data is encrypted what do you do?
WD usb3 3.5" external drives for backup is my current choice.
House burns down. Virus deletes your files. Controller error borks your filesystem.
RAID is not backup, and it never ever should be considered backup.
I use an online backup solution (CrashPlan) in addition to a few external drives I keep offsite (used to be at my office, now I'm using my locker at school). A handful of 4TB drives is not that expensive, and you can just back up every now and then and shove them anywhere.
The first needs a backup, the second needs a snapshot capable filesystem, the third can be treated or at least discovered by a end to end checksum capable filesystem.
Yes, you need backups but they are mostly outdated or do not cover a previous state whenever you need it. They are good for a real disaster. Mostly you need a trusted filesystem with save and read only snapshots.
- Joined
- Aug 3, 2004
- Messages
- 24,152
didnt read it all but do not use Raid 5 unless you got small ass drives or SSDs. Raid 10 for media server.
Raid 5 rebuilds take forever and as noted already 1 flipped bit and your rebuild fails... period..
raid 10 is only 1 drive more and does not parity calculations, can have upto 2 drives fail depending, better speeds on read and write and rebuilds take next to no time and you dont get the massive performance hit Raid5/6 get on when rebuilding.
Raid 5 rebuilds take forever and as noted already 1 flipped bit and your rebuild fails... period..
raid 10 is only 1 drive more and does not parity calculations, can have upto 2 drives fail depending, better speeds on read and write and rebuilds take next to no time and you dont get the massive performance hit Raid5/6 get on when rebuilding.
Last edited:
Format drives.
Robots rise against their masters, aliens attack, Godzilla strikes.
I'm really not worried about any of this it's more to do with convenience. I'm not keeping anything dire on here just if one drive fails be nice to have a backup. Why else would someone use Raid 1?
Uptime and performance. Weren't those motivations already mentioned in this thread?
I really didn't think you would see any performance benefit of Raid 1.
drescherjm
[H]F Junkie
- Joined
- Nov 19, 2008
- Messages
- 14,933
If you have a smart raid implementation you can get raid0 read performance from raid1.
D
Deleted member 126051
Guest
Yes. But that doesn't make it a "backup" solution. It's actually an uptime cheat.
Basically I'm a big fan of layered backup strategies.
System/Device (like NAS) drive for the original data. Either a mirrored device or rotating drives/tapes that can be removed from premises. And some form of off-site backup. Be it another device/server at another facility, or an online backup.
This way:
If the original device goes *POOF*, you have immediate access to a recent backup. And you can recover the few files that aren't ABSOLUTELY up to date from the online backup.
If your location goes *POOF*, you have access to recovered off-site drives, either locked in a safe-deposit box or at another facility. Again, you get the files that need to be updated from your online backup to flesh things out.
If all your localized devices go "poof", earthquake, nuke, coordinated robbery of all your facilities, whatever, you can simply reach out and grab your online backup. Get the files you need to get going immediately, and if you're using something like Backblaze, have them mail you a hard drive with the rest of your data. A the bandwidth of a postal-mailed hard drive on next-day is higher than pretty much ANY internet connection you'd be able to afford.
D
Deleted member 126051
Guest
Anything worth doing is worth doing right.
"My stuff isn't that important", until it's gone and you realize there was stuff in there that WAS important.
Why not be moderately zealous about doing your backup right, from the get go. Rather than becoming an uber-zealot later after you lose something because You Did It Wrong?
D
Deleted member 126051
Guest
Actually, you should see this with just about ANY RAID1 implementation unless something's wrong with your controller/software setup. It's usually write speed that suffers, as your MAXIMUM speed is that of a single spindle on the array, and is often considerably less.
read goes to both spinners @ the same time so you double bandwidth and reduce seek time (not in a half however)
brutalizer
[H]ard|Gawd
- Joined
- Oct 23, 2010
- Messages
- 1,603
Ok, I did not know this. I need to read more on ZFS apparently.
So, it seems that ZFS does do parity calculations, and after that, it does checksum calculations?
Do you have any information about ZFS doing parity calculations, how do you know this?
Very simple, easy to read, right in the source.
https://github.com/joyent/illumos-joyent/blob/master/usr/src/uts/common/fs/zfs/vdev_raidz.c
https://github.com/joyent/illumos-joyent/blob/master/usr/src/uts/common/fs/zfs/vdev_raidz.c
Ok, I did not know this. I need to read more on ZFS apparently.
So, it seems that ZFS does do parity calculations, and after that, it does checksum calculations?
Do you have any information about ZFS doing parity calculations, how do you know this?
How on earth did you think ZFS rebuilds the data when a random device fails in a RAIDZ configuration?
Parity is the ONLY way.
brutalizer
[H]ard|Gawd
- Joined
- Oct 23, 2010
- Messages
- 1,603
Oh yes, sorry, of course you are correct. I know that ZFS does parity calculations. I meant "XOR", not parity. I mistakenly switched those in my head.
Checksums are done at a higher level.
Data -> split into blocks -> apply checksum/metadata -> map blocks to vdevs (add parity data if needed for raidz) -> write data to each disk.
compression and dedup also live at the checksum level.
Should use a full zfs diagram but can't remember where one exists at the moment.
Data -> split into blocks -> apply checksum/metadata -> map blocks to vdevs (add parity data if needed for raidz) -> write data to each disk.
compression and dedup also live at the checksum level.
Should use a full zfs diagram but can't remember where one exists at the moment.