What make zfs a better choice for yourself?
Oh and JoeComp you can cool down, I get what he is saying. I also like to hear the pro and cons of both instead of just following what people say to do.
![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
Best Storage Solution?
- Thread starter HICKFARM
- Start date
What make zfs a better choice for yourself?
Oh and JoeComp you can cool down, I get what he is saying. I also like to hear the pro and cons of both instead of just following what people say to do.
Rectal Prolapse
Gawd
- Joined
- Jan 19, 2007
- Messages
- 575
JoeComp: Haha - but then I would have to pay royalties! 
Last edited:
Except that is not what he wrote. If you want to take things literally, you should get the quote right and pay attention to every word. That is literally what it means to take something literally.
Except that is not what he wrote. If you want to take things literally, you should get the quote right and pay attention to every word. That is literally what it means to take something literally.
Right, I realized it when you mentioned it previously. Doing it online, while the scenario I proposed would be doing it offline.
That's not OCE. That's a headache. And while you're deleting, recreating and restoring the zpool, you're effectively running without a backup - you're putting your data on the back foot as the backup copy becomes the only copy, vulnerable to further data loss until the restore process completes - which could be days or weeks depending on the size of your dataset. Certainly very time and effort intensive when all you want to do is add one more disk for more space. At that point you're no longer riding the horse, the horse is riding you.
Perspective: I currently have 20 x 4TB data drives, protected by 4 x 4TB disks set as parity (technically its 12+2 and 8+2, but let's say 20+4 for sake of simplicity). If I want to add another data drive:
SnapRAID: Add additional 4TB disk to line in snapraid config file, add drive to pool in Stablebit DrivePool. Completion time: 30 seconds
ZFS: Make sure backup is up to date, delete zpool, recreate zpool with additional 4TB disk, copy 65TB from backup to new pool, run a second verify pass on all 65TB to make sure data was written correctly and CRC matches backup. Completion time: assuming gigabit ethernet in and out of ZFS server -- Days? Weeks?
I realize most people aren't running data sets that big, but it gives an idea of the OCE headache with ZFS the larger you scale, unless ofcourse you want to put money into additional parity disks each time you add a vdev and expand that way. Meaning if you want to expand a raidz2 by one drive and maintain double parity, you end up having to buy 4 drives to do it. And once again, I'm talking within the context of home media storage. Business/enterprise is a different beast with different requirements.
Last edited:
That's my whole question too. No one yet has really given any good reasons/examples as to why zfs is the right choice. Other than it just is. It has some nice features most of which are also available in some way with other non-standard raids or software.
Every example with zfs involves so much more work/time/effort that it hardly seems worth it. And with zfs and it's version of standard raid, lose more than your fault tolerance and lose it all is what I mean, that it doesn't seem like a good choice either.
What reasons make zfs better? Don't just say because it can do this or that and snapraid/flexraid/other raid can't. Why is this or that even a good feature or useful? Why or how does it make anything easier/better. How does zfs make the server easier to setup and maintain?
No one has really addressed anything directly. Just that they use it and it works for them. No good reasons, it just does. They don't mind the long amounts of time to backup/restore/maintain the server. The long rebuild times when adding drives or doing other operations is not an issue. They have no trouble setting it up and using it and don't mind the chances with data loss of a more standard raid solution.
It's all just so the right choice without really giving details of why it's the right choice. Why not? What is it that makes those things I see as downsides and unwanted so worth it?
I think if the pro zfs posts would actually answer these things this conversation would go so much better. Because all I've gotten so far is that zfs is better because someone said so.
To be fair, I'd say that ZFS is not a terrible choice for home media serving, but it seems less than ideal for voluminous data like terabytes of music/movies/photos that changes infrequently, given the inherent advantages and malleability of snapshot based parity which in my opinion is much better suited for this usage scenario.
With striping based RAID systems you're sacrificing some configuration change flexibility for gains in performance and 24/7 uptime -- the latter of which aren't as important in a home setting when you're doing, say, serving videos over GigE.
Here's the bottom line for me, and what makes me sleep better ever since moving from striping based parity (HW RAID) to JBOD+Parity: The removal of the domino effect (aka "losing everything") in the event of a multi-disk failure scenario. I dont need the performance multiplier of striping, and lack of interdependency between disks is a huge advantage. Having a bunch of independent, plain NTFS formatted disks - protected by parity disks that are also independent - is everything for me. This distinction has made the difference in not needing to buy even more drives to cover every data drive 1:1 with a backup drive, since a lot of the data I already own on optical media. Meaning if more drives were to fail than I have parity drives to cover, I'm only stuck restoring one or a few dead disks worth of data from optical - and NOT the entire array.
Last edited:
My specific setup does not, Windows Storage Pools does have options for Parity, but I needed Storage over Parity currently. I also backup the stuff I do not want to loose to external HD and Tape Drives just in case.
TeeJayHoward
Limpness Supreme
- Joined
- Feb 8, 2005
- Messages
- 12,272
These are the reasons I chose ZFS over more common file systems:
Integrity - It protects against bit-rot. This is becoming more common, but when I chose ZFS, it was practically the only filesystem which did. This, along with all the more common features, was a biggie for me. I've got TB of data that would be a royal pain in the arse to recreate. I want it safe. This is also the reason I've got two basically identical systems. I use raidz because restoring the data takes about 3 days... Replacing a disk takes a lot less time.
Availability - My information is available regardless of hardware. I can pull the disks out and stick them in a brand new system, or a 10-year old system, and know that I can access my data.
Performance - In terms of file system performance, it was superior to any other option when I made my NAS. It's practically as fast as a $1000 RAID card, but doesn't cost a dime.
Cost - It's free, and it doesn't require hardware. I can use a cheap controller instead of an expensive RAID card.
I'm sure it's not for everyone, but for most home users it's a godsend. Other filesystems are catching up, and there are some features out there that I think ZFS needs to include, but right now, it's still got the best overall featureset.
Given that there are no advantages to ZFS RAIDZx for a home media fileserver, and a number of disadvantages, I stand by the fact that ZFS RAIDZx is a terrible choice for a home media fileserver.
Having high sequential read/write speeds is not an advantage for a home media fileserver -- there is no home media fileserving application that needs sequential speeds of several hundred MB/s. As for uptime, the most likely failure would be to have a single data drive fail, which with SnapRAID would make the data on that particular drive inaccessible during the hours it takes to restore the data. However, given that we are talking about a home media fileserver, being unable to access one of multiple data drives for a few hours is not an issue. So again, continuous 100% data availability of ZFS RAIDZx is not an advantage relevant to a home media fileserver.
ZFS is a great choice for many applications (mostly commercial), where those advantages (and others) make SnapRAID a terrible choice. But for a home media fileserver, ZFS RAIDZx is a terrible choice.
That is not the question he meant to ask. In context, it was fairly obvious he was asking why would someone choose ZFS RAIDZx over snapshot RAID (SnapRAID or FlexRAID) for a home media fileserver.
It depends very much on how much of a media server your box really is. I elected to run ZFS because of its superior read performance, its ability to snapshot without causing a huge performance penalty (on-the-fly) and its mere flexibility in expanding the pool. No doubt, I use it also as a file server on top of a media server which makes the choice obvious.
I personally would not recommend SnapRAID not because it is not tried and tested, but it merely appears to be "clumsy", and having to use another application to pool the drive together is a hassle. With hotswap bays and easy access to the RMA Center, it is much easier for me to pull a defective drive out and get an immediate replacement and allow the pool to resilver using a single command (or GUI) as opposed to having to fiddle around the SnapRAID configuration files.
Most importantly, I value uptime greatly and do not want to deal with unnecessary reconfiguration when drives fail, aside for issuing a zfs replace command with the new drive. It is more complex and time-consuming to set up, but the going concern aspect of managing is plain easier to me.
If you do not need the features of ZFS, do not have hotswap bays, have no need for comparatively higher read performance, have no desire in learning about ZFS and only intend to use it to serve media files > choose SnapRAID or the other variants that are simpler to manage.
If you want to deal with sophistication, learn about ZFS and potentially intend to use the pool as a datastore or file server > consider ZFS.
SnapRAID is built around the average consumption habits of the average user which might be more practical in your case of wanting to only buy hard drives when you run out of capacity, as opposed to purchasing 10 drives at a single go. What I'm really saying is, a ZFS configuration tend to require higher initial cash out flows.
No, it does not. I am only talking about the part of the fileserver that is a home media fileserver. For that part, ZFS RAIDZx is a terrible choice. For non-home media fileserver parts, ZFS RAIDZx could be an excellent choice, depending on the application, but that is not what we are discussing here.
Jesus Joe, relax. I personally don't like Microsoft products and don't want my data associated with it based upon my 20+ years experience. And because Flex and Snap Raid are dependant upon Microsoft is one reason why I didn't choose it.
Another reason I use ZFS is because I utilize the snapshot capabilities to replicate to another server in another state for disaster recovery support.
As I mentioned earlier, for over 20 years I've been in the storage industry, dealing almost exclusively with file systems, backup and recovery and disaster recovery. My ZFS system has been operational for over 3 years without a hiccup. Doesn't sound like a "terrible" choice to me.
Another reason I use ZFS is because I utilize the snapshot capabilities to replicate to another server in another state for disaster recovery support.
As I mentioned earlier, for over 20 years I've been in the storage industry, dealing almost exclusively with file systems, backup and recovery and disaster recovery. My ZFS system has been operational for over 3 years without a hiccup. Doesn't sound like a "terrible" choice to me.
Joe, you need to take a chill pill. the OP (or moderators) should be posting comments regarding what is relevant to the discussion or not, and afraid that is not you.
And are you proposing to run SnapRAID for the media file server portion and ZFS on the other?
drescherjm
[H]F Junkie
- Joined
- Nov 19, 2008
- Messages
- 14,941
snapraid is not dependent on microsoft. I mean it works on linux. I am at the moment considering putting snapraid on top of zfsonlinux (no redundancy) for the linux based htpc. I do not need pooling with MythTV.
Last edited:
No matter how many times I write it, some people seem to still not read what I wrote. I do not know how to write it any more clearly:
The fact is that for a home media fileserver, ZFS RAIDZx is a terrible choice.
Right. In addition to Windows and linux, SnapRAID can run under Mac OS X, OpenIndiana, Solaris, or BSD. The underlying filesystem for the data drives can be anything readable by the OS.
You are simply a troll.
False.
I am posting facts and correct information.
You are posting incorrect information.
3 years ago it was.
MrGuvernment
Fully [H]
- Joined
- Aug 3, 2004
- Messages
- 21,838
Lots of post, but just to say.,. DO NOT USE raid 5., ever, it is dead and VERY VERY useless for todays massive harddrives, your asking for data loss.
Raid 6 / Raid 10 min for todays drives and even those need backups!
ZFS is fine and other type if your using registered memory, unless eventually you want to loose most if not all of your data.
Raid 6 / Raid 10 min for todays drives and even those need backups!
ZFS is fine and other type if your using registered memory, unless eventually you want to loose most if not all of your data.
Incorrect? 3 years + without a problem. Damn, that solution is terrible.
False.
drescherjm
[H]F Junkie
- Joined
- Nov 19, 2008
- Messages
- 14,941
On the subject of snapraid. I see on Feb 24th the developer of snapraid has submitted patches to the linux kernel to implement up to 6 levels of parity in the linux kernel and btrfs
https://lkml.org/lkml/2014/2/24/555
To me this is very interesting for work (although I have recently migrated 1/2 of my 70TB at work to zfsonlinux)
https://lkml.org/lkml/2014/2/24/555
To me this is very interesting for work (although I have recently migrated 1/2 of my 70TB at work to zfsonlinux)
SnapRAID is tried and tested. It's been out several years, not as long as zfs true. But it has advanced much in those years and is quite stable. I see nothing "clumsy" about it. It's an easy setup. Which drive is parity? Which drives are data? Once those are defined it provides redundancy for the data and you're done. How is that a clumsy piece of software?
True it doesn't pool your drives. How does that make it clumsy? How much of a hassle is a single fstab entry to pool those drives together? Besides, SnapRAID isn't a drive pooling solution, it's redundancy. Apples and oranges there buddy.
SnapRAID also doesn't seem to have any performance hit if you update the parity while files are being accessed. I've never seen any issue while doing that.
With hot swap bays and easy access to the RMA Center all you need to do with SnapRAID is replace the drive and use a single command to restore your files through it's parity. No SnapRAID config files to fiddle with to do that. The only time any SnapRAID config file needs fiddled with is when you setup SnapRAID. You define the drives in that file. Done. Adding a new drive? Add an entry to the SnapRAID config. Done. Is that really hard to do? Is that such a hassle? A simple, single config entry for the drive. But if a single entry for a pool is a hassle for you then this would be too I guess.
I don't see dealing with zfs to be any kind of sophistication. It's different. Has some more features. It requires more setup and time to use. But sophisticated? Not the word I'd choose. But I see where you're going with that based on this and you other posts. Insulting us by implying we're of lower intelligence for not choosing zfs.
That SnapRAID is for the average user doing average things with their average habits. SnapRAID is for the poor who can't afford to buy 10 drives at a single go. That's your point right? Your intellectual and financial superiority to the other users of this forum? That's how your posts are reading anyhow. Not pro zfs with a great argument of it's better features but pro you and zfs is better because you say so.
What's wrong with buying drives as you need them? I see that as a big benefit for anyone even if you can buy 10 drives at a single go to not. Why put in more drives than you need? Buy what you need, add more as you need. If you only need 5 drives then why buy 10? When you need a 6th or 7th buy them. Prices will be lower by then and you'll save money. Not a dumb thing for anyone to do regardless if they can buy 10 drives at a single go to not. Also when adding a new drive takes a single little config entry and then a sync command to add it to the parity. What's so complicated about that? What's such a hassle about that?
Anyhow, this is what I get from your posts. Not that zfs is in any way a better choice. Not that you really gave much of an argument to convince anyone. So far superior read performance is about all you've said that is in favor of zfs and that could be highly debated based on other factors. OS, RAM, drive speed, network and so on. The rest has been insults to us readers and inaccurate info and insults about the alternatives.
Last edited:
brutalizer
[H]ard|Gawd
- Joined
- Oct 23, 2010
- Messages
- 1,602
First of all, there is no research on snapraid/flexraid/unraid/etc how safe they are against data corruption. There is research by CERN and others (big Enterprise players as NetApp) showing that it is very difficult to get a good data corruption protection. There are independent research from several professors in comp sci that shows ZFS seems to be safe in several research papers. There is research that shows NTFS being unsafe. And you can not just add checksums and hope they will give data integrity. Hard disks have lot of checksums and error correcting codes etc on their surface, and still they get data corruption. CERN says so too; you can not just add checksums - it must be designed right.
It is like cryptography - it is very difficult to get a safe solution, everything must be safe, the math, the algorithm, the OS, etc. Same with storage, you need a solution that controls everything. Research shows that if you have separate layers, filesystem / raidsystem / volume manager / etc then there might be data corruption when data passes through the boundaries. So if you have NTFS as filesystem, Unraid as parity, a third system as raid, hardware raid card with it's buggy firmware, etc - there are much more things that can introduce data corrutpion. ZFS controls everything: filesystem, raid, volume, etc - everything, so the data never leaves ZFS domain, so ZFS can always guarantee data integrity. ZFS controls the whole chain from RAM down to disk, without raid controllers.
Solutions that use many other layers can not guarantee data integirty - just read the research papers. Here are some research papers on data corruption and ZFS if you want to learn more:
http://en.wikipedia.org/wiki/ZFS#Data_integrity
There is a reason the big players in Enterprise storage business with storage servers costing millions of USD, has problems with data corruption. They spend years and billions of research trying to get safe solutions. If you could just add a unraid solution ontop a million dollar NetApp server - they would have done it long time ago and saved themselves all billions. Oracle, NetApp, IBM, etc - all these big storage players have 1000s of engineers trying to protect data. Amazon.com that stores huge amounts of data gets data corruption just about everywhere, all the time says an engineer. Read it. Interesting.
http://perspectives.mvdirona.com/20...ErrorsCorrectionsTrustOfDependentSystems.aspx
If someone believes some random Unraid developer can solve all these data corruption problem - I suggest you read the research before you open your mouth.
Other than data integrity, ZFS is a much better solution to me because I want a pooled storage. I want a big folder, say 10TB big or more with movies. I dont want 20 separate disks: "where was that movie again? On D: K: or P?". I want one mount point and several folders, that can grow how much I want. Not many different disks. It is hell to administer all disks and remember where I put a particular file, and which version is the newest.
There is a reason that in big Enterprise storage servers, they run ZFS. And there is a reason you find ZFS in huge Petabyte installations with 100s of disks. ZFS scales excellent, much better than unraid/flex/snap/etc. Can you imagine managing 100s of disks worth of data manually? You need an automatic system for that. You can not checksum files manually, nor remember which files are updated so you need to run checksums on those files, etc. All that have to be done automatically. And in a safe way. You will never see unraid/snapraid/flex/etc go outside small home media installations. No business will trust a home brewn solution by some random hacker. They need storage products built by 1.000s of engineers with experience of big data. You will never see a Petabyte storage server powered by NTFS and flexraid/snapraid/etc.
But sure, for a home media installation I think snapraid/unraid/flexraid/etc will suffice. But it doesnt scale beyond 10 disks or so, and definitely not beyond 100 or 1000s of disks. Which ZFS does (55 petabyte installations with lustre exist and 1Terabyte/sec). If you need to go serious, and really trust your data, and high performance, there is no question at all - ZFS it is. For small home media stuff you can choose both. The best might be to use snapraid ontop ZFS?
For my needs, flexraid/snapraid/unraid/etc - is a terrible choice. I see that marginally better than using plain NTFS. Maybe worse, because I need to update all files individually, that introduces much work for me.
It is like cryptography - it is very difficult to get a safe solution, everything must be safe, the math, the algorithm, the OS, etc. Same with storage, you need a solution that controls everything. Research shows that if you have separate layers, filesystem / raidsystem / volume manager / etc then there might be data corruption when data passes through the boundaries. So if you have NTFS as filesystem, Unraid as parity, a third system as raid, hardware raid card with it's buggy firmware, etc - there are much more things that can introduce data corrutpion. ZFS controls everything: filesystem, raid, volume, etc - everything, so the data never leaves ZFS domain, so ZFS can always guarantee data integrity. ZFS controls the whole chain from RAM down to disk, without raid controllers.
Solutions that use many other layers can not guarantee data integirty - just read the research papers. Here are some research papers on data corruption and ZFS if you want to learn more:
http://en.wikipedia.org/wiki/ZFS#Data_integrity
There is a reason the big players in Enterprise storage business with storage servers costing millions of USD, has problems with data corruption. They spend years and billions of research trying to get safe solutions. If you could just add a unraid solution ontop a million dollar NetApp server - they would have done it long time ago and saved themselves all billions. Oracle, NetApp, IBM, etc - all these big storage players have 1000s of engineers trying to protect data. Amazon.com that stores huge amounts of data gets data corruption just about everywhere, all the time says an engineer. Read it. Interesting.
http://perspectives.mvdirona.com/20...ErrorsCorrectionsTrustOfDependentSystems.aspx
If someone believes some random Unraid developer can solve all these data corruption problem - I suggest you read the research before you open your mouth.
Other than data integrity, ZFS is a much better solution to me because I want a pooled storage. I want a big folder, say 10TB big or more with movies. I dont want 20 separate disks: "where was that movie again? On D: K: or P?". I want one mount point and several folders, that can grow how much I want. Not many different disks. It is hell to administer all disks and remember where I put a particular file, and which version is the newest.
There is a reason that in big Enterprise storage servers, they run ZFS. And there is a reason you find ZFS in huge Petabyte installations with 100s of disks. ZFS scales excellent, much better than unraid/flex/snap/etc. Can you imagine managing 100s of disks worth of data manually? You need an automatic system for that. You can not checksum files manually, nor remember which files are updated so you need to run checksums on those files, etc. All that have to be done automatically. And in a safe way. You will never see unraid/snapraid/flex/etc go outside small home media installations. No business will trust a home brewn solution by some random hacker. They need storage products built by 1.000s of engineers with experience of big data. You will never see a Petabyte storage server powered by NTFS and flexraid/snapraid/etc.
But sure, for a home media installation I think snapraid/unraid/flexraid/etc will suffice. But it doesnt scale beyond 10 disks or so, and definitely not beyond 100 or 1000s of disks. Which ZFS does (55 petabyte installations with lustre exist and 1Terabyte/sec). If you need to go serious, and really trust your data, and high performance, there is no question at all - ZFS it is. For small home media stuff you can choose both. The best might be to use snapraid ontop ZFS?
For my needs, flexraid/snapraid/unraid/etc - is a terrible choice. I see that marginally better than using plain NTFS. Maybe worse, because I need to update all files individually, that introduces much work for me.
You've also totally missed the point of this thread. Home media server. Not Enterprise storage with Petabyte installations with 100s of disks. And there's more than one way to pool disks. So that fact than flexraid/snapraid/unraid/etc don't pool is not an issue. And they don't update files individually. One command and they all do. Low overhead for files that don't change on a constant basis. And I've built a snapraid with 4 parity and 30 data disks so I'm not sure where you get the 10 disk number at either.
Seems to be a recurring theme with the guy, a wall of offtopic goes up, its full of inaccuracies and he sits back and trollface.gif as everyone gets caught up arguing with him and the whole thread is derailed. Look no further than the zfs streaming thread.
Anyway pooling is beyond the scope of SnapRAID, and technically its not actually trying to compete with zfs in that regard, or anything else for that matter since its free and essentially a passion project for the developer. Technically, striping based raid isn't even disk pooling as much as it is disk combining. True pooling to me is the ability to add and remove disks at will and it doesnt impact the rest of the pool members with ripple effects.
The nice thing about pooling and parity being separate is you can choose best of breed for each, as they are modular, and in my experience Stablebit Drivepool has been flawless on Windows Server 2012R2, I have three pools of various quantities of drives, windows index service picks it up (which isn't true for some of the other competing jbod pooling software) and best of all when I go to read or write a file to or from the pool it only spins up the one disk rather than all pool members. Ive also configured it to fill up one disk at a time instead of round robin that would scatter files randomly across all disks, this all but eliminates folder splits and keeps related files together..
Last edited:
If you want Windows, any recent version of Windows, both clients (Windows 7/8/8.1) and servers (Server 2008 R2, 2012, 2012 R2) and Stablebit Drivepool will do the trick.
Drivepool is not free, but well worth the money ($20).
I'm currently running Drivepool on Server 2012 R2 with 16 disks totaling 28TB, it runs like a champ.
I was running Drivepool on my WHS 2011 box, decided to upgraded to Server 2012 R2, all I had to do was format the OS disk and install the new OS. Once I installed Drivepool, my entire pool came back instantly. It's really impressive.
Drivepool is not free, but well worth the money ($20).
I'm currently running Drivepool on Server 2012 R2 with 16 disks totaling 28TB, it runs like a champ.
I was running Drivepool on my WHS 2011 box, decided to upgraded to Server 2012 R2, all I had to do was format the OS disk and install the new OS. Once I installed Drivepool, my entire pool came back instantly. It's really impressive.
Windows Storage Spaces = I avoid it at all costs. It's simply not there yet, and I'm not sure it'll ever be there. It also suffers from the same problems as any striping based RAID where if you lose more disks than you have parity disks, you lose everything. I suppose there are some usage scenarios that it's -a- solution for, even if its not the best solution.
By and large its basically a iteration on the same Windows Dynamic Disks system that's existed forever - I'm talking about the ability to for example create a software RAID5 volume in Windows Disk Management. Only now its got a better GUI and has undergone some improvements.
They added the ability to do pseudo-tiered storage in 2012 R2, where you can assign all or part of an SSD to a storage space to cache writes, I played with that and it's interesting, but overall I just have a lot of apprehension about storage spaces.
By and large its basically a iteration on the same Windows Dynamic Disks system that's existed forever - I'm talking about the ability to for example create a software RAID5 volume in Windows Disk Management. Only now its got a better GUI and has undergone some improvements.
They added the ability to do pseudo-tiered storage in 2012 R2, where you can assign all or part of an SSD to a storage space to cache writes, I played with that and it's interesting, but overall I just have a lot of apprehension about storage spaces.
Copies=2 is worthless. Try adding setting copies=2, adding a single disk vdev, and then removing that disk. You're fucked.
Those are some wise words and I fully agree with it. Copies were implemented to mitigate read errors and not provide redundancy against disk failures.
The question was "Best Storage Solution" with focus on a home setup. While I would not say that data @ home is less valuable than @work, demands are sometimes others. The focus may be more on lowcost, expandability and low power.
Discussions like this are ending too often in comparing features of different solutions that cannot be compared as they completely targeting different problems - mainly questions that depend on filesystem features and questions that depend on raid features. Other reason of confusion is the fact that systems like ZFS combines the filesystem with a volume manager, raid, raid-management and share management where questions regarding raid can be mixed with questions regarding the filesystem.
Most importand basic of any storage solution is the filesystem that can be used on different platforms for example:
Apple OSX (HFS+, ZFS)
BSD (UFS, ZFS)
Linux (btrfs, ext, XFS, ZFS among others)
Solaris and derivates (ZFS)
Windows (ntfs, ReFS)
If you look at these filesystems, you can divide them into the "traditional" and "advanced ones" where I use the term advanced due to the two main features
-Realtime end to end checksums for data and metadata
Every file is quecksummed always and automatically and verified on every read - no undetected data error
- Copy On Write
There is no "inline update" of data. Every modified datablock is newly written. Only when the write was successful, the filesystem pointers are updated
This gives you the always consistent filesystem, no need of a fschk where old data can be kept as snaps without delay.
These two revolutionary fetaures are available on btrfs, ReFS and ZFS
I would call them mandatory for every high capacity storage system with a demand on data security.
This is completely independent from the next question: redundancy and backup
For this you should always do traditional backups (hold data on a second storage) on important files for disasters
(fire, server stolen etc) and availability (keep data intact in case of a simple disk failure).
For availabilty there are mainly two choices with the main advantages (missing on the other):
1. offline/ on demand systems like snapraid
- data are not striped over disks (performance is like singe disk)
- redundancy data is striped over disks and this is created on demand
- you can add any single disks
- disks are not reinitialized (can contain data)
- disks can be removed and read on another computer
- not used disks can sleep
- if more disks fail than the selected redundancy level only data on that disks is lost
2. realtime online/ striped systems with traditional raid (raid 5/6 or ZFS Z1-Z3)
- data and redundancy information is created always and in realtime and striped over all disks
- performance scale with number of disks
- adding single disks is not suggested without backup (raid5/6) or cannot be done
(on ZFS it is possible is a replace disks with larger disks or add another vdev to extend capacity)
- if more disks fail than the selected redundancy level (up to 3 disks on ZFS Z3), all data is lost
3. systems with snapshots adds the option of a file history or an undelete. This affects availability
but is not a raid question - more a filesystem question as this can be best done with copy on write filesystems.
Result:
This gives you two questions: What filesystem and what raid-type
Regarding the raid type, it is quite clear where they are usually used.
Offline systems like snapraid are perfect for a home mediaserver with only few changes in data because you need to run a "update redundancy" on every data modification that you want to be included in rednundancy. You are not restricted with disk usage and all unused disks can sleep. If a disk fails, you need to replace the disk and run a rebuild to read the data again.
Online systems are perfect if your focus is on performance and availability where all written data must be included in redundancy during every write or change and not on demand with a delay. If a disk fails, your data keeps available. You only need to care that no more disks fail than your redundancy level (up to 3 disks in a single raid/vdev)
Even a mix of these two raid-systems is an option. Snapraid for media data on many disks and any realtime raid up from a mirror for data that should be protected in realtime. Only variant that I would not use any longer with large capacity storage is raid-5 or Raid-Z1 as the chance of a total data loss is too high.
Discussions like this are ending too often in comparing features of different solutions that cannot be compared as they completely targeting different problems - mainly questions that depend on filesystem features and questions that depend on raid features. Other reason of confusion is the fact that systems like ZFS combines the filesystem with a volume manager, raid, raid-management and share management where questions regarding raid can be mixed with questions regarding the filesystem.
Most importand basic of any storage solution is the filesystem that can be used on different platforms for example:
Apple OSX (HFS+, ZFS)
BSD (UFS, ZFS)
Linux (btrfs, ext, XFS, ZFS among others)
Solaris and derivates (ZFS)
Windows (ntfs, ReFS)
If you look at these filesystems, you can divide them into the "traditional" and "advanced ones" where I use the term advanced due to the two main features
-Realtime end to end checksums for data and metadata
Every file is quecksummed always and automatically and verified on every read - no undetected data error
- Copy On Write
There is no "inline update" of data. Every modified datablock is newly written. Only when the write was successful, the filesystem pointers are updated
This gives you the always consistent filesystem, no need of a fschk where old data can be kept as snaps without delay.
These two revolutionary fetaures are available on btrfs, ReFS and ZFS
I would call them mandatory for every high capacity storage system with a demand on data security.
This is completely independent from the next question: redundancy and backup
For this you should always do traditional backups (hold data on a second storage) on important files for disasters
(fire, server stolen etc) and availability (keep data intact in case of a simple disk failure).
For availabilty there are mainly two choices with the main advantages (missing on the other):
1. offline/ on demand systems like snapraid
- data are not striped over disks (performance is like singe disk)
- redundancy data is striped over disks and this is created on demand
- you can add any single disks
- disks are not reinitialized (can contain data)
- disks can be removed and read on another computer
- not used disks can sleep
- if more disks fail than the selected redundancy level only data on that disks is lost
2. realtime online/ striped systems with traditional raid (raid 5/6 or ZFS Z1-Z3)
- data and redundancy information is created always and in realtime and striped over all disks
- performance scale with number of disks
- adding single disks is not suggested without backup (raid5/6) or cannot be done
(on ZFS it is possible is a replace disks with larger disks or add another vdev to extend capacity)
- if more disks fail than the selected redundancy level (up to 3 disks on ZFS Z3), all data is lost
3. systems with snapshots adds the option of a file history or an undelete. This affects availability
but is not a raid question - more a filesystem question as this can be best done with copy on write filesystems.
Result:
This gives you two questions: What filesystem and what raid-type
Regarding the raid type, it is quite clear where they are usually used.
Offline systems like snapraid are perfect for a home mediaserver with only few changes in data because you need to run a "update redundancy" on every data modification that you want to be included in rednundancy. You are not restricted with disk usage and all unused disks can sleep. If a disk fails, you need to replace the disk and run a rebuild to read the data again.
Online systems are perfect if your focus is on performance and availability where all written data must be included in redundancy during every write or change and not on demand with a delay. If a disk fails, your data keeps available. You only need to care that no more disks fail than your redundancy level (up to 3 disks in a single raid/vdev)
Even a mix of these two raid-systems is an option. Snapraid for media data on many disks and any realtime raid up from a mirror for data that should be protected in realtime. Only variant that I would not use any longer with large capacity storage is raid-5 or Raid-Z1 as the chance of a total data loss is too high.
Last edited: