Follow along with the video below to see how to install our site as a web app on your home screen.
Note: This feature may not be available in some browsers.
True. So far the implementation in FreeBSD 8 is working very well for me.If you care so much about your data to use ZFS, why would you then trust it to a barely maintained fuse plugin?
True. So far the implementation in FreeBSD 8 is working very well for me.
Let's tell you guys a bit about my setup, and perhaps you'll understand my love for ZFS:
I have 5 workstations running Ubuntu Linux, but none of them have internal harddrives. They all use iSCSI-on-root, meaning they mount the system drive over the network, and also boot from the network. That works with the motherboard's NIC just fine, no special hardware needed.
The server runs FreeBSD 8 and of course ZFS. I actually have two servers with the second being passive and offline for the most part; every night it switches on to create a snapshot, sync all ZFS filesystems with the master fileserver and shutdown again. This is a very safe solution in my eyes.
So back to iSCSI, the system disks of my 5 workstations reside on RAID-Z with HDDs, so read latency is normally quite high. But by adding a RAID0 of two small SSDs as cache device, the portions that are most accessed will stay on the SSD and have read latencies from the SSD (very low). It's like having RAM filecache, but then my 2x30GB SSDs have much more storage than my system's RAM (8GB).
So essentially, all my system drives run on the SSDs on the FreeBSD server. Writes are buffered anyway so have no latency. The SSDs are not very high speed, but the low read latency are still a huge improvement over responsiveness. The cache device is mostly read-only, and only gets updated as my usage pattern changes. So that also means it automatically adapts to my usage pattern. The system drives on the workstations are only 8GB large, and mount the larger (shared) filesystem through NFS.
So far, i'm very happy with this setup. ZFS can correct any corruption, i have snapshots so can always go back to an earlier date. And, paranoid as i am, i also have a complete mirrored slave fileserver as true backup, only without the SSDs.
I would very much like an update to 10Gbps though, as this bandwidth limit now also applies to the system drive. But affordable products that are supported by FreeBSD would probably take awhile, although a bunch of 10Gbps adapters are already supported, none of those use the 10GBaseT ("copper") standard. Still i hope 2010 will see some cheaper products implementing 10GBaseT, which works on my existing Cat6 cabling.
Still doesn't help RAID5. Still doesn't help with RAM bit flips either, as the hardware RAID controller assumes the data that it's been given is valid.All the hardware RAID controllers have an option to do scheduled scans of the stripe integrity. It is not necessary to check on every read, if you do a scan, for example, once a week.
Yes, they are on the RAID-Z array as zvol - so no ZFS filesystem just a zpool (the RAID-part of ZFS), the actual filesystem is Ext4 as that's what Ubuntu uses by default. So that means i have 5 zvol's x8GB of reserved space as a block device /dev/zvol/<name>. That device is then exposed to the iSCSI-target ("istgt") daemon in FreeBSD, which is not part of native ZFS; i believe in OpenSolaris it is. So that's it: Ubuntu workstation -> Network -> FreeBSD server -> SSD Cache -> zpool containing Ext4 filesystem.Neat setup! You are basically exporting iscsi luns from the ZFS server, correct?
Lovely setup. If you have the time, why don't you write a small how-to for it. I would love to replicate something like that, but I am lacking the time to scavenge together all the knowledge that I would need for such a project.True. So far the implementation in FreeBSD 8 is working very well for me.
....
The reason ZFS isn't in Linux is that the Linux license doesn't permit linking to non-GPL code (i.e., the CDDL-licensed ZFS code). Now, this raises the question "Why didn't Sun license ZFS under GPL", I guess, but as it stands it's Linux's license that stops its integration, not ZFS'. IIRC Windows would be on firm ground to include ZFS in their OS if they wanted to, provided that they release any changes they made to it (and no other part of their kernel!).I really like ZFS. I wish Sun had not taken such a proprietary path with it, but instead fully open sourced it so it could be made part of the linux kernel.
That doesn't make it easy. The problem is basically this: one block can be referenced in a bunch of datasets. You make a filesystem, put a file in it, snapshot it, clone that, take another snapshot, and so forth, and now a thousand pointers know where that block is supposed to be. Now you want to change where that block is located. Well, you can't just change the pointers one at a time to point to the new location. If something happens while this is going on (remember, all the filesystem operations are online: you can keep using the pool while they happen) that changes the block (or some copies of it) everything should see a consistent view of things. That means you need to find all the pointers to a given block and change them all in a single transaction.If that had happened, th community with have added reshaping support to it - there is nothing in the ZFS architecture that prevents it.
Like what? OpenSolaris doesn't have support for my integrated graphics card, but it runs fine in console mode. Support for disk and network controllers is fairly good, in my experience.The other issue is that OpenSolaris has limited driver support for a lot of hardware that folks like us like to use.
Intel or AMD CPUs, with ECC memory if you can get it. Intel NICs, LSI SAS controllers. Whatever disks you like.This makes it non-trivial to build a system that can run ZFS well.
GUI tools aren't everything. Being able to write down concisely how you got where you are ("zpool create foo raidz2 c1t0d0 ...", say) is quite nice. Remote administration over ssh is the most convenient system I've ever used... but I'll admit it's not easy to get used to.Plus there are a lack of GUI tools for management that folks here like.
I know senior people at Sun, and they know what needs to change to make it work really well for our type of use.
... or maybe the reason to buy Sun wasn't to spend $7.4 billion on things they wanted to throw out the window, but to get the existing, working, tested code.However, Sun's fortunes were less than bright, and the Oracle purchase has basically gutted ZFS development. Oracle has a next gen system called BTRFS that isn't bad, but the conflict means that ZFS will be EOL'd at some point.
Maybe they can get the whole ZFS code base open sourced and then it can live on past Sun, but Oracle isn't exactly that type of organization...
Here, look at the source!Wikipedia said:ZFS is implemented as open-source software, licensed under the Common Development and Distribution License (CDDL).
Saw this post today.No, with RAID 6 there is enough information to correct a bit flip on one drive, as I already explained. Perhaps the problem is that you do not understand RAID 6. RAID 6 does not use a simple parity checksum, but rather a kind of Reed-Solomon error correcting code with Galois fields math. The computation is complicated, but I think I explained clearly in my previous post how an error could be detected and corrected with RAID 6.
I just wanted to add: no, you are wrong.
What I am trying to say, is that raid-5 nor raid-6 is safe. In my link, there are lots of research papers that show hw raid is not safe. Read my link.No, you need to read what I wrote (and you quoted!). I said that RAID 6 has enough information to correct one bit flip. That is correct. Your point that RAID 6 cannot correct 2 or more bit flips is true, but does not invalidate my statement.
What I am trying to say, is that raid-5 nor raid-6 is safe. In my link, there are lots of research papers that show hw raid is not safe. Read my link.
With ZFS I dont understand why anyone would use hardware raid.
Seriously? Well, there is performance, convenience, and OS compatibility.
There are numerous ZFS benchmarks in this thread alone, sometime once the new Intel SSDs drop I'll be building a ZFS RAIDZ SSD array with at least 5 drives
Unless I'm willing to add another $XXXX price for a decent hardware raid card and battery backup unit ....
If you have even tried submesas zfsguru you have no idea how easy it is to set a zfs array up.... and I don't know what OS you are running that doesn't have samba (windows file sharing). My Win7 and mac OS X laptops connect to the zfs guru shares at home (or even the opensolaris and openindiana shares at work)
When something as complex as zfs gets easy enough for an technically challenged person to set up, and when they(the technically challenged) do go live with zfs(the unbreakable, never buggy, fixes the unfix-able, magical fs) with their data, that is when one is begging for massive data-loss for that person, as they disregard the most important thing, backups, as they don't see why that would be needed since it's a magical fs<period>
Still not bad for something you get for free, and even while utilizing all its protections which adds upkeep, it still manages to perform very decently, well above gigabit limits.I've never seen a ZFS benchmark where the performance is as high as I have seen from several good hardware RAID cards.
If this is your conclusion of our earlier discussion, then we still have a lot to talk about i guess. Variable stripesizes are not what makes ZFS slow; it is what makes random writes to RAIDZ/RAIDZ2 much faster than is possible with traditional RAID5 and RAID6, and it solves the write hole too, 2 wins in 1 blow.This is not surprising, since the ZFS architecture (variable stripe size, etc.) was not built for performance.
Can implement something like that easily, to let all LEDs light up in a pool, and you will know which disk has failed since it does not light up. Also you give names to your disks, so you may know which disk is in which casing beforehand and can write labels near the HDD swap bays.And even with a GUI, ZFS cannot compete with hardware RAID for convenience. Particularly with replacing failed drives, where the HW RAID card just lights up an LED next to the failed drive, and you swap it out, and everything proceeds automatically. This is helpful if you have racks full of hot swap bays.
I agree.Just because your application does not benefit from hardware RAID does not mean that there are no applications where hardware RAID is the best choice.
Can implement something like that easily, to let all LEDs light up in a pool, and you will know which disk has failed since it does not light up. Also you give names to your disks, so you may know which disk is in which casing beforehand and can write labels near the HDD swap bays.
Still not bad for something you get for free...
If that is a priority to you, to save a few minutes of your precious time, then by all means opt for the Hardware RAID solution.On hardware RAID, the failure LED comes on automatically, no need to implement anything, or even touch a keyboard.
Well from all the money you save (TLER disks, Hardware RAID controller, BBU) you surely can spend 120 euro on 8GiB DDR3 ECC DRAM; or 180 euro for 12GiB. And depending on what features you use, ZFS will run decently on low CPUs like an Atom. If you are going to do alot of things on the background on your multi-purpose NAS then you would want something more powerful.Nothing is free. ZFS comes with a huge memory footprint, an almost insatiable desire for more memory, and consumes a significant number of cycles on your CPU that you may need for other purposes.
It's amusing to see you're desperately trying to discredit ZFS, while ZFS is just so innovative and great that it basically sells itself! No pro-arguments needed; people are quickly convinced this is the next-generation storage that they want. Their only problem is accessibility; getting it to run on their systems, and lack of information sources and support. All these are valid arguments, but as each users needs are different those argument may be compelling or worthless.Perhaps you could get away with "something you get for less than hardware raid costs today". But the line above just calls you out as a Zealot and not objective.
Contrary to your claims here and elsewhere, TLER drives are not really needed for raid. TLER is an invention of WD in response to the fact that they overdid their retry algorithm. Many people have discussed this at length and shown that drives exist that do not suffer from this problem (e.g., Hitachi) and consumer grade disks are just fine. You conveniently ignore this every time you get onto your anti-raid crusade, even though it has been pointed out in some detail by the most respected members of this forum (odditory, nitro and others). Also, you really should actually read my posts before you respond to them. At no point have I said that ZFS is not appropriate for a dedicated NAS. As for the Atom, might be OK (although most Atom systems are too memory limited to actually run it safely - but that's quite another topic).Well from all the money you save (TLER disks, Hardware RAID controller, BBU) you surely can spend 120 euro on 8GiB DDR3 ECC DRAM; or 180 euro for 12GiB. And depending on what features you use, ZFS will run decently on low CPUs like an Atom. If you are going to do alot of things on the background on your multi-purpose NAS then you would want something more powerful.
If ZFS "sold itself" it would be everywhere. It isn't. Why? Because it does is not supported on mainstream OSs, it is a PITA to administer, the OSs that it does run on have limited hardware support, it is a memory hog, on and on. It is a great filesystem with some interesting features. But it is far from perfect for all users.It's amusing to see you're desperately trying to discredit ZFS, while ZFS is just so innovative and great that it basically sells itself! No pro-arguments needed; people are quickly convinced this is the next-generation storage that they want. Their only problem is accessibility; getting it to run on their systems, and lack of information sources and support. All these are valid arguments, but as each users needs are different those argument may be compelling or worthless.
Yup. Same is true of Raid - even though you spout off at every turn how horrible that is. Same is true of WHS. In fact reading posts here its pretty clear that there are lots of good choices out there - one size does not fit all.But just look at this forum; many people want to run ZFS, as it provides alot of benefits these home users, giving them reliable network storage with great cost-per-gigabyte as well.
Zealots need a counterweight. I don't hate ZFS, but it is also not the "one true religion".If you want to call me a zealot that's fine. I rather be a zealot than a hater.
I think you need to say "you don't find me preaching ZFS in topics that are clearly windows or WHS anymore". But it hasn't been that long...Oh and another thing. You don't find me preaching ZFS in topics that clearly are Windows or WHS storage domain. It is somewhat appalling to see a witchcraft happening against ZFS in a thread that started clearly in ZFS context, and not with the desire to make it into a ZFS versus whatever discussion. Wouldn't you consider it fair to create your own thread to 'discuss the inferior ZFS filesystem' and suck all the negative air out of this thread into yours, what do you say?
Silent corruption isn't as much of a real problem as it is a theoretical one...
On hardware RAID, the failure LED comes on automatically, no need to implement anything, or even touch a keyboard. And no need to play, 'which LED is not on' in a bank of hundreds of bays. And labeling each disk individually? That is not convenient. Hardware RAID cards are more convenient than ZFS for some applications, as I already said.
I see that you didnt read my OpenSolaris link that shows lots of research papers. The point of ZFS is not that it adds checksums - every storage solution adds checksums. You dont understand ZFS.Why? You write like there is some secret revelation in those papers. I am aware of the properties of various types of RAID. Nothing is completely safe. Yes, ZFS adds checksums. Good for ZFS.
See the CERN study, among others. Everything I said above, is taken from my OpenSolaris link I posted earlier. Here are lots of research papers that I have summarized in this post on OpenSolaris forum:Silent corruption isn't as much of a real problem as it is a theoretical one...
I see that you didnt read my OpenSolaris link that shows lots of research papers. The point of ZFS is not that it adds checksums - every storage solution adds checksums. You dont understand ZFS.
To be fair, there are some implementation of ZFS on machines which already do this. Both Nextena and Sun/Oracle ZFS based storage lines support this.
Why? You write like there is some secret revelation in those papers. I am aware of the properties of various types of RAID. Nothing is completely safe. Yes, ZFS adds checksums. Good for ZFS.
I just wanted to point out that ALL storage solutions add checksums in one way or the other. But no one did it right, until ZFS. Comp Sci researchers examine raid-5/raid-6/filesystems and show they are all flawed. On the other hand, they show ZFS seems to be safe.Come on! Of course I understand ZFS, and I am not going to read your links or everything you wrote below the line I have quoted. Yes, comprehensive checksums are the big feature of ZFS for preventing silent data corruption. No, not all storage systems do comprehensive checksums like ZFS does. No need to make this simple fact so complicated.
I just wanted to point out that ALL storage solutions add checksums in one way or the other. But no one did it right, until ZFS. Comp Sci researchers examine raid-5/raid-6/filesystems and show they are all flawed. On the other hand, they show ZFS seems to be safe.
I do not claim that.