ZFS Data Duplication question

D

darkeen

Supreme [H]ardness
Joined
Apr 12, 2004
Messages
4,267
Hey all,

I'm helping in the construction of a backup server for work. I'm wondering how much the data duplication helps. Does it happen to work better for text vs binary or vice verse? I'd rather avoid solaris if possible, but I hear the FUSE implementation under linux is pretty bad, anyone know how bad? Does anyone have any experience they can share?
 
By duplication you mean deduplication or redundancy? Deduplication is currently in alpha in FreeBSD 9.0-CURRENT so you have to use Solaris.

Here's a test on its real life performance: http://www.edugeek.net/forums/nix/49844-zfs-deduplication-dedup-vs-compression.html . It really depends on your data :) Incremental backups might work well but then again dedup is a new feature and I'd wait for it to become more scrutinized/benchmarked/etc. Although it's simple as a concept it adds more overhead and complexity.

As for zfs on linux I wouldn't use zfs-fuse on a production machine.
 
Last edited:
If you want stable storage, you do not want FUSE, you do not want dedupe but you would want a stable ZFS release. Most home users would have very little benefit from de-duplication anyway, and they are not comfortable buying 32GiB RAM + SSD to store dedupe tables, only in such configuration will it be decently fast.

Using an unstable ZFS release is like buying the safest car in the world and driving without safety belt; many protections won't matter if you're taking such a big risk. Do not run ZFS v28 and skip v21 de-dupe for now.
 
If you want stable storage, you do not want FUSE, you do not want dedupe but you would want a stable ZFS release.
Click to expand...

I would have to agree. I have recently tested zfs-fuse for a part of my /var/tmp where I generate my own customized distribution stages and releases for gentoo using my changes to metro.

https://github.com/funtoo/metro/wiki

Anyways during the io (coping thousands of small files) required to build a new release zfs-fuse crashes every time knocking the filesystem offline. I am now testing btrfs but that seems to be years away from zfs and the client tools are really immature.
 
Relativistic said:
By duplication you mean deduplication or redundancy?
Click to expand...
yes, my bad :p

The backups would probably be very scattered with many different people in different departments backing up "small" amount of data. Thanks for the info, I don't think we'll be using ZFS for this project.
 
You must log in or register to reply here.
Back
Top