Nexenta/Napp-it deduplication questions

ochadd

[H]ard|Gawd
2FA
Joined
May 9, 2008
Messages
1,469
Setup a NexentaCore/Napp-it server this afternoon and deduplication doesn't appear to be working. I'm expecting the DEDUP % to go up. I've restarted the box a number of times with no succcess. Each one of the highlighted files is an exact copy other than the name. Anything else I should check?

I'm hoping to use this as a backup target for around 4TB of server backup images if all works out.

zfs_napp_it.jpg

zfs_folder.jpg

zfs_settings.jpg


PS: Some performance numbers from my setup using Hitachi Deskstar 3TB drives attached to an Asus M4A7B-E AMD board:
Code:
Info	NAME	SIZE	Bonnie	Date(y.m.d)	File	Seq-Wr-Chr	%CPU	Seq-Write	%CPU	Seq-Rewr	%CPU	Seq-Rd-Chr	%CPU	Seq-Read	%CPU	Rnd Seeks	%CPU	Files	Seq-Create	Rnd-Create
2 disk Mirror	dedupemirror	2.72T	start	2011.10.20	7672M	103 MB/s	74	89 MB/s	13	56 MB/s	12	120 MB/s	95	160 MB/s	13	616.4/s	2	16	+++++/s	28519/s
4 disk Mirror	dedupemirror	5.44T	start	2011.10.20	7672M	134 MB/s	93	186 MB/s	27	89 MB/s	18	123 MB/s	98	293 MB/s	20	1077.7/s	5	16	+++++/s	24115/s
4 Disk RaidZ	poolraidz1	10.9T	start	2011.10.21	7672M	130 MB/s	92	160 MB/s	25	111 MB/s	25	124 MB/s	97	427 MB/s	47	566.0/s	3	16	+++++/s	31555/s
 
Last edited:
Did you copy the files after dedupe was turned in (i.e. you aren't expecting it to dedup an existing volume)?
 
BEWARE of ZFS deduplication. It is in beta stage. Do NOT use dedup yet.

There are a number of problems with ZFS dedup today. First, you need more than 1GB of RAM for every TB of disk space, otherwise performance will be abysmal. Second, destroying a snapshot can take days. etc etc

It is much much better to buy extra disks than to use dedup.

Spread the word: do not use ZFS dedup. Avoid like the plague. Read the wikipedia article on ZFS.

/ZFS fanboy
 
This first file had already been copied onto the volume before I figured out I needed to enable deduplication. Mistakenly thought enabling ZFS level 21 implied it. The second two files where copied after dedupe was turned on.
 
can you post the output from typing:
zpool list dedupemirror
and
zdb -DD dedupemirror

Also while I don't agree that you need to avoid dedupe "like the plague", brutalizers warning is very apropos. As he says you should except to loose 1GB of RAM per TB of deduplicated data (for the dedupe table essentially) - so make sure you have enough memory in your system that you won't crowd out your ARC, standard system memory usage, etc. (For your setup 16 would be fine, I wouldn't do it with 8). Also I would ensure that you have an L2ARC installed on the dedupe pool as well. The issue that many people run into (destroying snapshots, pools, performance issues, etc.) come form the dedupe table swapping out of memory - if you have to swap to disk to read data on a per block level about disk operations you are trying to perform you become massively IO bound. The dedupe table will fall into the L2ARC if you have one available, which while not as good as system memory, is still infinitely better than swapping to standard disk.

Basically as long as you have sufficient RAM and L2ARC for the amount of data you are trying to dedupe (i.e. don't just turn on dedupe for an entire 20TB pool without having 32+GB of ram) you will be okay performance wise.

Now Brutalizer also brings up the valid point that disks are currently dirt cheap as far as space is concerned, so you have to consider that with respect to dedupe's value proposition.

I currently have dedupe turned off for my main pool, but on for my backup & VM folders and I'm getting a ~2:1 ratio there. I just created and destroyed a snapshot without a problem. It's only up to about 600GB actual though, and I have 16GB of memory & an 80GB L2ARC.
 
I nuked the pool and recreated a RAIDZ setup and it's working now. Created pool, benchmarked it, restarted the box, created folder, copied files, now I'm seeing 2x dedupe. May have been because I had a file in there before enabling dedupe.

Command results because I was curious:
zpool list poolraidz1
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
poolraidz1 10.9T 5.76G 10.9T 0% 2.00x ONLINE -


zdb -DD poolraidz1
DDT-sha256-zap-duplicate: 35184 entries, size 299 on disk, 142 in core

DDT histogram (aggregated over all DDTs):

bucket allocated referenced
______ ______________________________ ______________________________
refcnt blocks LSIZE PSIZE DSIZE blocks LSIZE PSIZE DSIZE
------ ------ ----- ----- ----- ------ ----- ----- -----
2 34.4K 4.29G 4.29G 4.29G 68.7K 8.59G 8.59G 8.58G
Total 34.4K 4.29G 4.29G 4.29G 68.7K 8.59G 8.59G 8.58G

dedup = 2.00, compress = 1.00, copies = 1.00, dedup * compress / copies = 2.00
 
Back
Top