ZFS - Testing its features

W

war9200

Limp Gawd
Joined
Oct 9, 2010
Messages
273
Hi,

Incase anyone wants a quick place to checkout the differences between the various types of attributes you can set on ZFS in terms of deduplication and compression, read on..


root@atom:~# uname -a
SunOS atom 5.11 NexentaOS_134f i86pc i386 i86pc Solaris
root@atom:~#

Note:
1. 0 = Root volume, 1-5 for testing.

Code: 
root@atom:~# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c3t0d0 <DEFAULT cyl 18238 alt 2 hd 255 sec 63>
          /pci@0,0/pci15d9,60a@1f,2/disk@0,0
       1. c3t1d0 <DEFAULT cyl 18239 alt 2 hd 255 sec 63>
          /pci@0,0/pci15d9,60a@1f,2/disk@1,0
       2. c3t2d0 <DEFAULT cyl 18239 alt 2 hd 255 sec 63>
          /pci@0,0/pci15d9,60a@1f,2/disk@2,0
       3. c3t3d0 <DEFAULT cyl 18239 alt 2 hd 255 sec 63>
          /pci@0,0/pci15d9,60a@1f,2/disk@3,0
       4. c3t4d0 <DEFAULT cyl 18239 alt 2 hd 255 sec 63>
          /pci@0,0/pci15d9,60a@1f,2/disk@4,0
       5. c3t5d0 <DEFAULT cyl 18239 alt 2 hd 255 sec 63>
          /pci@0,0/pci15d9,60a@1f,2/disk@5,0
Specify disk (enter its number):
--

Make a number of pools:

1. regular ZFS storage
2. compressed ZFS storage
3. deduplicated ZFS storage
4. compressed and deduplicated ZFS storage (if possible)

--

root@atom:~# zpool create regular-zfs-pool c3t1d0
root@atom:~# zpool create compressed-zfs-pool c3t2d0
# nice it checks for this, to see if its been created or not
root@atom:~# zpool create deduplicated-zfs-pool c3t2d0
invalid vdev specification
use '-f' to override the following errors:
/dev/dsk/c3t2d0s0 is part of active ZFS pool compressed-zfs-pool. Please see zpool(1M).
root@atom:~# zpool create deduplicated-zfs-pool c3t3d0
root@atom:~# zpool create compressed-and-dedupe-zfs-pool c3t4d0
root@atom:~#

--

Code: 
root@atom:~# zpool list
NAME                             SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
compressed-and-dedupe-zfs-pool   139G   124K   139G     0%  1.00x  ONLINE  -
compressed-zfs-pool              139G   124K   139G     0%  1.00x  ONLINE  -
deduplicated-zfs-pool            139G   124K   139G     0%  1.00x  ONLINE  -
regular-zfs-pool                 139G   108K   139G     0%  1.00x  ONLINE  -
syspool                          139G  4.06G   135G     2%  1.00x  ONLINE  -
root@atom:~#

--

Show filesystems:

Code: 
root@atom:~# df -h
Filesystem             size   used  avail capacity  Mounted on
syspool/rootfs-nmu-000
                       137G  1006M   132G     1%    /
/devices                 0K     0K     0K     0%    /devices
/dev                     0K     0K     0K     0%    /dev
ctfs                     0K     0K     0K     0%    /system/contract
proc                     0K     0K     0K     0%    /proc
mnttab                   0K     0K     0K     0%    /etc/mnttab
swap                   3.8G   292K   3.8G     1%    /etc/svc/volatile
objfs                    0K     0K     0K     0%    /system/object
sharefs                  0K     0K     0K     0%    /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap1.so.1
                       133G  1006M   132G     1%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                   3.8G     4K   3.8G     1%    /tmp
swap                   3.8G    40K   3.8G     1%    /var/run

/dev/dsk/c0t0d0s2      7.6G   136M   7.5G     2%    /media/NO NAME
regular-zfs-pool       137G    31K   137G     1%    /regular-zfs-pool
compressed-zfs-pool    137G    31K   137G     1%    /compressed-zfs-pool
deduplicated-zfs-pool
                       137G    31K   137G     1%    /deduplicated-zfs-pool
compressed-and-dedupe-zfs-pool
                       137G    31K   137G     1%    /compressed-and-dedupe-zfs-pool
root@atom:~#

--

Options:
compression=on | off | lzjb | gzip | gzip-N


root@atom:~# zfs set compression=on compressed-zfs-pool
root@atom:~#

Easy enough..


root@atom:~# zfs set dedup=on deduplicated-zfs-pool
root@atom:~#

Also easy enough.

Ok, can it do both?

root@atom:~# zfs set compression=on compressed-and-dedupe-zfs-pool
root@atom:~# zfs set dedup=on compressed-and-dedupe-zfs-pool
root@atom:~#

Wow..

Now for the benchmarks!

=> Also included, time it takes to untar to each one..

root@atom:/regular-zfs-pool# time tar xf /root/linux-2.6.37.tar
real 0m14.199s
user 0m1.381s
sys 0m12.817s

root@atom:/compressed-zfs-pool# time tar xf /root/linux-2.6.37.tar
real 0m16.230s
user 0m1.449s
sys 0m14.762s

root@atom:/deduplicated-zfs-pool# time tar xf /root/linux-2.6.37.tar
real 0m19.161s
user 0m1.519s
sys 0m15.393s

root@atom:/compressed-and-dedupe-zfs-pool# time tar xf /root/linux-2.6.37.tar
real 0m19.636s
user 0m1.548s
sys 0m15.472s

RESULTS:
Code: 
root@atom:/compressed-and-dedupe-zfs-pool# zpool list
NAME                             SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
compressed-and-dedupe-zfs-pool   139G   230M   139G     0%  1.00x  ONLINE  -
compressed-zfs-pool              139G   214M   139G     0%  1.00x  ONLINE  -
deduplicated-zfs-pool            139G   437M   139G     0%  1.00x  ONLINE  -
regular-zfs-pool                 139G   420M   139G     0%  1.00x  ONLINE  -
root@atom:/compressed-and-dedupe-zfs-pool#
--

It appears the dedupe is NOT inline, it runs post-copy:

root@atom:/compressed-and-dedupe-zfs-pool# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
compressed-and-dedupe-zfs-pool 139G 223M 139G 0% 1.00x ONLINE -
compressed-zfs-pool 139G 214M 139G 0% 1.00x ONLINE -

deduplicated-zfs-pool 139G 429M 139G 0% 1.00x ONLINE -
regular-zfs-pool 139G 420M 139G 0% 1.00x ONLINE -
syspool 139G 4.46G 135G 3% 1.00x ONLINE -
root@atom:/compressed-and-dedupe-zfs-pool#

Notice it is getting smaller allocated for compressed & dedupe as well as
dedupe:

230M->223M (comp+dedupe)
437MB->429MB (dedupe, bigger than the actual data (see regular zfs pool)

--

Code: 
root@atom:/compressed-and-dedupe-zfs-pool# zpool list
NAME                             SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
compressed-and-dedupe-zfs-pool   139G   223M   139G     0%  1.00x  ONLINE  -
compressed-zfs-pool              139G   214M   139G     0%  1.00x  ONLINE  -
deduplicated-zfs-pool            139G   429M   139G     0%  1.00x  ONLINE  -
regular-zfs-pool                 139G   420M   139G     0%  1.00x  ONLINE  -
syspool                          139G  4.46G   135G     3%  1.00x  ONLINE  -

# Check some options

root@atom:/compressed-and-dedupe-zfs-pool# zfs get dedup deduplicated-zfs-pool
NAME PROPERTY VALUE SOURCE
deduplicated-zfs-pool dedup on local
root@atom:/compressed-and-dedupe-zfs-pool# zfs get dedup compressed-and-dedupe-zfs-pool
NAME PROPERTY VALUE SOURCE
compressed-and-dedupe-zfs-pool dedup on local
root@atom:/compressed-and-dedupe-zfs-pool# zfs get compression compressed-and-dedupe-zfs-pool
NAME PROPERTY VALUE SOURCE
compressed-and-dedupe-zfs-pool compression on local
root@atom:/compressed-and-dedupe-zfs-pool#

--

Total sizes:

Code: 
root@atom:/compressed-and-dedupe-zfs-pool# zpool list
NAME                             SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
compressed-and-dedupe-zfs-pool   139G   223M   139G     0%  1.00x  ONLINE  -
compressed-zfs-pool              139G   214M   139G     0%  1.00x  ONLINE  -
deduplicated-zfs-pool            139G   429M   139G     0%  1.00x  ONLINE  -
regular-zfs-pool                 139G   420M   139G     0%  1.00x  ONLINE  -
syspool                          139G  4.46G   135G     3%  1.00x  ONLINE  -
root@atom:/compressed-and-dedupe-zfs-pool#

--

OK, let's make a copy of the directory on each disk and time it and then check
the space again..

root@atom:~# for i in /regular-zfs-pool /compressed-zfs-pool /deduplicated-zfs-pool /compressed-and-dedupe-zfs-pool
> do
> cd $i
> echo in dir $i; time cp -r linux-2.6.37 linux-2.6.37.copy
> done
in dir /regular-zfs-pool

real 0m26.795s
user 0m0.958s
sys 0m14.747s
in dir /compressed-zfs-pool

real 0m19.094s
user 0m0.883s
sys 0m13.700s
in dir /deduplicated-zfs-pool

real 0m22.661s
user 0m0.859s
sys 0m13.276s
in dir /compressed-and-dedupe-zfs-pool


Wowee! Not quite realtime..

Code: 
   PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU PROCESS/NLWP
  4336 root        0K    0K sleep   99  -20   0:00:40  11% zpool-compresse/142
  4327 root        0K    0K sleep   99  -20   0:00:36 3.5% zpool-deduplica/142
  4313 root        0K    0K sleep   99  -20   0:00:23 1.1% zpool-compresse/142
  4304 root        0K    0K sleep   99  -20   0:00:13 0.3% zpool-regular-z/142
  4508 root     4224K 2764K cpu0    49    0   0:00:00 0.0% prstat/1

Will wait until these are done and then look at the disk space, but it APPEARS
de-dupe only works on a file/name(?) basis, it did not find ANY files/as
candidates for de-duplication within the kernel source tree (ZERO DE-DUPE),
when I copied the source tree again, then it gets 2.00x de-duplication,
well of course, but I am not going to store many copies of the same data on
the same disk

TOTALS:

Code: 
root@atom:/compressed-and-dedupe-zfs-pool# zpool list
NAME                             SIZE  ALLOC   FREE    CAP  DEDUP  HEALTH  ALTROOT
compressed-and-dedupe-zfs-pool   139G   238M   139G     0%  2.00x  ONLINE  -
compressed-zfs-pool              139G   429M   139G     0%  1.00x  ONLINE  -
deduplicated-zfs-pool            139G   441M   139G     0%  2.00x  ONLINE  -
regular-zfs-pool                 139G   840M   138G     0%  1.00x  ONLINE  -
syspool                          139G  4.46G   135G     3%  1.00x  ONLINE  -
root@atom:/compressed-and-dedupe-zfs-pool#

Code: 
840M: REGULAR POOL
429M: COMPRESSED POOL
441M: DEDUPLICATED POOL
238M: COMPRESSED AN DEDUPLICATED

--

This is of course a test scenario, but should give people a good overview
of the differnet ways ZFS can help save space on your disks, I need to test
on some actual/real datasets to see how much it can help in the various pools.
 
about dedup:

if you activate it, it will dedup all files you will copy onto this ZFS AFTERWARDS,
already stored files are not affected

to dedup already stored files, you have to create a new dedup-enabled pool/ZFS and
copy all files to it

ZFS dedup is not file but block based
for ex. if you have 10 text-documents that are nearly the same, the common part of them is stored only once. File-based dedup will not help in such a case.

The problem about such a sophisticated block-based dedup:
ZFS dedup is extremely CPU and RAM hungry.
You will need about 2,5 GB RAM per TB Data + a SSD Read Cache is suggested or it
may become unusable slow.

thats the same like with encryption on Solaris Express 11
you will need real power or a new CPU with encryption features

Only compression has not that problems with CPU-power

see also my thread about Solaris based Systems
http://hardforum.com/showthread.php?p=1036678184#post1036678184


gea
 
Last edited:
You must log in or register to reply here.
Back
Top