zrep modified for speed

[Flintstone]

Hi!

I don't know how many who use zrep (http://freecode.com/projects/zrep) but it is a great way to replicate stuff between ZFS boxes. Problem is that it uses SSH and that means performance suffers (atleast for me - I only get like 30MB/s). That's fine for small transfers but not for initial syncs.

Thus I've made a patch so that initial transfers can be made with mbuffer. I'm not suggesting using the patched script in production (it hasn't been tested a lot), but it works great as a way of getting your initial sync. As most of you know, using mbuffer will get you near gigabit (mine says 107-112MB/s when I transfer).

Patch against zrep 0.8.4 using the following command:

"patch zrep mbuffer.patch.zrep.0.8.4"

Please let me know if this works for you! Remeber to have mbuffer at both ends, and that ports 10000-10100 must be free (or modify the patch).

668a669,678
>
>       portnumber=0
>       while [ "$portnumber" -le 10000 ]
>       do
>         portnumber=$RANDOM
>         let "portnumber %= 10100"
>       done
>
>       echo Using port $portnumber on destination. Problems? Please make sure the range 10000-10100 is free or modify the script.
>
683,684c693,695
<               zfs send -p $snap |
<                 zrep_ssh $desthost zfs recv $MOUNTFILTER $READONLYPROP -F $destfs
---
>               zrep_ssh $desthost "echo \"mbuffer -m 1G -s 128k -I $portnumber | zfs recv $MOUNTFILTER $READONLYPROP -F $destfs > /dev/null 2>&1\" | at now"
>               sleep 3
>               zfs send -p $snap | mbuffer -m 1G -s 128k -O $desthost:$portnumber
689,690c700,702
<               zfs send $snap |
<                 zrep_ssh $desthost zfs recv $READONLYPROP -F $destfs
---
>               zrep_ssh $desthost "echo \"mbuffer -m 1G -s 128k -I $portnumber | zfs recv $READONLYPROP -F $destfs > /dev/null 2>&1\" | at now"
>               sleep 3
>               zfs send -p $snap | mbuffer -m 1G -s 128k -O $desthost:$portnumber

 

[Flintstone]

For anyone using this I have patched the latest release of zrep (1.0.1). Just follow instructions in my previous post to use the following patch

684a685,694
> 
>       portnumber=0
>       while [ "$portnumber" -le 10000 ]
>       do
>         portnumber=$RANDOM
>         let "portnumber %= 10100"
>       done
>  
>       echo Using port $portnumber on destination. Problems? Please make sure the range 10000-10100 is free or modify the script.
> 
744,745c754,758
<               zfs send -p $snap |
<                 zrep_ssh $desthost zfs recv $MOUNTFILTER $READONLYPROP -F $destfs
---
> #             zfs send -p $snap |
> #               zrep_ssh $desthost zfs recv $MOUNTFILTER $READONLYPROP -F $destfs
>               zrep_ssh $desthost "echo \"mbuffer -m 1G -s 128k -I $portnumber | zfs recv $MOUNTFILTER $READONLYPROP -F $destfs > /dev/null 2>&1\" | at now"
>               sleep 3
>               zfs send -p $snap | mbuffer -m 1G -s 128k -O $desthost:$portnumber
750,751c763,771
<               zfs send $snap |
<                 zrep_ssh $desthost zfs recv $READONLYPROP -F $destfs
---
> 
> #             zfs send $snap |
> #               zrep_ssh $desthost zfs recv $READONLYPROP -F $destfs
> 
>               zrep_ssh $desthost "echo \"mbuffer -m 1G -s 128k -I $portnumber | zfs recv $READONLYPROP -F $destfs > /dev/null 2>&1\" | at now"
>               sleep 3
>               zfs send -p $snap | mbuffer -m 1G -s 128k -O $desthost:$portnumber
> 
> 
1672a1693
>