• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

FreeNAS/ZFS pool hot plug question.

P

Phantum

[H]ard|Gawd
Joined
Jul 25, 2001
Messages
1,716
I've been Google'ing all day and have come up with nothing, maybe you folks can help me out.

So I've got an LSI 9211-8i in my FreeNAS box, connected to a Rackable SE3016. This unit houses 8x 3TB WD Reds and acts as my local 'archive' pool. All data here is then pushed to Crashplan using the plugin. What I'd like to do is bring this pool up once a week, rsync the data from my other pools to it, push it to Crashplan and then turn the rackable unit back off until next backup all without bringing the whole NAS down.

So how should I go about this? Do I have to export the pool every time I want to power off the unit? Then import the pool, do the backups, turn off the plugin, export the pool and turn the unit off? Or do I really have to bring the whole thing down just to turn off the unit and then bring it all back up? This 'archive' pool doesn't do anything that would require constant uptime and isn't accessible to any user other than root, which runs the rsync operation.

EDIT: I realize that by not exporting the pool FreeNAS will complain about the "missing" pool but it's not really a big deal because this pool isn't required for day-to-day operations. I've also tried connecting this unit to a FreeNAS guest on my ESXi box with pass through however I quickly realized that the performance of the rsync operation from the pools on my bare metal box to the guest was abysmal so that's out... unless anyone has any ideas on how to clear that up.
 
Last edited:
I can't really comment about FreeNAS/ZFS, but I can offer some insight to rsync.

rsync is slow. Really slow. It's convenient, but doggedly slow - it's main strength isn't file transfer speed, but in reducing required bandwidth and being super easy to implement script-side. You can speed it up somewhat by reducing the connection encryption type (it defaults to using SSH normally), or switching to something else like SCP.

If your just updating a small percentage of your data set, rsync may end up being ok after the initial data copy. But if large amounts of data are changing on your weekly backup, you may be better off using another tool than rsync.
 
Thanks Brian, I've actually come up with a number of tests (based on your thoughts here) that should help me target the underlying cause(s); such as VM misconfiguration, hardware/driver, bandwidth availability, software (rsync maybe), etc. So I'm organizing and outlining a procedure now. However I can tell you that I've never experienced the level of slowness over gigabit Ethernet that I'm seeing now; avg. 20MB/sec
 
SirMaster said:
If you are going from ZFS to ZFS you should be using ZFS send/recieve.
Click to expand...

Yes, ZFS to ZFS however the ZFS send/receive method is for snapshots. I want the data on my 'archive' pool to be the raw files, so I can pick and choose individual files upon restoring. This is a conscious choice and I'm doing it this way on purpose. Essentially the end result is the same but the method to achieving it is different. :D
 
Phantum said:
Yes, ZFS to ZFS however the ZFS send/receive method is for snapshots. I want the data on my 'archive' pool to be the raw files, so I can pick and choose individual files upon restoring. This is a conscious choice and I'm doing it this way on purpose. Essentially the end result is the same but the method to achieving it is different. :D
Click to expand...

Why can't you take a snapshot now and then just use cp or whatever to copy it all to your backup server and then from there on out take another snapshot and then use zfs send/recv whenever you want to update your backup copy with the changes since last snapshot.
 
SirMaster said:
Why can't you take a snapshot now and then just use cp or whatever to copy it all to your backup server and then from there on out take another snapshot and then use zfs send/recv whenever you want to update your backup copy with the changes since last snapshot.
Click to expand...

There's no reason I couldn't do that, but frankly it sounds a bit more complicated than a simple rsync operation.... And it's outside the scope of the issue at hand.

I made a similar post over at the FreeNAS Forum and the closest consensus reached was; turn off the plugin, export the pool, turn off the Rackable unit. At next backup cycle; turn the Rackable unit on, import the pool, scrub, turn on plugin, backup to Crashplan, repeat. We know that the SAS expander is hot-plugable but what we're not positive about is proper procedure for pool/system integrity of an 'external' pool, that isn't required 24/7, to 'pop' in and out of the system without bringing the system itself up and down.
 
You must log in or register to reply here.
Back
Top