Home Server - Backup Strategy (ZFS, VMDKs, File Level)

[DlStreamnet]

Hi guys,

For context - I set this ESXi whitebox up some time ago and it was always a test bed; until it wasn't. It now stores a whole bunch of non-critical content (backups, audio, ISO images). Recently my brother somehow installed CryptoWall which nuked about 800Gb of files and we were only able to recover from file level back ups. I'm now home for Christmas and would like to do some maintenance, and implement better backups.

Scenario:
ESXi Whitebox
5x 1TB Harddrives in ZFS Raidz1 (I believe; one failure is the limit)
1x 250GB hard drive for Windows, Open Indiana and PFSense
2x 500GB hard drives for workstation backups (via Server 2012 Essentials)
1x 3TB hard drive for file level backups of the ZFS pool

I have spare...
3x 250GB hard drives

The current 3TB drive simply copies files over, and if they are new or updated, overwrites the existing one. It is a direct mirror of the data.

Problem - the server backed up the 800GB of encrypted files. and nuked the 800gb of good files.

Solution - ?? I'm going to install one of the spare 250GB drives to take a back up of the 250GB Windows, Open Indiana and PFSense drive; I'm not sure how to back this up though. Copy the VMDK? For Windows, use the built in Windows-server backup?

I also seem to be missing shadowfiles when I tried to recover some data. Is there any considerations that need to be made before enabling this? Is there any point?

I understand ZFS has several mechanisms built in as well, so do I enable these and re-configure the pools? I'm thinking snapshots, but have no real understanding of how this works in terms of additional disk space taken up.

Lastly the 3TB drive...it currently mirrors, what is the best way to maximize the use of space? There is around 3TB used on the server, but i'd say 60% is non critical; so I'd rather have GOOD backups of some versus bad backups of all.

Any insight/reading material would be much appreciated. I would go and research all of this individually but the breadth of the problem has led me to the [H].

Thanks,
dL.

 

[_Gea]

You main problem seems.
Although ZFS is a versioning filesystem with readonly snaps, you need to do snaps to go back to former states.
Without snaps, you are lost with ZFS as you would have used a non-versioning filesystem.

A possible solution with ESXi and ZFS

-ESXi (USB stick or 250GB HD)
No backup required as it is reinstalled within minutes or use a boot clone

- You Storage VM (Openindiana, I would update to a newer Hipster or OmniOS) on 250GB HD ( local VMFS Filesystem)
No backup required as you can reinstall within minutes ex from an OVA template as you use it for storage only
(add no services like databases, webservers, mediaservers etc, if you need do this with VMs on ZFS with snaps)

- PFsense (local VMFS filesystem or ZFS storage via OpenIndiana and NFS with snaps)
Backup required, ex export as an ova template. If you use ZFS over NFS you can use snaps

- Windows backup
If you keep your data locally on Windows, sync them daily to a ZFS storage and enable ZFS autosnap
These snaps are readonly and accessable via Windows previous versions

A better approach is to hold all date on the ZFS storage as ZFS is more secure and robust than ntfs
Enable autosnap (Snaps is versioning and required to go back to a former version eq prior a CryptoWall infection
Use a raid z2 or 3way mirror for your important data as this allow a double disk failure.

Add a disaster backup of important data on two removable ZFS datapools (basic, mirror or raid.z) that you can keep
separately (fire, thiev, overvoltage) and that you use rotational. Add some snaps as well on the disaster backup.

For a perfect setup, use a second storage box on a differnt place in the house with zfs replication between them.
For easier configuration, you can use my napp-it web-ui on OpenIndiana/ Hipster/ Omnios or Oracle Solaris

 

[dilidolo]

You main problem seems.
Although ZFS is a versioning filesystem with readonly snaps, you need to do snaps to go back to former states.
Without snaps, you are lost with ZFS as you would have used a non-versioning filesystem.

A possible solution with ESXi and ZFS

-ESXi (USB stick or 250GB HD)
No backup required as it is reinstalled within minutes or use a boot clone

- You Storage VM (Openindiana, I would update to a newer Hipster or OmniOS) on 250GB HD ( local VMFS Filesystem)
No backup required as you can reinstall within minutes ex from an OVA template as you use it for storage only
(add no services like databases, webservers, mediaservers etc, if you need do this with VMs on ZFS with snaps)

- PFsense (local VMFS filesystem or ZFS storage via OpenIndiana and NFS with snaps)
Backup required, ex export as an ova template. If you use ZFS over NFS you can use snaps

- Windows backup
If you keep your data locally on Windows, sync them daily to a ZFS storage and enable ZFS autosnap
These snaps are readonly and accessable via Windows previous versions

A better approach is to hold all date on the ZFS storage as ZFS is more secure and robust than ntfs
Enable autosnap (Snaps is versioning and required to go back to a former version eq prior a CryptoWall infection
Use a raid z2 or 3way mirror for your important data as this allow a double disk failure.

Add a disaster backup of important data on two removable ZFS datapools (basic, mirror or raid.z) that you can keep
separately (fire, thiev, overvoltage) and that you use rotational. Add some snaps as well on the disaster backup.

For a perfect setup, use a second storage box on a differnt place in the house with zfs replication between them.
For easier configuration, you can use my napp-it web-ui on OpenIndiana/ Hipster/ Omnios or Oracle Solaris

Want want to comment on pfsense. My harddisk in pfsense box died yesterday and I had to replace it and do a reinstall. It's within minutes as well if you have ever backed up pfsense config.
- reinstall pfsense
- config lan ip
- log on to webui and restore config
- reboot, you are back in minutes.