Data Backup and Workflow for a One-Man Operation

[Michael Scarn]

Workstation
i7-950 : Gigabyte GA-X58A-UD3R : G Skill 6x 4GB DDR3 RAM : eVGA 460 GTX : Corsair AX750 : 2x Samsung F3 1TB : 2x WD Black 640GB : 1x WD Blue 500GB : 1x Seagate 500GB : and the following storage setup:

Internal
640GB (Operating System).….[OS & Apps]
640GB (OS Backup).…............[OS Images and Backup]
1TB (Media Primary).…............[Media, downloads, storage, etc]
1TB (Media Backup).................[Exact copy of ‘Media Primary’]
500GB (Cache).........................[Strictly caches, temps, and pagefiles]
500GB (Export).........................[Temp export disk for renders, also hosts VMware disk image]

External
1TB (Movies).............................[TV shows, Movies, Video in general]
1TB (Master Backup)................[Comines ‘Movies’, ‘Media Primary’ and ‘OS Backup’]
16GB (Flash Drive)...................[Most current projects and temporary file transfers]
‘Movies’ is currently in an external enclosure, accessed through eSATA
‘Master Backup’ is a hotswap, kept in a fireproof/waterproof safe when not being used

Other
DVDs.........................................[Movies, TV, and most things I can’t recreate]

‘Media Primary’
Apps (~30GB)............................[Games and apps I keep so I don’t need to redownload]
Documents (~1GB)...................[Random printer scan, excel, word documents]
eBooks (~1GB)..........................[Various PDF books and such]
Editing Content (~20GB)............[Custom graphics and motion for projects]
Icons (~1GB)..............................[I like making Icons, makes folders easier to remember]
Music (~5GB).............................[It’s music]
Pictures (~6GB).........................[Pictures that don’t belong to projects]
Projects (~70GB).......................[All project files and accompanying assets]
Video (~280GB)..........................[Everything from my ‘Movies’ drive, want to remove eventually]

Media Primary is the most important to me, and also will get the most access (and not just from my workstation computer). Currently, I only have my workstation as a computer. However, I am waiting for some power extension cables to finish my Pentium 4 case transplant to finish a FreeNAS build. I’ll also be looking into a laptop to do some work away from home with.

FreeNAS
- Use for network backup of workstation and laptop
- FTP access
- Local network media server (nothing a 1TB-2TB hard drive can’t handle)


Laptop
- I would like to get a dual hard drive setup, but only within reasonable cost. Hopefully setup as OS and Media drives. The ‘Video’ folder will be absent from it.
- I want to sync the ‘Projects’ folder, either from the Workstation or from the FreeNAS. That way I can have updated projects when I need to show clients on moments notice.
- Laptop will also be backup’d to the FreeNAS as well.


I do small graphic design projects, bit of video touch-ups and rendering, and occasional computer maintenance locally. Things are fairly casual right now, with no intention of getting super busy until I have properly configured backups and proper synchronization between computers. I also figure getting some sort of offsite backup will be a must, either through peer to peer crashplan or using an online bakcup service. Even then, it will only require the ‘Projects’ folder for the most part.

My workstation and NAS are behind a UPS. I don’t get floods where I’m at, few thunder storms. Theft is always an uncertainty wherever you live. I tend not to delete things, until I do a consolidation of my hard drive space (every few weeks). Regardless, I think snapshots would be important. I also like burning DVD’s of finished projects, just so I can hand them off to someone else easy enough if ever needed.

The ‘Projects’ and ‘Documents’ folder are the folders that I would like redundancy for. Everything else, as far as I can see, are recreatable (time consuming in many cases). I stick away from RAID when necessary. With as many backups as I have, there really doesn’t seem to be a need for them. I should also mention I turn off my Workstation when it is not in use or not rendering (or backing-up if that can’t be done during it’s regular on time) The NAS would then be on 24/7 then.

I guess I’ll stop there. My big questions then are:
- What software should I be running on my Workstation and Laptop for backing-up to the NAS?
- Can that same software sync my Laptop with my NAS or Workstation?
- For doing offsite backup, should I be using my NAS for that online service/crashplan?
- What methods do you guys use for some of the aspects I use my setup for?

Thanks very much you guys

 

[Michael Scarn]

I guess I'll spend a couple ours each day doing trial and error and see what works.

 

[ChrisBenn]

Well you didn't specify your OS, but I'm assuming Windows 7? The built in windows 7 backup is actually surprisingly decent. I had been using a WHS box for awhile (HP ex485) but ran out of room and moved to a Solaris/ZFS setup. I was contemplating running WHS in a VM, but really the built in backup was more than sufficient. As a bonus you get access to the "previous versions" dialog when using windows backup.

My initial reaction on reading your description is you have made it overly complicated. Do you really need a full local duplicate of your "Media Primary" for instance?

I would put together your NAS and consolidate most of your drives there. Get a fast SSD for your OS/Apps, and if you need a separate drive from video rendering (you mention "cache", not sure what that includes) leave on fast drive in there.

Basically you first off want to protect your data against disk failure, so this means some sort of raid type setup (in the NAS preferably). You can mirror your OS disk, or just make images with the built in backup to the NAS.

Next point is maintaining versions of your working data. For things like videos, music, probably not so important, but for work documents, graphics you are actively editing, etc. - well it can save you form deletions, bad edits, corruption, etc. You can achieve this redundancy locally (depending on what you run on your NAS - under ZFS I just have some rotating snapshots for my high value data).

Finally you have off-site storage. You could rotate disks to another location, but if you have a decent internet connection something like Backblaze, crashplan, carbonite would be a simpler choice. These services will also give you some versioning.

Just decide what level of backup/redundancy you need for your different data types and segregate stuff accordingly. I would simplify/consolidate all the disks you have hooked up to your main workstation into the absolute minimum there, and move what you need to to the nas.

Don't know if that answered your question, but you probably won't get too many replies as it's a very vague question - not sure if you want suggestions on how to do things, comments on your current situation, comments on your proposed solution, etc.

 

[Michael Scarn]

What I think I really needed was someone to point out that I was probably asking vague/wrong questions, so thank you very much ChrisBenn. And yes, currently I am running Windows 7 Ultimate. I feel stupid for not posting such a simple thing.

I guess I should say that what I am doing right now, is not what I want to be doing. In an effort to make things simpler, I would like to treat this entire case as a small business asking for data management advice. Forget about all tasks beyond the business oriented. I'll try to explain the purpose of what I have, and where I hope it can go.

I have six hard drives in my workstation, only 4 of them really being necessary. OS/Apps, Media, Export, and Cache. The purpose of 4 drives in for performance when working with video. The cache disk is strictly for anything temporary and requires no redundancy. Export is strictly for renders so it has its own dedicated writing (also requires no redundancy)

So again, to keep things simple, let's think simply in terms of the NAS as a network backup. I'm looking at backing up one folder on my 'Media Primary' drive to the NAS (ideally with versioning if possible). If that could be automatically run during work, that would be the "end-all" solution for me. Being able to work in the same way with a laptop, would also icing on the cake.

I'm just trying to imagine what the workflow of a small graphic design/video production business would be like, and closely emulate that within reason. I don't mind doing some testing and tweaking, but don't see sense in perfecting a "lost cause" of a solution. You are absolutely right, I've made this overcomplicated. The ideal situation would be: snapshots to a NAS every 15 minutes while I work, while keeping snapshots of everyday for a week, with a final snapshot being compiled of the previous week. I'll worry about integration with other workstations/laptops when I can figure that out.

So is that something that can be accomplished using FreeNAS with Windows 7?

 

[[LYL]Homer]

I only skimmed this, but my impression is that this is near as complicated as my setup before I went with WHS a few years ago. If you can rebuild/retask a box for most your drives and run WHS (verions 1, not the upcoming 'Vail' version 2 without drive extender) with full duplication it seems like it might solve much of your maintenance and backup 'mental overhead'.

My wife is the photographer between us so all of our photos are on her PC on a single disk as a 'master' copy. That disk is then manually copied to the WHS \Photos\ share which is duplicated. Her system also runs Mozy to back that all up to the cloud as well as a nighty WHS backup of everything (including another copy of the photos). This would a little easier if there was a reasonably priced WHS backup service, but this is our workaround.