Help me organize / get my shiznit in gear - about 10 TB+ of data and growing

hardware_failure

[H]ard|Gawd
Joined
Mar 21, 2008
Messages
1,370
Disclaimer: big read
TLDR; I have 2x new 14TB WD140EDGZ USB drives but no NAS yet,
14.png
I want the best way to safeguard 10TB+ of data between them. I have unlimited home backblaze plan and 2TB Dropbox. I have unlimited comcast gigabit (down) but its upload sucks ass at 40Mbps.


Environment: Typical geek household - I have a home network of various PCs and laptops and tons of all kinds of mobile junk.

Currently file sharing ~10TB of data from a 24TB RAID bare metal box running:
  • i3 ivy bridge
  • Win10 pro on an SATA SSD
  • Areca ARC-1880I controller
  • 8x Toshiba MD04ACA400 4TB drives
  • 24 TB RAID6
raid123.png


Current situation:
  • I have a ton of TV shows, movies, pr0n, apps, games, iso's I use for projects, big mix of junk. Also family photos and important stuff.
  • Mostly just generically sharing this with SMB authentication (which is a PITA - I have a DC but I found for simple home stuff running everything on a domain can be more of a PITA than its worth, especially with alot of kids that need easy access with mobile devices)
  • A while ago the PSU died on this thing and my heart skipped a beat. Fortunately replacing it brought it back to life.
  • I feel like this thing is a ticking time bomb and I want a better solution.
  • For the really important stuff I back it up in multiple cloud locations. I dont back up movies etc because thats at least like 1/2 of it and a s-ton of data.

Moving forward:

I just bought 2x 14 TB WD Easystores when they were on sale on BF for $200 each and Im very happy that I did. I havent tried to shuck them yet or anything.
I have dumped the contents of the RAID box to one of these USB Easystores. Using about 9 of 13 TB on it. Its just 1 lump 13TB basic windows NTFS partition on USB 3 interface.
I had an itchy trigger on one of those Synology DS220's that were on sale everywhere on BF but didnt bite AND NOW I REGRET it. 😡 Those deals seem to be GONE, cant find a good NAS on sale anywhere.
I dont like the BM RAID box in that its big, semi loud, draws alot of power, RAID6 could obviously fail, etc etc.
I have comcast 1gb down 40Mbps up unlimited. I would of course much prefer CCI or ATT fiber but neither is available in my area, this is the best I can get without going expensive business internet.


Cloud: I have unlimited home backblaze plan and 2TB Dropbox. I use the dropbox alot for documents and photos etc. I originally tried to backup media to backblaze from a win 2012 server setup and backblaze said yeah go f your self so I moved to Win10 pro. ATM I dont even use the backblaze because Im lazy and stuck at 40mbps up (and also basically Im an 1d10T). I tried uploading once a few years ago and just failed:

overview2.png



So.
What would you do?

ALSO - after a hardware / backup plan is worked out I need to find a way to organize my videos. They are just generically shared. Its dumb.
Plex? Synology Media Server (if I had one)? Kodi? Something else? I have wireless mini Win10 boxes
pc123.png
and these
remote.png
on all 5 of my TVs. The integrated intel video obviously sucks but the i5 CPUs software decode h.265 without even blinking. Also would be nice to have a super easy way to stream to the ton of iphones/ipads in the house.


Thank you kindly for reading and for any feedback!
 

Attachments

  • 2022-02-04_21-51-00.png
    2022-02-04_21-51-00.png
    23.6 KB · Views: 0
  • overview.png
    overview.png
    17.4 KB · Views: 0
I used backblaze until I needed to restore from them. Their restoration features suck (or at least did when I needed it a few years ago) if you have huge folders you want to restore, they limit the size and you have to use a crappy gui to manually select folders/files to restore while staying under the size limit. If they just added an automatic restore thing I would would probably consider them again.

Plex is pretty good for ease of use and compatibility. I would recommend it if your servers are powerful enough to transcode on the fly. An i3 ivy bridge cpu might work for your workload but it may not. It really depends on how many simultaneous streams you have and the quality of the content on disk (1080p or 4k). Since you already have HTPCs on your TVs I would use kodi on those so you're not transcoding locally and plex on mobile devices so you could transcode to lower bitrates as to not overload your upload (if you want remote access) and increase compatibility/ease of use.

I run truenas for my nas server, I really like it but it is also fairly limiting. The debian based releases are nicer and offer features like GPU passthrough but the freeBSD based releases have much more limited VM support and the debian releases cost money. I currently run truenas core, which is free but freeBSD based. I really like zfs and would recommend going that route but it is a fairly large undertaking to convert your NAS and the OS limitations might not suit your needs/preferences.

You could do offsite backups yourself with something like synctrayzor or rsync at a relative/friend's house and a NAS at their place. You can do the initial dump by shipping filled drives, then you just need to keep up with what you're adding. Their "nas" could be something as simple as installing drive bender and enabling file duplication (sort of raid 1) on the drive pool. Generally though I would re-consider what you actually need/want backed up offsite. Large data collections are usually fairly replaceable, you can probably only need to backup family photos/videos, maybe some software/work files. You could do your own offsite backups for everything and cloud backup just the mission critical stuff to help reduce how much you are uploading. You could also just do yearly (or whatever time frame you want) shipments of hard drives to update your large offsite backups.
 
You basically buy twice as much storage as you need to use. You use half to store your data, and you use the other half to store a backup copy.

The drives don't need to be in a NAS. You could use an old PC for example.

This can be as easy or complicated as you want to make it.
 
Back
Top