• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Expanding my data storage help

F

fhrosty

n00b
Joined
Oct 16, 2012
Messages
6
I have a setup that I'd like to grow, but I'm not sure what is the best way. Also, I posted this to Overclock.net, I hope that isn't bad. Just wanted to disclose that fact - different audience here.

Current Setup
I have 1 3TB internal Seagate drive (1TB platters - ST3000DM001) and 1 3TB external Seagate drive (same drive as the internal). I manually backup from the internal to external.

The data is video and images, but important media that would be very hard, if not impossible in some cases, to replace. I don't care too much about write speed. I do have some HD movies on here too, so the ability to play Blue Ray movies without pauses is nice.

Needs
  • At least 6TB storage
  • Backup that isn't in the same PC as the main media. I already have offsite backup of the most important documents
  • Uptime during the daytime


Plan / Options
  1. Buy another 3TB internal and external. Manually manage the media separation between the two 3TB drives
  2. RAID10, then 2 3TB HDDs for backup. Makes sense, but 18TB to store 6TB of data - is that the most efficient?
  3. Use the 3TBs as backups, make a new NAS / RAID out of smaller drives. Will this be a better long term solution?


I have an old PC that is a bit power-hungry, but could be used as a NAS / RAID, if needed. The CPU takes about 95W, so it wouldn't be a cheap or green solution, but its free for use.

Picking the best long term option takes priority. Once that is decided, I'd like to minimize the costs there as much as possible.

Which option - or a new one - should I go with? Also, any HDD recommendations?
 
Bluray rips only need several megabytes per second (24Mbps / 8 = 3 megabytes per second).

Personally I would look at RAID5 or an equivalent. It gives you reasonable assurance from hardware failure, but also gives you efficient space usage. I can't say that I would recommend the on-board RAID controller (if available). However finding an affordable RAID card that does hardware RAID 5 & supports >2.2TB drives is not that easy. Looking at about $90-100 off of EBay for a used server part (like an IBM or Dell controller). Speed does "suffer", but it's pretty irrelevant in media storage applications.

That being said, if you do not need the file server to run Windows, then ZFS is an excellent choice. You do not need any expensive hardware and you can add drives to the pool later on and the pool just grows. Plus it adds some additional protection of file integrity that hardware RAID will not do.
 
3x 4TB drives in RAID 5 will give you a resilient 8 TB. Rebuild your old PC with that in it. Just use XP or 7 and use Synctoy or Robocopy to back everything up.
 
Thanks for the replies. Two people mentioned RAID5 and it makes sense. Then I read articles like a "raid5 is dead" article like this and that scares me a bit. Is this truly an issue or hype?
 
How many disks will you have in your array? I would only use RAID5 for 4 to 6 disks. If you have more disks than that RAID6 (5 to about 14 disks) or other levels like RAID60.
 
drescherjm said:
How many disks will you have in your array? I would only use RAID5 for 4 to 6 disks. If you have more disks than that RAID6 (5 to about 14 disks) or other levels like RAID60.
Click to expand...

3 would make the most sense for 6TB of storage, although realistically I'll need to add 3-6TB more storage (1-2 disks, using multiples of 3) in 1-2 years. Please tell me if I'm being too concerned with this. Doing some quick math:

Byes read / URE = % failure chance
3x3TB disks in RAID5 = 4.8x10^12 / 10^14 = 48%
4x3TB disks in RAID5 = 6.4x10^12 / 10^14 = 64%
full calculation = 3 TB * (size of array -1 (b/c RAID5 = 1 disk)) * 8 bits per byte * 10^12 to convert to TBs. / URE, then convert to percentage​

So with RAID5 I'm already ~50% chance of not rebuilding. That doesn't take in
to consideration any recovery procedures that RAID has, and maybe that makes my concern unfounded and RAID5 worthwhile. I'm not experienced enough with RAID to know if that is true, or experienced enough with handling large amounts of data to know if I'm worrying too much.
 
fhrosty said:
Thanks for the replies. Two people mentioned RAID5 and it makes sense. Then I read articles like a "raid5 is dead" article like this and that scares me a bit. Is this truly an issue or hype?
Click to expand...

Hype. Totally applicable to your needs. They're speaking more to much larger data needs.

+1 on the Synology
 
I can tell you that those % chance of failure to rebuild from URE analysis that I have seen are very much over stated in my experience with 200 or so drives spinning 24/7/365 in multiple raid 5/6 arrays over the last 17 or so years.

Edit:

I believe the reason is 1 bit error in 10^14 bits that the drive manufacturers state is somewhat understated. The real error rate is somewhere between 1 error in 10^14 and 1 error in 10^15.

Also Synology uses linux software raid so I know a single bit error will not cause the recovery to totally fail.
 
Last edited:
I'll go with Raid5 on Synology or my old PC as a NAS + external drives as backups. Thanks everyone for your comments!
 
Also if you look at Enterprise drives these have URE rates of 1 in 10 ^15 and above. 1 in 10^15 is 1 bit error in 125 TB of data.
 
fhrosty said:
I'll go with Raid5 on Synology or my old PC as a NAS + external drives as backups. Thanks everyone for your comments!
Click to expand...

To give you an idea on what you can do with the Synology.

I use their software called "CloudStation" which syncs my Documents, Downloads, Pictures and Movies folder automagically to the NAS.

On the Synology NAS I run CrashPlan that is mapped to those same directories. CrashPlan is for offsite backups.

So, whenever or whatever I am working on that involves media, or critical folders that I chose are automatically backing up to the NAS over WiFI or LAN whenever I'm home working, and behind the scenes CrashPlan is offloading those data offsite. I don't have to do anything anymore.

Import pictures to iPhoto? Backed up to the NAS and CrashPlan
Create a movie? Backed up to the NAS and CrashPlan
 
RAID 5 "is" really scary. Lose more than one drive and you lose the data, unless of course if you have a backup. Also, RAID 5 writes are quite a bit slower unless you buy a very good controller. If you want a giant volume, do RAID 10, but otherwise, RAID 1 would be the way to go for the maximum amount of data protection with typical raid. You could go with a Dobro or similar that is unraid. Read more on their site for an explanation.
 
Raid5 with consumer 7200rpm 3 TB drives all in one filesystems is a bit suicidal. The rebuild time is enormous and if your remaining disks die from the additional load caused by the rebuild you have a problem.
 
uOpt said:
Raid5 with consumer 7200rpm 3 TB drives all in one filesystems is a bit suicidal. The rebuild time is enormous and if your remaining disks die from the additional load caused by the rebuild you have a problem.
Click to expand...

At work I had longer raid5 rebuild times (than the 12 or so hours it takes to rebuild using 3TB drives) with 250GB drives a decade ago.
 
You must log in or register to reply here.
Back
Top