• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Media Storage/Server question

S

Shifte

n00b
Joined
Mar 9, 2009
Messages
16
So I am in the process of building a file/media server, mainly for storing/streaming video and music. This server will feed to a HTPC, Gaming PC, Office PC and Laptop. I don't expect more than 2 devices to stream at a given time.

I estimate needing 8-10TB of storage, currently have from 4-5TB spread out over all machines.

My goal is to use the best possible set up to account for hard drive failure. My thoughts were to use Raid 6. Though I am pretty new to the whole scene and am unsure if this would affect video/audio streaming.

Would Raid 6 work/is it recommended? Should I not consider Raid at all just just throw in some drives?
 
What OS are you using?
What type of RAID? (software, hardware, Fake, etc)

RAID6 would be appropriate, but again, we need to know a little more about your setup.
ZFS, as said above, would also be a good choice.
 
Not really sure, as I said I am not extremely tech savvy but was planning on using WHS or Ubuntu.

Id welcome any suggestions from hardware to software. Tired of transferring files around depending on where they "ended up" would like to centralize and give myself a little room to grow.

Most of the storage is over half movies, with the rest being music and about a gig of pictures etc. Looking to store everything simply and send new movies/TV from the HTPC to the server with ease.
 
It depends on your budget and personal feelings and needs. I personally like hardware RAID6 with Areca cards, but mdadm RAID6 and ZFS are also good choices. Do you plan on using the server box as a workstation as well? Do you care which OS it runs?
 
Honestly, don't really care what OS it runs. All PCs in the house run W7 (Win7/XMBC on the HTPC)

Budget ranges between 0-$1k Id like to go less than 700, but have budgeted up to 1k.
 
Well, since you said you want 8-10 TB of space, RAID6/ZFS-Z2 would need 5 3TB drives for example (or 6 or 7 2TB drives), and at current prices the drives alone will cost you a grand or more, before you figure in any other parts (the rest of the computer, controller(s), etc.) Also, at the risk of sounding like a broken record in some posts.... RAID/ZFS IS NOT A BACKUP! If you care about the data, you should have some kind of backup plan in place. Decide how much your time is worth having to rerip all the stuff you have, and how much time it would take to find again if it is video you don't have physically.
 
Check also UnRaid, its good option for media storage and serving your network pcs, you can test it for free up to 3 hdds, if you find it fits what you are looking for, get the licence for 6 drives.
 
@mwroobel I actually plan to pick the hard drives up last, after prices calm down, some 2TB drives used to be 60-80 are now over 200.

I was planning on using 5 (6 if needed) 2TB drives. Once I have everything (minus the drives) I am building a rack to put in the basement. Cat5 runs from there to each of the rooms needed already.

Side note: I was looking at the Spinpoint drives does this sound like a good investment for my needs?

You guys have been a big help already thanks for any additional support and information you can provide.
 
Shifte said:
So I am in the process of building a file/media server, mainly for storing/streaming video and music. This server will feed to a HTPC, Gaming PC, Office PC and Laptop. I don't expect more than 2 devices to stream at a given time.

I estimate needing 8-10TB of storage, currently have from 4-5TB spread out over all machines.

My goal is to use the best possible set up to account for hard drive failure. My thoughts were to use Raid 6. Though I am pretty new to the whole scene and am unsure if this would affect video/audio streaming.

Would Raid 6 work/is it recommended? Should I not consider Raid at all just just throw in some drives?
Click to expand...

What is your backup strategy?
 
@Trepidati0n - Great question, honestly this was a big concern, with this much storage how much backup storage on say a external drive I would need? Currently, out of all my devices only one is backed up.

Could you provide a suggestion on how much backup storage space would be needed? I would estimate only doing backups Once a week at most.
 
How would he know how much data you value? The answer could be as much as 2 times the size of the raid array or only a few GB depending on what risk you want to take.
 
My Apologies, perhaps I misunderstood.

Most of my data is Ripped movies (Blu/DVD) and music (22GB), which when added up is an considerable time investment. I would like to take the least amount of risk involved, while still being affordable. 2 Times the size of the Array does not seem like a good way to go.

Would it be possible to use a single 3TB drive for backups? I am unfamiliar with what would be needed, as currently only one machine has backups done, where backups are about 30GB on a 1 TB machine (No where near full). Using this logic, 30gb per 1TB 3TB would be enough i think depending on real world sizes.
 
Would it be possible to use a single 3TB drive for backups?
Click to expand...

Only if your entire data set is less than 3TB.

where backups are about 30GB on a 1 TB machine (No where near full). Using this logic, 30gb per 1TB 3TB would be enough i think depending on real world sizes.
Click to expand...

Your example is a result of compression of compressible data. You are not going to get any compression on DVD rips (you do not gain much if any from compressing already compressed data) so the total amount of data you have stored you will need about that much space to make a single backup. In a work environment (stuff that absolutely can not be lost) I recommend at minimum 2 full backups.
 
Last edited:
Shifte said:
Would it be possible to use a single 3TB drive for backups? I am unfamiliar with what would be needed, as currently only one machine has backups done, where backups are about 30GB on a 1 TB machine (No where near full). Using this logic, 30gb per 1TB 3TB would be enough i think depending on real world sizes.
Click to expand...
UnRaid uses only 1 hdd for parity (you should chose the biggest drive), and if you lose more than 1 drive, you only lose what was on those drives, its not like raid, you also have the advantage of using different sized drives and can add as you need or replace, etc. Overall its a very nice media storage server, the only downside that i do see in it, has slow write speeds 30-40 mg/s, but you can add a cache drive outside the storage array, that will allow you to transfer at normal speeds and the server will then move it to the storage array.
 
Abula said:
UnRaid uses only 1 hdd for parity (you should chose the biggest drive), and if you lose more than 1 drive, you only lose what was on those drives, its not like raid, you also have the advantage of using different sized drives and can add as you need or replace, etc. Overall its a very nice media storage server, the only downside that i do see in it, has slow write speeds 30-40 mg/s, but you can add a cache drive outside the storage array, that will allow you to transfer at normal speeds and the server will then move it to the storage array.
Click to expand...

This does not solve the backup problem. Unless you mean making a second server in addition to the raid6 server containing a bunch of mismatched drives and using that as backup of the first server.
 
drescherjm said:
This does not solve the backup problem. Unless you mean making a second server in addition to the raid6 server containing a bunch of mismatched drives and using that as backup of the first server.
Click to expand...
Not trying to make it sound as unraid as backup solution, true backup will only be external to the server, in another server, single drives, tapes, BR, cloud storage, etc. His backup should be his legally owned BR and DVDs regardless, thats good enough, the unraid pairty will just serve as uptime for what he has on the server.
 
Abula said:
Not trying to make it sound as unraid as backup solution, true backup will only be external to the server, in another server, single drives, tapes, BR, cloud storage, etc. His backup should be his legally owned BR and DVDs regardless, thats good enough, the unraid pairty will just serve as uptime for what he has on the server.
Click to expand...

Ah I understand, yea, I think that a 2nd sever for backups would be a bit much, Since I do own them I think Ill have to suck it up and just use the server and watch for possible issues.

Any suggestions on how much extensive of a CPU and RAM i would need? I wouldn't expect to need much, but RAM might need be helpful to have higher amount.
 
Abula said:
UnRaid uses only 1 hdd for parity (you should chose the biggest drive), and if you lose more than 1 drive, you only lose what was on those drives, its not like raid, you also have the advantage of using different sized drives and can add as you need or replace, etc. Overall its a very nice media storage server, the only downside that i do see in it, has slow write speeds 30-40 mg/s, but you can add a cache drive outside the storage array, that will allow you to transfer at normal speeds and the server will then move it to the storage array.
Click to expand...

The other thing is that you do not get ZFS like data scrubbing and you still only have single-parity. With big drives I am basically 100% on double parity or RAID 1 at this point.

UnRaid 30-40MB/s speeds and the cache drive are a bit different. You can read from multiple drives but write is where you see a bottleneck because you are basically doing RAID-4 without NVRAM cache/ WAFL (e.g. NetApp although they no longer recommend single parity from what I gather)

One thought is:
RAID 6 now with physical media backups.
Wait 6 months for drive prices to get better, then build an Atom/ E-350-based backup server// get 2-3x 4TB drives for offline backup.
Downside: live 6-7 months with only double parity protection (really good) physical media backups.

Another idea is WHS V1. You can just consolidate local drives into the server. Less efficient from a storage perspective, but you get much higher data security than RAID 4 / 5 and it is very flexible with drive extender. No cache drive and you can do full gigabit speeds if the drive allows. My guess is people have licenses that would be inexpensive to purchase at this point if you couldn't find one. I actually have not had spindle disks on client machines in over two years now.

Finally, UnRaid is not a bad solution for a media server. I would suggest putting files that you do not want to lose in something like Amazon S3 if you go this route. Media files are one thing, tax records, work files and etc are another.
 
pjkenned said:
I would suggest putting files that you do not want to lose in something like Crashplan if you go this route. Media files are one thing, tax records, work files and etc are another.
Click to expand...

FTFY. Sorry, but S3 compared what else is out there makes me cringe in terms of cost.
 
RAID6 (or 5) isn't going to solve your issue. Your issue is you have a lot of data you want to keep safe. RAID isn't going to do that for you. RAID5/6 can help you utilize multiple drives efficiently and reliably, but it's not a backup solution.

The issue you need to work out is how you're going to back up 10 TB of information.

You can either build two separate RAID 5 arrays and rsync them (reasonable considering you probably won't make many changes), or back data up on portable drives and have a system to separate out the files (a volume of A-G titles, H-T, etc.).
 
Actually, question: Do you have all your origional media? DVDs/Bluray etc? Do you plan on keeping them?
 
Trepidati0n said:
FTFY. Sorry, but S3 compared what else is out there makes me cringe in terms of cost.
Click to expand...

Totally agree it is expensive. On the other hand, they have a fairly robust system and it is pretty awesome if you do not use it for media storage. Powerpoints, PDFs, Excel docs, and etc only take up so much space, especially if you compress/ encrypt the files.
 
You can get 2 4TB USB3 drives at Bestbuy for about $500, so for a cheap backup you could use those. That would save you the time of reripping the BDs, and if you MKV them it will save you a hell of a lot of transcoding/compression time.

http://www.bestbuy.com/site/Seagate...lack/3551503.p?id=1218413796012&skuId=3551503 x2 is about the least expensive 8TB of external local backup space you are going to find right now.
 
I think ZFS is the way to go with Raid2Z. This way 2 drives need to fail before you are close to being screwed. This would be my backup plan as data like that is not "priceless" in my eyes. ;)
 
If you want to use S3 for backup take a look at Tarsnap. Personally, I back up all my important data on crashplan (about 170gigs, grad school stuff, photos, etc).

TV/movies/mp3s are not backed up, not really worried about loosing them.
 
Wow great suggestions!

So sounds like ZFS may be the way to go. I could see using a online solution such as CrashPlan. I had not thought of this but it would be cost effective.

Could anyone provide additional resources for me to read up on ZFS and/or WHS, as these two solutions interest me the most.

Thanks for all your help thus far guys, learning alot!
 
Shifte said:
Wow great suggestions!

So sounds like ZFS may be the way to go. I could see using a online solution such as CrashPlan. I had not thought of this but it would be cost effective.

Could anyone provide additional resources for me to read up on ZFS and/or WHS, as these two solutions interest me the most.

Thanks for all your help thus far guys, learning alot!
Click to expand...

ZFS's biggest attraction right now (at least to me) is it protects against the most evil issue on the face of the planet..BIT ROT. Unforutnately we tend to collect lots of data and never really look at it again. But when we do...it may not "open". This sucks IMO. ZFS however tends to make this fairly painless. ZFS, also compared to WHS will make it easier to handle larger arrays (8+ drives). The article below sorta drives home the issue a bit (pun intended). More simply put. If you have pictures that are ~2MB..~1 in 3000 will get corrupted in a year w/o protection. In some cases hte pciture may still open...however, they may not. videos may start playing and stop. Sorta scary :(

http://www.zdnet.com/blog/storage/data-corruption-is-worse-than-you-know/191

WHS on the other hand is probably the easiest platform to manage (if only working with a few drives). Add something like FlexRAID and you get quite a bit of benefit.
 
So I went through some old hardware I had laying around..

Asus P5G41-M-LE Motherboard with a max of 4 Sata 3gb and 8gb Ram

E5300 Dual core Pentium 2.6.

Would these work good for my media server? The board is a mATX so i might be limited on case choices
 
You must log in or register to reply here.
Back
Top