What are you guys storing your media on?

I have all my media on 2x 2TB drives in my HTPC. I try to maintain (manually) a mirrored backup on my gaming PC, but that's down a drive since I added another to the HTPC for a recording drive.

I'm going to have to figure out some sort of home server solution. All I want is a mirror of everything in case of drive failure...
 
A parity solution makes more sense for media than mirroring. Parity can protect against a hard drive failure (and more if you use more parity) and gives much more space. Mirroring, imo, is best for really mission critical stuff, like documents. But I use cloud backup for that.
 
mdadm raid in Linux. I have 4.5TB of usable redundant space. There are 7 1TB drives in the array (1 is a hot spare). I store lot of other stuff on there such as VMs. I can grow that raid more by adding more drives too, without downtime.
 
Valaire said:
A parity solution makes more sense for media than mirroring. Parity can protect against a hard drive failure (and more if you use more parity) and gives much more space. Mirroring, imo, is best for really mission critical stuff, like documents. But I use cloud backup for that.
Click to expand...

Any way to create a parity solution just using plain Windows 7 and not using to use one of the software raid solutions?
 
AQ_OC said:
Any way to create a parity solution just using plain Windows 7 and not using to use one of the software raid solutions?
Click to expand...

Use a true hardware RAID card?
FlexRAID is an option as well.

Thats about it. Also, what is your picture suppose to say/show?
 
Flexraid, Hardware Raid. Flexraid has no striping, Raid 5 does. When the data is striped, if you lose multiple hard drives, defeating your parity scheme, you lose all your data. Without striping, if you lose multiple hard drives, you only lose what is on them.
 
Red Squirrel said:
mdadm raid in Linux. I have 4.5TB of usable redundant space. There are 7 1TB drives in the array (1 is a hot spare). I store lot of other stuff on there such as VMs. I can grow that raid more by adding more drives too, without downtime.
Click to expand...

+1 mdadm is a powerful tool for software raid in Linux. Though it does take a bit to become familiar with it.

I run Ubuntu Server and have a smaller 4 disk RAID-5 array. I haven't had any issues with it. I have experienced drive failures and I have recovered 100%. I'm actually going to add another 1TB HDD this weekend and expand my array!

I would actually recommend CentOS for you. Both Ubuntu and CentOS have virtualization options, but I think VirtualBox in CentOS is much easier to manage with the GUI. (Ubuntu Server is command-line only unless you install a GUI, which defeats the purpose.)

Just my 2 cents. :)
 
Do you guys generally play back your media over network streaming to a PC hooked up to your TV, or do you just hook the TV directly to the system with all the media on it?
 
Daggah said:
Do you guys generally play back your media over network streaming to a PC hooked up to your TV, or do you just hook the TV directly to the system with all the media on it?
Click to expand...
I generally stream things over the network to a 360 hooked up to the TV. For the few files that cannot be played by the 360, I hook up my laptop and stream media to that.
 
Danny Bui said:
Use a true hardware RAID card?
FlexRAID is an option as well.

Thats about it. Also, what is your picture suppose to say/show?
Click to expand...

It shows the mount points for all the hds in my system that comprise the 31 gb of storage. That's how I store my movies. All of that is shared over my network and available for viewing on my htpcs. I keep track of what movies is on which drive, so if a crash happens, I just rebuild that one drive. Since I mostly rip to iso using mymovies, the process won't take too much time. While I would like to have parity-type back up, I'd rather just use windows home sharing and stay away from any raid solution or spending money on a true NAS.
 
Daggah said:
Do you guys generally play back your media over network streaming to a PC hooked up to your TV, or do you just hook the TV directly to the system with all the media on it?
Click to expand...

I stream to two different htpc's and to my main computer, all over a gigabit network. Works very well. Mymovies is used to build the visuals into windows media center. I also have the mymovies app on both my iPad and iPhone, so I can call up movies with them on either htpc.
 
AQ_OC said:
It shows the mount points for all the hds in my system that comprise the 31 gb of storage. That's how I store my movies. All of that is shared over my network and available for viewing on my htpcs. I keep track of what movies is on which drive, so if a crash happens, I just rebuild that one drive. Since I mostly rip to iso using mymovies, the process won't take too much time. While I would like to have parity-type back up, I'd rather just use windows home sharing and stay away from any raid solution or spending money on a true NAS.
Click to expand...

Seems like a PITA to me. With 31TB I don't see why you wouldn't commit one drive to at least RAID 5 or two to RAID 6. While Windows itself might not allow the higher RAID levels, most motherboard softraid solutions do. Running either that or using a NAS solution would cost you $0 from a software point of view. I can understand the ripping to ISO isn't that difficult, but why go through it at all? Considering I've moved to 2TB drives even retrieving from backup 2TB of anything would be a pain.
 
kac77 said:
Seems like a PITA to me. With 31TB I don't see why you wouldn't commit one drive to at least RAID 5 or two to RAID 6. While Windows itself might not allow the higher RAID levels, most motherboard softraid solutions do. Running either that or using a NAS solution would cost you $0 from a software point of view. I can understand the ripping to ISO isn't that difficult, but why go through it at all? Considering I've moved to 2TB drives even retrieving from backup 2TB of anything would be a pain.
Click to expand...

I find raid solutions to be a PITA. That's why. And these 31TB are in use, only about 1TB free at the moment (I have more storage sitting on the shelf, though). Finally, this is so non-critical that I don't feel the strong need for backup. But if an easy solution presented itself, I might be motivated. But I'm certainly not going to redo this. And even if I had to rebuild a drive, it's not all that. All I would do is insert the disc and let mymovies do all of the work while I'm doing something else. In the meantime, I have some 28-29TB of other stuff to watch. Just as building it up was a low-effort affair, rebuilding a drive is as well.
 
Daggah said:
Do you guys generally play back your media over network streaming to a PC hooked up to your TV, or do you just hook the TV directly to the system with all the media on it?
Click to expand...

I have my media stored on my server PC (unRAID), which streams the stored media to my HTPC (xbmcLive) over the network.
 
Daggah said:
Do you guys generally play back your media over network streaming to a PC hooked up to your TV, or do you just hook the TV directly to the system with all the media on it?
Click to expand...

This is the HTPC forum. Streamed to an HTPC. :p
 
AQ_OC said:
I find raid solutions to be a PITA. That's why. And these 31TB are in use, only about 1TB free at the moment (I have more storage sitting on the shelf, though). Finally, this is so non-critical that I don't feel the strong need for backup. But if an easy solution presented itself, I might be motivated. But I'm certainly not going to redo this. And even if I had to rebuild a drive, it's not all that. All I would do is insert the disc and let mymovies do all of the work while I'm doing something else. In the meantime, I have some 28-29TB of other stuff to watch. Just as building it up was a low-effort affair, rebuilding a drive is as well.
Click to expand...

Different strokes for different folks I guess.
 
I stream over my gigabit network to a Boxee box and a WD media player, works well.
 
extide said:
I can't believe nobody has suggested ZFS yet!
Click to expand...

There have already been too many ways listed already, lol. Sucks when there isn't a consensus of "This is what is the best for what you want to do." I'm more confused now than before I created this thread. :(
 
officer921 said:
There have already been too many ways listed already, lol. Sucks when there isn't a consensus of "This is what is the best for what you want to do." I'm more confused now than before I created this thread. :(
Click to expand...

Yeah that's the situation today for home server OSes and data security: No general consensus besides "Don't use Window's built-in software RAID".
 
magao said:
Which part?

For the junctions/mountpoints I just use Link Shell Extension.

For upgrading (which obviously doesn't happen very often) I install the new drive (either internally, or using my SATA drive dock) and copy all the data I want to it. Then I run Junction Link Magic to list all junctions and delete any referring to the drive(s) I'm about to replace. Then I use the Disk Management section of Computer Management (right-click Computer, select Manage and change the drive letter(s) of the drives I'm about to replace to unused ones, and change the new drive to use one of the old drive letters. Then I recreate the junctions I want and uninstall the old drive(s).
Click to expand...

Thanks for this info, I had been wondering about this as well. (I am looking to move away from WHS and this would replace one of the few benefits of WHS)

AQ_OC said:
I find raid solutions to be a PITA. That's why. And these 31TB are in use, only about 1TB free at the moment (I have more storage sitting on the shelf, though). Finally, this is so non-critical that I don't feel the strong need for backup. But if an easy solution presented itself, I might be motivated. But I'm certainly not going to redo this. And even if I had to rebuild a drive, it's not all that. All I would do is insert the disc and let mymovies do all of the work while I'm doing something else. In the meantime, I have some 28-29TB of other stuff to watch. Just as building it up was a low-effort affair, rebuilding a drive is as well.
Click to expand...

Do you have a method of recording what is on each drive? That always seems like one of the pains if one were to lose a drive.... trying to figure out what was lost! (without having to wait until you go to watch it and it isn't there)
 
rhouck said:
Do you have a method of recording what is on each drive? That always seems like one of the pains if one were to lose a drive.... trying to figure out what was lost! (without having to wait until you go to watch it and it isn't there)
Click to expand...

I have an excel spreadsheet that I update whenever I rip new content. It comes in handy if I update a DVD title to blu ray. I track which disk, whether is is mkv, Iso, or folder and which type of folder it is in (rips, mkv, tv). Most tv goes to mkv, a movie go to rips, and music blu ray goes to mkv. This also helps if I want to transcode for my tablets when I get ready to travel. I just mount the iOS and run handbrake.
 
extide said:
I can't believe nobody has suggested ZFS yet!
Click to expand...

That's what I currently use. However, I'm not going to recommend something like that when it does take some knowledge to set up properly.
 
officer921 said:
There have already been too many ways listed already, lol. Sucks when there isn't a consensus of "This is what is the best for what you want to do." I'm more confused now than before I created this thread. :(
Click to expand...

I would say there's no consensus because there's no ONE correct way.What I can say is that:

A) Regardless of what setup you are running you should at least have your media on some kind of raid configuration. Whether that's FlexRaid, RAID (1, 5, or 6), software raid. or ZFS it's entirely up to you. But if you like your media then you need to run some sort of RAID not just for redundancy but because it also opens up flexibility when you start expanding. WHS (Windows) or mdadm+lvm (Linux) are extremely flexible. The former costs money, but is easier to setup. The latter is free but takes a little more skill. If you can't do either of those.... use FakeRaid (the one that comes with your MB). It's the cheapest option of all. It's quick to setup, easy to do, and costs you very little if your MB supports it.

B) Hard drives are expensive right now. Too expensive in my eyes (yes they've come down but not enough) So start small. A mirror (2 drives) will be cheaper to do, and will give you the basics of creating a RAID configuration. If two drives ain't no thang to you then buy 4 and start with a RAID 6. When doing RAID it's always better to think out your needs first and then create your array. The last thing you want to do is convert RAID configs.

C) If you are uneasy then go with what you know. However if you like learning something new then by all means go with what you want to do. Either way you should know how RAID works no matter what you choose to do.

I think that's it. Hopefully I've cleared up some confusion.
 
kac77 said:
A) Regardless of what setup you are running you should at least have your media on some kind of raid configuration. Whether that's FlexRaid, RAID (1, 5, or 6), software raid. or ZFS it's entirely up to you. But if you like your media then you need to run some sort of RAID not just for redundancy but because it also opens up flexibility when you start expanding.
Click to expand...

And yet there are many of us who think just the opposite. Who have had RAID setups and find they're a lot less flexible when you need to increase the amount of storage you have. Who don't feel a need for redundancy/backup for our media, because if we lose it we can get it back (e.g. re-rip or re-download it).

It's not just adding new drives into a pool (which ZFS makes very easy) but replacing existing drives with a larger one (or consolidating multiple drives into a single larger one). When you've already got 6 drives in a system, how do you upgrade one?

I've been through the pain and lost data in the past because I had a RAID system where when I set it up I thought I would be able to expand it - and then when upgrade time came found that I couldn't.
 
magao said:
And yet there are many of us who think just the opposite. Who have had RAID setups and find they're a lot less flexible when you need to increase the amount of storage you have. Who don't feel a need for redundancy/backup for our media, because if we lose it we can get it back (e.g. re-rip or re-download it).

It's not just adding new drives into a pool (which ZFS makes very easy) but replacing existing drives with a larger one (or consolidating multiple drives into a single larger one). When you've already got 6 drives in a system, how do you upgrade one?
Click to expand...

sudo apt-get install lvm2 (would be the easiest response) But if you're short on chassis space it wouldn't matter if you ran a file system called Shoe. You'll be in a bind either way. In the latter case you'll be transferring data to the new drive regardless of the solution either through RAID resizing, or straight up file transfer. I think you are trying to apply common problems of poor planning of storage with that of RAID itself. Poor planning whether you run anything on anything will bite you.

What if you turned on dedup on a storage pool of 10 TB with only 1GB of RAM? Would that be the fault of ZFS or poor planning? RAID is the same way. Had the person run a storage pool over the RAID collection, which many OSes allow you to do (for a reason) the problem of expanding storage isn't a problem at all. If you went mdadm it's not a problem either. If you went with good hardware raid controller it's easy there too. Basically if you plan before you dive in you won't spend more time or money than you need to. If you don't then it's really not the solution that's the problem but the person implementing it.

I did say before there is no one solution. They all have their pluses and minuses. However, poor planning can make any one of them an awful solution if not implemented correctly.

With that said, RAID and offshoots of it exist in every OS I can think of and every storage solution has some technology like it for the purpose of maximizing up time. It is NOT quicker to re-download or re-rip movies unless you've got hellacious internet access and the destination does as well or you've got a disc hopper where you can set it and walk away. In lieu of access to a DeLorean equipped with a flux capacitor in either case you'll be missing content until you pull the data off of your backup media.
 
I would never go non-raid. It just makes no sense to have a bunch of single drives. Not only can you loose data upon a drive failure, but it just makes organizing data harder. It's easier to just have one big pool and put everything in there. When you need more space you just add more drives and grow the array. None of "hmm which drive did I put that movie on again?"

A decent backup solution is a good idea, but if your data is not that important and you really can afford to re-get it, then you can omit the backups, but AT LEAST use raid!

Raid can be very flexible and provides better uptime during even big operations such as a resize. I actually watched a movie off my server when I was resizing the raid once. Flawless.

The only thing I really use single drives for is my backups. I treat them kinda like tapes and have a rotation.
 
I do like the idea of RAID and having one big pool. My problem is just figuring out which, of the MANY, options to use as the server's OS. After all of them mentioned here, I also stumbled across Amahi today, which looks interesting. Still have to read up on it more...
 
I like freeNAS my self easy to setup and I have yet to have problems. I have slowly been learning FreeBSD too been thinking of a major expantion soon. Planning is a big part of it. I don't think it really matters to much what you do for media just matters what it's worth to you. If all your movies tv shows ect not worth storing why even bother keeping it after you watched it. Now if you want to keep your data then no mater what you do you will want to back it up.

I personally tried Unraid and a few others people sudjested and I ended up going with freeNAS I find that my media server also serves up family pics and videos and sometimes I even install programs to run off it.

Cost might be the main reason to not go raid you can just as easly just buy 1HDD at a time and just back up the data when you need need too. So you need min of 3 HDD for RAID 5 that is around $300 for 4TB of storage if your going to do 8 drive array so you don't have to expand any time soon you are looking at over $800 for 14TB in RAID 5 or 12TB in RAID 6 that don't include the controller that supports 8 or more drives. If you buy everything new from newegg you could have $1000 or more in just 12 to 14 TB of storage. You still need to be able to back that up so to back up 14TB is 7 more drives add another $700 to the cost for a total of $1700 in 14TB of storage backed up and in Raid 5.
 
Last edited:
Nerdydave said:
I like freeNAS my self easy to setup and I have yet to have problems. I have slowly been learning FreeBSD too been thinking of a major expantion soon. Planning is a big part of it. I don't think it really matters to much what you do for media just matters what it's worth to you. If all your movies tv shows ect not worth storing why even bother keeping it after you watched it. Now if you want to keep your data then no mater what you do you will want to back it up.

I personally tried Unraid and a few others people sudjested and I ended up going with freeNAS I find that my media server also serves up family pics and videos and sometimes I even install programs to run off it.
Click to expand...

I'm looking into FreeNas as well. Amahi has caught my attention for the moment though. Seems to do everything the others do, appears user friendly, well supported, and has a large amount of plugins available. Including Sabnzbd, Sickbeard, and Couch Potato. All of which are a requirement for my server.

Unraid, Flexraid, and WHS 2011 are all off the table at the moment for various reasons. So I'm currently studying up on Amahi and FreeNas.
 
officer921 said:
I'm looking into FreeNas as well. Amahi has caught my attention for the moment though. Seems to do everything the others do, appears user friendly, well supported, and has a large amount of plugins available. Including Sabnzbd, Sickbeard, and Couch Potato. All of which are a requirement for my server.

Unraid, Flexraid, and WHS 2011 are all off the table at the moment for various reasons. So I'm currently studying up on Amahi and FreeNas.
Click to expand...
Awesome! Amahi seams really neat I never heard of them before I am checking it out too!
I have found trying to do to much with my file server just makes it in stable. What makes freenas nice there is not much else going on. I use another computer for DLNA granted I dont use DLNA much. My first server I used it for file serving live transcodes to mobile devices and transcoding movies for storage and it just created head aches lol just trying to do to much with the file server I guess
 
Nerdydave said:
Awesome! Amahi seams really neat I never heard of them before I am checking it out too!
I have found trying to do to much with my file server just makes it in stable. What makes freenas nice there is not much else going on. I use another computer for DLNA granted I dont use DLNA much. My first server I used it for file serving live transcodes to mobile devices and transcoding movies for storage and it just created head aches lol just trying to do to much with the file server I guess
Click to expand...

It looks like version 8.2 of FreeNas is going to enable plugin support, similar to what Amahi has. That makes it a harder choice. From what little I know, ZFS is pretty awesome and FreeNas has it.

Amahi seems to be a really user-friendly replacement for WHS. App support and disk pooling are both there. Still reading up, so not sure if RAID or anything is supported.
 
kac77 said:
With that said, RAID and offshoots of it exist in every OS I can think of and every storage solution has some technology like it for the purpose of maximizing up time. It is NOT quicker to re-download or re-rip movies unless you've got hellacious internet access and the destination does as well or you've got a disc hopper where you can set it and walk away. In lieu of access to a DeLorean equipped with a flux capacitor in either case you'll be missing content until you pull the data off of your backup media.
Click to expand...

I agree that a parity raid solution gives greater "up-time" in terms of having access to my full movie collection via streaming, but this is pretty low-importance stuff. All of it is on optical media so not having a drives worth of mostly ISOs isn't a huge issue to me. I have so much stuff in my collection that I'm really just a digital hoarder, because there is absolutely NO WAY I can watch all of this stuff. I have brand new movies on there that came out over the last year on blu that I have yet to watch (and I like it like this because that means there is always something I know I'm interested in that I can watch). AnydvdHD and My movies, along with a spreadsheet, makes rebuilding a drive a piece of cake. But yes, not as nice as a parity raid solution would be, but then that requires an entire drive for the parity (as I understand it, anyway). I have two 2TB drives in this system that I hope to one day replace with 3TB drives...just a swap & copy operation. I like the simplicity of just hanging the drives off the controller...no raid to worry about figuring. The mount points might cause someone else a moment of pause, but I know how I've got it set up, so it's simple for me.

I don't see the point of backing up a backup. Especially movies when the system is easy enough to simply rebuild.
 
I've looked at the all the different solutions, Napp-it, win2k8, freenas, etc. I ended up going the simple linux route. Here's why.

Reqs:
1. TimeMachine support and I wanted to do it via iSCSI
2. Simple samba support for file serving
3. Since I have under 2TB of data, mdadm RAID 1 is desirable
4. WakeOnLAN/ACPI suspend as I don't need 24x7
5. Scripting support
6. DR Replication

I chose CentOS 6.2 as it works with everything I require and has a very good support community. I ended up with 2 servers, the primary in the main house and the secondary in the guest house for DR/backup purposes(all connected via cat 6 gbe). Both servers are identical hardware, AMD 250u, 4 GB mem, 40 GB boot SSD, (2) samsung 1TB.

My main goal was TimeMachine as I needed to have an easy, scalable solution. I created a 500GB volume and it's been working great. The only issue I encountered here was I needed to be able to replicate this volume from primary to secondary or vice versa. I ended up using DRBD for this and it works wonderfully. Whenever I power on the secondary machine (via WakeOnLAN), DRBD automatically connects to the primary and only syncs the changed blocks from the primary. If the primary server is unavailable or is destroyed, I simply promote the secondary to primary and the iSCSI target is available and all my TimeMachine backups and samba shares are available. Really works well.

I used EXT4 for the file system which is simple, fast and easy. Samba was also a piece of cake to setup. Since I only currently require (2) 1 TB drives, it was really easy to create a simple Raid 1 setup. Rsync also works really well.

WakeOnLAN support is awesome (main reason why I didn't use freenas). Whenever I need the server(s) on, I send the magic packet(s) and voila, there they are. Whenever I want to suspend the server(s), I wrote a simple little script that I run and the servers go into suspend mode. Again, really works well for me.

Scripting was important for me as I have mentioned. I use it for the suspend action as well as for my rsync actions, which I use to sync the file systems between primary and secondary. I chose rsync over the DRBD option because rsync is really simple to implement and it works really well. I have backup scripts written, so when I ssh into the primary, I execute 1 script and it rsync's everything to the secondary. I could use cron here, but I don't require that automation yet. I also use the scripting to automatically sync my iTunes and Pictures from my macbook (yes, I know double redundancy here) so that every time I downloaded music from iTunes and/or sync'ed pics from my iPhone(s), I was guaranteed to get the changes over to the server(s). Again works really well.

All in all, this solution works great for me as I have a couple of htpc's that stream simultaneously without problem. Regarding htpc's, this solution also enabled me to free up some hard drives on my htpc's as I went to one small 30 GB ssd per htpc which allows me fast boot times and a single storage location for xbmc. Transfer rates are ~100 mb/sec and system idles at ~40 watts.
 
It all boils down three major variables and you need to decide which ones are really important to you.

Performance
Cost
Your Time

For example ZFS, has great performance and cost...BUT, you will need time (unless you aleady exist in a NIX world). A hardware RAID solution will have great performance and not suck up a lot of your itme BUT will cost oyu. A WHS system will have good cost and not suck up your itme...but performance will not be stellar. A prebuilt NAS will save you ALOT of time...but you will not get a good price and/or performance.

So..where do you sit?
 
You must log in or register to reply here.
Back
Top