Looking for input on my home media storage solution

A

Acejam

Weaksauce
Joined
Nov 4, 2009
Messages
65
Hey all,

Here's my current setup:

2 x 1TB Seagate 7200RPM drives in RAID0
1x 1.5TB WD MyBook e-SATA/USB/Firewire
1 x 80gb Intel X25-M SSD

Asus P6T Mobo
RAID Controller: Intel ICH10R
Intel i7 920 D0 @ 3.8ghz
6GB Corsair Dominator
Antec 1200 Case
Windows 7 Ultimate x64

Long story short - I was previously running my two Seagate drives in a RAID1, which limited my capacity to 1TB of data. About a week ago, I came close to almost running out of space, with only 15GB free. At that time, I backed up all of my data on the RAID1 array to my WD MyBook, and then converted my RAID1 array to a RAID0 array, effectively doubling my storage capacity. My OS is installed on my Intel SSD of course.However, this also increases the risk of loosing data. This RAID array is used primarily for storage, containing lots of music/mp3's, blu-ray movies, DVD movies, etc. Files range from 2MB up to 75GB.

I picked up the WD MyBook for cheap from BestBuy last year on a tax-free weekend here in MA. Even with e-SATA, this thing sucks and it keeps disconnecting randomly. From reviews on the net, I'm not alone - no wonder why BestBuy and other places no longer carry this item - WD stopped making it because it was a flop. It was pretty the only "Home" MyBook that had e-SATA in a large size. At the time, $120 was a deal. :eek:




Moving forward, I was at my local MicroCenter this past weekend and picked up some new drives. I found that they had Samsung HD203WI drives on sale for $109 each. I'm sure they could be found cheaper - but I decided to pick up 2 of them anyways. From NewEgg and other reviews, these F3 drives are very reliable and a great match for storage/RAID. My question is - how should I use these new drives? Should I pick up 2 more for a total of 4, and setup a RAID5 array?

Current thoughts/ideas:

2x1TB Seagate drives in RAID0 for 2TB of backup
2x2TB Samsung drives in RAID 0 for 4TB of storage

2x1TB Seagate drives in RAID0 for 2TB of backup
2x2TB Samsung drives in RAID1 for 2TB of storage

2x2TB Samsung drives in RAID1 for 2TB of backup
2x1TB Seagate drives in RAID0 for 2TB of storage



For now - 2TB should be OK. However, it's possible that I might exceed this within the next 3-6 months. I've basically begun ripping my media collection, which is the cause of the sudden increase in storage capacity. However - at the bare minimum, there's only 10GB or so of data that I can't loose, which includes vacation pictures, etc. Perhaps I should look into an online backup solution for that.

I'm also using the onboard Intel ICH10R raid controller for the time being. I've never had a failed disk or array failure, and this includes my older Asus P5T board as well, when I had my Core2Duo E6600.

All drives are currently being housed in my Antec 1200 case. I thought about building a ZFS or NFS setup from an old box but I'm not sure I need a dedicated box yet. However in 1-2 years time I plan on building a monster storage 24 bay hot-swappable rack server, once I save up the $$ and plan it out. (and once I move into a new place) I also don't have a HTPC yet - but I plan on building a front-end/back-end setup down the road. For now I'm using my PS3 with great success.

So, thoughts? Any and all input is greatly appreciated. I'm basically looking for an effective solution to increase storage capacity, reliability.

Thanks in advance,
Ace
 
Last edited:
You should consider FlexRAID. You could use one of your 2 TB HDDs as a parity drive (PPU), and the other HDDs for data drives, which would allow the loss of one drive without losing data, and give you a data capacity of 4 TB to start. You could easily expand it later by adding any number of 2 TB drives.

And you should definitely back up your irreplaceable files somewhere else than your media volume(s). The FlexRAID drives should be reserved for data that is not modified often (added files are fine, but modifying existing files requires recomputing the parity immediately to prevent data loss in case of failure).
 
john4200 said:
You should consider FlexRAID. You could use one of your 2 TB HDDs as a parity drive (PPU), and the other HDDs for data drives, which would allow the loss of one drive without losing data, and give you a data capacity of 4 TB to start. You could easily expand it later by adding any number of 2 TB drives.

And you should definitely back up your irreplaceable files somewhere else than your media volume(s). The FlexRAID drives should be reserved for data that is not modified often (added files are fine, but modifying existing files requires recomputing the parity immediately to prevent data loss in case of failure).
Click to expand...

I had seen FlexRAID mentioned in a few other threads - but I'm still trying to get a good grasp on exactly how it works. I've visited their site, but I'm still reading through it. Thanks for the recommendation.

I have all my data on my MyBook right now backed up. I think I'm going to unplug it later today, and then try formatting my 2x2TB Samsung drives into a RAID0 array - just to get some read/write numbers. (for testing/out of curiosity)

From what I can see so far - backing up a RAID0 array onto another RAID0 array is pretty much pointless - no?
 
Not completely pointless, since one RAID-0 could fail while the other survived, thus backing up the data. But it is riskier than other alternatives, and I would not recommend it for important data.

As for FlexRAID, the key things to note about it are that it is file-based (not block-device-based), it is not striped, and it is currently snapshot RAID (not live RAID).

Being file-based, it can fully utilize different sized HDDs in the same RAID. The only size requirement is that your parity disk(s) need to be at least as large as the largest disk in the RAID (more accurately, have as much capacity as the most-filled data disk in the RAID).

Being non-striped, the HDDs can be used independently if necessary (for example, if you lose 2 HDDs from a single-parity set, the data on the other HDDs is still intact), and you can expand the RAID without writing to the other HDDs in the RAID. Also, you will usually only be reading from one disk at a time, which saves power compared to a striped RAID where all disks are accessed in parallel for most reads.

Being snapshot RAID, you want to avoid modifying files already in the RAID at the last parity snapshot, since after modifying files you need to generate new parity immediately or risk losing data in the event of a disk failure. Adding new files is fine, since then you only lose the new files in the event of a disk failure before snapshot parity generation. But modifying files already in the RAID without updating the parity snapshot is risky.
 
Last edited:
I'm not sure if FlexRAID is such a great idea for my needs. In all honesty, I don't want to have to worry about modifying files and snapshots. Occasionally I will need to modify an existing file, or unzip/untar a file - creating 30GB in one shot.

I need to pick up an Intel 1000 NIC sometime this week for my ESXi lab build - I'm debating on picking up 2 more Samsung HD203WI's while I'm @ MicroCenter, throwing them in, and building a RAID 5 array.

I'd then take my "old" Seagate drives out, and throw them into some type of enclosure. Perhaps a DNS-321?
 
If you only do that occasionally, then you can just regenerate the parity right after you do it. But a better idea is to have a separate place where you store files that you modify. With your unzipping example, I assume that you mean you delete the .zip file after the data is extracted. For FlexRAID, you could have a scratch disk where you unzip the file, and then copy the unzipped data to your RAID. Or you could copy the .zip file to the RAID, unzip the data, and then delete the .zip file, as long as the parity snapshot does not get generated between the time you copy the .zip file to the RAID, and when you delete it (generally, people set the parity snapshot to take place daily in the early-morning hours).

The FlexRAID should really only be used for media files and other write-once, read-many type of files.
 
Last edited:
john4200 said:
If you only do that occasionally, then you can just regenerate the parity right after you do it. But a better idea is to have a separate place where you store files that you modify. With your unzipping example, I assume that you mean you delete the .zip file after the data is extracted. For FlexRAID, you could have a scratch disk where you unzip the file, and then copy the unzipped data to your RAID. Or you could copy the .zip file to the RAID, unzip the data, and then delete the .zip file, as long as the parity snapshot does not get generated between the time you copy the .zip file to the RAID, and when you delete it (generally, people set the parity snapshot to take place daily in the early-morning hours).

The FlexRAID should really only be used for media files and other write-once, read-many type of files.
Click to expand...

Respectfully, I think my point is that I don't want to even have to worry about what you just summarized, regarding FlexRAID. Sometimes, I will keep the zip/tar file around - just in case my extracted working copy gets messed up. Often times these can be zipped/tarred VM's set to a specific snapshot/working state.

What if I were to pick up two more Samsung drives and run a RAID 5 or RAID0+1? (or RAID10 even)

aca0f2d1531c431f10c7f222d1b413b1.png


RAID5 should give me 6TB of usable space. Would RAID5 have any issues streaming full 1080p HD movies to a PS3? No databases here....
 
Last edited:
Essentially anything can stream 1080p content. You can even do it from something like a USB flash drive.
 
Acejam said:
Sometimes, I will keep the zip/tar file around - just in case my extracted working copy gets messed up. Often times these can be zipped/tarred VM's set to a specific snapshot/working state.
Click to expand...

Keeping the zip file is not a problem. The problem comes in if your most recent snapshot included the zip file, and then you delete the zip file, and then you have a drive failure before the next snapshot.
 
FlexRAID is good if you have a bunch of large media files that do not change very often. It isn't completely hands off, but as previously stated you can use different size drives, its hardware independent, and its more flexible in some scenarios.

Real RAID, whether software or hardware, will be more 'automatic' with the data protection, but requires same-sized drives, may have hardware requirements, and a couple other exceptions here and there. I would never trust data I do not want to lose to a RAID 0, unless its in a RAID 10. RAID 0 is for speed; if you don't need speed, don't use it. Remember RAID is for availability and not backup; always have a separate backup of your important data.

Windows Home Server may be another option if you want the simplest method, but it has its drawbacks. I personally use WHS for the simplicity. ZFS is powerful and efficient if you are comfortable with Linux/BSD.

I believe the data rate for a typical, compressed 1080p stream tops out at 40 Mbit/s (5 MB/s) which is Blu-Ray. A typical hard drive has a data rate of 70-80 MB/s, or 20 times the data rate you need to send 1 stream. You do not need RAID to stream typical, compressed 1080p content.

-Cool-
 
Ok, this is all great to know, thanks guys. I've read that RAID5 can be a great redundant storage array - but isn't very good for database use. This is why I asked about the HD stream(s).

I'm thinking about picking up two more Samsung drives, going RAID5, and having 6TB of usable capacity. Then with my other (2) 1TB Seagate drives, I'll either run them in a RAID1 or RAID0 as a backup, and might run this on a D-Link DNS-321 or other NAS/backup setup.

Still researching...
 
Where RAID5 takes the biggest hit is in write speed. Databases depend on lots of writes, thus the performance issue. There's something to be said for not using a RAID5 at all, but to use a RAID10 instead. There you have a mirror of striped drives and you gain better reliability against multiple drive failures. Without taking the performance hit. What you don't get is the added capacity.

For delivering video in a home entertainment situation it really isn't going to matter which type you use. They're all more than capable of delivering the content fast enough for playback.
 
What would you say the pro's and con's of RAID10 vs. RAID0+1 are?

It sounds like RAID10 is faster - but does it provide true redundancy? My understanding from the simple usable capacity equations is that RAID10 would give me 4TB of usable space, whereas RAID5 would yield me 6TB.

With performance aside, for a storage solution it sounds like RAID5 would be more practical, no? Both setups appear to provide redundancy, however RAID5 has a parity disk, but also increases storage capacity, therefore having a better $/usable GB ratio.
 
There is no reason to choose RAID 01 over RAID 10. The fault-tolerance of 01 is much worse, and the performance is about the same.

capacity RAID 10 = N/2 * X
capacity RAID 5 = (N-1) * X

Personally, I would never even consider doing ICH RAID in your situation. If you do it, make sure you don't put anything on there that you don't have a good backup of or don't mind losing.
 
RAID10 is better than RAID5 for redundancy's sake. You can suffer a two drive failure and still keep running, whereas RAID5 can only lose one. That and it's faster. Yes, you'd get more storage out of a RAID5. I'll take better redundancy and great speed over slightly more storage any day.

That and I prefer to use a decent hardware RAID card. I went with an Areca 4 port to start with but upgraded to a 16 port for the sake of future expansion (and a deal too good to pass up on fleabay). I get better than single drive performance, even out of a RAID5 off it.
 
Oh and the warning about ICH or other motherboard RAID configurations should be heeded. Some of them WILL NOT ALLOW using 2TB drives as part of the array. It was quite the rude surprise to discover I could not get an nVidia motherboard to create a RAID5 using 2TB drives. I believe some of the older ICH setups have similar limitations.

Also consider you generally cannot boot from a greater than 2TB volume. You can certainly mount larger volumes after booting. Decent hardware RAID cards will let you create more than one volume from within a set of drives. So I have one machine set up with 4 2TB drives. There's an 80GB boot volume setup as a RAID10. The remainder is setup as a RAID5 for data. Works great. Few motherboard RAID setups will do this properly (most still run up against the 2TB volume limit).
 
RAID 10 fault tolerance is only slightly better than RAID 5. It is best to consider the worst case if your data is important. In the worst case, the second drive failure kills your array in RAID 10. And the worst case is not so unlikely, since to restore a RAID 10 after losing 1 disk, you need to completely read the other disk in the mirror pair, thus subjecting it to stress and exposing any bad sectors. That disk is therefore more likely than the others to be the next to fail.
 
Drives in arrays get beat up. One being used to handle the load due to failure is unlikely to be more stressful than any other regular use. And wouldn't a failure of a RAID5 cause more "stress" for two drives?

In no cases is maintaining operation in a failed state going to be free of stress, regardless of array type. So pardon me if I don't fully buy into that argument.

I've had a two drive failure in an array. It was a RAID10. I got the drives swapped out and rebuilding without having to take anything offline, or suffer the reload times of pulling that much data in from backups. Yes, the performance on the array took a hit. But the system was setup to have extra capacity to begin with. So users on the box never noticed a thing. Now there are hot spares for the controller to fail over to automatically.

An additional point to consider about swappable drives is the impact of removing units on the other still-spinning mechanisms. Having hot spares gives you the opportunity to postpone removing the defective drives until the machine can be brought down. The swappable drives thus help avoid having to spend extra time unscrewing a lot of things to rip and replace.
 
You missed the point, so I will repeat it more simply. In the worst case, RAID 10 arrays are lost on the 2nd drive failure, just like RAID 5 arrays. And the worst case is not unlikely.
 
No, I didn't miss your point. Your point is simply wrong. Two drives dying in a RAID5 *will* stop it. Two drives dying in a RAID10 will not. Is that simple enough for you now?

Neither is a good scenario. Two dead drives in a RAID5 means you MUST recover from backups. Two drives dead in RAID10 means you only have to rebuild from the already running array. Sure, worst case means you better have backups AND you better have an understanding of just how long it'll take to restore from them.
 
You are incorrect. My point is logically sound. I will simplify even more: in a RAID 10 array that has lost 1 drive, if you lose the other drive in the mirror pair, the entire array has failed. That is incontrovertible.
 
FIGHT FIGHT FIGHT!!!

Really, arguing about RAID X config vs RAID Y config makes me rofl.

If you want to use RAID, you should educate yourself on the options, and then chose the setup that best meets your uptime and performance needs. It's really that simple.

To the OP:

You're probably already aware that your CPU and RAM are way overkill for what sounds like a single user NAS. I suggest that you stray on the side of reliability and avoid RAID 0 - Unless you really want max our your 1gbps ethernet when transfering rips to this file server, you won't notice the performance during playback... I'd spend some time checking out ZFS if I were you.

Oh, and take the hard drive out of the external enclosure. I've done that with a few WD externals - you'll be happy you did.
 
Last edited:
styx0r said:
FIGHT FIGHT FIGHT!!!

Really, arguing about RAID X config vs RAID Y config makes me rofl.

If you want to use RAID, you should educate yourself on the options, and then chose the setup that best meets your uptime and performance needs. It's really that simple.

To the OP:

You're probably already aware that your CPU and RAM are way overkill for what sounds like a single user NAS. I suggest that you stray on the side of reliability and avoid RAID 0 - Unless you really want max our your 1gbps ethernet when transfering rips to this file server, you won't notice the performance during playback... I'd spend some time checking out ZFS if I were you.

Oh, and take the hard drive out of the external enclosure. I've done that with a few WD externals - you'll be happy you did.
Click to expand...


Good call about the external drive! It definitely wouldn't hurt to have an extra scratch or temp disk.

I've created the RAID0 array on the new drives and performed some benchmark testing on them. I will post up the numbers later. I'm pretty certain that I'm going to move away from RAID0 though, as I don't want to loose data. I WILL be keeping backups of the important data. I plan on scheduling a daily backup every morning around 2-3am, which will back up to a single SATA drive. The "important" data is only around 20-30GB. Perhaps I should pick up a Mozy online backup account while I'm at it.

Right now, besides my ESXi lab, this is my main computer. I use it for everything from gaming (Starcraft 2) to Photoshop, to web development & folding. Clearly, lots of people build huge NAS and dedicated media server, using ZFS, SAS, etc. For me, I don't see the point of a dedicated machine just yet. Right now, all I do is stream the occasional 1080p rip to my PS3 using PS3 Media Server. Therefore, I don't see why I can't just throw in a few more drives, and call it a day. If I'm streaming a movie, i'm not at my machine - so it handles this role with ease. Basically - this machine is doing it all for me, and so far it's been 100% rock solid.

In a couple years or so, I plan on building a large media server using some type of Norco rack setup. For now, these extra drives are holding me over until then.

Still debating between RAID10 or RAID5, but RAID10 sounds better for what I want...
 
Anymore input about RAID5 vs RAID10?

I'm pretty sure that I'm going to pick up 2 more HD203WI's this weekend. My local MicroCenter has them for $99 each now.

RAID10 = 4TB of usable space
RAID5 = 6TB of usable space

Performance isn't a huge concern. Like I said before, I'm backing up the really important stuff. So yes, there will be 1 or 2 TB's that I don't have backed up, and it would be nice to keep the array as reliable as possible. Should this make me lean towards RAID10?
 
Went to MicroCenter today and they only had 1 HD203WI in stock....ughhh. :(

They told me to check back on Monday since that is when they get deliveries. I checked their site before I left and it indicated 1 in stock, but I still went anyways since I needed to pick up a new switch for my ESXi lab.

NewEgg is also out of stock right now but I put my name on the auto-notify list. TigerDirect appears to have them - but I'm gonna hold it out and wait until Monday.
 
Acejam said:
Anymore input about RAID5 vs RAID10?

I'm pretty sure that I'm going to pick up 2 more HD203WI's this weekend. My local MicroCenter has them for $99 each now.

RAID10 = 4TB of usable space
RAID5 = 6TB of usable space

Performance isn't a huge concern. Like I said before, I'm backing up the really important stuff. So yes, there will be 1 or 2 TB's that I don't have backed up, and it would be nice to keep the array as reliable as possible. Should this make me lean towards RAID10?
Click to expand...

If performance isn't a huge concern, then why go with RAID 10? You get more space with the same drives with RAID 5.

-Cool-
 
john4200 said:
The Samsung HD204UIs are in stock at newegg, for $95 if you enter the promo code:

http://www.newegg.com/Product/Product.aspx?Item=N82E16822152245
Click to expand...

Wow....

I just checked MicroCenter's site...the HD203WI isn't even listed anymore. Like I mentioned before, they had 1 left in stock when I went there on Saturday, but the stores in other states still had a few here and there.

I just checked their site again, and it seems they have HD204UI's in stock now for $99 each. I'm debating on returning my HD203WI's, picking up 4 of the HD204UI's, going RAID5, and calling it a day. Do you think I will have any issues using my ICH10R on my Asus P6T board? I read a post on here from another member who used his ICH9R without issue. I'm on Windows 7 x64.

I thought about picking up Hitachi drives as well - but I'm not sure if they're worth it. They might spin a bit faster, but I've read they will use a bit more power, and tend to run on the warmer side.

Thoughts?
 
Just got back from MicroCenter. I swapped out my 2 HD203WI's and picked up 4 HD204UI's. Backing up important data now.

Now the question is - RAID5 or RAID10? Once I build this array, it would be nice to just "set it and forget it". I'm leaning towards RAID5 - so I have the extra space if ever needed in the future. (and since there's some redundancy, unlike RAID0)
 
Just did some basic benchmarks. Both tests were done using a "quick format" on the volume in Windows 7 x64. I also enabled write-back caching in the Intel Matrix Storage Console. (for both tests)

RAID10:

brandnew_4TB_RAID10.png


RAID5:

brandnew_4TB_RAID5.png



Thoughts?
 
Do I need to worry about aligning my own partitions using the ICH10R onboard Intel RAID? With a single drive, Windows 7 x64 should take care of it - but what about using the ICH10R controller in RAID mode?
 
You must log in or register to reply here.
Back
Top