New Media Server Build / Raid Controller Discussion

W

wardour

n00b
Joined
Sep 30, 2008
Messages
30
Background:
Around Christmas of last year I bit the bullet and decided to start building a media server. Before that my media was spread around anywhere I could put it. I started with 3x750gb drives in a linux raid 5, and have scaled up from there. Adding a drive or two / expanding the array every month or two.

I am up to 8 drives in Raid 5. Spread between a Promise 4 port SATA card, and on board controller. All the drives are Western Digital WD7500AAKS. Right now its all running on a Single core Opteron 148, in an Antec 1050 case.

I will be starting my build log tonight (first batch of hardware is arriving), but I am going with a lian li PC-V2000B Plus case, 2xLian Li 4 in 3 drive holders (uses 120mm fans which I prefer), and switching to an Asus nForce 650i board, with a q6600, and 2gb of ram for now.

I was looking for a decent budget raid controller for the setup. Mostly because the project doesn't really have a budget (more of an on going thing), but realistically I want to be able to drive 2 htpc's outputting 1080P content, as well as have data being continually written to the array (and processed). I will be running 2xNics one for each HTPC.

Reason for Post:
The reason I am writing this here now is, I have investigated so many potential options, that my head is swimming a bit. The 3 series Adaptec cards, and the Perc 5/i's seem to be the best bang ATM. Or possibly just sticking with linux software raid (I like the independence from hardware). I want to stick with Raid 5. My plans for now are to run 2 arrays, joined as a meta disk in Linux. I am going to use 1.5 TB drives for my future array (starting that in 2 weeks).

My board has 4xPCIE available, and the case supports 20 drives. (and I will find a place for an additional boot drive). So my eventual goal is 20 drives. 2 arrays of 8, and 1 array of 4. All combined into a meta disk.

My bandwidth needs are modest. I am not trying to be king of throughput. But I also don't want to be cutting it close, or if I eventually expend to 3 extenders, that I am upgrading everything.

I am going to post some benchmarks tonight, of my old setup, to see how much the hardware upgrade will effect linux raid (single to quad core, and Sata II over Sata I).

Right now I have a supermicro Sat2-mv coming, but I think I will be sending it back to the egg, in lieu of whatever you fine people suggest.


Also, just to belay the warnings. I have all 500GB of my irreplaceable data stored off site, and backed up to another machine.
 
Well I'd go then with the perc and hardware raid the configuration.

However, since this is a server, have you thought about looking into WHS? it will addres all your issues and you can expand whenever you feel like it with whatever controller you have and any sized drive you can throw at it.
 
I have years of *nix experience, and am very comfortable with the tools available in linux. I currently have no problems expanding my array when I want with mdadm, and raid tools. Basically the only difference in going hardware, is I only need to resize the meta disk / partition on each expansion (so it should be easier than it is now).

I am really impressed with Server 2008, and am using that as the os on both of my home theater pcs. (workstation edition mods). But I prefer linux for servers, because for me its easier to administrate (more experience).

Does anyone know how well ubuntu (in particular) supports the Perc card?

Also, I assume the best / only way to migrate to hardware raid, is to start a new array, and move the data over.

Also Ockie, do you think it would be better to go with one larger (12 port? 16port? card) for a bit more money, because of underwhelming perc 5i performance?

Right now, I am definitely underwhelmed by my current setup. I have 8 drives, and the second I start downloading something to the cluster, I can't watch 1080P content. I run a few sandboxed web servers for testing (software engineer), but nothing major. And I am not sure if I am running into controller bottlenecks or processor (probably both).

What do you all think of the Ciprico cards.

One thing I also noticed, is no one has posted any 256mb vs 512mb results for the perc 5i. If I order then, I will be throwing 512mb on them (have some spare ECC laying around), and i'll post some benches.
 
Well see you have an interesting scenario in that you want to be able to stream to multiple sources, and do writes simultaneously.

With Raid the data is scattered across all the drives so in order to read a single a file all the drives must be "reading" but if you want to write at the same time the heads on the drives will be bouncing back in forth between reading and writing because they will be doing this on separate parts of the platters.

I had this same Issue
I was running Software RAID10 on my server
All it did was stream media to my other computers, Print Server, and downloads torrents....Well when I got into HD stuff it wasnt working...I upgraded my network got new Video cards to offload processing from the CPU well it turned out that it was the RAID array.
About 2 Years ago I was on the microsoft connect site and saw WHS and I was intrigued.
I installed it on a test box and played with and began to really like it.
Well they way WHS works is kinda like JBOD so any given file is on a single drive, but it is all seen as a single volume. So I can play 3 HD movies simultaneously and be downloads and doing video conversion at the same time because the processes are using different spindles...WHS will write to a drive that is not being used so If I'm watching Live Free Die Hard and its on my 1tb Seagate it will write my torrent downloads to my WD640gb and I wont have any interuption.
 
I should be able to simulate the same thing using meta disks in linux as well. Id really like the redundancy that raid provides. I currently have 3tb of material, spread across 8 drives. There is a 100% chance that those drives are going to fail (fate of all mechanical devices). So I see the advantage of JBOD, but I would prefer a high performance raid that would allow me to do read writes.

Anyone else using raid successfully in a setup like this? (without going to more extreme sas / 10k->15k rpm)
 
WHS provides redundancy through a feature call "folder Duplication" and in my opinion is much better than raid because it allows me to decide what I want redundant...so lets say I have some home movies with my family I can have those duplicated and I can decided its not a big deal if I lose some Mp3s so I wont waste space there. You get to decide what is important.
 
Translation to linux: A cron script that duplicates files in certain locations. The issue I still see, is in your scenario, you will always be losing data when a drive fails, in my situation, I will only be losing data if 2 drives fail. (or an act of god). But more importantly I will theoretically get better throughput with Raid, than you will with single drives.

Another Idea, is to have a "staging" drive, or even use, say a larger hard drive for the OS, and use a partition for staging (download to there, and move from there to raid at 3am). Which is another solution.

The biggest reason for me to run linux from a practical standpoint, is I do software development, and need some sandboxed linux web servers. And I don't have room anymore for multiple boxes.

Anyone have any idea how bad my read performance will be hurt with writes being performed? Using Raid5 on either a Perc 5/i or an Adaptec 3085. Also how much better is the adaptec over the Perc?
 
On a Perc5/i with those requirements you are asking a lot

What you could do is what you were talking about the staging drive and then copying at night but honestly you prob wont be happy in the end and it will cost you money down the road. BTW do you have any sort of budget for this project?

You say you need linux for Development purposes, have you though about virtualization? It could make some of your processes faster. I have a couple Master VM's that when I want to test something I just clone the master and go I dont have to wait an hour to install and spend time setting things up its already done for me.
 
For raid controllers, I want to keep it under or around $500, for 16 drives. Which is why the Perc's are tops on my list. I know I am asking a lot, but from the numbers I have seen, it looks like they should do the job.

Software raid gets me close to what I need. I am looking at running a virtualization machine, but after looking at ESX its a pretty big commitment, and I currently don't have the hardware for that kind of undertaking. So it would need to be bought in.

It looks like the "easiest" way to setup ESX are to use OpenFiler as iScsi, and then have a whitebox ESX instance that connects to it. Which means two machines, instead of one, or buying more expensive supported controllers.

What kind of hardware would I need to setup a respectable ESX server (running 3 low usage web servers (centos), an instance of windows xp (for whenever I need to run an exe, or test a web site), a file server, and a giganews / torrent server? I am guessing at least 8gb of ram, and either quad or dual quad xeons.

I don't really have a budget, because right now I am not exactly certain what I am building. It started out as a board / case swap so I could add more drives. But after some research, I know I can get better throughput for not a lot of money ($350 for percs with cables), and If I can somehow go ESX server for not a huge investment, I would love to do that.

So what say you, what can I get by with? and what shouldn't I skimp on? I have a lot of hardware / software experience, but I am just learning about the low level virtualization game. Also are people recommending ESX 3.5 or 3.0 (due to issues with U2).
 
Yes ESX is a big undertaking and isnt always the best solution so heres what I would do

Run w/e flavor of OS that you like to run on your destop(linux, windows personally I would run XP as the host becasue you can RDP)...install VMware Workstation(its awesome) Take a Perc5/i and make a volume that will store your movies
Make another vlume on the Perc5 and it will house just your VM's (or if you want to get crazy grab some SSD's and RAID those)
Single Drive for boot not on Perc5(just preference)
You would need about 4-8gb of ram depending on how much ram you want to give your VM's(also your VM's can share/swap ram meaning you can allocate more than you have...I have 8gb but I can run 6 vm's with 2gb each because it will swap the memory when one needs it from one that is not in use.
Single Quad would do just fine...but a Dual Quad like I run you can do just about anything

What are you thinking for total space for movies, vm's etc.
 
my barebones ESX combo is for sale (for the OP, if he's interested). This issue is very close to the one that I'm having, as well. Ripping videos to disk for streaming, and needing to run mutliple VMs...and having the spindles to do it all efficiently (and within budget constraints) can be diffifcult.

FYI OP - if you're looking to do ESX for not a lot of $ (assuming you'll need some beefier hardware to tun it on, as well), then you should really be looking at ESXi, which is now free (but comes with more restrictive HCL, currently).
 
You must log in or register to reply here.
Back
Top