advice on a home file server

mason.kramer

Weaksauce
Joined
Mar 27, 2007
Messages
64
Hi there. I'm trying to configure a home file server. I will use the server to store a large library of assorted types of media content, including high-def video, which I'll be serving to my TV's via myth TV (or something like it). I'll also use it in conjunction with folder-synching software to give me added data security, and I will use it to make regular backups of my entire system. I might also use it to host my low-traffic blog from home. This will be a low traffic server. The most strenuous possible scenario that I can imagine is it three users simultaneously streaming HD video from it. I have no experience with servers and I don't know what is required for that. I want to do it as cheaply as possible, but I want it to work well. I don't want any stuttering, I don't want any crashing, I don't want any hassles. I just want it to do its job without any nursing. The less power it uses, the better.

Here are my questions:

1. How much bandwidth does a streaming 1080p HD video signal use?

2.1 Given the application I described above, how fast does the system have to be to provide a high quality of service? I want it to be fast enough and no faster. Right now, I've been basing my configurations on a socket AM2 mobo and looking at cheap X2 processors. I know there's a deal on an AM2 X2 1.9ghz brisbane proc at newegg right now, but I don't like that deal because it doesn't include a HSF. Instead I found this 1.9ghz brisbane in a retail box. Would that be enough, or with a 2.4ghz windsor X2 for $50 more? Which one eats more power (don't windsors have smaller power envelopes?)
2.2 What is the minimum amount of RAM required for this application?

3.1 Can I get away with motherboard based raid 5 or do I need to get a card? For instance, this mobo utilizes the 570 chipset which supports RAID 5. However I have seen benchmarks that indicate that the 570 chipset has very bad RAID 5 reading and writing speeds, that it caps at reading 250mb/sec no matter how many disks are in the array. Would that be enough speed? Besides speed, the other thing that worries me about using a motherboard-based RAID controller is failure. What happens if the motherboard fails? Do I lose the array? Could I plug the array into another 570-based board and recover it? Should I buy a second board and keep it in the box just in case? The primary purpose of this server is security.

3.2 If the 570 chipset won't do it for performance, that might indicate a need to get a much more expensive AIC solution. But, could I do this more cheaply by getting a core 2 solution which could utilize an intel chipset with a matrix chip? I hear that the matrix chip delivers good RAID 5 performance. If so, which board and proc do you recommend? (Something with onboard video would be best, as I am only using this to serve. Also, it needs to have at least 1000mb ethernet, and maybe dual channels, depending on the answer to #1). Once again, what do I do about board failure - would I need to get 2 of the same, or could I just find any matrix board years later if the mobo actually does fail?

4. I'd rather not get a card because they add a lot to the cost of the system. However I want high security and I feel like a card will give me that. For one thing, if a card fails, I feel like I could plug the array into another ard of the same type and get it to work. Could I plug the array into another card of a different make but with the same chip and get it to work? If I do get a card, how cheap can I go? Can I get something like this high point controller, which seems to be about $100 cheaper than any other PCI-e solution? Will it perform? What happens if the card fails, do I lose the array? Can I go with a PCI-e solution or is it worth it to spend more and get a server motherboard with a PCI-x slot?


I know this is a barrage of questions. Answers and advice are appreciated.

edit: by the way, I want to have a minimum of 6 drive connectors. 4 does not suit my requirements.
 
Hello

You are looking at a very powerful system for just media and as a fileserver. You can see my FS in sign which runs more than smoothly, three months uptime now and stable as a rock. Doesnt take much power except during boot.

1. Depends on what kind of signal it is (mpeg, vc-1, h.264 and so on) but as far as I know, nothing exceeds 40mbit/sec yet. So in theory, the highest quality should be 36Gb of data for a two hour movie. I've yet to see one of these, Lord of The Ring uses 27Gb (without extra material).

2.1 Any system above 500mhz/256mb ram should be able to stream 40mbit/sec, so for the basic idea of streaming you don't need much. However, for other general systemstability and performance I would aim a lot higher. There is nothing wrong with the AM2- kits but the older socket 939 gives the same performance for less money. You don't need dualcore either for its purpose. I run a small webserver and some other serversoftware on my rig as well without any problems what so ever.

2.2 Get at least 1gb of ram. Can't stress that enough, if you decide over cpu and ram - go for the ram.

3.1 Motherboard based raid always use the cpu for its calculations. These are very intensive and are one of the main reasons you would go for more power than a 500mhz/256mb combination. For what its worth, you should take a look at doing software raid5 in Windows XP or Windows 2003, which is supported from the beginning in the later, or take a look in your prefered Linux dist.

Now I'm very aware of that some users would much rather sell their mother than using software raid5, especially under Windows, but I do since the release of Windows Server 2003 and I never had any problems. The best part is that you can move the array from one installation to another and changing hardware on the way, which you can't with a motherboard based array - unless it is the same chipset and drivers (and of course the same amount, or more, of sata/pata connections). If your motherboard dies you need a new one, same with a controller - if it dies you will need an exact copy of it to start up the array again.

The speed of a software raid5 is more or less the same as a motherboard controlled raid5 and the cheaper raid5 cards out there. At least here in Sweden you need to give out what you pay for a decent Intel Core2Duo based computer with a Geforce 8800GTS graphic card to actually improve raid5- performance over a software raid. And if you ask me you should also get a backup card for that system so it is really expensive.

3.2 My first suggestion would be to recommend the DFI Lanparty UT motherboard for socket 939. Eight sata connections (sata2 even, I think) and two gigabitnetwork connections. I'm not sure of how the motherboards looks after socket 939 really as I haven't seen the need for an upgrade since so I can only recommend what I have myself. 100mbit is enough for streaming 1080p but consider the very small cost for a gbit network. It is not fun to move 50gb of files from one computer to another going at 12Mb/sec. My FS can write over 60Mb/sec on the raid5- arrays which is over five times the maximum speed of 100mbit.

4. If you decide for a card, go for pci-e and make sure to read up on it. Also make sure of what you have to do if/when it fails.

And as a general reminder: Raid5 is not meant as "backup", only increased security. Always remember to do backups!

Regards
 
Thanks so much for your suggestion to go with software. After reading this post I've done a little research I think that's definitely what I'm going to do. For server applications that need a very high throughput, hardware RAID is the only answer. For workstation and gaming platforms, you don't want your CPU to be spending time XORing. But for low performance server applications, it looks like software is great. The software performance that people have been reporting using linux RAID 5 is completely acceptable for my application, and there are two huge advantages: it's free, and it's not a failure point. I went with an AM2 socket because it doesn't make much sense to get an older technology socket right now. The price difference is nominal for an upgrade path (who knows, maybe they'll make an AM2 barcelona). Here is my fileserver configuration:

http://secure.newegg.com/NewVersion/wishlist/PublicWishDetail.asp?WishListNumber=4334285

The only question was: 500GB drives @ $.25/gb, or 750GB drives @ $.4/gb? Would it be cheaper to get three 750's now for $900, and have a maximum capcity of 4.5TB, or would it be cheaper to get four 500's now for $500, and have a maximum capacity of 3TB? I decided on the latter to get costs down. If, down the road, I need to expand capacity, I have the option of getting a SATA add in adaptor card for a couple of hundred bucks to double my capacity.
 
Sounds like a wise plan. To convince you even more, the amount of disks are more important than having a fast cpu in this case. More disks will dramatically improve overall speed for you, the mentioned FS in my sign reads faster than my Gbit network can handle and writes in mentioned 60Mb/sec which is absolutely perfect for a small homeserver. I can't see how you would be disappointed. I'm going with 8x300gb in two arrays which is a perfect setup for me. Using a pci-e controllercard for the rest of the disks and speed is still fine. Only problems occur when more than two three users writes to the arrays at the same time, it can go horrible slow. But we are only two in the house plus its only four computers so nothing that really bothers me.
 
1. How much bandwidth does a streaming 1080p HD video signal use?
hd-bandwidth.PNG

This is a screen cap of Haali's Media Splitter playing a copy of Casino Royale in 1080p h.264. This particular cap is from the first part of the chase scene at the beginning of the movie; if you're familiar with it, this is from the guy at the snake pit to the top of the crane. As you can see, there's well under 30 mbit/s involved, and this is quite a lot of motion, in 1080p. I was streaming it over 100 mbit lan with no problems. Even uncompressed Blu-Ray content is under 50 mbit.
2.1 Given the application I described above, how fast does the system have to be to provide a high quality of service?
My "streaming-from" machine is a dual p3 at 866 mHz. It's way overkill. A Pentium 133 can generally saturate a hundred megabit line with static content.
2.2 What is the minimum amount of RAM required for this application?
Not much, depending on OS. 512 is plenty for XP, less even for Linux.
3.1 Can I get away with motherboard based raid 5 or do I need to get a card?
I would shy away from motherboard-based RAID. If you later want to change the board out for a different model, you'd have to restore from backups and all that jazz. Having a separate card is one step better in terms of upgrade potential. In my opinion, though, the epitome of migratability is OS-based raid. Linux Software Raid or Solaris' ZFS are good choices.
...reading 250mb/sec no matter how many disks are in the array. Would that be enough speed?
Way, way overkill. Even gigabit ethernet is only 120ish MB/s, and the content is only a max of 6 megabytes per second as calculated above.
4. I'd rather not get a card because they add a lot to the cost of the system.
If you go with OS-based raid as mentioned above, you can get a AOC-SAT2-MV8, which is a cheap software-only controller for only $112 at the moment. I've got two, they're nice ;) They go in PCI slots fine, even though they're inherently PCI-X. This means you'd be capped at 100 megabytes and change, but as noted above, this isn't much of a problem.
Can I get something like this high point controller, which seems to be about $100 cheaper than any other PCI-e solution? Will it perform? What happens if the card fails, do I lose the array? Can I go with a PCI-e solution or is it worth it to spend more and get a server motherboard with a PCI-x slot?
The Highpoint isn't bad. If you're running Windows that's probably your best option. Otherwise look into the Supermicro card linked above and software raid from your OS.
I know this is a barrage of questions. Answers and advice are appreciated.
Thanks for asking good questions. It's nice to try answer them all at once rather than all spread out ;)
 
Hehe thanks, I never learn when to use capital letters when it comes to this with bits and bytes :)
 
Now that I've figured out the hardware and the platform, I realize that I don't know anything about setting up an intranet file server. I think what I want is a domain server. Is that correct? Can anyone point me to a good primer on setting up a domain? I have macs and windows in my house, will that be a problem?
 
Now that I've figured out the hardware and the platform, I realize that I don't know anything about setting up an intranet file server. I think what I want is a domain server. Is that correct?
Depending on how much work you are willing to put in now and how willing to learn you are, a domain is a nice thing to have in a home. I know nothing about MACs, but in a Windows/ Linux environment a domain can provide you with a unified login, meaning that you'd be using the same account on all your machines. This also helps alleviate the pain of sharing files, synchronizing passwords etc.

Since Windows Server is expensive, you could consider using Linux as a domain server. It may even be possible that it supports a Win/ Mac mixed environment:
http://weblog.bignerdranch.com/?p=6
http://www.steve-lacey.com/blogarchives/2006/11/linux_as_a_wind.shtml

googling "samba domain server" also yields some good resutls. If this is your first time working with all of this, why not ``test drive'' it with VMWare? The server version is free.
 
Since Windows Server is expensive, you could consider using Linux as a domain server.


:p that works, since I'm already setting up linux to use linux software raid. In fact, I guess I'd have to do it on Linux!

Thanks for the links. I got linux running last night, now I just need to get a test array set up, but in order to do that I have to shrink some NTFS file partitions so I can borrow some space off of my windows drives and put them into a test array. Once that is done...I get to learn all about primary domain controllers.
 
^ i dont blame you, see how Effin big it was....i have to get back to work anyway!


bigger is better (sata)
 
Back
Top