Specing a storage server for high def video

E

Eiolon

Gawd
Joined
Apr 6, 2005
Messages
928
I was wondering if anyone could give me pointers of what I am looking for in a server for storing high def video that will also be streamed? I am interested in knowing if multiple processors would be utilized or if a single processor is okay? What about cores? Will a large amount of RAM be beneficial? What about RAID? Redundency isn't an issue, but speed is important. Storage expansion in the future would be nice too.

I have about a $10,000 budget and we are contracted with Dell but if we can make a case for a different vendor that can be arranged.
 
It depends on how it is streamed. I know with my streaming severs (Flash,Silverlight,and Real) The trans-coding is done at the client side and the server just streams the video. If that's the case 95% of the time network will be your bottleneck. Single CPU will be just fine if the server isn't trans coding. I would spend the money on RAM and Disk. In my experience sata disk is just fine for streaming servers.

I'm in the process of replacing my Dell 2950's that have been doing all my streaming for the last 4 years with a single quad core and 2 gig of ram. With new R510s. I think I have dual quad cores and 12 gig of ram specked solely because it fit the budget. I would be comfortable with a single quad core and 12 gig of ram.

As for disks I usually will run Raid6 with near line SAS.
 
How much TB? Enough to max out 10k$?

How many clients streaming? How much mbps per stream?

Are you sure redundancy isn't an issue? You are okay with streams being unavailable?

This is what I came up with, I'd probably go dual g34 though :)

16 x http://www.newegg.com/Product/Product.aspx?Item=N82E16822145298
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16814102874
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16817139007
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16813182230
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16816151037
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16811219033
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16811997301
4 x http://www.newegg.com/Product/Product.aspx?Item=N82E16820148354
2 x http://www.newegg.com/Product/Product.aspx?Item=N82E16820167013
1 x http://www.newegg.com/Product/Product.aspx?Item=N82E16819105267

6700$ total for hardware

Run RAID2Z on that bitch with sub.mesa's help
14 x 2tb = 28tb of space. Run dedupe and compression enabled because you got 12 cores and 32gb of ram BALLER!

No for real though I'd try to find just a way to get 24 sata II ports for cheap, a raid card is useless for ZFS
 
Why is speed important?

A single hard drive has no problems in maintaining several HD data streams. (Is this bit mapped HD or is it encoded for OTA?) No need to have RAID unless you need more data streams than an individual drive will support.

You can do the math to determine how many data streams your network will support. Add enough capacity to get the number of data streams you need.

I guess for $10K you have big plans.
 
GeorgeHR said:
Why is speed important?

A single hard drive has no problems in maintaining several HD data streams. (Is this bit mapped HD or is it encoded for OTA?) No need to have RAID unless you need more data streams than an individual drive will support.

You can do the math to determine how many data streams your network will support. Add enough capacity to get the number of data streams you need.

I guess for $10K you have big plans.
Click to expand...


I'm guessing this is a companies streaming server. RAID is nice so that your all your drives show as one big partition and you don't have several mount points for the streaming server
 
I store my bluray rips on my windows home server. Using a Core 2 Duo e6600 for a CPU but prior to that I was using a P4 3.4ghz. 2gb of ram. Old DFI board now, 975x chipset. Was my old desktop. Using a variety of drives as WHS is ok with differing drives. But most of them now are WD green drives.

Doesn't take any power to stream video. I play it on my HTPC which handles the decoding and such so the server just had to send it there. I actually have two HTPC's and can stream to both of them 1080p bluray rips without issue.
 
DeChache said:
I'm guessing this is a companies streaming server. RAID is nice so that your all your drives show as one big partition and you don't have several mount points for the streaming server
Click to expand...
I think we are all guessing about the relevant issues.

For $10K one can put together a really massive system. The actual needs might be 4TB of storage today and 100TB by the end of the year and very little access. Or it might be a system with 4TB of storage with 20 users all day long. Very different systems.

We just don't know.
 
GeorgeHR said:
No need to have RAID unless you need more data streams than an individual drive will support.
Click to expand...

DeChache said:
I'm guessing this is a companies streaming server. RAID is nice so that your all your drives show as one big partition and you don't have several mount points for the streaming server
Click to expand...

I will second vraa's suggestion for RAID-Z2. RAID-Z2 is like RAID5 with two parity drives. Also, the ZFS file system is amazingly simple for snapshotting, rollbacks, sharing over nfs/smb, enabling compression, etc. I have it set up for my fileserver and it works like a charm.

Solaris 11 Express has the latest version of ZFS.
 
Thanks for all your comments everyone. There is clearly lots to consider.

The project is being handed to me in bits and pieces, but here is what I know. The high def video will be streamed to about 50 guaranteed clients. It will be streamed on a fiber based wide area network, and additional clients over the Internet may also have access to the video. That number is unknown.

Transcoding will most likely be done on the client end, but I will need to talk to our multimedia developers to confirm.

Redundency is not a huge issue and can be sacrificed if money can be put towards performance or storage of the server if that would be beneficial. Otherwise, redundency is okay to have.

What we currently have is about 4TB of footage created, we estimate another 12TB by the end of next year. We would definitely want room for expansion. As we delve into the realm of 3D I am unaware of how that will affect storage needs.

I appreciate all your advice on this as all this media stuff is relatively new to me.
 
black0ut said:
I will second vraa's suggestion for RAID-Z2. RAID-Z2 is like RAID5 with two parity drives. Also, the ZFS file system is amazingly simple for snapshotting, rollbacks, sharing over nfs/smb, enabling compression, etc. I have it set up for my fileserver and it works like a charm.

Solaris 11 Express has the latest version of ZFS.
Click to expand...


While ZFS is a worthy solution. I think we need OP to clarify what OS he intends to run. I have to question adding the complexity of a second server if he is going to run run windows. Its just one more thing to fail.

Eiolon said:
Thanks for all your comments everyone. There is clearly lots to consider.

The project is being handed to me in bits and pieces, but here is what I know. The high def video will be streamed to about 50 guaranteed clients. It will be streamed on a fiber based wide area network, and additional clients over the Internet may also have access to the video. That number is unknown.

Transcoding will most likely be done on the client end, but I will need to talk to our multimedia developers to confirm.

Redundency is not a huge issue and can be sacrificed if money can be put towards performance or storage of the server if that would be beneficial. Otherwise, redundency is okay to have.

What we currently have is about 4TB of footage created, we estimate another 12TB by the end of next year. We would definitely want room for expansion. As we delve into the realm of 3D I am unaware of how that will affect storage needs.

I appreciate all your advice on this as all this media stuff is relatively new to me.
Click to expand...

And here comes the part when I stick my foot into my mouth and ZFS starts to look really good. Because while I said above its adding complexity with more than one server I come advising on a single server with a DAS raid shelf :p

What I would do I think is buy something like a Dell R410 and then a Powervault MD1000 with as much storage as you can afford right now. The Powervault systems allow you to either just add disk (If you have open bays left) when you need more storage or add a whole need storage shelf to the system and add storage that way.

If its any kind of a business I always ask 2 questions what is the simplest way to achieve my goal and what is going to be the easiest to support in the long run. In business when something breaks I want to be able to call and ask somebody how do I fix this. I want to run ZFS for my file server at home. But I do not know enough about it to recommend it for a business environment
 
My company is in bed with Microsoft so it's hard to imagine Server 2008 NOT being used, but we are exploring unknown territory to us all, so if another OS is more practical, then that will heavily be considered.
 
Make everyone happy, split your budget into half and build 1 ZFS dumb storage box and 1 Win2k8 box to do whatever you guys need to do on top of it. You could even run that shit in a vm.
 
If Windows 2008 must be used, recommend going with a dedicated RAID controller.
Controller choice is up to you, I personally like 3ware for high reliability, LSI 9260-8i for speed/SSDs.
 
I work with a firm who develops in 3D for feature length movies. I can give you their perspective. 1. Stay with one platform, this allows you to minimize support and maintenance,
2. Up time is critical, having a bunch of developers sitting around is expensive. 3. You will have to have an in-and -out box, massive storage, as your developers work on projects.
4. This is one environment where speed = money, 10 GB network might be worth the cost, as 3D and video files are huge. You can lose quite a bit in productivity in transferring large files slowly.
 
You must log in or register to reply here.
Back
Top