Need recommendations for video workstation->server transfers

CC_Pirate

Limp Gawd
Joined
Aug 1, 2005
Messages
463
Hey guys,

At work we are setting up an enterprise level video capture and encoding system. Five workstations will capture and edit video (SD) for now. This video will then be encoded by 8 servers at a high rate. The servers are provided by Dell and all have Xeon CPUs and RAID 1+0 setups for storage.

I need advice on how to transfer the video data from the workstations to the servers. Gigabit LAN will not provide enough throughput, and some kind of shared fiberoptic bus will be too expensive. Therefore we have decided to transfer the data manually using external hotswappable eSATA drives. This will actually work quite well we believe, as it will provide plenty of throughput, be fairly inexpensive, and the physical work will not be a big deal.

So I need advice on what equipment to get:
1. We will only need to transfer less than 100GB at a time, but read/write rates are crucial, so I was thinking about getting something like a WD Velociraptor drive (fastest SATA drive out there, right), and putting it in an external eSATA enclosure.
Is this a good idea, or is it better to buy a pre-configured external drive with eSata support? I just don't think there are any of those with the read/write rates we want other than the larger and more expensive multi-drive RAID arrays.
2. We need eSATA controller cards in the servers that support HotSwap. The servers have PCIe slots. What should we be looking for? What will be the best brands/models?

Any other input and suggestions are greatly appreciated.

Thanks!
 
eSATA won't be much faster than gigabit ethernet with a velociraptor drive honestly. Get some decent multi-port NICs in your machines and then use teaming.
 
Intel makes a 4-port PCI Express Gigabit card. I haven't used it myself, though I am using the single port server NICs teamed and my hard drives limit my transfers before the network does.
 
You sound like you're trying to do something on too limited of a budget. You also say you're doing SD video "for now". Whatever ghetto rigged solution you come up with no most likely will just meet your needs.. maybe.. but then when you expand or go to HD, you'll need something 4x better. It's better to do something right the first time than to have to spend 2x, 3x, 4x, or even 5x as much money later on.

Do it right so the non-linear editors capture directly to a fiber channel array, and that array is hooked directly into the server cluster. If you're doing high speed encoding, especially with HD video in the future, you're going to need a pretty good RAID setup. Even with standard def material you can max out a small 2 or 3 drive raid array if you're doing team encoding or multiple online captures/edits.. especially when you get into multi-layered video.
 
Hmm, ok, but what kind of switch setup would I need to avoid bottlenecks there then? There will be 8 servers and 8 workstations, with the workstations feeding the servers.
 
For what it sounds like you are doing, look at the Ciprico MediaVault series, or G-Technology series arrays. They'll support multi-layered video from multiple NLE machines in standard def, or you can do a few video streams in HD. Beyond that, you're getting into really expensive territory. Like well over $25,000 expensive territory. With what you seem to want to do, it's going to be darn near it. I'd honestly contact one of their customer representatives. I'd recommend Ciprico first. They're an excellent company do deal with, and they can set you up with something for your specific application.
 
Thanks for the input.
We're not necessarily on a very limited budget, it is just that I would think a fiberchannel setup really is overkill for what we are doing, especially when there is a one to one relationship between feeding workstation and encoding server. The servers will all have their own RAID 10 arrays made up of 1TB disks.

So with a fiber channel array, would all workstations and servers share the same storage?

thanks.
 
I guess it all depends how easy you want it to be, or ghetto rigged you want it to be.

Easy/proper solution - go with a fiber channel array that has enough speed for your clients and the server to read/write to at high speeds. Yes, the clients/server share the fiber, and they're all connected with a fiber switch.

Ghetto solution - go with each individual machine having it's own little raid array using an external raid port card, then walk it over to your servers which would the same raid card you can plug into, then have it do your work, unplug it, and take it back.

If you ever move to HD video, you'd be really surprised how much space and speed you'll need, especially with multi-layered HD video. Do it the ghetto way, and most likely you'll be needing to do some things over again and spending more money. Do it the right way and you'll be all set for a smooth transition and have the proper read/write speeds you need along with enough storage.
 
I get your point. But wouldn't a high speed GB LAN with teamed NICs on all the peers provide the throughput we would need here, as suggested above?
 
You are certainly free to give it a try and report back with your results. ;)
 
Hehe, I will.
No but seriously, I appreciate your input, but after doing some quick research and math here, fiberchannel cards in all machines and sufficient FC array storage will end up being way too expensive.

And so will a 10Gigabit LAN setup be.

However, I am more compelled by the idea of teaming NICs and using a regular GB LAN for transfers. If each machine teams two or three ethernet ports, and we have a enterprise level switch to support it all, then each workstation should have a theoretical throughput of 2 or 3 Gbps to the servers, which should be plenty. This seems like a reasonably affordable solution that can be much more automated than manually transferring via external drives.

Am I right?
Can anyone provide some input on potential drawbacks (with the exception of futureproofing and inferiority vs fiber etc), and what hardware I might want to look into?

Thanks
 
The drawbacks for the way you want to do it are you're going to have to manually do everything which wastes valuable time. Also, you'll have to deal with network overhead and saturation from multiple clients using it at once. I'm going to make an educated guess here from my personal experience and say you're going to run into disk speed problems once you start into multi-layered video, especially HD when you decide to do that. Keeping all the client machines' shit together is going to be a fun task, especially when it needs to be rendered/encoded somewhere else and multiple people are using it. Teaming connections may work for transfers under specific conditions and platforms, but it usually isn't in a video edit suite scenario, hence why everyone in their applications uses fiber.

If you go fiber, it is basically like you're working off the server. The all the client bullshit gets taken out of the equation. You have everything centralized and you don't have to worry about having a raid array in each client machine. (if you think you're going to edit off a single disk solution, you won't without running into all sorts of disk speed problems..) Everything goes over the fiber 4gbps link back and forth just like you're working on your own machine. Once you're done with your project and you want it encoded or whatever, you use the editing app to tell the server to render & encode it all, and boom, you're done. No manual transfers, no training workers how to do it, no setup problems, very little if no maintenance at all, etc.
 
Yes, what you are saying makes complete sense. It will just be a matter of advantages vs price. But it is up for discussion.

Thanks!
 
Sure, since he doesn't have a budget, that sounds great. Fiber-- the proper solution, already sounds like it's over budget.
 
Go with quad port pci-e 1GB NICs for your servers (and any workstations that need more than 100MB a second of transfer capability), team the ports and get a gigabit switch that supports link aggregation (LACP). Teaming the NICs without LACP would be worthless as you are limited to only Transmit Load Balancing (with failover) without switch support and TLB is limited to the max speed of one link per connection thread.

We use HP Procurve 1800-24G switches in our test lab for iSCSI storage shared out from a server running Solaris. We have four 14drive arrays spread accross two RAID cards in the machine served out through a pair of quad port NICs (although we are only using 4 ports across both cards at the moment). That combined with jumbo packets gives us the capability to push around 400MB/sec combined bandwidth out across the network to multiple ESX and Windows test servers. Its cheap and relatively easy to setup on these switches. Of course someone is going to mention Cisco but frankly on a small budget they are just too steep for a managed gigabit switch that supports link aggregation and vlans from what I've seen. That being said we do run all Cisco on our production network and just went with procurve for our server test lab as its easy for us server guys to self manage and inexpensive to get the port density that we needed.
 
I should also mention that although there seems to be a bit of a belief that fiber is the end all for this application its far more likely that any shop worth their salt is moving towards iSCSI/FC-IP and 10GB ethernet rather than fiber for the ease of combined network/data distribution to their workstations. The considerably lower cost of 10GB ethernet vs fiber just makes the option much more compelling esp for something thats going to a workstation that isn't sitting in the datacenter near a fiber switch. Its definitely the way IT shops are starting to move for their SAN storage as it just doesn't make sense to maintain a set of MDS's in addition to a pair of server server switches. Sounds like even 10Gb ethernet isn't in the books for you yet but I honestly don't think you'd be going wrong trying aggregated 1Gb links for your purposes.
 
Back
Top