Media server // NAS questions

[mrmandude]

Howdy,

I'm looking into a home media server/nas option. I'm currently stuck with lots of questions, and despite hounding various forums for answers, I'm still in the alpha stages of getting it all together.

Ideally I'd like to have an NAS and media server (quite possibly Plex media server) contained seamlessly within a single box. I'm not looking for a huge amount of SATA/SAS ports; 6 (5+OS) or 8 (7+OS) would be ideal. The disks would be 4TB WD Red or Seagate NAS and I'd like to have single disk parity with 5 or two disk parity with 7. The box would have to be able to handle transcoding to 3 devices at a time - and Plex's ability to transcode and push through WAN is a huge selling point for me as I'd like to allow access to the pool to friends/family. I have a desktop that can handle the heavier encode/decode duties.

I'm currently considering building a small box with an AMD FM2 motherboard with 6 onboard SATA ports, utilizing 1 for an SSD and the remaining 5 for a RAID 5 array with single disk parity on mdadm Ubuntu or perhaps Debian/CentOS. I'm not new to the world of Linux but anything Sun derived is completely foreign territory to me. And what about the PMServer? I'm thinking that it would be wiser to separate the two such that I can run the PMServer in a VM within Linux to instance another Linux OS running PMS, but I'm not sure that's necessary. Alternatively, I could go the vSphere bare metal route and run two VM's. Ideally, I'd prefer not having two separate boxes as that just inflates cost and further complicates matters.

Thus far -
Motherboard - http://www.newegg.com/Product/Product.aspx?Item=N82E16813131972
CPU - http://www.newegg.com/Product/Product.aspx?Item=N82E16819113288
RAM - I have a spare 1.35V G.Skill 2x2GB set that I can utilize
HDD - http://www.newegg.com/Product/Product.aspx?Item=N82E16822236599
PSU - anything cheap and frugal
Case - Haven't decided yet. I like the look of the Fractal Node 304 but Lian Li's mATX offers better air flow and room for 8.


Some questions:

- What exactly would I be looking at for hardware? How much RAM and what platform and processor? I've read that PMS ignores QuickSync and adheres to a straight CPU approach.

- And what about ESXi? I've fiddled around with VMs, but would a bare metal approach with IOMMU/VT-d passthrough be the better option here? Should I just run both on the same OS?

- What file system and OS should I be looking at?

- Is ECC memory required? It would mostly be a large media pool with some photos and business related files/folders, the latter of which is uploaded to the cloud so data integrity/redundancy for that stuff isn't exactly necessary.

- Is ZFS the better approach here? I wouldn't be looking to expand simply because I don't want the hassle nor do I need that much storage. In the future I'd probably just build a new box and copy everything over. Deduplication isn't something I'm particularly interested in as I see that as mostly a mostly-for-backups feature but the compression aspect interests me if it can save me some space. I understand that with ZFS it's highly recommended to go with ECC memory

- unRAID? FlexRAID? Any-other-RAID solutions that would suit better?

- Would an off-the-shelf NAS suffice? Ideally I'd like to build a box (and software RAID) as it's going to be cheaper. There's just something about paying >$500 for a box built around an Intel Atom that absolutely infuriates me. But if it can handle transcodes and can fulfill my needs at a reasonable enough price then I'm all ears.

I don't have a hard budget, but I'd like to spend <$800 for the hardware pre-HDDs. It isn't going to serve as a home lab or ESXi playground. Transcoding and funneling data through multiple RJ-45s are the only tasks that it'll be asked to do.

*I'm not against buying an HP Microserver either. 4 HDDs + boot drive might suffice

Any helpful comments and suggestions would be highly appreciated.

 

[Zedicus]

thats a nice build for the project. i do not think a prebuilt NAS would give you the functionality that you need.

i would probably just use linux software raid and as long as you are serious about not needing expandability, then i would not mess with any VM set up. i would probably set this up with debian stable (you need a stable environment and it is a server so bleeding edge is irrelevant) and install the apps you need right on the box and go on down the road.

this lets you not use ECC, ECC is recommended for ZFS, and simplifies the OS install. if you want it webmanaged add webmin. you will end up with a NAS like box with the power of a server.

 

[Zedicus]

your using PMS and Plex?

 

[mrmandude]

your using PMS and Plex?

Well, I'd need both the back end and the front end (PMS + PMC) for transcoding and viewing on a TV. The TV end would be handled via a Roku box (one room) and an HTPC (second room).

I forgot to mention this above, but 720p transcodes would be sufficient so that would decrease the processing requirements.

 

[Zedicus]

gosh i was thinking of this PMS http://www.ps3mediaserver.org/

not the fact the plex services are known as PMS and PMC.

 

[_Gea]

this lets you not use ECC, ECC is recommended for ZFS, and simplifies the OS install. .

??
ECC is a mechanism to detect RAM failures. They even happen by chance,
more probably if you use large RAM.

There is only one relation to ZFS.
If you use the best and safest filesystem on earth, you do not want to loose data
because of a stupid Ram failure.

 

[mrmandude]

??
ECC is a mechanism to detect RAM failures. They even happen by chance,
more probably if you use large RAM.

There is only one relation to ZFS.
If you use the best and safest filesystem on earth, you do not want to loose data
because of a stupid Ram failure.

I'm close to putting an X next to ZFS. The cost and the hardware requirements would just be too high. Add to that, the features are more focused on huge arrays for enterprise where a flipped bit is a much more serious problem. For media storage the compression wouldn't be as helpful and neither would the ECC or the deduplication and AES encryption.

 

[Zedicus]

should have been a . as 'and simplify the OS install' was not related to the ECC but more related to the non VM install.

 

[mrmandude]

should have been a . as 'and simplify the OS install' was not related to the ECC but more related to the non VM install.

Both, really. I don't see the added security // enterprise features as necessary; and if I did I'd use it for redundancy with ECC, otherwise what's the point? And, yea, I'd have to fire up a VM for PMS unless I go with ZFS on Linux, but I really don't need ZFS.

I'm still not sure mdadm with RAID 5 or RAID 6 is the right choice. Does anyone have any experience running linux software raid + PMS on a single machine/OS? I'm mainly concerned about software stability and hardware requirements.

 

[Zedicus]

my current box is a debian serverwith a mix of MDADM and hardware arrays. its a seedbox, transcoder, media share, webserver, etc. it has a 6 core AMD cpu and 4 gigs of ram and never gets above 30% utilization unless transcoding blueray. but the arrays read and write 100% of the time. seriously, 24/7/365, forever. even when i am doing that none of the other services slow down. and even when transcoding i have never been over 50% utilization on my ram.

and i am running all this on the host OS.

ZFS WILL utilize ram, its not the OS that needs it really. if it were me (it is, i am building a new box too) and i were planning on using ZFS i would virtualize. if i did not need ZFS i would stick to my current design and add more space.

for UPNP and UDNA i use the link i mentioned for ps3 media server. it streams to anything not just PS3. and it does on the fly transcoding.

 

[mrmandude]

That helps quite a bit. Thanks

 

[brutalizer]

ZFS requires 2GB RAM or so. Not more. I dont understand why you consider ZFS builds expensive? No huge amounts of RAM is needed, no hardware raid cards, etc. No nothing, the software does it all for you.

With that said, if you are doing strictly media, then Snapraid/Flexraid/etc might be a better fit for you. As long as your data is not vulnerable to bit flips, there is no need for ZFS actually.

 

[Zedicus]

i think bit flip and or minor normal corruption is the exact reason to use ZFS even if it was strictly a media server.

 

[mrmandude]

i think bit flip and or minor normal corruption is the exact reason to use ZFS even if it was strictly a media server.

That would mean I'd have to spend more on the platform to get it ECC compatible and wouldn't be able to use the RAM I have lying around.

It's not that it's a bad idea, it's just unnecessary for a media server as I see it. A flipped bit isn't going to be the end of the world. The compression aspect is definitely more interesting, but how much space I'd save given what I'm storing, I'm not sure.

 

[Zedicus]

i think bit flip and or minor normal corruption is the exact reason to use ZFS even if it was strictly a media server.

i think bit flip and or minor normal corruption is the exact reason to use ZFS even if it was strictly a media server, for me.

fixed.

you have to weigh your options, cost, and goals, and select your own path.