What to do... NAS.. virtualize, or not.

To put my "NAS" in a VM or not.

Potential VM machine: Dual E5-2620 v1 ESXI Host, running Win VM Desktop, Win VM + Blue Iris + 6 cameras/monitoring (3-5 more coming), NAS to run network shares, iscsi, plex and or other media stuff, if not then another VM for that. 10gig dedicated for iSCSI/SAN, 10gig for LAN, potential 2x10GIG iSCSI & 4x1GIG general LAN. Hardware would be: 5x5TB WD RED, 4x Intel S3700 for VM hosts, misc other SSD array or HD array for iSCSI targets. (Hardware RAID6 & SSD Caching)

I'm not sure I want to throw my NAS into a VM, and call it good or keep it 100% separate, or partially separate IE: it still does plex/media stuff.

Option #2. Intel E3-1220 v2 w/16 or 32gb RAM. Running NAS / OS on bare metal. Intel 4x1Gb NIC. (Could possibly do 2 if I can do RAID card in there too).
Potential downfalls are:
- no 10Gig (let alone 2) so if any SSD array is added as an iSCSI target it won't live up to potential, or large spinning array.
- heavy-IO load won't work as well, but shouldn't be a concern for home work/lab.
- lower ram limit (should be fine for HW raid)
(Hardware RAID6, w/cache & SSD Caching for spinning arrays)

--- I'm not set on any NAS / OS software yet. Not sure which way to go.
------ I do want the ability to run my media server stuff (new to this stuff) in the same OS
------ I do want the ability to run automated backups to eSATA, netWork Shares, iSCSI, and USB attached devices (if VM then in pass through). Configured individually/numerous backups to various locations.
--- I'm using 12GB/s LSI RAID Cards so I have no interest in "software raid", please don't mention it.
--- Either system will have BBU on the RAID card & entire system w/shut-down command for data integrity.



Raid is not a backup... I know. I have TAPE, and BR and other media for various backups.

I wouldn't virtualize a NAS, if all you want is storage for it then keep it a separate machine using what-ever OS you are most comfortable with.

I assume you're going RAID5 w/hot spare? Or RAID6?

Just remember, rebuilds are going to be an absolute bitch.

RAID6. Might add more disks or hot spares later, plenty room in the chassis.

shetu said:

https://forums.freenas.org/index.ph...nas-in-production-as-a-virtual-machine.12484/

Click to expand...

Thanks, I've seen that.
Maybe it will help others. Looking for people experience on this forum, and not stuck on using "FreeNAS" either. There's a LOT of options for NAS.

I think Option 2 is a good idea.

What os? What software?

Anyone have opinions on this or is everyone using software raid?

It isn't recommended to mix ZFS RAID with hardware RAID. It is recommended that you place your hardware RAID controller in JBOD mode and let ZFS handle the RAID. According to Wikipedia: ZFS can not fully protect the user's data when using a hardware RAID controller, as it is not able to perform the automatic self-healing unless it controls the redundancy of the disks and data. ZFS prefers direct, exclusive access to the disks, with nothing in between that interferes. If the user insists on using hardware-level RAID, the controller should be configured as JBOD mode (i.e. turn off RAID-functionality) for ZFS to be able to guarantee data integrity. Note that hardware RAID configured as JBOD may still detach disks that do not respond in time; and as such may require TLER/CCTL/ERC-enabled disks to prevent drive dropouts. These limitations do not apply when using a non-RAID controller, which is the preferred method of supplying disks to ZFS.

ToddW2 said:

What os? What software?
Anyone have opinions on this or is everyone using software raid?

Click to expand...

If you want to use Windows + NTFS, hardware raid is a good solution,
otherwise a new generation software raid like ZFS is superiour.

While I prefer Solaris based ZFS solutions, you can use web-based storage appliances ex
on BSD (FreeNAS, ZFSGuru) or Linux (ZoL)

For Solaris or its free fork Illumos with OmniOS you can use my napp-it or NexentaStor as another option.

For a virtual NAS/SAN, i use ESXi + OmniOS for several years now and offer a ready to use ZFS storage VM as a free download. http://www.napp-it.org/doc/downloads/napp-in-one.pdf

_Gea said:

If you want to use Windows + NTFS, hardware raid is a good solution,
otherwise a new generation software raid like ZFS is superiour.

While I prefer Solaris based ZFS solutions, you can use web-based storage appliances ex
on BSD (FreeNAS, ZFSGuru) or Linux (ZoL)

For Solaris or its free fork Illumos with OmniOS you can use my napp-it or NexentaStor as another option.

For a virtual NAS/SAN, i use ESXi + OmniOS for several years now and offer a ready to use ZFS storage VM as a free download. http://www.napp-it.org/doc/downloads/napp-in-one.pdf

Click to expand...

I got your files as one to test a week ago, def. on the list.... but why not ESXI + HW Raid Pass Through to the OmniOS VM runnnig NapIT? Or is it only ZFS, no HW raid option?

ZFS handles its all better than Hardware RAID cards. Hardware RAID interferes with ZFS.

I'm using the ESXI All-in-one with OmniOS VM-Nappit. Running with an IBM M1015 controller flashed to IT Mode.

ToddW2 said:

I got your files as one to test a week ago, def. on the list.... but why not ESXI + HW Raid Pass Through to the OmniOS VM runnnig NapIT? Or is it only ZFS, no HW raid option?

Click to expand...

You can use a hardware raid to secure a local ESXi datastore but definitly not for ZFS. In such a case, ZFS may detect some real data corruption (your hardware raid cannot because it does not know the end to end checksums) but cannot repair because this would require access to the single disks. It complains about checksum errors/ data corruption and in the extreme it offlines your raid array while your hardware raid assumes that everything is ok.

This thread has actually been helpful to me. I was contemplating running a base Linux, and using KVM to spin up some kind of NAS for my storage. That might be out since it isn't recommended.

The alternative was to just present the disks to the base OS, and I was undecided whether to use MDADM RAID6 or ZFS.

I have 72GB RAM on my home server. My original plan was to have the base OS and some VM's.

FWIW I've decided to go with Gea's setup... ESXI All-in-one with OmniOS VM-Nappit, and use my E5-2620 system. I did add more ram for ARC, total ram now is 192gb.

I'm still deciding how to handle my media serving and may try to run something in Windows on the Blue Iris Windows VM since that's a security VM it's 1. a priority and 2. should be down least as long as Blue Iris plays nice with whatever software I use. I'm also thinking of using PMS (PLEX) w/Ubuntu VM. Also going to try http://mediatomb.cc/ too for those interested in something that can be AIO.

Also, I'm planning to use 64GB for ARC, how much should I allocate to the OmniOS VM then?

What about the PMS Ubuntu VM, what kind of resources (RAM) does PLEX really need? I'd imagine as long as you have enough for oS duties (very minimal) and fitting what you're streaming &/or transcoding into RAM it shouldn't really matter? IE: 4-8gb depending on # of streams expected at once, probably way overkill too for 1-2 streams I'd imagine.