Set Up my First ESXI! Need some Advice and Guidence on Storage

webby88

n00b
Joined
Apr 16, 2012
Messages
37
After much research I finally set up ESXI 6.5 on this Dell PowerEdge T710 that I got (for free). Everything up until this point has sort of played out as I expected; Installed ESXI from a USB Stick to another one internally plugged into the motherboard (this system didn't come with the built-in internal SD card slot).

My question at this point is about storage; This server has 8x 3.5 hotswap hard drive bays connected to a Dell SAS 6/iR integrated controller (sadly, no PERC), and it came with 4x 250gb drives and 2x 1tb drives.

I know that the SAS 6/iR can do RAID, but am unsure what would be the best route. My goal is to host a Windows 2008 Server for some basic accounting software shared over MSTSC, a Plex Media Server and two FreePBX instalations.

For the Media end of it I have a Synology ds1813 NAS on which I would like to store all media, but aside from that I ahve no idea what would be the best place to store the VM images and the local data to go along with it.

So from this point on what would some of you with some real virtualization experience do?
 
Does your SAS 6/IR have any cache memory?

If not, then performance will be pretty horrible if you attempt any kind of RAID 5 or better, so I'd stick to mirroring.

That said, 4x 250 GB drives are pretty worthless. I'd throw them away and either add additional 1TB drives, or just operate with the pair you have in a mirror, especially if the ds1813 is going to hold most your media. Heck, the ds1813 can even serve up iSCSI volumes and you could boot your VMs directly from the NAS.
 
Does your SAS 6/IR have any cache memory?

If not, then performance will be pretty horrible if you attempt any kind of RAID 5 or better, so I'd stick to mirroring.

That said, 4x 250 GB drives are pretty worthless. I'd throw them away and either add additional 1TB drives, or just operate with the pair you have in a mirror, especially if the ds1813 is going to hold most your media. Heck, the ds1813 can even serve up iSCSI volumes and you could boot your VMs directly from the NAS.
After posting I did some reading on the SAS 6/iR and discovered that it is handicapped quite badly. First, its throughput is just 3Gbps and not 6. Then, it is limited to just RAID 0 or 1 plus one support a total of two global hotspares - one per volume.

So this card seems quite limiting. A quick check on eBay I see that there are h700 PERC 6i cards available from around ~$200, and those are capable of full 6Gbps, have proper caching and battery backup, plus can do quite a few different combos of RAID.

About my Synology, as I understand it, I cannot run both an iSCSI volume alongside NFS. Is that right? (Need standard file sharing for local file sharing and backup.)

I am thinking that I would need to keep the VM images local and have some local shared storage, but all data would be kept on a NAS. Is this the right plan?
 
It can really be done either way.

First off, getting the better RAID controller is a super good idea if you plan on having the local storage.

You've got room for 8 disks in the Dell, so armed with the decent controller you can pretty much run the same capacities there that you can shove into the ds1813. Things on the local controller in the Dell will be *significantly* faster in operation than running across the network, but for media storage that may not be an issue. There won't be much of an intrinsic benefit to storing things on one device versus the other; if it was me? I'd run everything local on the server so I only have one device to worry about, or maybe run primary storage on the Dell and send backups to the Synology.

As for iSCSI vs NFS, depending on how you carve out volumes on the Synology you can do both simultaneously, or not. ESXi can also store VMs on NFS shares as well, though I personally don't have any experience doing so.

The *least expensive* method is likely to ignore local storage entirely - don't buy the H700 - and to instead use the ds1813 to store the VM files and boot them from there via iSCSI or NFS.
The *fastest* method would be to ignore the Synology ds1813, buy the H700, and operate entirely locally.
The *best* - but most expensive - method would be to do both - run locally and back up to the ds1813 - but that requires purchasing the H700, and potentially twice as many disks so you can populate both the Dell and the DS1813.
 
It can really be done either way.

First off, getting the better RAID controller is a super good idea if you plan on having the local storage.

You've got room for 8 disks in the Dell, so armed with the decent controller you can pretty much run the same capacities there that you can shove into the ds1813. Things on the local controller in the Dell will be *significantly* faster in operation than running across the network, but for media storage that may not be an issue. There won't be much of an intrinsic benefit to storing things on one device versus the other; if it was me? I'd run everything local on the server so I only have one device to worry about, or maybe run primary storage on the Dell and send backups to the Synology.

As for iSCSI vs NFS, depending on how you carve out volumes on the Synology you can do both simultaneously, or not. ESXi can also store VMs on NFS shares as well, though I personally don't have any experience doing so.

The *least expensive* method is likely to ignore local storage entirely - don't buy the H700 - and to instead use the ds1813 to store the VM files and boot them from there via iSCSI or NFS.
The *fastest* method would be to ignore the Synology ds1813, buy the H700, and operate entirely locally.
The *best* - but most expensive - method would be to do both - run locally and back up to the ds1813 - but that requires purchasing the H700, and potentially twice as many disks so you can populate both the Dell and the DS1813.
That makes sense. I actually have an old Linksys/Cisco NAS with 4 x 2tb drives which I can repurpose into 4 of those 8 bay's and RAID 5 them for media storage and sharing, then use the existing 2 x 1tb for storing the VM's and completely discard those 4 x 250gb drives. All I would need is the h700.

Would I be able to still serve files off the T710 over the local network in such a setup? Or would I need to set up a VM for file sharing? Is it at all worth it?
 
You need a VM to be running to host files over the network. ESXi is a Type 1 Hypervisor. All it knows how to do is interact with the hardware in the system. You need an OS to be running to host the files. To be honest, in my server room at work we have a few servers that have the SAS 6iR Controller. I know there are some out there that would blast me for it, but depending on what its being used for you can run perfectly fine with the 6iR.

Here is something to think about as well. How many Virtual Machines are you going to be running tomorrow/1month/1 year from now? What most people confuse with Internet speed also falls over into disk access speed as well. Lets say you are going to be running those 4 VMs that you mentioned. Out of all those, the one that I see being the most intensive is the Plex Media box. Depending on the end player, it could require a lot of transcoding ( CPU not disk) to make the playback stable and clean. I'm running a quad-core Intel Atom CPU with 6GB of RAM and 1GB nic. I can stream 1080p quality media to 6 machines before it starts choking. That T710 obviously has a more power CPU than an Intel Atom.

That being said, Your 710 will handle those VMs with no issues either local on the SAS6iR or connected via iSCSi or NFS. You can also have Datastores connected with both protocols to one ESXi host as well. Also I'm pretty sure you could offload that PLEX media server onto the Synology disk server and run it there. I have a friend running PLEX on his and it works pretty well. I also believe its worth it in the end. Virtualization gives you the freedom to run what you need, test, and expand in the future all on 1 machine. You got lucky and someone gave you a pretty powerful box. Depending on what came in the box when you got it, it has plenty of room for expansion and future proofing.

I also have a Dell PERC H700 if you want it. I had to buy a new Mainboard for a SQL server and it came with the H700 even though I already had one.
 
That will be fine to run a ESXi host and a couple VM's, performance won't be the best but I've used much worse for work environments.

Something to be aware of is VMware only gives you 60 days in evaluation mode. 60 days might sound like a lot, but trust me it goes in the blink of an eye. After that you'll either need to acquire a license $$$$ or blow it all away and start over all again. Or I guess there are other options (arrr matey), but I just got used to rebuilding every other month. I was doing nested hosts too so that was kind of a chore. If you choose other options (and this may just be me being paranoid) you might not want the host to have internet access, which means your VMs won't either. I have no proof of black vans full of VMware auditors raiding anyone, but I've heard some interesting stories.
 
That will be fine to run a ESXi host and a couple VM's, performance won't be the best but I've used much worse for work environments.

Something to be aware of is VMware only gives you 60 days in evaluation mode. 60 days might sound like a lot, but trust me it goes in the blink of an eye. After that you'll either need to acquire a license $$$$ or blow it all away and start over all again. Or I guess there are other options (arrr matey), but I just got used to rebuilding every other month. I was doing nested hosts too so that was kind of a chore. If you choose other options (and this may just be me being paranoid) you might not want the host to have internet access, which means your VMs won't either. I have no proof of black vans full of VMware auditors raiding anyone, but I've heard some interesting stories.

You can get a free esxi key from vmware to get past the 60 days.
 
You can get a free esxi key from vmware to get past the 60 days.

I was not aware they were handing out ESXi keys. I do remember getting a 1 year Workstation license when I passed my VCP the first time, but that was several years ago.

edit - I googled it and I do see this : http://www.virten.net/2016/11/free-esxi-6-5-how-to-download-and-get-license-keys/

Doesn't include vCenter which I would need for my testing etc, but I doubt the OP cares about that for home use.
 
Yep. As long as you don't need vCenter, then the free ESXi is pretty capable. If you end up needing vCenter, the essentials bundle (around $570) comes with 6 CPU sockets worth of ESXi licensing and a vCenter key. For a single host, it won't do much for you aside from make some things easier (GUI to clone a VM being a big example) but if you have multiple hosts with shared storage, the vmotion capability is greatly appreciated from the licensed copies.
 
Also I'm pretty sure you could offload that PLEX media server onto the Synology disk server and run it there. I have a friend running PLEX on his and it works pretty well. I also believe its worth it in the end. Virtualization gives you the freedom to run what you need, test, and expand in the future all on 1 machine.
My NAS is holding a massive collection of photos and other very important files spanning some 11Tb. I have crashplan running on the box and it is always backing up, chewing up all the max available 4Gb of ram. I have looked into using it for Plex and other media applications and decided that it is too important to risk anything that may destabilize it, so putting some other hardware such as this T710 to use seems like a better idea.

I also have a Dell PERC H700 if you want it. I had to buy a new Mainboard for a SQL server and it came with the H700 even though I already had one.
I would love to take it off you. PM me :).

Something to be aware of is VMware only gives you 60 days in evaluation mode.
As the others have already jumped in and pointed out that VMWare gives a key for one ESXi Installation with two sockets and unlimited cores. The web based GUI seems nice and capable for the things that I would need.

Truth be told that I have looked at Xen being that it is free, but for the my intended use it really isn't necessary.
 
My NAS is holding a massive collection of photos and other very important files spanning some 11Tb. I have crashplan running on the box and it is always backing up, chewing up all the max available 4Gb of ram. I have looked into using it for Plex and other media applications and decided that it is too important to risk anything that may destabilize it, so putting some other hardware such as this T710 to use seems like a better idea.


I would love to take it off you. PM me :).


As the others have already jumped in and pointed out that VMWare gives a key for one ESXi Installation with two sockets and unlimited cores. The web based GUI seems nice and capable for the things that I would need.

Truth be told that I have looked at Xen being that it is free, but for the my intended use it really isn't necessary.



As Grayson said, you can get a Free ESXI key from VMware to run your box without a trial period. They have been handing out Free licenses since at least version 4.0 ( when I started with VMWare). The trial period is like most software; they let you try ALL that the software can do, then you need a Permanent ( Free or paid for) license to keep the box running. The only feature that the free key gives you is Virtual Symmetric Processing which doesnt affect you other then letting you have a license key to continue to run the server. VSP essentially allows you to assign Virtual CPUs to a VM and the system can utilize more than 1 physical CPU to assign processing cores.

Check out this link to see what the versions offer you. The 2 that I have found most useful over the years is VM vMotion and Storage vMotion.

https://www.vmware.com/products/vsphere.html


Also this is knowledge for everyone; The Current version of VMWare Hypervisor ( Free edition) has no limit on Physical CPUs or RAM any longer. The Free Version is called Hypervisor and the paid versions are just called ESXi.

https://www.vmware.com/products/vsphere-hypervisor.html
https://www.vmware.com/products/vsphere-hypervisor.html
expand tech specs to verify.
 
Also this is knowledge for everyone; The Current version of VMWare Hypervisor ( Free edition) has no limit on Physical CPUs or RAM any longer. The Free Version is called Hypervisor and the paid versions are just called ESXi.

https://www.vmware.com/products/vsphere-hypervisor.html

expand tech specs to verify.
What are the practical differences between this free version and ESXi? (aside obviously for the unlimited CPU's and RAM)

Also, how do other free offerings stack up against those from VMWare?
 
I was not aware they were handing out ESXi keys. I do remember getting a 1 year Workstation license when I passed my VCP the first time, but that was several years ago.

edit - I googled it and I do see this : http://www.virten.net/2016/11/free-esxi-6-5-how-to-download-and-get-license-keys/

Doesn't include vCenter which I would need for my testing etc, but I doubt the OP cares about that for home use.

For $200/yr or so, you can get a license that is pretty much everything you might want. https://www.vmug.com/evalexperience
 
What are the practical differences between this free version and ESXi? (aside obviously for the unlimited CPU's and RAM)

Also, how do other free offerings stack up against those from VMWare?

If you have more than one physical host, you want the paid edition for the vMotion capabilities.
If you plan on using third party tools that integrate with VMWare - like Veeam for backups - then you need the paid edition because they don't play ball with the freebie license.

If you're a home user with one server, you probably don't need the paid edition.

As for comparisons to other software, I've used Xen back in the 5.6 and 6 era, which was years back. It was arranged differently, but was otherwise somewhat feature comparable. I prefer ESX between the two.
 
You need a VM to be running to host files over the network. ESXi is a Type 1 Hypervisor. All it knows how to do is interact with the hardware in the system. You need an OS to be running to host the files.
I understand that the hypervisor is just a layer to interact with the hardware. I was wondering what would be the most resource efficient and speedy way to share that (possibly RAID5) storage volume over the network?

If you plan on using third party tools that integrate with VMWare - like Veeam for backups - then you need the paid edition because they don't play ball with the freebie license.
What type of backup's are we talking about? The actual virtual machine images? The ESXi settings themselves? If so, what would be the solution to good backup on the freebie versions?
 
What type of backup's are we talking about? The actual virtual machine images? The ESXi settings themselves? If so, what would be the solution to good backup on the freebie versions?

The actual VM images. The ESXi installation itself contains very little vital data - which is why often it's installed on non-redundant USB sticks or SD cards. The VMs are presumably the important bits.

Veeam has an advantage in that it can back up *any* VM, and the VM itself doesn't even know the backup is taking place because it happens at the ESXi level. This requires a licensed ESXi installation.

Your other options for backing up the VMs are mostly just to back up the VMs as if they were any other computer; copy the data somewhere else, run local backup software like Shadowprotect or Reflect, whatever you're comfortable with.
 
The actual VM images. The ESXi installation itself contains very little vital data - which is why often it's installed on non-redundant USB sticks or SD cards. The VMs are presumably the important bits.

Veeam has an advantage in that it can back up *any* VM, and the VM itself doesn't even know the backup is taking place because it happens at the ESXi level. This requires a licensed ESXi installation.

Your other options for backing up the VMs are mostly just to back up the VMs as if they were any other computer; copy the data somewhere else, run local backup software like Shadowprotect or Reflect, whatever you're comfortable with.
I see that Veeam has a really robust free edition of their backup software for VM's (https://www.veeam.com/virtual-machine-backup-solution-free.html), but I guess the issue is that the free ESXi does not have vSphere which is what Veeam needs?
 
I see that Veeam has a really robust free edition of their backup software for VM's (https://www.veeam.com/virtual-machine-backup-solution-free.html), but I guess the issue is that the free ESXi does not have vSphere which is what Veeam needs?

I think you meant "vCenter" instead of "vSphere" - VMWare's terminology can be confusing. vSphere is sort of the name for the whole product suite, ESXi is the hypervisor itself, vCenter is the management server. vCenter is not required for Veeam, but a licensed ESXi installation is. Or at least it was - I'll be honest, I haven't re-read their system requirements since Veeam 6.5 and it's possible they've changed. They certainly never had a free edition when last I looked, and I have no clue as to its capabilities or requirements.
 
Back
Top