Virtual File Server?

[tjf]

Hi

I have at home 4TB Linux softraid file server (RAID5 5x1TB HDD) that is also used as firewall, mail server, web server, Asterisk PBX + a few virtual machines in VMware Server. It is quite a support nighmare. I want to switch to ESX, and virtualize all this services to virtual appliances. It should also be safer. I already built a new machine with dual Xeon SuperMicro motherboard and started to play with ESX.

However my big problem is the file server. I do not want to keep it separate from the ESX server, mostly for power efficiency reasons. I want redundancy (RAID5) as I do not want to loose the data in case of single disk failure. Also in the case of HW failure (motherboard, SATA controler), I want to recover the data easily. Therefore my questions is how to do it?

I see a few options:

1. Buy RAID controler (Perc 5/i) and create one 4 TB HW raid volume. Any idea how to avoid 2TB virtual disk limit? Should I split it to two disks and combine them together in the file server virtual machine (Openfiler)? If Perc 5/i dies on me, how do I recover the data?

2. Create on each HDD a virtual drive and create softraid in virtual fileserver (Openfiler?). Will there be a bix performance hit?

3. Instead of ESX use Hyper-V and let the host Windows server system act as the file server. This way I could have soft raid without the need to buy RAID controler.

Is there any other way or should I forget it and build a separate NAS?

Thanks

Tomas

 

[jtg1993]

Use Esx and give your 5x1TB hard drives direct access to a linux vm just for fileserver/raid.

 

[Adidas4275]

using virtual drives for file serving can be done, but it seems to be something that is unnecessary and could cause issues when something goes wrong

phyisicall drive access is better, VM workstation provides physical access also

 

[djBon2112]

You obviously have experience in Linux, so why not use Xen? Use the Dom0 as the fileserver and virtualize the rest of the servers.

 

[tjf]

I am not sure I understand the suggestions about giving the physical access to the virtual machine. I thought I cannot use Raw Device Mapping with SATA disk in ESX.

Edit: I figured it out:

vmkfstools -r /vmfs/devices/disks/vmhba33\:0\:0\:0 disk1.vmdk

will map the whole raw disk to a virtual disk. Pretty neat. Now it is time to do some write and read tests.

 

[Ndel]

I am not sure I understand the suggestions about giving the physical access to the virtual machine. I thought I cannot use Raw Device Mapping with SATA disk in ESX.

Edit: I figured it out:

vmkfstools -r /vmfs/devices/disks/vmhba33\:0\:0\:0 disk1.vmdk

will map the whole raw disk to a virtual disk. Pretty neat. Now it is time to do some write and read tests.

I am having the same problem, the command you provided does not work for me though, can you provide me with some assistance please?

 

[spectrumbx]

I am having the same problem, the command you provided does not work for me though, can you provide me with some assistance please?

+1

vmkfstools -r /vmfs/devices/disks/vmhba33\:0\:0\:0 disk1.vmdk

How safe is doing this?

VMware is moving away from supporting raw local disks (the feature was removed from VMware Server starting with version 2.0).
I really want to run ESXi, but I don't want to run a separate box for my file server.
I certainly don't want to host my data in vmdk files either.

Could you provide more info on how all this work and how safe it is?

 

[Ndel]

+1


How safe is doing this?

VMware is moving away from supporting raw local disks (the feature was removed from VMware Server starting with version 2.0).
I really want to run ESXi, but I don't want to run a separate box for my file server.
I certainly don't want to host my data in vmdk files either.

Could you provide more info on how all this work and how safe it is?

safe, thanks for the above poster in pointing me to the right direction

 

[tjf]

You obviously have experience in Linux, so why not use Xen? Use the Dom0 as the fileserver and virtualize the rest of the servers.

I tried that with XenServer, but I would have to run samba server in Dom0 which I do not want to, as it is not safe. This was the main reason I want to virtualize all the services to separate appliances so they cannot affect each other. In this way if there is something wrong with samba I do not have to take whole server down (which has happened to me).

 

[tjf]

I also tried Windows Hyper-V Server. What a nightmare to connect to it remotely. Microsoft is really behind in virtualization. On the other hand the physical access to disks was very easy, but then to access my mdadm RAID5 I had to create virtual linux machine, which was again troublesome, as Microsoft only supports SUSE Linux.

The RAID access speed were the same for ESX and Hyper-V, so I have made my choice - ESX it is.

 

[Thuleman]

I also tried Windows Hyper-V Server [...] I had to create virtual linux machine, which was again troublesome, as Microsoft only supports SUSE Linux.

Not sure what you mean my troublesome. Pretty much any Linux distro installs without any trouble on Hyper-V. The integration tools are not available for every distro under the sun, but the non-enterprise user has very little use for them anyway.

 

[djBon2112]

I tried that with XenServer, but I would have to run samba server in Dom0 which I do not want to, as it is not safe. This was the main reason I want to virtualize all the services to separate appliances so they cannot affect each other. In this way if there is something wrong with samba I do not have to take whole server down (which has happened to me).

You shouldn't ever have to take down the entire server because of Samba... just kill the daemon. But I see your point. You can give DomUs direct disk access by using tap:aio:/dev/sdX in the DomU config file if I'm mistaken though.

 

[dbwillis]

/rant, one thing I hate about Xenserver is during setup it makes your HDD's into one large JBOD, although you can choose one HDD to install on, after its installed, its not easy (ie GUI) to add another HDD

/rant, ESX and HyperV do this easily

 

[tjf]

Not sure what you mean my troublesome. Pretty much any Linux distro installs without any trouble on Hyper-V. The integration tools are not available for every distro under the sun, but the non-enterprise user has very little use for them anyway.

Maybe I am not patient enough, but first I tried Debian. It was sooo slooow and I got stuck during the install process. Then I tried 'supported' SUSE. First I got OpenSUSE and it did not find network card. Then I tried SUSE Enterprise (3GB download). It worked but the network card had to be in legacy mode. Getting the integration tools is not trivial either. But the biggest showstoper was getting the Hyper-V core to be managed remotely. I was able to do it only with beta R2. And the end result was not as pretty as ESX. So I gave up. I will try Hyper-V in a year or so. I am sure Microsoft will catch up with the competition.

 

[tjf]

You shouldn't ever have to take down the entire server because of Samba... just kill the daemon. But I see your point. You can give DomUs direct disk access by using tap:aio:/dev/sdX in the DomU config file if I'm mistaken though.

It happened to me once. For some reason I could not connect to Samba (linux with VM Server). I restarted the daemon many times but it still did not work. I had to reboot the whole server to get it to work. I do not want to get to the same situation again.

 

[djBon2112]

/rant, one thing I hate about Xenserver is during setup it makes your HDD's into one large JBOD, although you can choose one HDD to install on, after its installed, its not easy (ie GUI) to add another HDD

/rant, ESX and HyperV do this easily

I have to ask... what is this Xenserver? I have an Ubuntu Server 8.04 install that I set up and configured, then I installed the xen-ubuntu-server package. Is Xenserver a seperate distro?

 

[Bookmage]

http://citrix.com/English/ps2/products/product.asp?contentID=683148
vm by citrix
http://en.wikipedia.org/wiki/Xen

we ran into ubuntu compatability problems with MS Hyper-V... ie it hung on install or couldnt find the right device/network/controller... suse did install but we had some minor issue that required a workaround not a fix... so we gave up on it for now... it had some cool features and was easy to use but its late to the game and still needs improvements... So we'll try again in a year or so...

 

[Zlash]

Maybe I am not patient enough, but first I tried Debian. It was sooo slooow and I got stuck during the install process. Then I tried 'supported' SUSE. First I got OpenSUSE and it did not find network card. Then I tried SUSE Enterprise (3GB download). It worked but the network card had to be in legacy mode. Getting the integration tools is not trivial either. But the biggest showstoper was getting the Hyper-V core to be managed remotely. I was able to do it only with beta R2. And the end result was not as pretty as ESX. So I gave up. I will try Hyper-V in a year or so. I am sure Microsoft will catch up with the competition.

Debian works great, you defintely need to install the services or you will have issues with the system clock, etc. And also for Linux you need to remove the default network adapter and add a legacy one and it fires right up. This is for hyper-v...

 

[nessus]

But the biggest showstoper was getting the Hyper-V core to be managed remotely. I was able to do it only with beta R2. And the end result was not as pretty as ESX. So I gave up. I will try Hyper-V in a year or so. I am sure Microsoft will catch up with the competition.

One of the MS development team released 2 visual basic scripts that make all the changes for you on Hyper-V server, and a Vista client machine for supporting remote access with non-administrative users.

http://blogs.technet.com/jhoward/ar...ure-hyper-v-remote-management-in-seconds.aspx

There is also a fairly nice Vista Gadget for controlling your VMs made by a third party.

http://mindre.net/

Otherwise, you have to make on Azman security entry, give permissions to two additional WMI subtrees, and open the firewall on the non-domain joined host machine, unless you use the local admin account to connect remotely. Then make the corresponding firewall changes on your remote workstation.

If your client machine is using Windows Live One care and Vista, you have to make a rule change in Live One care's interface (135 TCP allow incoming) even though it uses the Vista firewall, and the rule is created there, until the matching rule is defined in Windows Live One Care, it doesn't actually work. Finding this drove me nuts when I was trying to figure out why my remote client couldn't connect.

That said, with VMWare for this situauation, direct mapped drives are the recommended solution for avoiding the 2 TB limit with ESX and ESXi.