Looking For Ideas On Server Software Architecture

S

S-F

Gawd
Joined
Aug 5, 2010
Messages
671
Hello all,

I just ordered some new server hardware to replace my decade old system. It's got plenty of processing power and about 32 GB of RAM. This server will be used mainly for media, so I don't really need all of the power this thing has, but I'm future-proofing a bit. I need to have a Windows instance that can access USB ports to run my home automation software and connected devices. I also plan to run a VM with NextCloud. I currently have 4 x 8TB drives with 3 in a pool using Stablebit DrivePool. I have SnapRAID using the 4th disk for parity. I'm not married to any of this aside from the HA software needs.

I'm open to the idea of installing ESXi and having a Winblows VM and an Ubuntu VM for NextCloud. If I'm going that route I might as well use Napp-it in another VM, but only if I can get new drives and keep my current NTFS ones as backup. Hell. Keep my entire current server as backup. So it's an option if BestBuy has another crazy good sale on shuckable 12TB drives.

All that said, I'm open to other options that I might not have thought about. And suggestions? Thoughts? Criticisms? (This is the internet where so many people feel disembodied from each other and often act like assholes, so please be reasonable with the criticisms)











P.S.
[H]ardForum fucking rocks!
 
I like proxmox over vmware any more. And you might as well consider zfs. Rather it is on the host, or seperate like nappit, of freenas.. There is a half dozen standard ways to skin this cat. Then you could go way outside and deploy openattic on xen or something if you just want to be weird.
 
+1 to Proxmox. Unless you want/need ESXi for learning/lab purposes, I can't think of any reason why it would be better for a home (or probably even SOHO) setup.
 
I'm not familiar with Proxmox. I'm planning on using one particular VM image that was created with VMWare player. Will that work? I guess an alternative is Hyper-V as there is an image available for that. Is that an option in Server 2019?
 
There is a guide to morph VMware clones to KVM. I have never tried but i hear it works fine. I would also avoid hyper-v for a home setup.
 
S-F said:
I'm not familiar with Proxmox. I'm planning on using one particular VM image that was created with VMWare player. Will that work? I guess an alternative is Hyper-V as there is an image available for that. Is that an option in Server 2019?
Click to expand...
I used proxmox for a while; it's pretty powerful. I recently dropped it in favor of just running Ubuntu with docker which works just fine for my needs. It had a lot more features but I just didn't need them. I feel docker is much simpler for my use case and I can easily build my own docker images if I need something specific.
I run a few Minecraft servers (very simple to spin up, as they are all the same image with just a different directory passed as a parameter in my compose file. I also have a file server (samba), visual studio code server (web based server so I can program on any computer on my network without having to install software), and a few other things like Plex media (with a TV capture card for live TV and dvr). Like I said, no dogs on proxmox as it worked well when I used it but some things I didn't care for (the way it handled disks/drives was annoying for me). Provisioning disk space is cool, but it's much simpler to just share the entire drive instead of having to these at the space it may use. I can easily move my stuff around with docker by moving the directory, updating the compose entry, and restarting).
 
I haven't ruled out Proxmox yet, haven't really had the time to look into it. Here's what I'm currently thinking about doing: ESXi, a VM for Windows Server 2019 (Does Proxmox have the same hardware passthrough as ESXi? I absolutely need to be able to pass my USB ports to Windows.), a VM for the napp-it VM (so I don't have to screw with learning all that stuff at once), and a VM for the NextCloud Ubuntu (again, so I don't have to screw around with figuring out Linux all at once) as well as Colabra. I'm looking to get off Google's and Microsoft's teets here as much as possible. Doing this all without utilizing the prefab VM's would probably end up crushing my technically challenged soul.

I've never run ESXi or Proxmox. Given the above scenario does everyone still think that Proxmox would be a better choice, what with having to transform .ova files and whatnot?

Also, I only got 32 GB of RAM. Think it's enough? It sounds crazy to my ears to say that.
 
S-F said:
I haven't ruled out Proxmox yet, haven't really had the time to look into it. Here's what I'm currently thinking about doing: ESXi, a VM for Windows Server 2019 (Does Proxmox have the same hardware passthrough as ESXi? I absolutely need to be able to pass my USB ports to Windows.), a VM for the napp-it VM (so I don't have to screw with learning all that stuff at once), and a VM for the NextCloud Ubuntu (again, so I don't have to screw around with figuring out Linux all at once) as well as Colabra. I'm looking to get off Google's and Microsoft's teets here as much as possible. Doing this all without utilizing the prefab VM's would probably end up crushing my technically challenged soul.

I've never run ESXi or Proxmox. Given the above scenario does everyone still think that Proxmox would be a better choice, what with having to transform .ova files and whatnot?

Also, I only got 32 GB of RAM. Think it's enough? It sounds crazy to my ears to say that.
Click to expand...
Absolutely you can pass through hardware, last I remember it was a little bit of hand writing a config file, but pretty simple. I was passing my WinTV card through for Plex. I was running my 12/24 with 24GB of ram without issues.... I have since upgraded to 96GB and still barely use 4GB, lol. The downside of proxmox is if your using containers it's kind of difficult to make your own (last I checked a year ago). A VM is no issue obviously, but containers are much lighter weight when you don't need a full VM.
 
Its not crazy at all, I'm running 128g in my main box and 64g in my backup box, barely using half of either and a large portion of the 128 is for transcode cache :D.
 
proxmox device passthrough is easy. USB device passthrough is super easy.
honestly if you have not done any previous work with a hypervisor, proxmox with its pre-rolled GUI is probably the simpler choice.
32G is fine for this project.
it looks like proxmox supports direct import of an ova now. there may be some driver config needed after the import is the only thing i have seen as a possible issue.
 
I've heard that with Proxmox you can't install the VMs on the same drive as the OS itself. True? If so I'm going to have to get a new SSD just for the VMs.
 
same physical drive, yes you can.
it will partition the drive in a way that will look wierd but the OS goes in its own mini zfs partition, there can be space for VMs on the same drive in a separate partition. (default it builds the drive exactly like that)

also you could always grab a USB3 stick and install proxmox on it. that is a supported config too.
 
Beautiful! I'll just use an old Intel SSD for Proxmox, napp-it and Pi-Hole. I'll install Nextcloud and Colabra on the array. Thanks for all the great advice guys.

Anything else I should be keeping in mind?
 
i pass through real nics to certain VMs. the network likes it better. those include my DC, firewall, vpn server if it is separate. not required, but performance will improve.
install a random older large drive or large USB as a tank for ISOs and a place to make clones of the VMs too. clones are nice so you can test out changes and recover immediately if something goes sideways.
i think i have a script somewhere to remove the proxmox nag screen if it bothers you.
 
Zedicus said:
i pass through real nics to certain VMs. the network likes it better. those include my DC, firewall, vpn server if it is separate. not required, but performance will improve.
install a random older large drive or large USB as a tank for ISOs and a place to make clones of the VMs too. clones are nice so you can test out changes and recover immediately if something goes sideways.
i think i have a script somewhere to remove the proxmox nag screen if it bothers you.
Click to expand...


Ah. That's a good idea about the clones. And I'd love that script. I've watched some tutorials and it looks a little annoying.

Thanks again for all the input.
 
Zedicus said:
same physical drive, yes you can.
it will partition the drive in a way that will look wierd but the OS goes in its own mini zfs partition, there can be space for VMs on the same drive in a separate partition. (default it builds the drive exactly like that)

also you could always grab a USB3 stick and install proxmox on it. that is a supported config too.
Click to expand...

I finally have Proxmox up and running. Now, how do I utilize the rest of the OS drive for VM storage? I only really need to install napp-it.
 
can you post the drive layout?
'local' at the bottom of the left should automatically allow VM installation. unless you made it so small in the installer that they will not fit. defaults in the installer usually are passable.
 
Zedicus said:
can you post the drive layout?
'local' at the bottom of the left should automatically allow VM installation. unless you made it so small in the installer that they will not fit. defaults in the installer usually are passable.
Click to expand...


I have a local and a local-lvm......... I think?
 

Attachments

  • Untitled.jpg
    Untitled.jpg
    82.7 KB · Views: 0
go here

prxmx.JPG
mine is disabled so it is empty, i enabled it to get this screen cap.
 
Proxmox seems Interesting and I may play around with it. I had just built a Linux (Mint) machine to be able to use KVM since I already have a full ESXi server in the basement. Unfortunately, I am horrible with Linux and abandoned using KVM pretty quickly for what I needed and went with Hyper-V on my Win 10 machine. Hyper-V on Windows 10 has crapalicious pass through support (it's pretty non-existent). I guess I will convert that Mint machine into a proxmox machine to play with it a bit.
 
ESXi is known to be faster than ProxMox, has a lower footprint and has better OS support especially for Windows and Unix clients. The plus of ProxMox is when you mainly have Linux clients with many lightweight Docker Containers.

In an environment with only one or few Linux clients and Windows or Unix like a Solarish/Unix based ZFS NAS appliance, the full virtualisation approach of ESXi is better as Docker offers only advantages with many containers. ESXi has more features than Proxmox and the superiour vmxnet3 virtual nics. Proxmox offers many ESXi features for free what is the main reason for its success but especially when combined with a ZFS storage VM, the free ESXi is more than enough.

btw
NextCloud is a solution to built a Google Gsuite alike solution on your own. It is a very complex type of software that combines everything known to be vulnerable and bugfix/security sensitive like webserver, sql, php and many add ons up to office and client sync tools. You must virtualise if you want a stable base system.

If you only want an Internet cloud for yourself or a group to backup/sync files between clients or to share files via anonymous links, I would not use such a software monster like NextCloud but an Amazon S3 compatible simple storage server service like minIO that is much faster than NextCloud. Unlike Nextstep the whole server service is a single file without any dependencies. More or less minIO is idiot proof to to configure and setup - no need to virtualise. You can use minIO to add S3 sharing to your ZFS filer on a per filesystem base where you use NFS and SMB locally and S3 for Internet/Cloud sharing, see https://forums.servethehome.com/index.php?threads/amazon-s3-compatible-zfs-cloud-with-minio.27524/
 
Last edited:
Esxi was faster 10 years ago. Esxi, especially the free variant, is not feature comparable to proxmox. Docker is not proxmox primary target audience. Gae, you have a great platform and a huge user base, i do not think you need to bother with the 1% of people looking at other platforms.
 
I have Proxmox up and running. I got a second SSD for VMs. I have the napp-it .ova and my NextCloud .ova. How do I get them on Proxmox? When I Google "Proxmox .ova" all I find is complicated business. Can I just put them on a USB drive and import them? I'm sort of thinking that ESXi might be easier. I don't know though. I'm pretty new to all this.
 
Zedicus said:
Esxi was faster 10 years ago. Esxi, especially the free variant, is not feature comparable to proxmox. Docker is not proxmox primary target audience. Gae, you have a great platform and a huge user base, i do not think you need to bother with the 1% of people looking at other platforms.
Click to expand...

Proxmox has its use cases as a VM platform but is not a decent base for a Unix storage appliance like napp-it. Currently if you want to use it as a ZFS filer, you should learn CLI and SAMBA.
 
_Gea said:
Proxmox has its use cases as a VM platform but is not a decent base for a Unix storage appliance like napp-it. Currently if you want to use it as a ZFS filer, you should learn CLI and SAMBA.
Click to expand...
Should I just use ESXi instead? That was my plan originally.
 
if you want to use napp-it and a Solaris/Unix based ZFS storage appliance, yes for sure.
 
if you specifically want to use Napp-it, i would go the pre-canned route.
if all you need is a VM host, ZFS, device passthrough, and a bare metal hypervisor, and OVA support, proxmox can do that, and has other advantages.
 
Zedicus said:
if you specifically want to use Napp-it, i would go the pre-canned route.
if all you need is a VM host, ZFS, device passthrough, and a bare metal hypervisor, and OVA support, proxmox can do that, and has other advantages.
Click to expand...


Yes, yes.

How?

How do I run a .ova?
 
S-F said:
I've heard that with Proxmox you can't install the VMs on the same drive as the OS itself. True? If so I'm going to have to get a new SSD just for the VMs.
Click to expand...
Hmm... I can't confirm or deny this, but I don't recall having any restrictions. I did have a separate OS drive from data drive, so I cannot promise you anything. I have an SSD for my OS drive (2565GB), then I have a 6-disk RAID 10 for my data drive, as well as a single 8TB USB for backups and non-essentials. I could have used a 32GB OS drive and been fine honestly, but I had a 256 from another PC I was upgrading.
 
Zedicus said:
https://www.itsfullofstars.de/2019/07/import-ova-as-proxmox-vm/
there is a good guide on importing OVAs.

did you get your local storage set up? you never responded on that.
Click to expand...


Yeah, I just got a second SSD for VMs. Server hardware has been rolling in to my house like a herd of turtles. Shipping is apparently overtaxed these days. I also have all of my RAM installed (64 GB) and 8 x 8 TB HDDs so I'm ready to go.

I had already seen that guide. It seems like a bit of work. Is it worth it? Going through all that to run an .ova when it's native to VMWare? At the moment I'm putting my NextCloud .ova on a USB drive to try to load on Proxmox.


Also the SSD I installed Proxmox on is only something like 80 GB and the install took almost all of it so trying to utilize the remainder for VM storage is kinda pointless.


EDIT:
That guide doesn't specify how to target the location of the .ova. The initial command to unzip it fails. Please be gentle with me here guys. I'm a complete Nix N00b but I do need to learn this stuff for when I ditch Windows 7 in the near future.
 
Last edited:
Is the source of the nextcload ova a backup from another environment, or a new install? If new install, can you use one of their other installers?

Also, the 'installed size' will fit on a 16gb flash drive, not sure what is on your 80g drive if not some sort of default iso or vm storage.

I was a certified vmware support person before going proxmox. I can help with iether.
 
Last edited:
Zedicus said:
Is the source of the nextcload ova a backup from another environment, or a new install? If new install, can you use one of their other installers?

Also, the 'installed size' will fit on a 16gb flash drive, not sure what is on your 80g drive if not some sort of default iso or vm storage.

I was a certified vmware support person before going proxmox. I can help with iether.
Click to expand...


Jesus! I went through a very painful process of downloading ESXi (The download button wasn't responding for me in any browser) and now it won't even load! I loaded the .iso onto a USB drive with Rufus and after the boot process I immediately see a screen that says "Remove media and press any key to reboot". When I do so it just boots into Proxmox....... WTF?
 
S-F said:
Jesus! I went through a very painful process of downloading ESXi (The download button wasn't responding for me in any browser) and now it won't even load! I loaded the .iso onto a USB drive with Rufus and after the boot process I immediately see a screen that says "Remove media and press any key to reboot". When I do so it just boots into Proxmox....... WTF?
Click to expand...

rufus will not correctly (reliably?) build an installer from the VMWARE iso. i THINK YUMI https://www.pendrivelinux.com/yumi-multiboot-usb-creator/ will but it has been a while since i had to fight through that.

also, why are you changing now?
do you have goals in mind?
you are changing fundamental parts of the set up with out even completing 1 attempt. not an efficient way to build a server environment.
 
^ i normally just do the media creation run twice with rufus and its generally fine
 
Zedicus said:
rufus will not correctly (reliably?) build an installer from the VMWARE iso. i THINK YUMI https://www.pendrivelinux.com/yumi-multiboot-usb-creator/ will but it has been a while since i had to fight through that.

also, why are you changing now?
do you have goals in mind?
you are changing fundamental parts of the set up with out even completing 1 attempt. not an efficient way to build a server environment.
Click to expand...

I switched because I hit a wall in importing the NextCloud .ova and was just thinking that ESXi would be easier in that respect. Not so though. It doesn't even have drivers for my HDD controller. I'm happy to use whatever works and am happy to learn.

I got stuck at the first command. I think it's because I have the .ova on a USB drive but I don't know.

tar -xzvf NextCloudvm.ova

I have two .ovas that are already configured. NextCloud and napp-it. Everything else I can setup from .isos.

YUMI worked by the way. Thanks for that one.
 
You seem to pick and choose what you do based on a randomness i can't figure out. Please consult google for future information as i can not keep up with what you are doing/have tried.
 
Man! I don't know what I'm doing. I'm trying to learn. This is not my world. Otherwise I'd just use VirtualBox on some Windows install. I have been Googling for HOURS over days. I'm very grateful for your advice thus far.

If anyone can point me to a guide I'd love it. I just want to make this work and start to learn.
 
You must log in or register to reply here.
Back
Top