ESXi and x570 NVME Raid

pillagenburn

[H]ard|Gawd
Joined
Oct 3, 2006
Messages
1,271
Looking at digging up a x570 board and running nvme RAID-0 - I'd also like to run ESXi 6.7 or 7.0

Is this even possible? I won't be using this as the boot volume (SATA disk for this).
 
Depends on what you are trying to do. Hw raid solutions can be used as the boot partition or as disks for os.

If you need to install a driver for the raid solution you will have to pass the controller through to the VM and install the drivers through the os.

If esxi can see the virtual disk right off the bat I would throw the boot volume on the raid as well if you are using this as a desktop or workstation.
 
Depends on what you are trying to do. Hw raid solutions can be used as the boot partition or as disks for os.

If you need to install a driver for the raid solution you will have to pass the controller through to the VM and install the drivers through the os.

If esxi can see the virtual disk right off the bat I would throw the boot volume on the raid as well if you are using this as a desktop or workstation.

Thanks for the response. I'm using this purely as database storage and OS boot won't be necessary. I plan to try to use quad SN850 in RAID-0 and backup to a much slower 2TB samsung nvme ssd. I just want to know if anyone has done something like this and if the RAID volume showed up as a usable adapter/device/datastore on ESXi. If so I'm going to go dig up a x570 board and throw my 3900x on it and a quad m.2 pci-e x16 card that I have.
 
Thanks for the response. I'm using this purely as database storage and OS boot won't be necessary. I plan to try to use quad SN850 in RAID-0 and backup to a much slower 2TB samsung nvme ssd. I just want to know if anyone has done something like this and if the RAID volume showed up as a usable adapter/device/datastore on ESXi. If so I'm going to go dig up a x570 board and throw my 3900x on it and a quad m.2 pci-e x16 card that I have.
ive had it go both ways. notably a raid 5 of sas ssds on a perc 6i raid card worked great and was reconized properly as a virtual driver. The same config with a differnt intel raid card did not. If you only need the drives avalible to one vm you have the option to passthrough the pcie raid controller to the vm and install the drivers there. That may not work depending on how you are trying to do backups.

external pcie cards normally pass through pretty well, I have had mixed results with onboard x570 stuff or onboard raid controllers.

To clarify, you are using the raid through the board and just using the quad pcie card as a adapter or is that the raid controller?
 
ive had it go both ways. notably a raid 5 of sas ssds on a perc 6i raid card worked great and was reconized properly as a virtual driver. The same config with a differnt intel raid card did not. If you only need the drives avalible to one vm you have the option to passthrough the pcie raid controller to the vm and install the drivers there. That may not work depending on how you are trying to do backups.

external pcie cards normally pass through pretty well, I have had mixed results with onboard x570 stuff or onboard raid controllers.

To clarify, you are using the raid through the board and just using the quad pcie card as a adapter or is that the raid controller?

Yes I'm planning to try to use the quad m.2 card through the on-board RAID. The quad m.2 card itself isn't a raid card just x4/x4/x4/x4 bifurcated.
 
Yes I'm planning to try to use the quad m.2 card through the on-board RAID. The quad m.2 card itself isn't a raid card just x4/x4/x4/x4 bifurcated.
gotcha, in that case i have to say its probably a crap shoot as to if it will work properly. I dont think passthrough will be a option. Either the virtual disk is properly reconized, there is a good linux driver you can install to esxi (rare), or you can passthough the indiviudal m2 drives and do software raid (not ideal for preformance on pcie drives).

Honestly you will be making a tradeoff for performance running 4 pcie drives in raid 0 anyways (i dont think the onboard controllers are very good?). Not ideal unless you know your workload will benefit. something like this would be closer to ideal
https://www.ebay.com/itm/3040911550...Bs2qtF3rx08/lhAGWb0ic2eOpRrI|tkp:BFBMksvAuZpg
 
As an eBay Associate, HardForum may earn from qualifying purchases.
gotcha, in that case i have to say its probably a crap shoot as to if it will work properly. I dont think passthrough will be a option. Either the virtual disk is properly reconized, there is a good linux driver you can install to esxi (rare), or you can passthough the indiviudal m2 drives and do software raid (not ideal for preformance on pcie drives).

Honestly you will be making a tradeoff for performance running 4 pcie drives in raid 0 anyways (i dont think the onboard controllers are very good?). Not ideal unless you know your workload will benefit. something like this would be closer to ideal
https://www.ebay.com/itm/304091155096?hash=item46cd3eca98:g:J5oAAOSwhGphCRkW&amdata=enc:AQAHAAAA4OwP7BuId/38YTJ8teuXjhnuH2lrokvptQihUMlLGFYcq7oqGqXZaZvzboeMNvzGKn7VD9l96sMnq7/KHl/kcxY6bndCfWWJ70X3RbYJa1ijxOZiyU44yaclbx2J6ET7lts781Ta6NsXoF/+L9UwByJdNk+uir+icRqP9cLanjuZSMsLCAqdMPbcXuk3Xu9+CDZ/vUb0LAPoL9bMSikRw1CcLZN1Ad7huFHb16MwOK2APQPLhLQq6F/Q/gbXTGPlBKY3Q6/1rzhJpA/QBs2qtF3rx08/lhAGWb0ic2eOpRrI|tkp:BFBMksvAuZpg

I'll be doing heavy database work in a very time-sensitive format so I'm trying to get a PCI-E 4.0 setup going as I think it will do what I need it to.

One option would be to use a debian or ubuntu-based hypervisor.... or Hyper-V but I'm trying to stay away from Microsoft as much as possible.

edit: I'm just going to pass through and softraid the whole thing. I've got a 3900x so it should have enough horsepower to deal with it.
 
Last edited:
As an eBay Associate, HardForum may earn from qualifying purchases.
I'll be doing heavy database work in a very time-sensitive format so I'm trying to get a PCI-E 4.0 setup going as I think it will do what I need it to.

One option would be to use a debian or ubuntu-based hypervisor.... or Hyper-V but I'm trying to stay away from Microsoft as much as possible.

edit: I'm just going to pass through and softraid the whole thing. I've got a 3900x so it should have enough horsepower to deal with it.
Im more concerned about latency vs thouroghput for softwarte raid. raid 0 doesnt take much power but the onboard nvme controllers do a much better job of deligating loads then relying on software or external hardware. part of this is just due to the interconnect lengths the data has to pass through.

Is it a option to just do software jbod? I think the proformnance would excel in everything but the simplistic sequential loads.
 
Im more concerned about latency vs thouroghput for softwarte raid. raid 0 doesnt take much power but the onboard nvme controllers do a much better job of deligating loads then relying on software or external hardware. part of this is just due to the interconnect lengths the data has to pass through.

Is it a option to just do software jbod? I think the proformnance would excel in everything but the simplistic sequential loads.

I'll test both and see - this is only going to be a database playpen and the data will get backed up routinely to the "slower" secondary SSD. I just need super-fast storage to blow through massive datasets.
 
Most well optimized DB's use RAM, not disk when configured properly, i mean if your lacking on ram, but also most DB read writes are small subsets not large gigabyte transfers so raid 0 may not give much improvement, vs 2 seperate NVMe with data on one, logs/temp on another.
 
Most well optimized DB's use RAM, not disk when configured properly, i mean if your lacking on ram, but also most DB read writes are small subsets not large gigabyte transfers so raid 0 may not give much improvement, vs 2 seperate NVMe with data on one, logs/temp on another.
Yeah I can't afford the optimal amount of RAM for this - I've got 64GB right now and really I should upgarde to 128 at some point but... not gonna happen right now.

I thought about buying a metric sh*t ton of old DDR3 to put on an old v2 Xeon board with a couple of 10+ core Xeons but I don't think that would have worked very well.
 
Back
Top