Just got a dual XEON x58 platform, few questions

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
Hey guys, I just got a Supermicro X8DTU board, dual x5620 XEONs, and 24gb of ddr3-1333 ECC memory off another member here.

I was wondering, is there any reason why I would be unable to put dual x5650 or other 6 core 12 thread xeon chips on this board? I was looking and I can get dual x5650's for very cheap off ebay.

Also the memory seems to be running slower than it should. It shows up as DDR3-1333 but the frequency is only at DDR3-1066 speeds. In the BIOS I don't see anything obvious that would limit it to this speed.

Any thoughts on which OS I should run on this? I have access to server 2016 but was considering just putting 10 on it and running virtualbox VM's through it. Is there any advantage or disadvantage to doing this versus installing 2016 as the base OS? There is some software I wanted to try that will not run on server 2016, so I figured I could just run 10 and run the server OS in a VM.
 

Grebuloner

Gawd
Joined
Jul 31, 2009
Messages
864
You should have no problem putting in whatever 56xx Xeons you want into the board. Plug the full model number into Supermicro's website beforehand and see if there are bios updates to be had, but they are top notch at supporting all the enterprise CPUs that fit a socket. Keep in mind you won't be able to overclock anything, so scour for deals on the faster CPUs (X5670, X5675, -80, -90), you'll be surprised at what you can find for cheap now.

The E5620 only supports DDR3-1066 which is why it's running at the slower speed. Once you get your faster 6 core, your memory will run at the rated speed.

You'd be fine on Win 10 Pro. Really it just comes down to what you intend to do with the machine and if a higher order OS (ie server) fits the bill. If you have apps that won't run on it, why bother?
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
You should have no problem putting in whatever 56xx Xeons you want into the board. Plug the full model number into Supermicro's website beforehand and see if there are bios updates to be had, but they are top notch at supporting all the enterprise CPUs that fit a socket. Keep in mind you won't be able to overclock anything, so scour for deals on the faster CPUs (X5670, X5675, -80, -90), you'll be surprised at what you can find for cheap now.

The E5620 only supports DDR3-1066 which is why it's running at the slower speed. Once you get your faster 6 core, your memory will run at the rated speed.

You'd be fine on Win 10 Pro. Really it just comes down to what you intend to do with the machine and if a higher order OS (ie server) fits the bill. If you have apps that won't run on it, why bother?
I found a deal for 2 x5650 for $20. The cheapest I could find anything above that was about $40 for 2 x5670's, so I figured f it, I'm sure that the x5650 is enough anyways. Can always upgrade again later I guess.

Yeah, I noticed the speed supported on Intel Ark was ddr3-1066 and I figured that was it.

As for the OS, I was mainly just curious if there would be any limitations to running windows 10 as the base platform, but I think it will be fine.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I am trying to setup a VM in virtualbox, but it seems like it is refusing to run virtualbox across both CPU's. I have access to all 8 cores but if I try and use more than 4 the VM blue screens with error dpc_watchdog_violation
watching cpu usage shows that when launching virtualbox, the cpu usage on one CPU starts getting used while the other one stays idle. I ran cinebench and both CPU's got pegged at 100% so it's not the hardware. Never really used a dual socket setup before. Guess I need to do a bit more research to see if virtualbox is the right solution to be using on this system.
 

Grebuloner

Gawd
Joined
Jul 31, 2009
Messages
864
I found a deal for 2 x5650 for $20. The cheapest I could find anything above that was about $40 for 2 x5670's, so I figured f it, I'm sure that the x5650 is enough anyways. Can always upgrade again later I guess.

Yeah, I noticed the speed supported on Intel Ark was ddr3-1066 and I figured that was it.

As for the OS, I was mainly just curious if there would be any limitations to running windows 10 as the base platform, but I think it will be fine.
2 for $20? Damn, that is quite a deal.
 

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
19,363
Ya, compared to 2-3 years ago when X5650's were still going for $50 a pop at the cheapest if you were lucky!
 

Grebuloner

Gawd
Joined
Jul 31, 2009
Messages
864
Ya, compared to 2-3 years ago when X5650's were still going for $50 a pop at the cheapest if you were lucky!
I see the motherboards on ebay are also coming down to reasonable prices as well, much better than the $50+ Xeon and $200 UP motherboard. I bought a full X58/X5675/etc. combo on FS for $150 this week. There are still a couple subprojects at PrimeGrid that don't use AVX so it will join my other X5675 in that fun for as long as they can be useful.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I see the motherboards on ebay are also coming down to reasonable prices as well, much better than the $50+ Xeon and $200 UP motherboard. I bought a full X58/X5675/etc. combo on FS for $150 this week. There are still a couple subprojects at PrimeGrid that don't use AVX so it will join my other X5675 in that fun for as long as they can be useful.
I paid $60 shipped for this board, dual CPU and 24gb ddr3-1333. I HAD to buy it at that price.
 
D

Deleted member 88227

Guest
Damn nice find!

Gonna run some BOINC on that bad boy???
 

Grebuloner

Gawd
Joined
Jul 31, 2009
Messages
864
I paid $60 shipped for this board, dual CPU and 24gb ddr3-1333. I HAD to buy it at that price.
OH DAMN.

Wish I could help you with your VM issues, my 2P Opteron died years ago so I haven't played with anything like that in a long time...maybe when Xeon Platinums come down sub-$100 in 10 years. Some light googling suggests playing with your bios's NUMA settings (it was new tech with Nehalem, so there should be plenty of options to tweak), or even that there is a hard drive incompatibility at play. You can also try unchecking VT-x in the VM setup and see what happens.
 
D

Deleted member 88227

Guest
None of the 4P Optys that I run will allow me to run more than 4 CPUs on a VM as well. Even though I have 48 or 64 cores available across 4 CPUs it maxes out at 4 virtual CPUs per VM.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
None of the 4P Optys that I run will allow me to run more than 4 CPUs on a VM as well. Even though I have 48 or 64 cores available across 4 CPUs it maxes out at 4 virtual CPUs per VM.
I can select all 8 cores, but the VM just crashes and freezes. Wonder if its a limitation of virtualbox or some other thing.
 
D

Deleted member 88227

Guest
I think it's a limitation of VM. I can also select beyond 4 cores/cpus per VM, but anything over 4 and the VM will not work.

Running these VMs in Windows Server 2016 by the way. Others on the [H] BOINC team that run multiple VMs have similar results as well.

If there is a trick to running a VM with more than 4 cores, we don't know it. I would love to figure it out though.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I think it's a limitation of VM. I can also select beyond 4 cores/cpus per VM, but anything over 4 and the VM will not work.

Running these VMs in Windows Server 2016 by the way. Others on the [H] BOINC team that run multiple VMs have similar results as well.

If there is a trick to running a VM with more than 4 cores, we don't know it. I would love to figure it out though.
It's 100% possible to run more than 4 cores on a VM. Something screwy is going on. Are you using virtualbox or something else?
 
D

Deleted member 88227

Guest
I'm using VirtualBox as well. I can spin up 10 VMs each with 4 CPUs, but if any of them have more than 4 CPUs they wont work. They'll either never load, crash after loading, or something else. Usually, they just hang at the loading VM screen.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I'm using VirtualBox as well. I can spin up 10 VMs each with 4 CPUs, but if any of them have more than 4 CPUs they wont work. They'll either never load, crash after loading, or something else. Usually, they just hang at the loading VM screen.
Do you CPU's have more than 4 cores each? I am pretty sure the limitation is that virtualbox won't use 2 sockets for 1 vm
 
D

Deleted member 88227

Guest
One of my systems is a 4 socket system. It has 4x 12 core CPUs. Another is also a 4 socket system with 4x 16 CPUs.
 

DeChache

Supreme [H]ardness
Joined
Oct 30, 2005
Messages
7,069
I think it's a limitation of VM. I can also select beyond 4 cores/cpus per VM, but anything over 4 and the VM will not work.

Running these VMs in Windows Server 2016 by the way. Others on the [H] BOINC team that run multiple VMs have similar results as well.

If there is a trick to running a VM with more than 4 cores, we don't know it. I would love to figure it out though.
If your on Server 2016 use Hyper-V have ran 24 Core VMs with out issues there. I suspect the problem is VirtualBox and Oracle wanting you to pay for the pro edition.

Just say no to Oracle.
 
D

Deleted member 88227

Guest
Never tried using Hyper-V. Will look into that though as running 10x 4 core VMs it a pita especially with the amount of RAM I need to give each VM for the project + Windows.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I'm working on trying to get everything going on Hyper-V. So far it seems to be pretty easy. I have been able to assign as many cores as I want to a VM with no issues, so definitely a good start.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
Well I just ran into a pretty stupid issue with Hyper-v, I tried to shut down my server and had to stop my VM's. When I tried to turn them on next, it duplicated the hard drive file and basically ran out of storage on my SSD. I am not sure how to resolve this. It seems like this is a checkpoint trying to restore itself but I didn't even try to do that so I'm not sure why it would behave like this. bleh. Hopefully I can fix it without too much issue
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I tried turning it off but I can't get the vhdx file to stop growing, and as a result there is not enough storage space. I tried merging the files but it fails saying unable to set parent. IDK, I'm probably just going to recreate the VM, and disable snapshots (why the f does microsoft call them checkpoints :X)
 

DeChache

Supreme [H]ardness
Joined
Oct 30, 2005
Messages
7,069
I tried turning it off but I can't get the vhdx file to stop growing, and as a result there is not enough storage space. I tried merging the files but it fails saying unable to set parent. IDK, I'm probably just going to recreate the VM, and disable snapshots (why the f does microsoft call them checkpoints :X)
Because snapshot is VMware speak and they have to be different. I think it was trademarked for a while too.

Did you create an expanding disk or a fixed size one. It might not be a checkpoint issue. But you should shutdown merge/delete the snaps and then disable the feature.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
Because snapshot is VMware speak and they have to be different. I think it was trademarked for a while too.

Did you create an expanding disk or a fixed size one. It might not be a checkpoint issue. But you should shutdown merge/delete the snaps and then disable the feature.
I did the default option (expanding). I am unable to merge the disk, unable to shrink it, everything just fails because there is no storage space left on the drive. bleh.
 

elite.mafia

[H]F Junkie
Joined
Aug 23, 2004
Messages
13,232
I ended up just redoing everything and disabling checkpoints.... Still not sure if I want to keep my host OS as windows 10, I think it might make more sense just having hyper-v as the base OS but is there a way to manage it through the snap-in and not through powershell/cmd ? Anyone have a guide
 

jrobdog

Limp Gawd
Joined
Dec 4, 2006
Messages
298
Have you considered unraid? Have no problems running over 4 cores in a VM. It uses KVM.
 

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
19,363
Use ESXI, so much better in almost every aspect, reason why VMware owns the virtualization world, outside of KVM on amazon :D
 

rtangwai

[H]ard|Gawd
Joined
Jul 26, 2007
Messages
1,369
Use ESXI, so much better in almost every aspect, reason why VMware owns the virtualization world, outside of KVM on amazon :D
ESXi is touchy about old hardware. OP has X5620's, they are not supported past ESXi 6.5 (current version is 6.7). I bet the onboard Intel NICs are also unsupported.

I know this because I gave up on ESXi on my dual X5650 on a SuperMicro X8DAi motherboard due to those very issues.

On the same hardware Hyper-V 2019 works perfectly (I do not do any passthrough which helps a lot).

As a bonus not having to run the vCSA to do updates and such means I save 10GB RAM and a fair bit of storage space.
 
Last edited:

MrGuvernment

Fully [H]
Joined
Aug 3, 2004
Messages
19,363
ESXi will work fine on X5650s and that chipset (not sure what your supermicro board had for nics....), I have it running on a R610 with dual X5690's with integrated broadcom's, Not supported vs wont work are 2 very different things. ESXi 6.7 notes it wont support soon those processors but that usually means they will not officially support it on their hardware list.

Also to update ESXI you can do it via CLI. You dont need vCenter unless you want all the fancy bells and whistles of DRS / HA
 

Ready4Dis

[H]ard|Gawd
Joined
Nov 4, 2015
Messages
1,993
I put 2x L5640's in my server due to it not supporting the higher wattages and 24 threads with 96gb was more than enough for my home use even at slightly lower speeds. I have a different system, but double check the power requirements would be the only thing that could cause it not to work.
 

Ready4Dis

[H]ard|Gawd
Joined
Nov 4, 2015
Messages
1,993
Oh, forgot to mention, I am running proxmox not esxi, so not sure about that part.
 
Top