Just got a dual XEON x58 platform, few questions

Discussion in 'Multiprocessing Systems' started by elite.mafia, Feb 2, 2019.

  1. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    Hey guys, I just got a Supermicro X8DTU board, dual x5620 XEONs, and 24gb of ddr3-1333 ECC memory off another member here.

    I was wondering, is there any reason why I would be unable to put dual x5650 or other 6 core 12 thread xeon chips on this board? I was looking and I can get dual x5650's for very cheap off ebay.

    Also the memory seems to be running slower than it should. It shows up as DDR3-1333 but the frequency is only at DDR3-1066 speeds. In the BIOS I don't see anything obvious that would limit it to this speed.

    Any thoughts on which OS I should run on this? I have access to server 2016 but was considering just putting 10 on it and running virtualbox VM's through it. Is there any advantage or disadvantage to doing this versus installing 2016 as the base OS? There is some software I wanted to try that will not run on server 2016, so I figured I could just run 10 and run the server OS in a VM.
     
  2. Grebuloner

    Grebuloner Gawd

    Messages:
    616
    Joined:
    Jul 31, 2009
    You should have no problem putting in whatever 56xx Xeons you want into the board. Plug the full model number into Supermicro's website beforehand and see if there are bios updates to be had, but they are top notch at supporting all the enterprise CPUs that fit a socket. Keep in mind you won't be able to overclock anything, so scour for deals on the faster CPUs (X5670, X5675, -80, -90), you'll be surprised at what you can find for cheap now.

    The E5620 only supports DDR3-1066 which is why it's running at the slower speed. Once you get your faster 6 core, your memory will run at the rated speed.

    You'd be fine on Win 10 Pro. Really it just comes down to what you intend to do with the machine and if a higher order OS (ie server) fits the bill. If you have apps that won't run on it, why bother?
     
    EvilAlchemist and auntjemima like this.
  3. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I found a deal for 2 x5650 for $20. The cheapest I could find anything above that was about $40 for 2 x5670's, so I figured f it, I'm sure that the x5650 is enough anyways. Can always upgrade again later I guess.

    Yeah, I noticed the speed supported on Intel Ark was ddr3-1066 and I figured that was it.

    As for the OS, I was mainly just curious if there would be any limitations to running windows 10 as the base platform, but I think it will be fine.
     
    EvilAlchemist likes this.
  4. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I am trying to setup a VM in virtualbox, but it seems like it is refusing to run virtualbox across both CPU's. I have access to all 8 cores but if I try and use more than 4 the VM blue screens with error dpc_watchdog_violation
    watching cpu usage shows that when launching virtualbox, the cpu usage on one CPU starts getting used while the other one stays idle. I ran cinebench and both CPU's got pegged at 100% so it's not the hardware. Never really used a dual socket setup before. Guess I need to do a bit more research to see if virtualbox is the right solution to be using on this system.
     
  5. Grebuloner

    Grebuloner Gawd

    Messages:
    616
    Joined:
    Jul 31, 2009
    2 for $20? Damn, that is quite a deal.
     
  6. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
  7. MrGuvernment

    MrGuvernment [H]ard as it Gets

    Messages:
    19,161
    Joined:
    Aug 3, 2004
    Ya, compared to 2-3 years ago when X5650's were still going for $50 a pop at the cheapest if you were lucky!
     
  8. Grebuloner

    Grebuloner Gawd

    Messages:
    616
    Joined:
    Jul 31, 2009
    I see the motherboards on ebay are also coming down to reasonable prices as well, much better than the $50+ Xeon and $200 UP motherboard. I bought a full X58/X5675/etc. combo on FS for $150 this week. There are still a couple subprojects at PrimeGrid that don't use AVX so it will join my other X5675 in that fun for as long as they can be useful.
     
  9. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I paid $60 shipped for this board, dual CPU and 24gb ddr3-1333. I HAD to buy it at that price.
     
  10. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    21,749
    Joined:
    Aug 14, 2004
    Damn nice find!

    Gonna run some BOINC on that bad boy???
     
  11. Grebuloner

    Grebuloner Gawd

    Messages:
    616
    Joined:
    Jul 31, 2009
    OH DAMN.

    Wish I could help you with your VM issues, my 2P Opteron died years ago so I haven't played with anything like that in a long time...maybe when Xeon Platinums come down sub-$100 in 10 years. Some light googling suggests playing with your bios's NUMA settings (it was new tech with Nehalem, so there should be plenty of options to tweak), or even that there is a hard drive incompatibility at play. You can also try unchecking VT-x in the VM setup and see what happens.
     
  12. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    21,749
    Joined:
    Aug 14, 2004
    None of the 4P Optys that I run will allow me to run more than 4 CPUs on a VM as well. Even though I have 48 or 64 cores available across 4 CPUs it maxes out at 4 virtual CPUs per VM.
     
  13. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I can select all 8 cores, but the VM just crashes and freezes. Wonder if its a limitation of virtualbox or some other thing.
     
  14. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    21,749
    Joined:
    Aug 14, 2004
    I think it's a limitation of VM. I can also select beyond 4 cores/cpus per VM, but anything over 4 and the VM will not work.

    Running these VMs in Windows Server 2016 by the way. Others on the [H] BOINC team that run multiple VMs have similar results as well.

    If there is a trick to running a VM with more than 4 cores, we don't know it. I would love to figure it out though.
     
  15. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    It's 100% possible to run more than 4 cores on a VM. Something screwy is going on. Are you using virtualbox or something else?
     
  16. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    21,749
    Joined:
    Aug 14, 2004
    I'm using VirtualBox as well. I can spin up 10 VMs each with 4 CPUs, but if any of them have more than 4 CPUs they wont work. They'll either never load, crash after loading, or something else. Usually, they just hang at the loading VM screen.
     
  17. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    Do you CPU's have more than 4 cores each? I am pretty sure the limitation is that virtualbox won't use 2 sockets for 1 vm
     
  18. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    21,749
    Joined:
    Aug 14, 2004
    One of my systems is a 4 socket system. It has 4x 12 core CPUs. Another is also a 4 socket system with 4x 16 CPUs.
     
    elite.mafia likes this.
  19. DeChache

    DeChache The ONE - Your Ignorance Annoys Me

    Messages:
    6,832
    Joined:
    Oct 30, 2005
    If your on Server 2016 use Hyper-V have ran 24 Core VMs with out issues there. I suspect the problem is VirtualBox and Oracle wanting you to pay for the pro edition.

    Just say no to Oracle.
     
    Orddie likes this.
  20. Skillz

    Skillz [H]ard DCOTY 2017

    Messages:
    21,749
    Joined:
    Aug 14, 2004
    Never tried using Hyper-V. Will look into that though as running 10x 4 core VMs it a pita especially with the amount of RAM I need to give each VM for the project + Windows.
     
  21. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I'm working on trying to get everything going on Hyper-V. So far it seems to be pretty easy. I have been able to assign as many cores as I want to a VM with no issues, so definitely a good start.
     
  22. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    Well I just ran into a pretty stupid issue with Hyper-v, I tried to shut down my server and had to stop my VM's. When I tried to turn them on next, it duplicated the hard drive file and basically ran out of storage on my SSD. I am not sure how to resolve this. It seems like this is a checkpoint trying to restore itself but I didn't even try to do that so I'm not sure why it would behave like this. bleh. Hopefully I can fix it without too much issue
     
  23. DeChache

    DeChache The ONE - Your Ignorance Annoys Me

    Messages:
    6,832
    Joined:
    Oct 30, 2005
  24. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I tried turning it off but I can't get the vhdx file to stop growing, and as a result there is not enough storage space. I tried merging the files but it fails saying unable to set parent. IDK, I'm probably just going to recreate the VM, and disable snapshots (why the f does microsoft call them checkpoints :X)
     
  25. DeChache

    DeChache The ONE - Your Ignorance Annoys Me

    Messages:
    6,832
    Joined:
    Oct 30, 2005
    Because snapshot is VMware speak and they have to be different. I think it was trademarked for a while too.

    Did you create an expanding disk or a fixed size one. It might not be a checkpoint issue. But you should shutdown merge/delete the snaps and then disable the feature.
     
  26. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I did the default option (expanding). I am unable to merge the disk, unable to shrink it, everything just fails because there is no storage space left on the drive. bleh.
     
  27. elite.mafia

    elite.mafia Broke Back [H]

    Messages:
    11,397
    Joined:
    Aug 23, 2004
    I ended up just redoing everything and disabling checkpoints.... Still not sure if I want to keep my host OS as windows 10, I think it might make more sense just having hyper-v as the base OS but is there a way to manage it through the snap-in and not through powershell/cmd ? Anyone have a guide
     
  28. jrobdog

    jrobdog Limp Gawd

    Messages:
    286
    Joined:
    Dec 4, 2006
    Have you considered unraid? Have no problems running over 4 cores in a VM. It uses KVM.
     
  29. MrGuvernment

    MrGuvernment [H]ard as it Gets

    Messages:
    19,161
    Joined:
    Aug 3, 2004
    Use ESXI, so much better in almost every aspect, reason why VMware owns the virtualization world, outside of KVM on amazon :D
     
  30. rtangwai

    rtangwai [H]ard|Gawd

    Messages:
    1,368
    Joined:
    Jul 26, 2007
    ESXi is touchy about old hardware. OP has X5620's, they are not supported past ESXi 6.5 (current version is 6.7). I bet the onboard Intel NICs are also unsupported.

    I know this because I gave up on ESXi on my dual X5650 on a SuperMicro X8DAi motherboard due to those very issues.

    On the same hardware Hyper-V 2019 works perfectly (I do not do any passthrough which helps a lot).

    As a bonus not having to run the vCSA to do updates and such means I save 10GB RAM and a fair bit of storage space.
     
    Last edited: Jun 27, 2019
  31. MrGuvernment

    MrGuvernment [H]ard as it Gets

    Messages:
    19,161
    Joined:
    Aug 3, 2004
    ESXi will work fine on X5650s and that chipset (not sure what your supermicro board had for nics....), I have it running on a R610 with dual X5690's with integrated broadcom's, Not supported vs wont work are 2 very different things. ESXi 6.7 notes it wont support soon those processors but that usually means they will not officially support it on their hardware list.

    Also to update ESXI you can do it via CLI. You dont need vCenter unless you want all the fancy bells and whistles of DRS / HA