• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Need help with Linux VM setup

M

may i be worthy

[H]ard|DCer of the Month - December 2010
Joined
Aug 17, 2010
Messages
1,186
Here is the thing. I use my 2 SR2 renderslaves to both fold and render 3D. Folding is 90% of the time. Rendering is only 10%, but pays for the whole enterprise. ;)

For folding I want to run Ubuntu, for rendering I need Win7. (Won't work in Wine) Right now I run Ubuntu on the weekend, and Win7 during the week.

Win7 Host running Ubuntu as guest inside a VM sucks for folding on 2p / 24 threads. So what about the other way around? I want to try running Ubuntu fulltime due to high levels of awesomeness, and Win7 inside a VM to render when needed. No rebooting. Just tried Virtualbox, but could not get it to boot the VM let alone install Win7.

Goal is reasonable performance (80%+ of native) and reliability. I don't need the power often, but when I do it had better deliver.

So short of Stanford releasing a 64 bit FAH core that is NUMA aware, I need some advice for my project.:cool:
 
nitrobass24 said:
Why dont you just dual boot?
Click to expand...

That is what I am doing now.

The issues are that booting has to be while I am awake and home, and is a PITA. Work units run from under two hours for SMP to 17 to 24 hours for bigadv. So I have to be finishing up on Linux by Sunday lunch to make sure I am good to go by Monday morning.

The kind of work I am doing at the moment is very light on for render time - but vital to have it online and available during the work day.

I just hate losing 25% of my PPD being in Windows. I could get therapy to learn to just let it go and relax, or I could continue to be a member of [H]. ;)
 
nitrobass24 said:
Why dont you just dual boot?
Click to expand...

He doesn't want to shut down the machine or be stuck waiting for WUs to finish when he needs the two slave machines for rendering.

I ran the Linux VMWare client with a WinXP VM for a while around a year ago, and it worked fine. I think you still need the 3.0 version for 8 thread compatibility. You would have three virtual Windows machine on each slave, each with their own unique IP address. Assume that is all your rendering software needs, it should work - each would load the machine 33%, all three going full bore would be 100%. Since you used to have 6 or so Q6600s doing the job, I don't think that the number of "machines" will cause you issue. If you can't find the V3.0 Linux VMWare client, I am sure I have it around somewhere and could send it to you when I get home this evening.
 
Interesting idea - that never occurred to me - but I am using distributed bucket rendering - all 3 rigs rendering the same large frame at once, so I want 24 threads. Network rendering animations - one frame per rig would be ok under that scenario.... but big issue is RAM - I did not design my farm that way - the godbox has 24GB and the slaves only 12GB, which is healthy, but not enough to do 3x per box. (and probably saturate the network. )

And the other reason I went with 3x SR2s rather than 1 SR2 and 4 or 5 1p rigs for less money was the joy of not having to maintain that many installs. I imagine there would be cloning and renaming going on, but still.

I will read up on using Xen to run 2 guests....
 
what issue are you having with virtual box ?

Is the program just not starting?

i've run multiple instances of windows 7/vista/xp in Virtual Box on ubuntu (9.10, 10.04, 10.10) with no issues..
 
I've never had much luck with virtualbox from the repositories - have you tried grabbing the latest version on the download site?
 
Not that I've tried doing it, but how about running vmplayer in ubuntu with Windows 7 in it?
Other option may be to run ESXi, set up both boxes as VMs, use remote desktop (or Vsphere client to access the Windows 7 machine). Give the Windows 7 box highest priority, and the ubuntu box lowest priority to the CPU, and let them go to town. <-- I do this at home on my Q9400. Runs my home domain server, a clean Windows XP VPN Client for work, and a ubuntu Folding client.
 
The problem is the number of cores limitation with VMWare (8) or the fact the Virtual Box is not very good. MIBW needs a Windows 7 VM that will use all 24 cores of his SR-2s when needed. I do not know of a way to do that.
 
I have no idea if this will work but I will put it out there.

What if the folding directory was on NTFS and mounted in linux. Then when running lInux he is folding the units in that folder. He stops the work unit reboots to Windows and it picks up where it left off.

Would this work? I have no idea. maybe something has to be symlinked in??? I have no idea but this way he wouldn't have to wait for the unit to end just reboot at the beginning and end of the work day.
 
nwrtarget said:
What if the folding directory was on NTFS and mounted in linux. Then when running lInux he is folding the units in that folder. He stops the work unit reboots to Windows and it picks up where it left off.
Click to expand...
The work units are packaged during assignment to only work on the host OS that it detected during assignment. When the work unit starts folding on say Linux, it expects to find the Linux binary of the core. You can't download a WU on Linux and move it to Windows.
 
MIBW
Good luck with Xen. Looks like it should at least be possible there. Would you fold on the OS that supports XEN or on a guess OS?
 
Lord_of_Doors said:
I've never had much luck with virtualbox from the repositories - have you tried grabbing the latest version on the download site?
Click to expand...

That was the latest from the site. Quite possible I havent configured the VM correctly - I did what worked for the other way around (Win host, ubuntu guest). Well kind of worked, performance sucked.

W.Feather said:
try new version of Virtual Box , or possibly a bad .iso for windows
Click to expand...

Never got as far as installing the guest OS.

nwrtarget said:
MIBW
Good luck with Xen. Looks like it should at least be possible there. Would you fold on the OS that supports XEN or on a guess OS?
Click to expand...

Too early to tell - I dont know enough about it. I imagine I would want to keep Ubuntu folding - it was hard enough to find a Linux distro that screams on SR2s. But it might be all about overhead. I gather that would mean 2 Guests plus overseer host.

cactus said:
With ESXi you can increase the amount of cores per vCPU. Would that be a way around the 8 cpu limit musky is talking about?
Click to expand...

Will check that out, thanks for the tip. Because what Vmware really need is to split VMware into more f*cking products. My head spins after reading that website.
 
Hmm, need to get my head around this - I gather some of them are teeny tiny OS that are basically hosts and managers. I don't know if that ill be enough to mess up Ubuntu performance.

The ideal is a high performance vm that can run from within the known working ubuntu would be best. I wish there was a simple VMware high performance version - there might be, but I cannot find even a basic comparison table that shows how many cores etc. Am I the only person in the world that might find that useful? jeez.
 
musky said:
ESXi is an OS, though. You would run it instead of Ubuntu. It is a Lunux variant of some sort, but i know nothing about it.
Click to expand...

Actually it's not - ESX used to have a management console that was actually a VM running on ESX, that was based on redhat - but VMware basically ripped out anything Linux related. Its a pure Type 1 hypervisor - no underlying OS, the hypervisor runs on bare metal.

MIBW, if you go the ESX path, you may want to look at vSphere Essentials. Its about a AUS$4-5K outlay, tax deductible presumably, but gives you serious manageability of the servers and VMs. Freebie ESXi will work, but a lot more over head. The only thing nagging at me is if you can create a VM with 24 virtual cores.

H.
 
Thanks for that. I should have mentioned, this has to be zero cost - it is a stopgap solution until the 64bit Windows client comes out. I consider myself reasonably [H]ard, but not $4k on software to get another 80K ppd. ;)

/internal conversation on

Or I could just buy a G34 system... stop it. No stop it.

/off
 
MIBW,

I don't know rendering software - but are three renderers with 8 cores each, able to operate anywhere near the renderer with 24 threads ?

H.
 
Haitch said:
MIBW,

I don't know rendering software - but are three renderers with 8 cores each, able to operate anywhere near the renderer with 24 threads ?

H.
Click to expand...

yes and no, but mostly no due to triple the ram requirement, and likely network bandwidth. And operator headaches maintaining. :cool:
 
Since I still cant seem to get my SR-2 running right overclocked, I can test the ESXi stuff tomorrow at stock. I'll see if I can get the free version of ESXi working with a 24 thread W7 install and a 24 thread U10.10 install.
 
thanks cactus, will watch that with interest.

Just had a thought. How does VMware handle HT and the 8 core limit?... I forget that I am running a 12 core beast, with HT.

Can a VMware use 8 real cores of my 12 without disabling HT on the host? That might change the dynamic a bit - try a 8 core / 4 core split with 2 VMs?
 
may i be worthy said:
Hmm, need to get my head around this - I gather some of them are teeny tiny OS that are basically hosts and managers. I don't know if that ill be enough to mess up Ubuntu performance.

The ideal is a high performance vm that can run from within the known working ubuntu would be best. I wish there was a simple VMware high performance version - there might be, but I cannot find even a basic comparison table that shows how many cores etc. Am I the only person in the world that might find that useful? jeez.
Click to expand...

Not the only one, I went onto the VMware website last night seeing if there was any info that would be of use to you. But no, I couldn't find anything useful there at all that would tell you what version to run. looks to me like its a website designed for those who know what they want, not someone wanting to find stuff out to make a decision
 
I would look at citrix xenserver free edition

Its is my understanding that ESXi will only allow a guest with up to 8vCPUs
 
nitrobass24 said:
I would look at citrix xenserver free edition

Its is my understanding that ESXi will only allow a guest with up to 8vCPUs
Click to expand...

That was mine a well...and that is 8 total, physical or virtual. To use up all 24, you would need three VMs.
 
MIBW I'll be watching your progress with interest. I have a similar dream of running Linux on my SR2 with a Windows guest OS. I don't need such high performance from my Windows VM though, so that probably makes my requirements significantly lighter than yours.

may i be worthy said:
Can a VMware use 8 real cores of my 12 without disabling HT on the host?
Click to expand...

My understanding (and what I've been told when I asked a similar question to some experts years ago) is that the "real" cores will always be used to the extent possible. So, if you send your CPUs a job that only requires 8 threads, it will essentially perform like a box with 8 physical cores.

Hyperthreading is designed to be a boost, not a hindrance. It was conceived when we had single core CPUs. If there happened to be a 2-thread workload heading toward the CPU, hyperthreading kicked in to boost the performance a bit. In the neighborhood of 10-15%
 
I can confirm that musky and nitro are correct with ESXi. You are allowed only 8 total threads per VM.
After some searching, I have found that it looks to be possible with XenServer to go up to 32 with a Linux VM. I don't have time to try today, nor do I have any experience with Xen.

The Xen info was found from these:
Concise >8vCPU how-to
Thread talking about it on the Citrix support forum
This is how to make more "cores" per socket in Xen, this will be needed if you want to run W7 because it is limited to 2 sockets

MIBW hope this helps,
Max
 
Thanks guys - glad i did not waste my time more with VMware... off to mothers day lunch..
 
original.jpg


After much faffing around, I finally have a nearly working Win7 install in a KVM virtual machine on Ubuntu. Just need to get it to run on more than 2 cores...

While Xen looks like the ultimate option, I have tried KVM first within host Ubuntu - as this way I know the folding will be fine, it will just be a matter of tuning Windows VM. This means I can fiddle and try things while folding, rather than start over with installing XEN, then Ubuntu and Win7. (days of faffing - and high risk of not getting the delicate Ubuntu & BFS magic working again)

Followed KVM install instructions here : https://help.ubuntu.com/community/KVM/Installation

  1. Headache one - I stupidly installed Ubuntu to a small partition. There are all sorts of wierd permission related errors, even running as root if you try to make a VM on another partition. Finally worked to use a separate 100gb Ext3 partition, but created without taking ownership of the file system.
  2. Headache Two - installing Win7 from an ISO had issues, finally did it from a CDROM and it worked.
  3. Headache Three - VM installed and works fine so far with networking etc, but once I rebooted it it died - after much faffing I learnt you need to have the VM CD-ROM disabled or else it would not boot without the CD still in the drive. :rolleyes:
  4. Headache 4 - the biggie - only using 2 CPUs.

I have not done guest drivers - I am not sure they are needed if the network is running ok.

original.jpg


As you can see I have selected 24 CPUs in the KVM GUI, and manually entering pinning 0-23 and 0-11,12-23 had no effect.

Googling it - I think the issue is that Windows is seeing two sockets - the licensed limit for Win7 Ultimate - whereas what we want KVM to show it is 2 sockets of 6 cores and 12 threads each.

Curiously the device manager shows 24 virtual CPUs as you can see in the first screenie.

Topology fix needed.

I found this page which looks like it has the answers, but I am lost as how to implement them in an XML config file - I made this whole thing with the GUI. Like I would know where an XML file lives. Pfft.

I ran # virsh capabilities which gives me this:

Code: 
virsh # capabilities
<capabilities>

  <host>
    <uuid>00020003-0004-0005-0006-000700080009</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Nehalem</model>
      <vendor>Intel</vendor>
      <topology sockets='2' cores='6' threads='2'/>
      <feature name='rdtscp'/>
      <feature name='pdpe1gb'/>
      <feature name='dca'/>
      <feature name='xtpr'/>
      <feature name='tm2'/>
      <feature name='est'/>
      <feature name='vmx'/>
      <feature name='ds_cpl'/>
      <feature name='monitor'/>
      <feature name='pbe'/>
      <feature name='tm'/>
      <feature name='ht'/>
      <feature name='ss'/>
      <feature name='acpi'/>
      <feature name='ds'/>
      <feature name='vme'/>
    </cpu>
    <migration_features>
      <live/>
      <uri_transports>
        <uri_transport>tcp</uri_transport>
      </uri_transports>
    </migration_features>
    <secmodel>
      <model>apparmor</model>
      <doi>0</doi>
    </secmodel>
  </host>

  <guest>
    <os_type>hvm</os_type>
    <arch name='i686'>
      <wordsize>32</wordsize>
      <emulator>/usr/bin/qemu</emulator>
      <machine>pc-0.12</machine>
      <machine canonical='pc-0.12'>pc</machine>
      <machine>pc-0.11</machine>
      <machine>pc-0.10</machine>
      <machine>isapc</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/bin/kvm</emulator>
        <machine>pc-0.12</machine>
        <machine canonical='pc-0.12'>pc</machine>
        <machine>pc-0.11</machine>
        <machine>pc-0.10</machine>
        <machine>isapc</machine>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <pae/>
      <nonpae/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
    </features>
  </guest>

  <guest>
    <os_type>hvm</os_type>
    <arch name='x86_64'>
      <wordsize>64</wordsize>
      <emulator>/usr/bin/qemu-system-x86_64</emulator>
      <machine>pc-0.12</machine>
      <machine canonical='pc-0.12'>pc</machine>
      <machine>pc-0.11</machine>
      <machine>pc-0.10</machine>
      <machine>isapc</machine>
      <domain type='qemu'>
      </domain>
      <domain type='kvm'>
        <emulator>/usr/bin/kvm</emulator>
        <machine>pc-0.12</machine>
        <machine canonical='pc-0.12'>pc</machine>
        <machine>pc-0.11</machine>
        <machine>pc-0.10</machine>
        <machine>isapc</machine>
      </domain>
    </arch>
    <features>
      <cpuselection/>
      <acpi default='on' toggle='yes'/>
      <apic default='on' toggle='no'/>
    </features>
  </guest>

</capabilities>

So, dear [H], I need some Ubuntu-KVM-awesomeness handholding to get this thing configured right. We are close, I can smell it! :cool:
 
Shouldn't this:

<topology sockets='2' cores='6' threads='2'/>

be this:

<topology sockets='2' cores='6' threads=1'2'/>

or maybe

<topology sockets='2' cores='12' threads='24'/>

Never worked with this, so I am blantantly guessing...
 
Your guess beats my linux noobiosity. Ta.

I was just reading the https://help.ubuntu.com/community/KVM/Managing

Going to try and spit out that xml and give this a go:

Editing the attributes of a Virtual Machine
libvirt stores it's configuration as xml in '/etc/libvirt/qemu'. The xml is easy to understand, and is similar to VMware *.vmx files. While it is possible to edit these files in place and restart libvirt-bin for the changes to take affect, the recommended method for modifying the attributes of a virtual machine is via virsh (or virt-manager, if it supports changing the hardware you want to change). The concept is simple:

export (aka 'dump') the xml of the virtual machine you want to edit
edit the xml
import (aka 'define') the xml
For example, to edit the machine named 'foo' (you can get a list of your machines with 'virsh list --all'), do:

$ virsh dumpxml foo > /tmp/foo.xml
(edit /tmp/foo.xml as needed)
$ virsh define /tmp/foo.xml
Click to expand...
 
http://serverfault.com/questions/10...der-linux-kvm-not-use-all-the-virtual-process

I have edited the xml to include the

<vcpu>24</vcpu>
<cpu>
<topology> sockets='2' cores='6' threads='2'</topology>
</cpu>

But nothing has changed... wondering if I have to redo win7, or if it is a kernel problem.

On libvirt 0.8.3, if you type:

virsh capabilities | grep topology
it will list the topology of the host:

<topology sockets='1' cores='4' threads='1'/>
The numbers refer to sockets, cores per socket, and threads per core. Add this line to the cpu entry in the xml file to allow windows to use all 4 cores, e.g.:

<vcpu>4</vcpu>
<cpu>
<topology sockets='1' cores='4' threads='1'/>
</cpu>
Click to expand...
 
Uninstall the CPUs in the VM (Device Manager) and re-boot it to pick up the newly available cores? "vcpu" is a common VM term, so that part sounds right. But, when Win7 was installed, you did not have the vcpus available for the guest OS to pick up.

Guessing here....
 
Further reading here (bottom post) looks like ther might be a flag to force the issue - so that Win7 does not think it sees virtual cpus

But this "args: -cpu host" everyone mentions may not be for this flavour of KVM - confused as where to put it. They say in the vm-config - but I thought that is what I am editing, and it would need xml tags around it...

- I have added all this to the config, but no luck.

Code: 
<vcpu>24</vcpu> 
    <cpu match='exact'>
      <arch>x86_64</arch>
      <model>Nehalem</model>
      <vendor>Intel</vendor>
      <topology sockets='2' cores='6' threads='2'/>
      <feature name='rdtscp'/>
      <feature name='pdpe1gb'/>
      <feature name='dca'/>
      <feature name='xtpr'/>
      <feature name='tm2'/>
      <feature name='est'/>
      <feature name='vmx'/>
      <feature name='ds_cpl'/>
      <feature name='monitor'/>
      <feature name='pbe'/>
      <feature name='tm'/>
      <feature name='ht'/>
      <feature name='ss'/>
      <feature name='acpi'/>
      <feature name='ds'/>
      <feature name='vme'/>
      <feature policy='disable' name='lahf_lm'/>
     </cpu>

Just tried this


Code: 
dave@sr2c:~$ kvm -cpu ?
x86           qemu64
x86           phenom
x86         core2duo
x86            kvm64
x86           qemu32
x86          coreduo
x86              486
x86          pentium
x86         pentium2
x86         pentium3
x86           athlon
x86             n270

No mention of Nehalem... hmmm, am I using the latest thing?
 
You must log in or register to reply here.
Back
Top