atrociously slow performance - ubuntu/workstation

T

Thuleman

Supreme [H]ardness
Joined
Apr 13, 2004
Messages
5,833
Fortran code (numerical analysis model) that uses a single CPU only.

physical hardware, Opteron 885, 6GB RAM, smp-lustre.1.6.7.2-perfctr.2.6.28-cve.4
execution time: 1m 49s

VM, Workstation on W7 64, Q9550 single cpu assigned, 3GB RAM, dedicated HD
execution time: 4m 57s

Not cool. The disk for the VM isn't pass through, it's NTFS with a 350 GB datastore for the VM and no other uses.

So I am a little bit at a loss here. I know that disk I/O is slower when using a Workstation VM, but this is just unacceptably slow. The Fortran code is disk write intensive but still the VM should at least perform in the same neighborhood as the physical machine, no?

I don't have much experience with Workstation other than having a couple test VMs myself but I was under the impression that it is a solid product but these numbers are just unacceptable.

This doesn't appear to be a memory issue as the 3GB on the VM are not fully utilized during a model run.
 
VMWare Tools are installed. The VM boot subjectively feels ok, certainly doesn't seem 2-3x slower than a physical Linux box.

If I had any indication that direct disk access would speed things up I'd go for an reinstall and try it out, but I distinctly remember a discussion here that essentially concluded that the VMDK overhead is basically negligible in a single disk configuration.
 
How is the performance inside KVM? I'm guessing you are running VMWare as it doesn't require amd-v for windows guests, no ?
 
What kind of writes does Fortran do? Small random? Is your disk partition-aligned? Latest version of VSphere and Workstation really do benefit from multiple cores assigned to guest if gueset app takes advantage of threading:

WPrime32 score of my VSphere guest with 4 virtual cores assigned (Dual Xeon 5650's):

f081a223.jpg


Same test with only 2 virtual cores assigned:

5e383d81.jpg


My alienware M11xR2 i520 with 4 cores assigned:

59755083.jpg


My alienware M11xR2 i520 with 2 cores assigned:

66c6afc5.jpg


Get this, my native score is actually slower than my virtual one, 4 virtual vs 4 real........well dual core w/hyperthreading (still trying to figure that out):

4c387f7e.jpg
 
Robstar said:
How is the performance inside KVM? I'm guessing you are running VMWare as it doesn't require amd-v for windows guests, no ?
Click to expand...
The host is Windows 7 the guest is Linux. Never had any issues with VMware Workstation before and we do have a site license for it so it was the natural choice.

MassiveOverkill said:
What kind of writes does Fortran do? Small random? Is your disk partition-aligned? Latest version of VSphere and Workstation really do benefit from multiple cores assigned to guest if gueset app takes advantage of threading:
Click to expand...
The writes are larger sequential writes of model output, so each file is several dozen MB and the OS is basically idle while the model is running, no other actions are performed on the VM. I was under the impression that the latest Workstation aligns disks by default, I should look into that later on today.

I don't think threading will really matter much, I didn't actually look at the code but I am fairly certain that it is highly sequential code because the guy who wrote it isn't a programmer but a climate scientist. I am not actually running this myself, just set it up for a coworker. Will grab the code later on today and have a look to see what's what.
 
How did you dedicate the hard drive? Is there a way to do pci-passthrough in vmware workstation? If so, do this and you should see much better performance...
 
So I looked at the code and it's not threaded and the writes are sequential. I am currently monkeying around in the code to add some output that will indicate just where exactly the things slow down.

Robstar said:
How did you dedicate the hard drive? Is there a way to do pci-passthrough in vmware workstation? If so, do this and you should see much better performance...
Click to expand...

Currently I have one physical disk "dedicated" in a way that it's not being used for anything in the host OS but it is NTFS formatted and the only files on it are the VM files. Workstation does allow for passthrough and if the issue turns out to be disk I/O I will reinstall the guest OS using the passthrough disk setup and see whether that improves things.
 
You are using VMDK files that are provisioned to the full disk size to start, and not dynamically expanding disks, correct?
 
Yeah it's a thick disk.

I eliminated all write statements in the code so no data is written to file at all, no temp files are used, etc.

The runtime when running on the VM is now 3m 52s. Getting rid of allt he writes only saved me about a minute. It's still significantly slower computing than it is on the physical hardware. In fact, I put Ubuntu on a P4 2Ghz old Dell and it runs in just under 2 minutes on there.

This is nothing short of crazy and amazing at the same time. Very strange and somewhat frustrating.
 
Just as an update on this, I have now tried this with Workstation 7 and Workstaton 6.5 on a couple different Intel QXXXX CPUs and the results are the same, as in running the code is significantly slower on the VMs than it is on the physical machines.

Next up running it on ESX to see whether this is a Workstation problem or a VMware problem. I'll be able to put in a support ticket once this runs on ESX which may help this along.

I also have been thinking about to maybe start a blog with this issue. I don't blog at all, I tend to not have the time for it, but this is puzzling and aggravating enough to maybe make it worth to put well documented test cases online in case someone else runs into the same issue.
 
Thuleman said:
Just as an update on this, I have now tried this with Workstation 7 and Workstaton 6.5 on a couple different Intel QXXXX CPUs and the results are the same, as in running the code is significantly slower on the VMs than it is on the physical machines.

Next up running it on ESX to see whether this is a Workstation problem or a VMware problem. I'll be able to put in a support ticket once this runs on ESX which may help this along.

I also have been thinking about to maybe start a blog with this issue. I don't blog at all, I tend to not have the time for it, but this is puzzling and aggravating enough to maybe make it worth to put well documented test cases online in case someone else runs into the same issue.
Click to expand...

Keep us up to date. I'm very interested in the outcome...
 
You must log in or register to reply here.
Back
Top