• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

How is VMWare Workstation I/O working?

A

afatkulin

n00b
Joined
Oct 23, 2012
Messages
9
I'm running VMWare Workstation 8 on top of a storage capable of doing 200K+ IOPS. The problem is I can't get more than 15K IOPS from a Linux guest no matter how man vCPU or iothreads I'm trying to use. Everything just piles up with wio as far as Linux guest sees it with the same 15K IOPS total.

Now if I run a second VM (and so on) I can get 15K IOPS in each. So three VMs can happily push 45K IOPS combined no problem.

It appears to me that VMWare workstation has some choke through which all I/O for a single VM has to go and it is not scaling well. Anybody is familiar with how IO is done? Does it have to flow through some sort of single-threaded process in the hypervisor which limits how much a single VM can push?
 
No, but depending on how your guest is set up, it may have a queue limitation. That sounds like about what the queue in a standard LSI parallel card can do. Give it a second disk on a second SCSI controller and see if it can suddenly push 30,000 IOPS (Between the two). Also, make sure the elevator for your kernel is set to no-op.
 
lopoetve said:
No, but depending on how your guest is set up, it may have a queue limitation. That sounds like about what the queue in a standard LSI parallel card can do. Give it a second disk on a second SCSI controller and see if it can suddenly push 30,000 IOPS (Between the two). Also, make sure the elevator for your kernel is set to no-op.
Click to expand...

Thanks for the reply. The storage is an iSCSI disk hooked from NAS so there are no controllers involved on the host where VMWare is running.
 
ND40oz said:
2nd virtual controller and virtual disk
Click to expand...

Well there are no virtual controllers in VMWare Workstation unless I'm missing something. I did try the test the multiple virtual disks but they don't make a difference. That's what lead me to believe the IO must be choking somewhere on the hypervisor level. I can scale IO using multiple VMs off of the same iSCSI disk no problem, it just doesn't scale for single VM.
 
There has to be a virtual controller of some kind or the VM wouldn't be able to access storage.

Assign the second disk to node 1:0 instead of 0:2 and it'll add a second controller, at least it does in ESXi. I think that part of the UI is the same for workstation.

Not that this will help you if you need to pull >15K IOPS from a single virtual disk, but I think the idea was as a testing step.
 
Are you doing the test over a single 1Gb link? 8k block size on your file system perhaps?

1Gb link / 8KB block size = ~15,000 IOPs
 
Child of Wonder said:
Are you doing the test over a single 1Gb link? 8k block size on your file system perhaps?

1Gb link / 8KB block size = ~15,000 IOPs
Click to expand...

I appreciate the reply but did you read the question? If it was over a single 1GbE link then multiple VMs won't be able to go over that limit wouldn't they?

Anyway, it's a 2x10GbE links set up with iSCSI MPIO.
 
afatkulin said:
I appreciate the reply but did you read the question? If it was over a single 1GbE link then multiple VMs won't be able to go over that limit wouldn't they?

Anyway, it's a 2x10GbE links set up with iSCSI MPIO.
Click to expand...

You're right, I completely glossed over the part where you've tested multiple VMs simultaneously. :eek:

What is your host OS? I assume what you're doing is attaching the host OS to the iSCSI disk then placing the VMDKs on the iSCSI disk?

As lopo said, this could be a limitation of the virtual SCSI adapter that is presented to the guest OS. Have you tried other guest OS's? If this occurs on, for example, a Windows guest as well then the limitation must be with the virtual SCSI adapter presented to the guest (VMware Workstation itself).
 
markwo said:
There has to be a virtual controller of some kind or the VM wouldn't be able to access storage.

Assign the second disk to node 1:0 instead of 0:2 and it'll add a second controller, at least it does in ESXi. I think that part of the UI is the same for workstation.

Not that this will help you if you need to pull >15K IOPS from a single virtual disk, but I think the idea was as a testing step.
Click to expand...

OK, I can go into "advanced" settings for the disk and specify "virtual device node" that way, for some reason it doesn't ask for it when creating a disk and that's why I missed it. That's seems to be it as I can get 15k from each virtual controller. The only problem is there is only 4 of them max.
 
afatkulin said:
Well there are no virtual controllers in VMWare Workstation unless I'm missing something. I did try the test the multiple virtual disks but they don't make a difference. That's what lead me to believe the IO must be choking somewhere on the hypervisor level. I can scale IO using multiple VMs off of the same iSCSI disk no problem, it just doesn't scale for single VM.
Click to expand...

It's been a few years since I used Workstation, but with ESX/ESXi to get a second controller added, choose to add a new hard disk. When you get to the advanced options section and you get to choose the virtual device node, pick SCSI (1:0). Then finish the creation of the disk, you'll see the 2nd SCSI controller show up. You can add up to 3 additional controllers this way by choosing nodes that don't have a controller yet.

Edit: nm, should have read through the rest of the thread more carefully, already mentioned.
 
afatkulin said:
OK, I can go into "advanced" settings for the disk and specify "virtual device node" that way, for some reason it doesn't ask for it when creating a disk and that's why I missed it. That's seems to be it as I can get 15k from each virtual controller. The only problem is there is only 4 of them max.
Click to expand...

Yeah, that's a PCI limitation (virtual hardware). You could increase the queue in the guest, if it's possible for it to handle more - or use the pvscsi adapter (if Wks has it - I ahven't used it in years).
 
Yes in workstation you can select from an IDE controller or two different scsi controllers. In the VM you need to edit the Virtual Machine itself and then there will be a tab with all the hardware. You can add and change the controllers through there.
 
You must log in or register to reply here.
Back
Top