Scalable PHP apps, benchmarking in a VM lab

[nry]

I am a PHP developer who primarily works with Zend Framework with mySQL, I don't have much experience with building scaling applications. This seems to be in increasing demand from my reoccurring clients now due to their higher profile clients and essentially higher traffic websites...

So far have two projects under my belt which were built to run on AWS with RDS, ElastiCache, Multiple EC2 instances and a load balancer and these have all run perfectly since day 1. Code was fairly optimized for this with all assets being stored in S3 with the EC2 instances having non persistent storage. Full opcode caching, minimising disk reads, using Nginx, PHP-FPM for the web server and caching DB queries where ever possible.

What I am trying to achieve here is the best practices when building scalable apps, which I think I am already on the right path to. As well as being able to test them fully before handing them over to the sys admin.

What I would like to be able to do is benchmark these apps in my VMWare lab, by this I mean setting up a load balancer, multiple web servers, multiple DB servers. Then have multiple VMs which simulate load on the app.
Maybe with the load generating VMs running selenium test cases I use
Is there any software out there designed to do such a thing?
How can I easily spin up additional web instances within my lab?
Something with graphs always wins me over lol

My lab consists of (or will by the end of the week...) so have a fair few resources to work with

NAS 
 - 8x 3TB in RAID6 (2 of)
 - 8x 1TB in RAID10
 - 10GbE

ESXi 1
 - Xeon E3 1245
 - 32GB RAM
 - 10GbE

ESXi 2
 - AMD FX8150
 - 32GB RAM
 - 10GbE

ESXi 3
 - Xeon E3 1245
 - 16GB RAM
 - 1GbE

ESXi 4
 - C2D E6750
 - 8GB RAM
 - 1GbE

Some software which I have found on my own, but all would need bringing together
Vargrant - http://www.vagrantup.com/
Chef - http://www.opscode.com/chef/
Fake S3 - https://github.com/jubos/fake-s3

Currently to test and optimize my apps I use a combination of Xdebug with webgrind
Apache ab on a single instance VM
All of which seems to help me find problems for smaller scale apps.

So in summary I am probably a little bit mental and going about this completely the wrong way, but could just do with pointing in the right direction.

Any help or suggestions would be grateful

 

[nry]

No one?

 

[Parmenides]

With PHP you don't need benchmarks, you just need to throw more hardware at it or go with an auto-scaling cloud service. (joking)

I am curious for an answer too.

 

[mikeblas]

I'm not too sure what your specific question(s) is/are. i note that you're using VMs to test, which is fine; but I'm not usre you want to spin up a VM for each web server instance you have. You can run multiple instances of Apache (each listening on a different port) then make sure your test client can communicate to different ports. That reduces the need for all sorts of VMs and memory load they'll cause.

I'm not sure if the point of the VMs is to emulate some aspect of your production environment, but ...

Are you doing functional testing or strictly load testing?

 

[nry]

It's more of a learning exercise on building these apps correctly.
As well as understanding how to setup all the interlinked systems

The vm's are to simulate production environment

I would like to load test using functional tests but understand the resources required for this might be through the roof.

Have done some research and I should be able to use chef to create new instances of what I require
Jmeter looks like it supports multiple remote instances to create a load.