When we re-ran the load test, the scalability was much improved, certainly enough for proof of concept. Scott's employers were then able to move ahead with the project, and Scott has since been setting up real servers in the company's data center and finishing up his application.
As I'm not intimately familiar with the whole gamut of load-testing software, I'm not really in a position to give NeoLoad a set of numerical scores for a formal review. I can say that NeoLoad had no trouble telling us what we needed to know, even though the company doesn't have an "agent" for our application server. Just seeing the CPU and memory loads for the VMs running the Web application server told us quite a bit, and combined with the response time measurements and the MySQL diagnostics gave us a fairly good picture of what kind of hardware the retailer would need. It also provided some clues about where we might find additional performance and scalability improvements in our software.
What follows is a visual, step-by-step tour of our process. Click each screen for a closer view.
1. Defining user populations. In this case we only needed one kind of user.