Exclusive: e-Test 8.0 earns very good marks

With the latest release of e-Test Suite, Empirix continues its tradition of providing Web site developers with point-and-click simplicity that makes creating and executing test scripts a breeze. However, the new version only partially addresses one of my major gripes about the 7.0 incarnation: the lack of an integrated scripting language to allow for true customization and extensibility of the test scripts.

Free IT resource Hear how top CIOs turn change into a competitive advantage. Sponsored by HP Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld Empirix e-Test Suite 8.0 Empirix, empirix.com Very Good  8.0 criteria score weight Manageability 8 25% Scalability 8 25% Configurability 7 20% Setup 9 20% Value 8 10% Cost: $30,000 for a single e-Load agent seat license with 100 virtual users; $5,000 per seat for e-Tester Platforms: Supports Windows for script development (e-Tester) and most major server OSes and application back ends (Windows, Linux, Unix, WebSphere, BEA, Oracle, and so on). Bottom Line: e-Test Suite 8.0 brings more of the same in terms of ease-of-use while greatly expanding the platform's reach with the inclusion of a revised Java-based agent architecture and expanded back-end server support. Although still not as well-integrated or capable as higher-end products such as Segue's SilkPerfomer, Empirix is nonetheless taking e-Test in the right direction by opening it up and expanding its horizons. By far the easiest of the Web-application testing solutions to master, e-Test is the ideal solution for customers seeking a quick return on their investment. About our Reviews and Scoring Methodology

Version 8.0 introduces a more scalable, Java-based client simulation agent that accepts Java source code as its scripting input. Although this is an improvement over the previous version (which did not provide access to script source code), the feature is still not fully integrated. For example, there is no code editor within the e-Tester script-recording tool. Customers must use an external IDE, such as the one being developed as part of the open source Eclipse project, to edit the code, which is created in parallel with the proprietary e-Tester script code.

That criticism aside, e-Test Suite 8.0 delivers some attractive if incremental improvements over the previous release. The most important of these is the complete rewrite of the e-Load UI. e-Load is Empirix’s automated workload-generation service. Using scripts created in the product’s e-Tester recording tool, e-Load allows you to simulate multiple, concurrent virtual users for stress testing and more.

In its previous incarnation e-Load was a native 32-bit Windows app that ran locally on the system with e-Tester and the rest of the suite. Version 8.0 effectively decouples the UI from the underlying workload engine, allowing you to access, configure, and execute workload packages from any PC via a Web browser.

The ramifications of this change are significant. Not only does it make e-Test more scalable (separating business logic from UI logic is always a good thing); it also makes e-Load a multiuser environment. Multiple users can now connect to the workload agent server and observe, in real time, the results of the currently executing scenario. They can also start their own independent scenario and thus share the agent server’s workload engine. The advantages, in terms of collaborative opportunities and the ability to share feedback within a group, are obvious.

I took the new e-Load for a test-drive on my own development workstation. Using an embedded Web server running off of port 8088, I was able to access the new e-Load UI through IE and quickly schedule a simple test scenario I had created previously in e-Tester. The UI itself was easy to navigate, making good use of tabs, toolbars, and various dynamic elements. Clearly defined field labels and logical element groupings helped guide me through the configuration process.

When I was ready to execute the scenario I simply clicked the big green button labeled Run Test, itself a dynamic UI element (it won’t work until it detects that a viable test configuration has been defined). I was then presented with a grid of active virtual users where I could observe as the user count was ramped up according to the specifications I provided. Other than a few UI blemishes (some of the Web page graphic elements didn’t display properly at 1,600 by 1,200 pixels) and the occasional required Web page refresh, the UI was clean and generally well thought out.

One welcome beneficiary of the e-Load UI makeover is the e-Test Suite ServerStats function. ServerStats allows you to attach a series of predefined performance-counter configurations to your test scenario, thus augmenting the results with additional data on Web server performance, database behavior, and similar infrastructure-level diagnostics.

In the previous version, ServerStats always was never particularly well-integrated with e-Load, a blemish addressed quite effectively with Version 8.0. Now ServerStats is an integral part of the new e-Load Web UI, with a link to a pop-up configuration window prominently displayed within the AutoPilot agent settings page. Empirix has also expanded ServerStats coverage to include popular Web application servers such as IBM WebSphere, BEA WebLogic, and Oracle, and you can now mix and match grouped functions (profiles) with individual counters to further tune the collection process.

Otherwise, much of the e-Test Suite remains relatively unchanged. The e-Tester script recorder, for example, is still one of the easiest tools of its kind to master, with an intuitive selection of toolbar buttons and copious use of contextual help and other cues. In fact, it took me less than five minutes to create a new script in e-Tester and then deploy it for testing via e-Load. Empirix truly understands the concept of rapid application development.

Of course, now that Empirix has shown what it accomplishes by Web-enabling e-Load, it will be interesting to see which other elements will be decoupled from the suite. I would like to see a Web-hosted version of e-Tester, perhaps with some of the same collaborative options found in e-Load. The ability to remotely observe, in real time, as a script is being developed -- with some sort of “markup” capability so team members can contribute their thoughts and ideas -- would be an attractive addition to an otherwise well-rounded solution.

Empirix e-Test Suite remains one of the easiest-to-use Web-application testing platforms. The entire development environment is geared toward getting you up and running quickly, and the addition of a decoupled, Web-based e-Load UI further extends the platform into the multiuser arena. My only remaining gripe concerns the poorly integrated Java support, something that Empirix plans to address in future releases. Still, this shortcoming takes nothing away from the many positive qualities that continue to define the e-Test Suite solution.