Three open source Web service testing tools get high marks
Capable soapUI, TestMaker, and WebInject toolsets shine once you conquer their learning curves
Projects are arranged hierarchically and contain one or more TestSuites, which contain one or more TestCases, which in turn contain one or more test steps. The actual work – sending requests, receiving responses, analyzing results, and altering test execution flow – happens at the test step level. TestCases gather and organize the steps need to perform a specific operation on the target. TestSuites gather TestCases into larger aggregates that exercise a particular area of a Web service (such as the operations necessary to order a book). You can create new TestSuites, TestCases, and test steps by right-clicking on the parent node in the project's tree and selecting New from the pop-up context menu.
Then soapUI determines a given test's success or failure by assertions that you attach to test responses. There are a half dozen assertions to choose from, ranging from a "simple contains" test -- which succeeds if a provided string is matched -- to "XPath matching," which succeeds if an arbitrarily complex XPath expression matches the response.
Click for larger view.
The Groovy test step (I am not making that name up) is soapUI's most powerful. Here, Groovy refers to the lightweight Javalike scripting language. A Groovy test step executes whatever Groovy code you want to place in it, which means that it can do pretty much anything that can be done in Groovy. The Groovy code in such a test step has access to the soapUI framework. For example, a Groovy test step might read information from a database (via JDBC), compare that information with a previous test step's response, and alter the execution flow of the TestCase accordingly – perhaps even executing a different TestCase.
Aside from the functional testing, SoapUI will also unleash load tests on a Web service. Each load test consists of the execution of one or more TestCases, and can be tuned to simulate various scenarios. You can, for example, control whether the test executes for a specific amount of time or a specific number of iterations, whether the activity executes in “bursts” or varies linearly over time, and so on.
When the load test is complete, a Load Test Editor provides a raft of statistics for each TestCase: number of executions; minimum, maximum, and average execution time; and others. You can even examine the results graphically on the Statistics Diagram page.
It is easy to get soapUI running; it will take you almost no time to build a basic project and construct rudimentary tests. My one gripe with the tool is that there is no contextual help in the system, which makes figuring out what your options are in particular areas of the application difficult. Nevertheless, the documentation provided is quite good, so any initial confusion evaporates with continued use.