The LISA documentation makes a big deal of no-code test development, as if that is the high road to simultaneously simplifying and accelerating test development. Perhaps, but while LISA's pure UI-approach does have the benefit of live interaction and is more accessible to QA engineers inexperienced at coding, it has limitations that a tool with easier access to scripting does not. Some testing nuts can only be cracked by a well-sharpened piece of code.
Mindreef SOAPscope Server 6.0
SOAPscope Server, like QEngine, is a thin-client-based tool. Behind SOAPscope's browser UI is a Tomcat server, girded by an RDBMS (relational database management system) that can be MySQL, Oracle, Microsoft SQL Server, or the embedded Apache Derby database. (Derby is supplied with SOAPscope but not recommended for even moderately large installations.)
SOAPscope Server's service spaces are the overarching containers of testing assets. An administrator will use service spaces to organize users into groups. Within a service space, member users can create one or more workspaces in which to store their, well, work.
Inside a workspace you'll find WSDL contracts, tests, notes, and other ancillary material needed to support actual testing. Typically, a workspace corresponds to a WSDL: When you create a new workspace, the first prompt you encounter is for a WSDL URL. You can, however, add more WSDL contracts to the workspace once it is created. Once you've imported a WSDL into a workspace, you can begin adding messages to that workspace. A message is really a SOAP request/response pair, created when you invoke a Web method on a WSDL. The invocation also optionally creates an "action" within the workspace.
The distinction between "message" and "action" gets a bit tricky. Messages are a kind of journal of your interactions with the Web service; each message is a request/response pair stored in a list each time you invoke a Web method. Actions are also messages, but they are kept in a separate area in the GUI. More importantly, actions can be arranged in an arbitrary sequence and replayed in that sequence. Such a replayed sequence is a test script. SOAPscope lets you fortify scripts with realistic behavior; for example, you can pass values among actions in a script so that a subsequent request is altered based on a preceding response.
Messages become actions when you enable a Recording flag (from a menu selection), so you tweak a message until you arrive at a request/response pair that you've determined will make an acceptable test instance. Then you turn on Recording and invoke the message; the associated action is created in the actions section, where it can be incorporated into a script.
Invoking a Web method to generate messages and actions is fall-off-a-log easy. SOAPscope is a very visual environment; similar to LISA, there is little or no programming involved. Request inputs can be either fixed values (entered manually), global values, or fed in from a data grid (such as a small spreadsheet built into SOAPscope). Also, SOAPscope can perform load testing, but you must first purchase the Mindreef Load Check module. The workloads employed by a load test are scripts imported from workspaces, and they're configured to expose the target Web service to different quantities of virtual clients executing those scripts.
SOAPscope can test the client side of Web services via the SOAPscope Server. You create a set of requests and responses – a request/response pair in this context is called a reaction – and store these pairs with the SOAPscope Server. Point the client to be tested at the server and submit requests. The server will search through the reactions; when it locates one with a stored request that matches the incoming request, it sends back the associated response. The upshot is a mock Web service – not particularly elaborate, but effective.
SOAPscope's documentation is first-rate, particularly helpful to anyone new to SOAP and Web services. And noncoding QA engineers will welcome the tool's code-free environment.
Parasoft SOAtest 5.1
As with other tools, Parasoft’s SOAtest will jump-start a project by automatically creating test cases. Point SOAtest at a WSDL; the tool will crawl it and build a set of positive and negative tests for each Web method on the WSDL, so SOAtest's initial boost propels your testing further from the launch pad than the other tools.
SOAtest's auto-generated tests, though minimal, are instantly executable and are actual tests, not completely empty test skeletons. Each test node in the tool's explorer is accompanied by a “traffic object” subnode that retains the most recent request/response pair generated by that test. So you can execute a test, then click on the traffic object to examine the exact exchange in Literal (unadulterated XML), Tree (a navigable tree explorer), or Element (pseudo code) representation.
All the tests associated with a single Web service are represented as a test suite node, and within that will be yet more test suite nodes, each representing tests on a particular Web method. Only when you get down to single request/response nodes do suites become simply tests.
Although Parasoft follows the standard pattern of representing projects in an explorer tree, nonleaf nodes in the tree are not simply containers. Adjusting attributes on a parent node alters the behavior of all child nodes accordingly. For example, select the test suite node corresponding to a complete Web service, and a tabbed configuration window appears. From that window, you can alter the execution order of all child test suites, setting them to either run sequentially or in parallel, or specifying that, say, Test B execute only if Test A succeeds. Hosts of other attributes – what version of SOAP to use (1.1 or 1.2), message encoding, default timeout, and more – are similarly alterable. This makes for quick and easy manipulation of project-wide behavior.
When you identify a WSDL on which to create a collection of test suites, you must tell SOAtest of the messaging pattern – synchronous or asynchronous – employed, and SOAtest will create tests suited to the pattern. Also, you can set SOAtest to enforce policy configurations on a project's WSDLs and SOAP messages. A policy configuration is a set of predefined rules, roughly analogous to coding standards. SOAtest will alert you when a WSDL or SOAP message violates one of the rules, so policy configuration provide a means of enforcing best practices among all your organizations Web-service assets.
SOAtest lets you turn a set of functional tests (which is how tests start their lives) into a load test with near flick-of-a-switch ease. Simply select the Load Tests item from the menu and choose how the load is applied (that is, whether the number of virtual users increases linearly, is steady, or tracks a more complicated curve). Once the load testing has started, you can view real-time graphs that plot the current load, as well as request-response turnaround times. You can halt the testing at any point, change the subset of suites being used, and restart.
SOAtest is overrun with useful support tools. Examples are an XML beautifier, various XML encryption tools, and a module that manages callbacks for asynchronous HTTP testing. SOAtest also supports client-side testing. Point it to a WSDL, and it will examine the WSDL structure and create "stubs" that will mimic the Web service's behavior. SOAtest handles testing coming and going, and it has an overall feel of maturity missing from several of the other tools.
Peering through the suds
This is a complicated field from which to draw a favorite. If your testing involves more than just Web services, and your development is primarily Java, then tools such as LISA or SOAtest are worth considering, in that they'll let you drive two nails with one hammer. If, however, you are only interested in SOAP-based Web service testing, and your QA staff is relatively new to the technology, SOAPscope is the obvious choice.
On the other hand, your choice may be based more on your philosophy of testing. Specifically, which is better: coding tests or building them visually (in other words, from menu and button selections within a GUI)? Tools such as SOAPSonar, LISA, and SOAPscope have chosen the latter to escape the former. But shielding the user from code necessarily places that user in a constrained environment. Those tools tacitly assume that their GUIs are flexible enough to meet all or most testing demands. I am not so sure. I favor the tools – QEngine and SOAtest – whose scripting capabilities are easily accessed. And of those two, SOAtest does the better overall job.
Finally, you should take advantage of the fact that almost all of these tools – SOAPscope being the notable exception – offer free versions that enable you to sample their capabilities with no time limitations.
Ease of use (20.0%)
Overall Score (100%)
|AdventNet QEngine 6.8||8.0||7.0||7.0||8.0||7.0||7.0|
|Crosscheck Networks SOAPSonar 3.0.5||8.0||8.0||8.0||8.0||8.0||9.0|
|iTKO LISA 3.6e||9.0||7.0||7.0||8.0||7.0||8.0|
|Mindreef SOAPscope Server 6.0||8.0||7.0||8.0||8.0||9.0||7.0|
|Parasoft SOAtest 5.1||9.0||8.0||8.0||8.0||8.0||9.0|
Having trouble installing and setting up Win10? You aren’t alone. Here are many of the most common...
Win7 Update scans got you fuming? Here’s how to make the most of Microsoft’s 'magic' speed-up patch
Picking an Android phone can be difficult, but we're here to help. These are the top Android phones you...
Sponsored by Intel
Sponsored by Puppet
In fact, wait as long as Microsoft will let you, since this is mostly a minor upgrade
The demise of Visual Studio LightSwitch shouldn’t prevent power users from building line-of-business...
Google's container orchestration platform can now scale to massive clusters
IT expertise is no match for execs' stubbornness and agendas, even under dire circumstances