Three open source Web service testing tools get high marks
Capable soapUI, TestMaker, and WebInject toolsets shine once you conquer their learning curves
WebInject's single greatest characteristic is its simplicity. Once you've gotten the hang of WebInject's XML commands, you can build, modify, and extend test cases quickly. The entire documentation consists of a one long Web page that can be read in a single sitting. However, that one page of documentation sometimes leaves you wondering how to proceed. Furthermore, you'll need a moderately good understanding of the SOAP protocol, as well as a tool to extract a Web service response's POST body, which you'll need to build test cases.
At your service
These three tools place themselves along the spectrum from quick and easy to complex and powerful. Do you need to assemble some code quickly for hurling tests at your Web service? WebInject is the logical choice; you'll be buffeting your Web service code in an afternoon. Do you need a high-end tool that lets you create powerful tests that can be extended to draw upon other system resources – the filesystem, databases, e-mail, and such? Then roll up your sleeves and plow into TestMaker. Grab a Jython manual first, though, and prepare yourself for some heavy lifting.
I prefer the middle balance struck by soapUI. The skeletal tests created by soapUI's wizard were easier to flesh out than those built by TestMaker. And, if I needed to do something elaborate and off the wall, I could always call upon soapUI's Groovy capabilities – funny name aside, they do their job well.
In terms of how these products compare to commercial Web service testing tools, I'd say it's a mixed bag. They are, of course, inexpensive (free), and work well for easy to moderately-difficult jobs; on the other hand, they're somewhat less user-friendly than commercial tools and if you need to do something complex, you have to build it yourself. TestMaker comes closest to looking like a commercial product, but having to learn Jython means that it takes longer to set up some tests than it would with, say, Mindreef's SoapScope. soapUI is a tad less professional looking, but makes up for it by allowing you to construct useable tests without having to program. WebInject, however, is definitely a developer's tool. You need to know SOAP to use it well, and it isn't going to be as capable as soapUI or TestMaker because its test cases are very much driven by templates.