Kapow's screen-scraper integrates, assembles your portal information with data from other online sites
Kapow’s Web Integration Platform version 6.0 is one of the best examples of these central-server solutions. The suite is a big, automatic screen scraper that assembles the information into a portal, aggregating information from many different sites in a way that makes it easy for users to absorb.
The Web Integration Platform could be a hit with big IT shops that build information portals for employees and clients. I’ve seen a number of cases where portal projects bog down because one division doesn’t want to open up its databases and systems. One simple, easy-to-use connection system would be wonderful, but that means getting all parts of a company to support this central vision.
Kapow’s solution avoids the politics by offering a system of code-capturing robots that operate at the lowest-common denominator: HTML-marked up text. These robots are experts at extracting information from internal and external Web pages, and usually do not require much cooperation from the source.
The central server schedules the robots and aggregates their results. If someone goes to a portal page, the server will fire up the right robots to clip the correct information before bundling it together. This information can be cached temporarily or stored in a database for a long-term view.
Most users won’t need to worry about this language because Kapow includes a sophisticated workstation for taking Web sites apart. After you provide the URL, the Kapow suite loads the Web site and displays it in a section of the RoboMaker UI. You can then start snipping and cutting from the site by pointing and clicking on the parts you want. The HTML and the language for extracting the HTML appears in a window alongside the Web site.
The robot instructions are at the top of the UI; they’re built with a fairly traditional visual language, and you can add loops and branches. The result looks like a standard flowchart, although there are many special features tuned to the nature of HTML -- one loop command, for instance, will extract all but the top row of a table.
The new features still won’t work at the most extreme Web sites, however. I’ve written AJAX pages that will calculate and rewrite tables after the user clicks a button; this type of page can’t be scraped easily.
I tested Kapow’s platform by building several robots and sending them off to collect information. The visual robot-building tool is surprisingly simple, yet powerful enough to handle many of the standard extraction jobs that it will be given.
Although it is nominally written in Java (Kapow has a partnership with BEA and also distributes a .Net version), most users will be able to build robots without knowing any Java. I suspect that some experienced programmers will be frustrated at times when they want to do something like produce odd Unicode characters, but average users will be able to develop much of the portal without help.
Kapow’s Web Integration Platform will find its greatest traction in two places: large shops with many legacy systems and centers of corporate intelligence. The developers in charge of linking the legacy systems will like the fact that they can scrape a screen without reprogramming that system. It may not be elegant to leave all of the old code in the path, but it could be a speedy integration solution.
Groups responsible for producing corporate dashboards and assembling intelligence will also appreciate Kapow’s wide-ranging site-scraping abilities. I could see someone in the hotel business using a system like this to watch the price of competitor’s hotel rooms.
Web Integration Platform version 6.0 is a well-polished mechanism for extracting data. If you need to gather the results from many different Web sites, this may be the fastest way to get your job done.
Ease of development (30.0%)
Overall Score (100%)
|Kapow RoboSuite 6.0||9.0||8.0||9.0||8.0||8.0|
Looking for the missing free copy icon? It's been replaced. There's a new direct link that works like a...
Supreme Court's decision is bad news for developers targeting the U.S. market, who will now have to...
The transition from command line to line-of-command requires a new mind-set -- and a thick skin
If an 'independent' code review says a product is totally secure, you aren't hearing the full story
A spate of projects from IBM's DeveloperWorks Open portal covers everything from improving Spark...
Built for development teams, Git can’t meet enterprise scalability and security requirements on its own...
AWS's developer-focused approach is one lesson enterprises should glean from the cloud leader