Web-scouring robots anchor Kapow
Kapow's screen-scraper integrates, assembles your portal information with data from other online sitesFollow @peterwayner
Kapow’s Web Integration Platform version 6.0 is one of the best examples of these central-server solutions. The suite is a big, automatic screen scraper that assembles the information into a portal, aggregating information from many different sites in a way that makes it easy for users to absorb.
The Web Integration Platform could be a hit with big IT shops that build information portals for employees and clients. I’ve seen a number of cases where portal projects bog down because one division doesn’t want to open up its databases and systems. One simple, easy-to-use connection system would be wonderful, but that means getting all parts of a company to support this central vision.
Kapow’s solution avoids the politics by offering a system of code-capturing robots that operate at the lowest-common denominator: HTML-marked up text. These robots are experts at extracting information from internal and external Web pages, and usually do not require much cooperation from the source.
The central server schedules the robots and aggregates their results. If someone goes to a portal page, the server will fire up the right robots to clip the correct information before bundling it together. This information can be cached temporarily or stored in a database for a long-term view.
Most users won’t need to worry about this language because Kapow includes a sophisticated workstation for taking Web sites apart. After you provide the URL, the Kapow suite loads the Web site and displays it in a section of the RoboMaker UI. You can then start snipping and cutting from the site by pointing and clicking on the parts you want. The HTML and the language for extracting the HTML appears in a window alongside the Web site.
The robot instructions are at the top of the UI; they’re built with a fairly traditional visual language, and you can add loops and branches. The result looks like a standard flowchart, although there are many special features tuned to the nature of HTML -- one loop command, for instance, will extract all but the top row of a table.