Kapow's screen-scraper integrates, assembles your portal information with data from other online sites
Kapow’s Web Integration Platform version 6.0 is one of the best examples of these central-server solutions. The suite is a big, automatic screen scraper that assembles the information into a portal, aggregating information from many different sites in a way that makes it easy for users to absorb.
The Web Integration Platform could be a hit with big IT shops that build information portals for employees and clients. I’ve seen a number of cases where portal projects bog down because one division doesn’t want to open up its databases and systems. One simple, easy-to-use connection system would be wonderful, but that means getting all parts of a company to support this central vision.
Kapow’s solution avoids the politics by offering a system of code-capturing robots that operate at the lowest-common denominator: HTML-marked up text. These robots are experts at extracting information from internal and external Web pages, and usually do not require much cooperation from the source.
The central server schedules the robots and aggregates their results. If someone goes to a portal page, the server will fire up the right robots to clip the correct information before bundling it together. This information can be cached temporarily or stored in a database for a long-term view.
Most users won’t need to worry about this language because Kapow includes a sophisticated workstation for taking Web sites apart. After you provide the URL, the Kapow suite loads the Web site and displays it in a section of the RoboMaker UI. You can then start snipping and cutting from the site by pointing and clicking on the parts you want. The HTML and the language for extracting the HTML appears in a window alongside the Web site.
The robot instructions are at the top of the UI; they’re built with a fairly traditional visual language, and you can add loops and branches. The result looks like a standard flowchart, although there are many special features tuned to the nature of HTML -- one loop command, for instance, will extract all but the top row of a table.
The new features still won’t work at the most extreme Web sites, however. I’ve written AJAX pages that will calculate and rewrite tables after the user clicks a button; this type of page can’t be scraped easily.
I tested Kapow’s platform by building several robots and sending them off to collect information. The visual robot-building tool is surprisingly simple, yet powerful enough to handle many of the standard extraction jobs that it will be given.
Although it is nominally written in Java (Kapow has a partnership with BEA and also distributes a .Net version), most users will be able to build robots without knowing any Java. I suspect that some experienced programmers will be frustrated at times when they want to do something like produce odd Unicode characters, but average users will be able to develop much of the portal without help.
Kapow’s Web Integration Platform will find its greatest traction in two places: large shops with many legacy systems and centers of corporate intelligence. The developers in charge of linking the legacy systems will like the fact that they can scrape a screen without reprogramming that system. It may not be elegant to leave all of the old code in the path, but it could be a speedy integration solution.
Groups responsible for producing corporate dashboards and assembling intelligence will also appreciate Kapow’s wide-ranging site-scraping abilities. I could see someone in the hotel business using a system like this to watch the price of competitor’s hotel rooms.
Web Integration Platform version 6.0 is a well-polished mechanism for extracting data. If you need to gather the results from many different Web sites, this may be the fastest way to get your job done.
Ease of development (30.0%)
Overall Score (100%)
|Kapow RoboSuite 6.0||9.0||8.0||9.0||8.0||8.0|
Microsoft buried a Get Windows 10 ad generator inside this month's Internet Explorer security patch for...
Hot or not? From the Web to the motherboard to the training ground, get the scoop on what's in and...
Microsoft’s 'Fall Update' promised to put the finishing touches on Windows 10 -- it doesn’t
From full-blown IDEs to essential resource utilities, these Android apps bring powerful programming...
Here are 10 in-demand tech skills that IT pros should master, according to research from Dice.com
The open source Futhark makes it easier to program for GPUs that speed up machine learning and other...
These full-fledged free-tier services and indispensable utilities will have your API up and running...