Using RSS for data integration

Last week at the O’Reilly Emerging Technology Conference, Jeff Bezos announced OpenSearch, an API that enables third parties to inject their own live search results into Amazon’s A9.com. I didn’t attend ETech this year, but that cloud had a silver lining: I was able to dive right in and do an OpenSearch implementation.

As fate would have it, I’d just finished a quick hack to reorganize the output of InfoWorld’s Ultraseek search engine. I named my little project “InfoWorld power search” because it delivers a dense page of titles, categorized by story type. When I heard about OpenSearch, I wondered how hard it would be to integrate my new view of InfoWorld search as a “column” in A9. As I soon learned, it’s almost trivial.

Step one was to add an alternate interface that would return results as a lightly modified RSS 2.0 feed. It differs from a vanilla feed only by a couple of A9-specific tags. These are neatly encapsulated in an OpenSearch namespace, and they demonstrate a fact about RSS 2.0 that has been too little known and exploited: It’s extensible.

Step two was to register the service, which I did by describing it by pointing A9 at a document written in an A9-specified XML format.

That’s all there was to it. On my blog, you can find a screenshot of a two-column A9 results page. A9’s own Web search results appear in the left column, InfoWorld’s on the right. If you visit A9.com, you can add InfoWorld to your set of columns. Excellent!

OpenSearch is interesting in lots of ways, but here I want to focus on its use of RSS. A9 doesn’t subscribe to my search-results feed in the way Bloglines or FeedDemon or NetNewsWire would. It doesn’t poll for changes. Instead it sends a request to my site when an A9 user with an active InfoWorld column performs a search. The response packet I send back just happens to be formatted as RSS 2.0, but from A9’s perspective, it could be any XML format.

Why RSS 2.0, then? Because it creates network effects that go way beyond the point-to-point relationships between A9 and its search partners. The work I did to export RSS 2.0 search results served double duty. It accomplished the integration with A9, but it also dramatically expanded InfoWorld’s RSS surface area. Now, for the first time, you can subscribe to any InfoWorld search in a feed reader. Want to be notified when the next review of a VoIP product shows up at InfoWorld.com? Run the query, and subscribe to its results.

Most people nowadays use RSS for person-to-person communication. You know the pattern: When a publisher posts a blog item, subscribers are alerted. A growing number of folks are also using RSS for process-to-person communication. Subscribing to searches is the best example of this pattern.

A9’s use of RSS for process-to-process communication represents a third pattern. We’ll be seeing a lot more of it. Not because RSS enables process integration in special ways -- it doesn’t -- but rather because RSS helps us blur the boundaries between human network and process networks.

To be honest, I wasn’t even planning to enable RSS subscription to InfoWorld search. It just came for free. When that happens, it’s a sign that things are deeply right.