APIs, protocols, and rogue plumbers

My local bank is switching from one online bill-payment system to another. I’m looking forward to the new system, which will be an improvement on the current one, but I wasn’t expecting this:

Free IT resource TechNet: More ways to know it, share it, and keep it running. Sponsored by Microsoft Free IT resource Attend the SOA Executive Forum: Breaking SOA Bottlenecks SOAExecForum.com/may2007 Sponsored by InfoWorld

“IMPORTANT NOTICE FOR CURRENT BILL PAYMENT CUSTOMERS: When we switch to this new system you will be required to re-enter the payee data you have set up under our current system.”

Nobody likes to have to send this message, and nobody likes to receive it. Pre-Web, the sender might have been justified in saying, “My hands are tied. Our partner doesn’t offer that feature. There’s nothing I can do about it.” And pre-Web, the recipient might have had to accept that explanation.

But since 1994 or so, things have been different. Integrating two Web-based systems, if only by brute force, is not only necessary but possible. If I can log in to both systems and drive them interactively, I can write a script to join them programmatically. Every Web application can be tortured into behaving like a Web service, even when its creator never intended it to.

The details aren’t pretty. Were I to attack this particular problem, I’d start by firing up Proxomitron-SSL, a local Web proxy that terminates SSL connections and displays HTTP headers and requests. Then I’d log into applications 1 and 2, fetch a few records from app 1, transfer them to app 2, and observe the protocol. Finally, I’d write a script to mimic the session setup (set a cookie, send an HTTP authorization header), and enact the transfer between app 1 and app 2 (some pattern of HTTP GET or POST requests).

I used to get a subversive kick out of doing this kind of thing. It made me feel like Robert De Niro’s Harry Tuttle, the rogue plumber in Terry Gilliam’s dystopian fantasy, Brazil. But that thrill is long gone. Web services exist so we won’t have to unclog the pipes this way any more. HTML screen-scraping just feels like the dirty job it always was.

The technical superiority of the Web services approach is well understood, but I don’t think we fully grasp the political impact.

Consider the dramatis personae in my little morality play: B, my local bank; V, the value-added reseller who supplies B’s software; and S, the service provider whose bill presentment and bill-payment system V is reselling to B. When I got the bad-news message from B, I called to inquire. B said S had provided no tool for importing payee data. Then I called S. Turns out there is such a tool. S pointed to V, citing communication difficulties and V’s organizational flux as reasons why V hadn’t known about or been willing to implement the import tool for B’s customers.

I haven’t tracked down somebody at V, but you get the picture. The political hot potato here is a software tool that implements a private API. Use of the API is contingent on access to the tool because, well, that’s how we’ve always done things. But not for much longer, I hope.

The inexorable logic of Web services sets aside APIs in favor of protocols. XML messages flowing through the pipes enact those protocols. Anyone with authorized access to that plumbing will be able to monitor and inject messages quite easily, and everyone will know that’s true.

If B or S or V can’t unclog the pipes, we’ll elect Harry Tuttle, and he’ll do it without even getting his hands dirty.