Playing the Internet scales

The blog world was enriched last week when Tim Bray's Weblog appeared on the scene. The co-founder and CTO of Antarctica Systems, Bray co-edited the XML 1.0 specification and remains deeply engaged in Web development. "It's not like it used to be," Bray observed in the hours following the launch of his blog, while "hovering over the access log like an expectant mother." The most dramatic change from a few years ago? The effect of RSS (Rich Site Summary/Really Simple Syndication) newsreaders:

"I'm seeing maybe four or five hits a minute on the RSS feed," Bray says. "When you consider the number of sites out there with RSS feeds and the number of people who subscribe to a bunch of them, we're talking some pretty serious traffic here. Architecturally, this seems pretty dumb, and you have to worry whether or not it's going to scale."  

Point taken. Although I've been publishing and subscribing to RSS feeds for four years now, I've noticed a change just recently. One Web site I manage used to attract more visitors by way of its e-mail newsletter than by way of its RSS feed. Three months ago, things tipped dramatically in favor of RSS.

My guess is that's just the start. RSS readers -- for example, Radio UserLand, NetNewsWire, AmphetaDesk, and NewzCrawler -- used to be pretty far off the beaten track. Last week, though, I started using Greg Reinacker's NewsGator -- a .Net-based plug-in for Outlook 2000 or 2002. If you're an Outlook user, this is probably the best and most natural way to tap into the vital stream of information that Weblogs deliver. If you're wondering where to start, just point NewsGator at somebody's OPML (Outline Processor Markup Language) subscriptions file -- mine, for example -- to read the same feeds as I.

For RSS, integration with Outlook is a huge opportunity to break into the mainstream. But as Bray points out, "Yes, Houston, we [potentially] have a problem." I, for example, subscribe to about 100 feeds. During the day, I migrate from one Windows box running NewsGator to another running Radio UserLand to a TiBook running NetNewsWire. Each of these RSS readers (also known as aggregators) polls each of its subscribed feeds hourly. A few months back, this sucked up a lot more bandwidth than it does today. Aggregators pulled whole feeds, then looked for changes until Simon Fell, the developer of PocketSoap, made this modest proposal: