Maybe you didn't notice, but Gmail went down for about 100 minutes Tuesday, depriving millions of the Gfaithful of their beloved e-mail service.
[ Google has its hands full with a variety of issues these days, including Internet anonymity: "Skanks for nothing: Google must identify 'anonymous' blogger" | Stay up to date on Robert X. Cringely's musings and observations with InfoWorld's Notes from the Underground newsletter. ]
The gnashing of teeth and the renting of garments could be heard clear across Twitterville. Like these representative twits -- err, tweets:
Words and phrases like “apocalypse” and “digital terrorism” really did come to mind when I realized the #gfail.
gmail is down? im having an anxiety attack. i cant function without it.
No Gmail, Facebook is twitchy and Twitter is slow. My life may end just now…
Actually, as several Tweeters pointed out, Google's POP and IMAP servers were working just fine. It was the Web interface that did a face plant. But to the Twitterati who couldn't quite grasp that concept, it seemed the world had ended, at least temporarily.
Turns out the problem was fairly prosaic, though at the scale Google operates, even a hangnail can look like a fatal condition. The Google team took a few Gmail Web servers offline for routine maintenance, the servers that remained online got too slow and shut themselves down, and the resulting traffic overwhelmed the machines still left standing.
In a post to The Official Gmail Blog, "Site Reliability Czar" Ben Treynor writes:
The Gmail engineering team was alerted to the failures within seconds (we take monitoring very seriously). After establishing that the core problem was insufficient available capacity, the team brought a LOT of additional request routers online (flexible capacity is one of the advantages of Google's architecture), distributed the traffic across the request routers, and the Gmail web interface came back online.