If you haven't heard, San Francisco is being held hostage. At least, the city's new network is being held hostage. It seems that Terry Childs, a disgruntled network admin took it upon himself to lock out all the other admins from "the city's new FiberWAN network," and is currently hanging out in jail, holding the keys to San Francisco's kingdom.
There have been many articles written about this event, and they all share an obscene lack of detail. The "network" as used in these pieces could be interpreted as just about anything from one or more servers, the network switches and routers, some storage servers, or any combination thereof. This quote from an IDG news item unfortunately doesn't offer much clarity: "The new FiberWAN handles city payroll files, jail bookings, law enforcement documents, and official e-mail for San Francisco." There are an awful lot of moving parts in that description. We obviously don't know what part(s) they're talking about. Thus, it's terribly difficult to draw a clear picture of what's actually transpired.
A clue as to the actual nature of the lockout has come from statements that it might cost the city "millions of dollars" to unlock the system.
Unless Childs managed to install BIOS or kernel-level disk encryption on all the servers or stuff M80s in the server drive bays, there's no way that the cost of "unlocking" the network would run into the millions of dollars. Since officials are talking publicly about bringing in Cisco experts to undo the damage, it may be safe to assume that what Childs did was change the login to some or all of the routers and switches running the network. Now, being a veteran Cisco network architect, I can tell you that there's no way that a network of this size should have been built with only local passwords on the switches and routers. Using TACACS+ or RADIUS to control admin logins isn't just a good idea, it's the only way to handle authentication on a network of this size. Perhaps one of these methods was in use, but Childs modified the configurations to use only local logins. We can't know for sure, but we can speculate.