My friends Paul and Robin Laudanski at CastleCops have been under a huge DDoS attack for over a week. The attack has initiated sustained malicious loads over 1GB/s.
[ RogerGrimes's column is now a blog! Get the latest IT security news from the Security Adviser blog. ]
While that number is incredible in itself, it’s just on the high side of average. Some DDoS attacks come in at 10 GB/s and last for months. Many Web sites, including those dedicated to fighting spam, phishing, and malware in general, have been completely pushed off the Internet forever by DDoS attacks. The bad guys have often won.
I talked to VeriSign about the latest DNS root server attack in last week's column and mentioned that I was surprised to learn that the DNS infrastructure still isn’t more inherently resistant to DDoS attacks, even after all these years and attacks. With this information and CastleCops' situation in mind, I figured that now is a good time to cover botnets and DDoS attacks.
Bots on the Brain
Bots are the more intelligent C&C (command-and-control) versions of yesterday’s remote-access Trojan. There is a definitive architecture to bots, bot nets, and their damage. Bots are coded by either an individual or a team. Several recent disassembled bots revealed a development architecture that rivals legitimate application projects with phases, testing, bug fixing, and development forks.
Most bots use one or more attacks to break into remote machines. Patched and unpatched Microsoft vulnerabilities are popular on the client side; flawed PHP programs are a common target to compromise Web servers. The bot is often coded to a particular specification (what it will do, how it breaks in, etc.) from a malicious requestor or created and sold speculatively like housing investments in the physical world.
Once coded, the bot is used to compromise multiple machines and make a bot network (or botnet). The number of exploited PCs in a botnet may range from a few thousand to over 100,000 machines, all under one person's (or one team's) command and control. Several experts have speculated that up to one-fourth of all computers on the Internet could be under the control of a bot at one point or time.
Bots are auto-updating and always moving. When they break into a computer, they immediately maliciously modify the host and then contact another previously exploited site (called a "mothership") to download more code. The newly downloaded program then contacts another server, and the whole cycle continues over and over again until the final program receives its instructions (DDoS, spam, password stealing, etc.).
All of the bots in a botnet are run from a handful of mothership servers, known as C&C servers. Hackers connect to the C&C servers to issue instructions to the compromised computers. Take down the C&C servers, and you’ve effectively killed the botnet or the hacker's control of the botnet -- at least temporarily.
The botnet is then used by the attacker (or sold or rented) to accomplish some sort of malicious activity. Denial of service attacks are very common, but bots can also steal user passwords, send spam or phishing e-mails, sniff traffic, steal identities, or affect the outcome of online polls (some people believe that's how Ruben won over Clay).
A 30,000-bot network can easily generate 1GB/s of malicious traffic, as CastleCops unfortunately knows. How many Web sites can withstand that type of attack? Not many, but there are defenses beyond simply ramping up bandwidth.
The most common defense is to filter out bad traffic at the Web site router or upstream neighbor. By the time it is filtered out at the Web site, the DDoS attack is usually already affecting all other servers and clients sharing the same ISP bandwidth pipe, so upstream filtering is better for all. Unfortunately, that takes additional coordination, and not all ISPs have the resources to deal with DDoS attacks.
Furthermore, it can be difficult to differentiate between legitimate and malicious traffic, and bots often used spoofed origination IP addresses to make it even more difficult. DDoS attacks using spoofed IP addresses can be stopped with ISP egress filtering as detailed in RFC 2827, written in May 2000 by my friend Paul Ferguson.
Unfortunately, it's been seven years, and the majority of ISPs have yet to implement RFC 2827's basic instructions. I don't have hope that they ever will without government regulation.
A botnet can also use legitimate IP addresses or send requests that mimic legitimate requests. Some DDoS attacks are known as HTTP recursion attacks because they pretend to be a legitimate customer but request every possible Web page, thereby overwhelming the server.
These attacks are specifically customized for the Web site target, requesting pages that actually exist on the server. They also send requests at a slow pace, from one per second to one per minute, of course multiplied by tens of thousands of malicious requesting clients. The idea is to force very legitimate-looking requests, which are difficult to mass filter out without affecting legitimate customer requests.
Spend your time protecting against HTTP attacks, and the attacker will just take out your DNS services or the upstream router. Catch the criminal, and odds are they are likely to be treated more harshly for a DUI than for disrupting your business.
I’m surprised, and saddened, that our anti-DDoS defenses haven't improved significantly over the last decade. Sure, there are many anti-DDOS vendors and solutions, but they, like many other defenses, have a hard time implementing a solid solution across the board. The real weakness is our unauthenticated Internet. But as Paul and Robin of CastleCops said, "We're in this for the long haul. We aren't going to be intimidated. We aren't going to go away."