Malware-fighting firewalls miss the mark

It's important to note that the attacks used in the test (Mu Dynamics' Published Vulnerability Attacks, drawn from the US-CERT database) were all exploits of known vulnerabilities (no "zero day" surprises) in a wide range of popular operating systems, applications, and protocols (Microsoft Windows, Internet Explorer, Cisco IOS, Apache, SQL, ICMP, SSH, and so on). We threw the full range of exploits at our UTMs, about 600 attacks in all, but the UTMs should have been designed to thwart such threats. And still hundreds were allowed to pass through.

Why did the UTMs miss so many exploits? We don't know, but we suspect that (apart from the SonicWall) they lack the horsepower to perform the necessary deep packet inspection while under load. At the same time the UTMs handled our attacks, we were pushing the limits of their throughput with legitimate traffic. The upshot is, although the vendors have packed these devices with additional gateway security functions, clearly many UTMs are still strictly firewalls at heart.

[ Read more about InfoWorld's UTM acid test and the test tools: "How to stress a UTM" | "Ixia IxLoad's multithreaded testing" | "Mu's Internet attacks in a can." ]

UTM functions require gobs of processing in order to peek into packets to look for malware, so it should be no surprise that the devices -- all except the Astaro -- took a significant hit in throughput when under attack. Compared to their maximum throughput without attacks, the WatchGuard took a 45 percent hit, the ZyXel 36 percent, and the SonicWall 23 percent. The Astaro, which blocked the fewest attacks in our test, barely lost a step when under attack -- a surprisingly tiny 2 percent dip from maximum throughput. (See chart below.) Generally, however, you should be prepared for huge hits to throughput when you turn on all of the security functions of a UTM. You are not getting a wire speed device. On the plus side, unless your WAN link is a gigabit Ethernet feed, you may never notice the slowdown.

Recognizing a winner
Despite the poor attack blocking test results, each of these "UTMs" will serve as perfectly effective firewalls and VPN appliances, if they're installed and administered properly. There were no crushing disappointments here and no products that we have to warn companies not to consider. Instead, there were four variations on firewall competence, with enough brilliance thrown in to make life good for the security folks at quite a few mid-sized enterprises.