How to stress a UTM

We challenged the Astaro, SonicWall, WatchGuard, and ZyXel appliances with a maximum dose of legitimate traffic, 200 VPNs, and hundreds of Internet attacks, all at the same time

For our scenario-based test of the Astaro, SonicWall, WatchGuard, and ZyXel UTMs, we simulated a representative corporation with 200 branch offices all connecting back to headquarters for various services. Unlike previous firewall tests, we tested all three major firewall functions -- Internet services, secure remote access, and malware blocking -- simultaneously, in order to better represent the workloads these devices face when deployed in the real world.

First, using the Ixia IxLoad system, we ran a mix of HTTP, FTP, POP, and SMTP traffic through three firewall interfaces: LAN to WAN to simulate Web browsing by employees; LAN To DMZ to simulate employees updating and querying public servers; and WAN to DMZ to simulate to simulate external users interacting with the company's public servers and employees on the road accessing e-mail. This also created a baseline on which to compare performance.

Second, again using the IxLoad, we ran a mix of HTTP, FTP, POP, and SMTP traffic through each of 200 VPNs, simulating 10 users at each branch office accessing intranet servers on the LAN. This allowed us to see how overall throughput was affected by VPN activity, and completed our legitimate traffic baseline for our fictitious company.

[ When is a UTM not a UTM? Read the overall results of the InfoWorld Test Center's great UTM challenge. Read the reviews: Astaro Security Gateway 425 | SonicWall NSA E7500 | WatchGuard Firebox Peak X5500e | ZyXel ZyWall USG1000. Compare the UTMs feature by feature. ]

Third, we added malware to the mix, using Mu Dynamics' Mu-4000 and Published Vulnerability Attacks module to test the UTMs' attack blocking capabilities. Attacks were launched against the WAN interface to simulate bot traffic and other external threats, and then from the LAN interface to simulate an outbreak from an infected laptop being plugged in behind the firewall.

Feeds and misdeeds
By laying a baseline of traffic across multiple firewall interfaces, adding traffic from 200 VPNs, and then hitting the UTM with roughly 600 attacks, we were able to determine how a stream of attacks affected overall throughput. We weren't surprised that the performance hit was typically substantial. Oddly, the Astaro system suffered a mere 2% drop, albeit while also failing to block more than 400 of our roughly 600 attacks.

Naturally, one of our main goals was to find out just how well these UTMs would handle the nearly constant attacks typically found on public Internet connections. To this end, we enlisted the help of Mu Dynamics and its Mu-4000 Analyzer. This unique test tool has the ability to generate millions of attacks based upon published vulnerabilities as defined by folks like U.S. CERT (Computer Emergency Readiness Team) to exercise the deep packet inspection capabilities of each UTM. (Although the MU-4000 can also "fuzz" these attacks to assess how well the UTMs could cope with variants or "zero dayattacks," we did not expose the UTMs to these attack mutations.) Mu Dynamics is so confident that it can break through a security device that the company even provides script-controllable power outlets on the Analyzer so that it can reboot the device after it's been locked up.