How we tested

To test the firewall functionality of our four entries -- Ingate Firewall 1400, Toshiba Magnia SG20, Nokia IP380, and Enterasys XSR-3250 Security Router -- we employed technology and test engineers from Ixia Communications. We used an Ixia 1600 chassis loaded with Ixia’s TXS Ethernet Load Modules. Ixia configured its 1600 chassis with 64 Ethernet ports, 24 of these being 10/100/1000 capable to ensure full load on any gigabit-capable devices.

As a midstop for all generated traffic, we employed a Cisco Catalyst 4507R routing switch chassis. The Catalyst 4507R chassis came configured with 78 10/100/1000 ports, 48 10/100 ports, and dual Supervisor II controllers for management. Configuring the 4507R for the test was simple, requiring merely a 50-percent segregation of the 10/100/1000 ports to simulate private and public environments.

The Ixia 1600 was configured using Ixia’s WebLoad software to generate stateful traffic using a variety of protocols, including HTTP, FTP, SMTP, and TCP. Traffic based on a mix of these protocols was used to first generate a baseline throughput number for each device. We then ran the same traffic loads, but in four iterations that also included a different application-level attack. Ixia’s Webload software is capable of eight separate attack modes; we used only Ping of Death, Smurf, Syn, and Teardrop to get an idea of how well our firewalls would respond to a variety of DDoS (Distributed Denial of Service) attacks.

While all the devices were able to defend against each attack to some degree, their measure of success in our results was in how much each attack affected the other legitimate traffic throughput flowing the firewall at the same time. The firewalls best able to defend against the attacks without dropping legitimate traffic received the highest scores. In addition to these security and performance capabilities, products were also evaluated on how easy they were to set up and configure, and on the power and flexibility of their management tools.