Intrusion protection systems are newcomers on the security scene, and somewhat of a challenge to test. I was primarily concerned with initial setup, ease of configuration and administration, throughput and latency under load, and of course the performance of the unit when under attack.
To test the network performance of the IDP-100, I first measured throughput and latency to several Windows, Linux and Solaris servers without the unit, then placed the unit in-line -- bridging, routing, and configured to proxy ARP requests -- and measured again. Following this baseline, I used Nessus and Nmap to throw thousands of attacks at the servers behind the IDP-100 while running constant 100Mb TCP and UDP streams through the device. I also took the IDP-100 out of the lab and placed it in-line on a production network, monitoring its performance and event logging over a two-day period.
Perhaps the most important aspects of an intrusion protection system are administration and manageability. These are also some of the most subjective elements of testing. I perused the configuration options from stem to stern, looking for weaknesses in policy generation, signature validity, signature creation and logging, but also to get a good feel of the management console layout and function. The management console was tested on both Windows and Linux, with both interfaces used alternatively throughout the tests.