Innovative IPSes resist our attacks

See correction at end of article

There's more than one way to snare malicious network traffic, and the more methods your IPS uses to detect bad streams, the better your chances of keeping your network productive. McAfee IntruShield 2.1 and NFR Sentivist 5.0 each combine numerous forms of detection to thwart known and unknown threats, hidden exploits, and dangers such as worm outbreaks and DoS attacks.

McAfee and NFR have also incorporated unique features that should turn some heads. In McAfee's case, the ability to decrypt and inspect SSL-encrypted traffic will be useful to customers wanting to protect e-commerce sites, and virtual interfaces allow fine-tuning of security policies for specific assets. For its part, NFR offers an interface that is wonderfully effective at zeroing in on important events, as well as an intelligent threat-indexing technique that helps eliminate false positives and speeds the deployment of in-line blocking.

McAfee IntruShield 2.1

Unlike traditional intrusion detection and prevention offerings whose signatures are based on exploits, McAfee's IntruShield uses signatures based on vulnerabilities, an approach that should prevent new strains of known attacks from sneaking past its sensors. As a result of McAfee's extensive threat library, network-based attack events are remarkably well documented. Event descriptions include CVE (Common Vulnerabilities and Exposures) references, vendor patch references (Microsoft bulletins), and even steps to mitigate specific attacks.

Additionally, McAfee includes an interface and solid manual for advanced users to create their own signatures, which proves helpful for customizing filters to your specific environment. We found that after a few hours of studying we were able to create a simple signature that looked for the request of a Web page named "test.cgi," an indication that someone may be hacking your Web site. Although we would have liked the work to be easier, we were satisfied with the end results.

When it comes to fending off attacks, one of the key tools at IntruShield's disposal is protocol parsing. Because conventional detection engines such as Snort typically incorporate port numbers -- 80/tcp for HTTP, for instance -- into their signatures, they wouldn't look for IM traffic on port 80, for example, unless you explicitly created a signature to do it. Protocol parsing allows IntruShield to apply protocol-specific signatures to traffic on any port. This functionality enables the IPS not only to spot policy violations such as IM conversations being sent over HTTP but also to identify the application running a specific network service, allowing you to apply different rules to Apache and IIS traffic, for example. IntruShield maintains traffic profiles on network hosts, so if a host begins using abnormal amounts of bandwidth or spews excessive packets, the IPS can throttle the usage to prevent a DoS.

By combining buffer overflow protection, protocol anomaly detection, vulnerability-based signatures, and rate-limiting DoS protection, McAfee believes it has created a device that will protect against zero-day attacks. For good measure, IntruShield looks for shell code where it shouldn't be, a patented technique that enables the device to prevent attackers from executing malicious commands even if they manage to penetrate the network. IntruShield also includes a statistical anomaly profile engine that looks for unusual spikes in network traffic. Both these techniques will unearth many new types of attacks, including worm outbreaks.

To put IntruShield to the test, we used our favorite tool, Core Impact, throwing 15 attacks from the SANS/FBI Top 20 at the IPS. The IntruShield successfully blocked all of our attack attempts. Additionally, we viewed an Apache Web page with no problem as IntruShield filtered out our buffer overflow attempts, showing the device could differentiate bad traffic from good even when it originates from the same IP address. IntruShield is capable of countermeasures, including sending TCP resets, changing access control lists on network devices, and sending ICMP (Internet Control Message Protocol) unreachables to fool attackers into thinking that network resources don't exist.

Other welcome features include spyware and VoIP protections and the capability of inspecting encrypted traffic. Using the private keys of your SSL-protected Web servers, IntruShield decrypts a mirror of the SSL traffic, checks it for attacks, and passes the original traffic to the server if it is clean and valid.

IntruShield takes an interesting approach to configuring protection policies. Assets are grouped into virtual interfaces comprised of IP ranges or VLAN tags and appear as separate network adapters. Although virtual interfaces introduce complexity, they support a high degree of granularity in policy management, allowing you to apply different policies to different network segments or specific hosts, all in a single sensor.

Setting up the sensor and management console took less than an hour and was relatively painless. The management server runs on Windows 2000 or Windows Server 2003, providing a sleek Java interface that provides in-depth attack information and produces greatly customizable graphical reports. Setting up and configuring the sensor is straightforward, and we found that McAfee has eased policy creation by including preconfigured profiles.

To extend IntruShield's visibility into the enterprise, the system is capable of using Entercept clients -- McAfee's host-based IPS -- as sensors. Entercept events are forwarded to IntruShield at the alert viewer level, which allows for correlation of events across multiple sensors and multiple network segments.

NFR Sentivist 5.0

We've seen many intrusion prevention systems throughout the years, but very few with a face as pretty as NFR's, whose new front-end component, called the Timeline view, offers an extremely useful 10,000-foot view of overall security posture. But NFR Sentivist isn't just a pretty face among otherwise drab competition. Sentivist combines several methods for detecting threats and policy violations, including vulnerability-based signature analysis, behavioral anomaly, protocol anomaly, and rate-based techniques. Sentivist can also import vulnerability data from the open source Nessus scanner, leveraging the data to prioritize alerts, notify admins of changes to the network, and ensure that appropriate signatures are deployed.

One interesting piece of NFR technology is called Confidence Indexing, which uses information such as network service, attack frequency, distribution of local hosts, and bandwidth utilization to establish the likelihood that given traffic is malicious. When the Confidence Index exceeds an administrator's threshold, the traffic can be blocked. This approach is extremely effective not only in minimizing false positives, but also in preventing "low and slow" attacks, or attacks that may take many days but use very little traffic.

Confidence Indexing also speeds deployment, allowing you to start blocking malicious traffic right out of the box. Still, it will take some planning and tuning to get the most out of Sentivist. The good news is that the interface guides you through the configuration process lickety-split, and changing enterprise policies can be done from a central server.

The bad news is that basic installation is time-consuming. It took us two hours -- with good knowledge of Red Hat -- to get the management server running on Linux (Solaris is the other option), and getting the sensors to talk to the management server was so problematic that we needed to phone the NFR help desk to work through it.

Built on FreeBSD, the sensors are available as preconfigured appliances or in a software-only configuration. 

Notably, management servers can be configured hierarchically so that a central Sentivist server could process all enterprise alarms while correlating alerts across multiple Sentivist servers throughout a geographically dispersed organization. When an IP is flagged as malicious in one branch, it can be blacklisted across the entire organization in near real time, although Sentivist cannot change access control lists on firewalls and other network devices. You can also whitelist hosts to ease administration and reduce false positives.

The Java-based interface allowed us to filter or pivot across all relevant data points quickly. Reports were also stellar. The Timeline view, which presents a right-to-left scrolling line of events along with a color-coded severity rating, is extremely helpful during incident response. Not only could we quickly drill from the top-level view down to the actual packet causing the alert, we could easily create filtered Timeline views for critical assets.

Setting up an aggressive Sentivist policy, we threw our SANS/FBI Top 20 exploits at the IPS but failed to exploit the target on the other side of the sensor. During testing, the Sentivist reporting console showed usefully detailed information, although its reports lacked the corrective actions provided by McAfee IntruShield. While monitoring live traffic on our production network, the solution revealed exploit attempts, internal bandwidth hogging, and users authenticating to unsecured protocols such as Telnet. Sentivist also did a very good job of minimizing false positives. We attribute Sentivist's solid traffic segregation to well-written detection signatures and the secret sauce in Confidence Indexing.

Sentivist guards against zero-day exploits through the use of behavioral signatures, which we found to be an effective approach to mitigating worms. Furthermore, as does IntruShield, Sentivist detects attacks of one protocol hidden within another. It also provides limited spyware protection but doesn't yet guard against VoIP-based attacks.

The learning curve for creating custom signatures using NFR N-Code signature language is mild. Unlike with McAfee's complex signature script, we wrote a signature to detect a popular online game in less than 15 minutes.

In Sentivist, NFR has developed a solid IPS with an extremely refined UI, although it doesn't yet offer the breadth of protections provided by IntruShield or competing solutions from TippingPoint and Sourcefire. And though configuring the IPS itself is a breeze, basic installation should be smoother.

McAfee IntruShield shines with virtualization capabilities, SSL decryption, shell code detection, an easy-to-use Web interface, and solid reporting. Version 3.1, which we've seen but not tested, now allows you to incorporate vulnerability assessment data from Foundstone and Nessus. However, because the data is used only to determine threat relevance and not severity, the integration is not as useful as it could be.

Finally, although both IntruShield and Sentivist provide some level of protection against new kinds of attacks, we would have liked to see service anomaly detection capabilities such as those in Lancope's StealthWatch and Sourcefire's RNA. The ability to spot deviations from normal network behavior is the best protection against zero-day attacks we've seen so far.

Correction:
This article has been changed to correct misstatements regarding competing products' use of signatures based on known exploits.