Application whitelisting review: CoreTrace Bouncer

CoreTrace's Bouncer 5 is application control and more. Bouncer is the only product in InfoWorld's review that successfully protected against buffer overflows. It also offers write protection of whitelisted files and does a nice job of handling updates to controlled applications.

A great-looking GUI, good reporting, and secure sessions between clients and the management server round out the rich feature set. However, Bouncer doesn't cover all program file types, notably those written in interpreted languages such as Python, PHP, or Java.

Started in early 2008, Bouncer is made up of a Windows XP Embedded management appliance and supports clients running Windows NT 4 SP6a and later and Solaris 7 through 10. The extra features and security considerations put into this product are evident from the start.

Logging into Bouncer's Control Center management console [screen image] requires a two-factor USB access token and either physical access to the management appliance or a Remote Desktop Protocol (RDP) session. Connections between the management console and clients are IPSec protected with PKI certificates. This is all automated in the setup of clients and server, and it does not use the normal Windows implementations.

The use of client certificates also aids monitoring. Clients can get new IP addresses, new network interfaces, new names, and so on, yet still be identified and tracked through the use of the certificate. Clients automatically check back in to the management console every 60 seconds using heartbeat packets across two high-numbered UDP ports, or you can schedule the connections for finer-grained control.

Managed computers are collected into groups known as Security Configurations. In fact, calling groups of computers Security Configurations is one of the few minor weaknesses of an otherwise top-of-the-class product. To be fair, Security Configurations are really the grouping of computers along with their defined treatment. But a simpler label would avoid potential confusion.

Test Center Scorecard
	30%	15%	25%	10%	20%
CoreTrace Bouncer 5	9	9	9	8	9	8.9 Very Good

Three Security Configurations are provided out of the box -- All Installed Systems, Pending Systems, and Unsecured Systems -- but administrators are encouraged to make their own custom groupings. Each Security Configuration (i.e., group) will have its own Bouncer settings and Policy Components defined.

Policy Components are built around the concept of trusted change. Administrators can define Trusted Applications (applications that are allowed to run), Trusted Digital Signatures (all applications signed by the same digital signature can run), Trusted Network Shares (any application in a trusted location can run), and Trusted Users (trusted users can run any program). Each managed computer will inherit the policy components defined for its Security Configuration.

Computers, users, and groups can be enumerated from Active Directory. Each computer can be scanned to generate a new whitelist baseline or another predefined policy can be applied. When generating a new baseline, Bouncer can create whitelisting rules for all binaries (including .EXE, .DLL, .COM, .DRV, .SYS, .CPL, .OCX, .DEV, .MANIFEST, .FON, 16-bit apps, and batch files), and these rules can be enforced on the system drive only or on all drives. Other file types (scripts, text files, and so on) must be added manually and become part of the Custom Policy component.

Only executable binaries and batch files can be prevented from executing. Like most of its competitors, Bouncer cannot prevent scripts or Java programs from executing without blocking the scripting engine or Java Virtual Machine. This means you may be forced into an all-or-nothing decision for non-executable file types.

Whitelisted files become write-protected by Bouncer's own kernel drivers. Although files can be copied, they cannot be modified, renamed, or deleted. This is an interesting feature that no other competitor has. Besides preventing computer viruses, it could be used to prevent unauthorized modification of security-related files like the DNS Hosts file, which is often maliciously manipulated by malware programs. Bouncer can be put into Learning mode, which allows it to generate whitelisting rules but not block any executions. The generated whitelisting rules can be added to any Security Configuration profile to allow or deny execution. End-user denial messages are customizable for each Security Configuration.

Whitelist rules for individual file types can be defined using three attributes: path, size, and the file's SHA-1 hash fingerprint [screen image]. Machines can be scanned for publisher digital signatures and the identified publisher then added to the Trusted Applications policy. This can be done on the individual computer level or on a baseline computer, then pushed out to other clients. File instances can also be searched across managed computers.

One of the biggest challenges for any whitelisting product is handling complicated product updates. Here Bouncer shines. First, any update operating under a Trusted User, Trusted Application, or Trusted Network Share is allowed to run, and the new whitelisting rule is generated. Bouncer can even handle multiboot, chained installs and major service pack updates, automatically generating the necessary new rules.

Bouncer goes even further in one seemingly small step that, although not unique among the products in this review, means big things. Any trusted application is allowed to install other applications. For example, administrators could trust the Windows Update service, Microsoft's Systems Management Server or Systems Center Configuration Manager, or their regular, controlled patching program. Any program installed using those predefined trusted pathways is automatically trusted and a new whitelist rule is generated. This allows companies to officially sanction their primary installer application without having to manually update the whitelist rules.

Buffer overflow protection is an excellent feature that is found in Bouncer and only one other competitor, McAfee Application Control. Many whitelisting applications cannot thwart buffer overruns that could be used to exploit an existing, authorized service. Most application control programs instead focus on preventing the execution of any malicious code that an attacker might run after the buffer overflow is successful. Unfortunately, not all exploits that rely on buffer overruns make use of executables. In fact, two of the world's most successful computer worms, SQL Slammer and IIS Code Red, did not modify existing services or write new files to disk.

Bouncer includes kernel memory write protection that is intended stop those types of buffer overflow programs. In my testing, Bouncer blocked my buffer overflow attempts every time. Even more surprising, Bouncer was able to stop a buffer overflow attack that was already running before Bouncer was activated. For these tests I used common buffer overflows and payload mechanisms, defined in Metasploit, to push rogue VNC consoles and reverse command shells to the target system. The exploit and payload went off just fine while Bouncer was disabled. Then I enforced Bouncer's policies and the agent immediately cut off the rogue access.

You can watch a demo video of Bouncer's buffer overflow protection at CoreTrace's Web site. I'm not sure of this feature's accuracy; I didn't run every Metasploit buffer overflow against it. I can only tell you that Bouncer provides better protection against buffer overruns than its competitors do.

Reporting is another strong point. Client events can be sent using syslog to any syslog-enabled product or to an optional centralized reporting server. The Bouncer reporting server runs Microsoft Reporting Services on Microsoft SQL Server or SQL Server Express. The reporting server comes with many customized reports under four different main report categories: Auditing (system type events), Configuration (used to compare policies between clients), Inventory (nodes, workstation info, versions, and so on), and Security Events (client whitelisting activity). Naturally, customers can also use Microsoft Reporting Services to write custom reports.

This story, "Application whitelisting review: CoreTrace Bouncer," and reviews of competing products from Bit9, Lumension, McAfee, SignaCert, and Microsoft, were originally published at InfoWorld.com. Follow the latest developments in information security and endpoint security at InfoWorld.com.