Solutions to data overload

When I wrote about the problems of data overload for administrators a few weeks ago, readers reminded me (sometimes constructively) that there are a number of products that might solve the problem (see “Data, data everywhere,” Jan. 6, page 30).

Yes, intrusion detection systems do indeed provide floods of data -- most of it useless. But they also provide useful data that can tell you if your enterprise is being compromised. Likewise, vulnerability assessment products work well on individual servers, workstations, and some network devices.

But in an enterprise with thousands of these, how do you sort out which vulnerabilities are important, and which you can worry about later? After all, no security staff has the people required to check out every possible vulnerability, make the decision as to whether or not it should be corrected, then perform the fix. Those days ended about the time Thomas Watson was predicting that the world would need as many as 12 computers.

So, acting on your constructive suggestions, we decided to take a look at some software packages that would make life easier for managers, yet would not miss events that could be significant threats to an enterprise. We've already placed two such products in the InfoWorld Test Center . The first is ArcSight, a package that consolidates data from a wide variety of sources -- including intrusion detection software such as Snort, firewalls such as Pix -- and from server log files.

When ArcSight retrieves the information it needs, it eliminates everything that doesn't appear to be a real threat and presents the results to the administrator.

The second product we installed was eEye's enterprise management software for its Retina vulnerability scanner. In the past, Retina, although very capable, was only able to scan for vulnerabilities that could be seen over the network from its server. The enterprise solution allows you to consolidate the reports from all of your Retina scanners, even those on separate networks. From this you can see the big picture then decide which vulnerabilities need immediate attention.

These are only two potential solutions to the problem of data everywhere. Whether they're the best solutions -- or even solutions you should examine seriously -- remains to be seen. We're still testing these products and others in this area, and we'll let you know more when we complete our tests.

These products are important for two reasons. First, there's always much more going on in your network than you'll ever really know. Second, there will never be enough people on your staff to do everything that must be done.

At first it appears we need some sort of enterprise networking version of the Heisenberg Uncertainty Principle, except that most of us already know we can't know everything about our networks. Even with excellent products that perform capable data consolidation, we'll never know it all -- but with products that work well enough, we won't need to.