Imagine how delighted your end-users would be if the help desk called them ahead of time to let them know they were having a hardware or software problem that was just starting to manifest itself. Wouldn't it be nice to catch hackers before they were successful? Can you imagine a world in which your purchasing department was alerted to buy additional hard drives before they ran out of free space?
Are log files a waste of time? The exact opposite is true. Logging, if appropriately configured and managed, will save you and your company time and money. The best-run organizations live on a diet of event log baselines and proactive responses, and you can too.
Log management 101
In a nutshell, logging allows you to quantitatively and proactively measure the overall health of your environment, from a security perspective, for auditing and compliance, for systems management, and for application tuning and troubleshooting. These basics will get you started.
Security monitoring. Most of the literature surrounding computer logging talks about monitoring events to lower your security risk. Logging can alert incident response teams to prevent malicious hacking in the first place -- or at least send in the cavalry as quickly as possible after an exploitative event has occurred to minimize damage and start forensic investigations.
Logging security events for intrusion detection and forensics, which is often the main reason administrators get into log management, requires specialized advice. You can start by reading NIST's Special Publication 800-92, "Guide to Computer Security Log Management." Released in September 2006, it's unusually easy to read for a NIST (National Institute of Standards and Technology) publication and extremely useful for deploying event log management systems in the real world. It's considered the gospel in this small corner of the computer security world.
The NIST guide steps through all of the essentials of log file management: identifying the threats and risks to your environment; determining policies for logging, auditing, and handling logs; collating, indexing, and normalizing logs for analysis; defining and generating alerts and actions for critical events; and defining reports and metrics for management review. From putting log management infrastructure and processes into place to reviewing and archiving logs, it leaves no stone unturned.
Auditing and compliance. As motives for instituting log management programs, auditing and compliance are becoming as important as traditional security requirements. Most industry regulatory guidelines now define specific security events that must be monitored. When the right audit policy has been enabled across all required computers, and the appropriate log management system is in place, most companies will pass that portion of a compliance review. On the other hand, the lack of an acceptable security auditing policy can raise suspicion that the right controls are lacking, which may have legal implications.
Systems management. The best-run shops understand the value of logging and use it for systems management. These organizations create baselines of normal operating activity and events, and they set up alerts triggered by excessive deviations. Many environments execute simple ping connectivity tests to monitor which devices are online and which unexpectedly dropped and need to be investigated. Other places embrace the full richness that logs provide.
If a hard drive begins to move too many bad sectors, even before a complete crash occurs, the log administrator has a replacement drive ready to roll. If network activity spikes unexpectedly, administrators are aware of the problem before the inevitable complaints about slowness arrive. A sustained traffic hit may be a worm or a denial-of-service attack. If a server or SAN crashes, the help desk knows about it before users start to call in.
Read more about how tap into log files in InfoWorld's free PDF report, "Log File Analysis Deep Dive," including:
- Application tuning and troubleshooting
- Choosing the right log management software
- The log management life cycle
- Pulling off a successful event management program
Read more about storage and managing enterprise data in InfoWorld's Data Explosion Channel.