Living the log management lifecycle
When evaluating log management solutions, keep the many phases of successful log management in mind
Follow @rogeragrimesAnalyzing network security events for intrusion detection and forensics is a good and popular reason to implement log management, but it's not the only reason. Auditing and compliance are becoming just as important as traditional security requirements, while the best-run IT shops understand the value of logging for systems and application management.
Regardless of the purpose behind logging, the log management process has several distinct phases: policy definition, configuration, collection, normalization, indexing, storage, correlation, baselining, alerting, and reporting. You may see the various phases summarized in different ways, but the lifecycle is always the same. When choosing a log management solution (see "InfoWorld review: Better network security, compliance with log management"), you'll want to evaluate the product features and capabilities with the whole process in mind.
Policy definition. Policy definition means determining what you're going to audit and alert on. Is your company interested in security event detection, operations and application management, or compliance auditing? Will you be auditing just workstations and servers or also applications and network devices?
Configuration. When you decide what and why you want to audit, you then need to detail what log events will help you achieve those goals. Many log management vendors provide "suites" or "packages" that attempt to provide predefined, built-in configurations to support various goals, although I did not find any product that was as inclusive as needed. Each user will have to review what the product is able to capture and alert on, then define additional capturing to fit the demands of their environment. Configuration is the act of translating your audit policies into actionable information capture.
On a related note, you can click here to download a detailed listing of events that should be monitored and reported on in Microsoft Windows networks: Windows Security Event IDs [Excel file].
Collection. Data collection involves sending log message events from clients to the log management server. Most products provide agentless data collection or require that client events be forwarded to the server. Most log management products provide agent software to assist with data collection in cases where agentless collection doesn't make sense.
