Appliance-based solution archives, analyzes multiple log file types
There are a million and one log-management tools running in networks all over the world. Some are simple file-searching utilities. Some dig a little deeper. But all are focused on a specific log file from a specific service or network device. They don’t always talk to one another, however, which creates havoc for admins.
LogLogic aims to bring all these disparate log files together in a palatable, sustainable way -- and they largely succeed with the Series 3 appliances.
Under the Hood
LogLogic’s solution is a collection of two appliances: The LX series serves as the initial log-file destination and the console for reporting, while the ST series handles log-file archiving.
Both appliance families are built on standard 2U and 3U rack-mount server hardware platforms, leveraging a Linux core, MySQL, Apache Tomcat, and Java to deliver the whole solution. The kernel is a recent 2.6.12 version, and LogLogic does not prohibit root-level SSH log-ins to the appliances.The hardware itself is quite robust, with both appliances sporting redundant power supplies and a 3ware 8500-series SATA RAID controller.
The LX 2000 I had in the lab was designed only to hold data for 90 days in a high-volume environment, so it had roughly 500GB of storage in a RAID 5 array of four 160GB drives, 4GB of RAM, and two 2.8GHz Intel Xeon CPUs.
The ST 3000 provides storage for the LX’s archived log data. It’s equipped with 2GB of RAM, two 2.8GHz Intel Xeon CPUs, and roughly 2TB of storage across a RAID 5 array of nine 250GB drives, with a mirrored set of boot drives. Both appliances have a significant amount of internal disk, but can be integrated into SAN and NAS environments for more archival storage.
Set ’Em Up, Knock ’Em Down
Setting up the appliances is straightforward, with a serial console connection necessary to configure IP information. Further configuration and admin tasks are done via the Web interface. The Web UI is reasonably fast and well-rendered, but occasionally has a touch of that well-known Java sluggishness.
You can quickly access high-level info using LogLogic’s status dashboard, which shows the current mps (messages per second) rate, existing alerts, system performance, and total message counters. Another click shows all defined syslog sources and their individual message counters, and a Java-based real-time syslog viewer applet shows the current syslog message streams as they flow into the box.
One of the appliances’ nicer features is their log source autodetection capability. To configure a device to log to the LogLogic appliance, you need only to point the device’s syslog output to the appliance’s IP address. When the appliance sees traffic from that source, it is automatically classified according to the determined data type.
LogLogic supports quite a few log formats, from routers, IDSes, and firewalls to more specific formats such as Microsoft Exchange and IAS and even Squid proxy logs. Log files can be imported directly into the LX 2000 via HTTP, HTTPS, SCP, FTP, or SFTP, although log-type restrictions can reduce this feature’s usefulness. However, the direct importing is hugely important when dealing with non-syslog services such as Exchange, as this process can be cumbersome without it.
In the lab, I configured an LX 2000 and an ST 3000 for log-file archiving and began blasting syslog messages at the LX 2000 from a variety of simulated sources. Using a homegrown syslog injection script, I achieved and verified the top-end 3,000 mps rate on the LX 2000. The promised 150 percent burst rate was almost spot-on as well: I maxed out at 4,450 mps, and sustained that level indefinitely. The ST 3000’s top-end rate is purported to be 50,000 mps; while I couldn’t generate enough traffic to approach that level, I did get it up to just under 10,000 mps.
In production, the LX 2000 acts as the syslog server for all local devices and periodically archives data by compressing the log data and transmitting it to the ST 3000 via TCP. This setup means you can place LX 2000s at remote sites to collect log data generated there, significantly reducing the traffic that must flow back across the WAN to the main datacenter.
All this horsepower and back-end work is useless without good reporting tools. LogLogic’s UI includes predefined real-time and summary reports that are readily available in the navigation tree. Reports can be generated on the fly with up-to-the-minute data and can be exported as CSV, HTML, PDF, or printable layout. Summary and scheduled reports are also available.
The predefined reports are actually quite useful: pulling up a list of accepted connections from all known Cisco PIX log sources is as simple as selecting that report, choosing a single PIX (or all of them), and defining a time period. The resulting report can be dissected further with pseudo-SQL search capabilities that allow for more fine-grained reporting or drilling down to a specific event. The recently added Exchange e-mail monitoring is also a boon, with predefined reports showing e-mail usage by user activity, size, and raw SMTP communications.
I found LogLogic’s power and relative lack of complexity intriguing. I would like to see more standard log-file support, such as Linux/BSD firewall log files, Sendmail/Postfix/Exim Mail Server logs, and perhaps an interface to define custom log formats. (You can import unsupported log files, but the options for parsing and reporting are limited.) Otherwise, LogLogic’s Series 3 appliances are ready to make a molehill out of your mountain of logs....
Ease of use (20.0%)
Overall Score (100%)
|LogLogic Series 3 LX and ST Appliances||9.0||8.0||7.0||8.0||9.0||9.0|
You may still be better off sticking with Win7 or Win8.1, given the wide range of ongoing Win10...
Early results look promising: the many-hours-long Win7 waits may be behind us
Now that we're down to the wire, many upgraders report that the installer hangs. If this happens to...
Want to get started in machine learning? Google has you covered with high-quality data sets, both big...
Some of the best third-party PC software around has been usurped by native Windows 10 tools and...
The company is removing UML support in Visual Studio 15 due to a lack of usage
VMware private clouds will gain elasticity from the Amazon-VMware offering, but customers may...