It's fair to say that I learned a lot about log management through Splunk. A few years ago, I used Splunk for a variety of computer security applications, most notably to collect and forward events across a wide range of machines. Picking it up again two years later, I was pleasantly surprised to find that Splunk has become feature-rich and very handy beyond its early origins.
For this review, I installed Splunk 4.1.2 across a few different platforms. Splunk comes as a single installer, available for Windows (XP and later), Linux, Unix, BSD, Mac, and a few other operating systems, including a few of the most popular network devices.
Depending on how you decide to use Splunk, all the components can be installed on a single computer; at a minimum, Splunk requires dual processors and 8GB of RAM. In addition, the various components can be spread across multiple computers. Indexers host the Splunk data store and provide indexing services for local and remote data sources. Stored data is compressed to half its original size. Search heads, forwarders, deployment servers, and high-availability components can also be deployed in a distributed implementation. I installed all components on single servers since I wasn't testing enterprise performance. Online and downloadable documentation is particularly good.
Splunk: Log collection and management
Installation was as simple as clicking Next, Next, and Finish. Once installed, Splunk is accessed through an HTTPS Web interface using TCP port 8000 by default, a command-line interface, or a custom third-party UI (if purchased or downloaded separately). The screen below shows the main Splunk Manager interface. Many of the available features, including reports, searches, and dashboard views, depend on which Splunk applications are installed. There is a healthy Splunk development community, and many of the Splunk add-ons are available for free.
When monitoring a folder, users can choose from more than 50 source types, including OS X, Snort, Asterisk, and several generic options. These source options can be selected on several of the different input types. Individual files within a selected folder can be included or excluded using regex (regular expression) syntax.
Long used across a very wide range of data sources, Splunk is very adept at handling unstructured data sources, allowing strong reporting and statistics to emerge, whereas many other solutions require that unstructured data be normalized before strong analysis can begin. New searches and reports (ad hoc or scheduled) can easily be created, either by inputting regex expressions or copying and modifying an existing object.
Windows Vista and later Windows versions have dozens of individual event log views beyond the legacy Application, Security, and System logs present in earlier Windows. As the screen image shows, Splunk allows you to choose one or more of the individual logs, which is not something most competitors easily allow.
Splunk: Log searching and reporting
With the default Windows application installed, Splunk comes with 23 default searches and reports, which is limited compared to the competition. Many more reports and views can be downloaded for free. In addition, Splunk offers numerous premium solutions, including the Enterprise Security and PCI Compliance Suites.
Data points in each view or report can be clicked on to drill into more detail or different timelines. Each returned search can be modified on the fly to narrow or widen the scope. Top-value reports for a particular time period or overall can be queried and presented with a single click.
New reports, with eight different chart types to choose from, can easily be created with the Splunk Report Builder. You can save the report to run again later, export it to CSV, XML, or JSON formats, print it, or send users a link that allows them to run the report at a later date.
Splunk allows for some of the most granular user role definitions I've seen. In addition to four default roles (admin, can_delete, power, and user), you can easily create custom roles by drawing on more than 40 different capabilities or configuring additional restrictions. LDAP authentication can be implemented instead of relying on the built-in authentication database for usernames and passwords.
Splunk comes with 16 XML-defined views and more can be created. Each view can be linked to a user role. You can also create custom navigation menus, although major changes (beyond simple button additions) require modifying the existing XML code or creating new XML code from scratch. Items can be added to an existing menu without XML editing, and dashboards can be built using a graphical tool. This customization isn't nearly as user friendly as most of the competition, but Splunk's extensibility through XML is great for users who want ultimate control.
The Splunk I once knew is all grown up, full of features and functionality. It doesn't contain all the bells and whistles of the competition, but it has the core components most users need.
See additional log management reviews: