The rest of the world
While ELK is a powerful stack, it's not meant to be the be-all and end-all. As such, the creators have taken care to provide interoperability with the rest of the world. Logstash currently bundles output connectors for 60 or more different systems. The range of possible outputs includes such diverse possibilities as AWS S3 buckets, IRC, Solr, MongoDB, Redis, Riak, XMPP, and many more.
Sissel points out that Logstash can be used as part of a more complex analytics workflow, such as complex event processing with Esper, Storm, or S4 -- or even batch processing with Hadoop. While Logstash does not include an HDFS output connector today, Sissel says it may arrive in the future, "if we see community demand for it."
Another case where Logstash is more appropriately used as a complement to other tools is the "document ingestion" scenario. Logstash really is an event/log based system, and it'd be an awkward fit for trying to crawl and consume a document repository and load those documents into Elasticsearch. In such a scenario, a cleaner solution involves using ManifoldCF or Nutch to handle "document" data, with Logstash as a peer to handle event/log-oriented data.
Open source and support
Logstash is fully open source and licensed under the business-friendly Apache License Version 2.0 (ALv2). Source code is available at GitHub. Downloads of both Logstash and the rest of the ELK Stack components are available at elasticsearch.org.
Organizations can receive support from the engineers that built Logstash and the ELK stack by subscribing to annual support offerings from Elasticsearch, Inc. ELK Stack subscriptions also include free licenses to Marvel, a real-time monitoring system for ELK deployments.
Elasticsearch, Inc. has seen revenue growth of over 400 percent year over year and reached nearly 6 million downloads. Given the company's track record over the past few years -- and the history of the founders as contributors to projects like Logstash and Apache Lucene -- it's fair to expect a steady stream of innovative new products from Elasticsearch in the future.
This article, "What's new in Logstash and why you should care," was originally published at InfoWorld.com. Keep up on the latest news in application development and read more of Andrew Oliver's Strategic Developer blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.