Google Cloud to host open source Elasticsearch

The popular alternative to Splunk will be available as a managed service on Google Cloud Platform

Current Job Listings

The Elastic stack—Elasticsearch, Kibana, and Logstash—has become a powerful open source alternative for doing real-time searches on generated data like logs. Now Google is turning one of them into a cloud commodity.

Google has partnered with Elasticsearch BV, the group that commercially supports the Elasticsearch stack’s cloud—called Elastic Cloud, appropriately enough—and is preparing to offer managed editions of Elastic Cloud on Google Cloud Platform.

It’s another example of Google offering a recognizable open source project as a managed service instead of its custom product.

Managed instances of Elasticsearch, such as Elastic Cloud, come with the latest versions of each component in the stack, as well as the X-Pack features only available in the commercial edition of the stack: data security, alerting, monitoring, report generation, and graph analysis of the data. A blog post on Elasticsearch’s site implies the offering will be available “in the second half of 2017.”

Being open source is a key part of Elasticsearch’s appeal. It provides an alternative to commercial products like Splunk, but has full commercial support as well, if needed. Elasticsearch has providing that kind of support for some time and provides hosted instances for its product on AWS.

There are two potential models for how Google Cloud Platform will make Elastic Cloud available. The first is via machine images that run in Google Compute Engine, essentially a variation on the way Elasticsearch offers official machine images for its product in AWS Marketplace.

The other would be in a way similar to the native Elasticsearch service that Amazon provides. In other words, as a fully managed service (for example, failed nodes are automatically replaced) with tight integration to other Amazon offerings like IAM or S3 snapshot backups. This second model seems the one Google is most likely to follow, as it provides the kind of seamless integration Google has offered with its cloud services.

Some of Google Cloud Platform’s other options also come from an open source project. Cloud SQL, for instance, provides managed instances of MySQL and PostgreSQL. Container Engine is essentially hosted Kubernetes, which comes as little surprise since Kubernetes is a Google product. And Cloud Dataproc provides Spark and Hadoop—two notoriously hard-to-manage products—as managed services.

All these products have large existing user bases, so offering them as managed options on Google Cloud Platform is a sensible way to broaden that platform’s appeal and get more kinds of users on board. 

Aside from the convenience that comes with managed Elasticsearch stacks, Google is also likely to play up the enhanced security. Earlier this year, ransomware groups attacked more than 600 unprotected instances of Elasticsearch and held hostage the data found there. As with MongoDB instances before them, Elasticsearch stacks haven’t been as aggressively protected as other resources.

Hosted instances of Elasticsearch can go a long way toward providing that protection, whether they’re provided by Google or another company. Now Google can invite Elasticsearch users to come on board and leave the driving to them.