After six years, container orchestration system Apache Mesos has hit 1.0, with a refined API and broad-spectrum support for different container types.
The 1.0 release comes as the other major container orchestration solution, Google's Kubernetes, has risen to the point where OpenStack is being reworked to use it as a deployment technology.
But Apache Mesos is striving for different goals. Kubernetes deals with containers, but Mesos' ambitions involve running clusters as well, with containers as simply one possible ingredient.
How to get to 1.0
Ben Hindman, co-creator of Apache Mesos and current chief architect at Mesosphere, described in a phone interview how getting Mesos to a full 1.0 release was about having a refined, stable, consistent, and broadly useful API.
The 1.0 API is HTTP-based, but not RESTful; it's closer in spirit to Google's GRPC system for remote procedure calls. It was also built to replace libraries provided for different languages, meaning future changes to the API can be made without as much breakage.
Mesos 1.0 also adds what Hindman called the "unified containerizer" -- a universal container runtime designed to work with Docker, CoreOS's rkt format, the appc image format created as part of the Open Container Initiative, and any others that might arise in the future.
Originally, the containerizer was devised to run Docker images back when the Docker daemon was still not very stable. The actual runtimes for each of these formats was not used, but Mesos leverages the containerizer to provide features that aren't native to those runtimes in the first place. Recently, Nvidia added support for GPU-powered applications by way of a Docker plugin, but Mesos' runtime is allegedly able to provide the same support natively.
Version 1.0 also has planned support for Mesos on Windows. It's well-timed, given Microsoft's release of a container-supporting version of Windows Server later this year.
Where to go next
To make clear how Mesos stands apart from Kubernetes, Hindman stated that while the two projects are in roughly the same space, Mesos runs as a lower-level component and encompasses a different class of problems. While Kubernetes deals mainly with container management, Mesos is about running clustered systems that have a great deal of state.
One way to illustrate this: Kubernetes can run on Mesos, and the code used to do this has since been released as an open source project. Among the folks who picked up on it are IBM, which Hindman says has used Mesos to run Kubernetes as well as big data workloads like Spark. (Mesosphere's data center management solution DCOS, which uses Mesos, was designed to do this as well.)
"Mesos really came from this area where people were thinking about cluster management with respect to big data technology," said Hindman. That was where the project started, but its maturity was "all about containerization," during its trials at Twitter.
Where Mesos goes from here, though, is about far bigger issues than containers at scale alone. The changes rolled out for 1.0, said Hindman, make it possible to think about what's next, like multitenancy and multiregion workloads, and how Mesos can address both containers and what's beneath them.
"Getting containers up and running is kind of the easy part." Hindman said. "But making it so that people can actually operate their systems on top is challenging."