Cloud convenience is killing the open source database

Thanks to stalwarts like MySQL, MongoDB, and Cassandra, the database realm has been a haven for open source -- but the cloud is changing all that

Cloud convenience is killing the open source database
Credit: NOAA

Open source has never been more important or, ironically, irrelevant. As developers increasingly embrace the cloud to shorten time to market, they're speeding past open source, making it even harder to build an open source business. 

After all, if open source were largely a way for developers to skirt legal and purchasing departments to get the software they needed when they needed it, the cloud ups that convenience to the nth degree. In Accel's annual business review, the vaunted venture capital firm writes: "'Product' is no longer just the bits of software, it's also how the software is sold, supported, and made successful." The cloud is changing the way all software is consumed, including open source.

The cloud is eating your database

The database market, where open source databases like MySQL and Postgres have dominated adoption for years, makes the shift to cloud readily apparent. Even among this open source elite, the cloud is making waves. More recently, MongoDB, Cassandra, and Redis have climbed to rank among the top 10 most popular databases, as shown in DB-Engines most recent findings.

If we drill down into relative growth, yes, cool-kid NoSQL vendors like MongoDB and Cassandra are growing fast relative to open source RDBMS leaders MySQL and Postgres. However, they pale in comparison to the absolutely torrid growth of cloud databases like Amazon DynamoDB or Microsoft Azure DocumentDB, as data reveals:

database jobs

Of course, this is relative growth, and the Amazon and Microsoft Azure databases have the benefit of starting from comparatively small bases. But Amazon DynamoDB, for example, is a top-25 database under DB-Engines' ranking, so it's hardly an unknown.

Make it easy on me

One way to explain this uptick in adoption across these databases is their appeal to developers. As noted in an Andreessen Horowitz podcast, developers are the new go-to-market for many leading businesses like Atlassian or MongoDB because they offer a way to route around the cumbersome institutions of legacy software procurement:

andreessen horowitz database podcast Andreessen Horowitz

The first companies to try this approach focused on open source, but the perfection of the model is the "as a service" business. Companies like Amazon offer a way around traditional procurement vehicles, minus the headache of assembly. As RedMonk analyst Stephen O'Grady has asked: "If you select Nginx, Kubernetes, Docker, Chef, and MongoDB, for example, what assurances do you have that these all work reliably together?" On their own, the answer is "none." But in the cloud, the answer changes.

Developers, fed up with cobbling together proprietary or open source solutions themselves, increasingly are comfortable letting AWS, Microsoft Azure, or Google Cloud remove that complexity for them. Or as O'Grady pithily puts it in a separate post: "Choice is an overhead, overhead that is multiplied with each additional choice a user has to make." The cloud removes choice (bad!), thereby increasing developer productivity (good!).

It may also remove open source.

Freedom to dump freedom

In this convenience-first world, open source licensing may not matter much. As DataStax executive Patrick McFadin told me in an interview, in the future "developers will use APIs and will be charged by the call or megabyte. The argument of what type of license will disappear as a result."

Not that everyone is happy about this. For some, like Weaveworks founder and CEO Alexis Richardson, this brave new world of cloud has "everything you need, except freedom." This, however, depends on how one defines freedom. If freedom begins and ends with the license governing the software, perhaps it's true that the cloud kills open source freedom.

But if the larger concern is with a "freedom to get stuff done," then the cloud has the potential to unlock dramatically more freedom than the 1s and 0s of open source code ever could. This is the realization that Capital One CIO Rob Alexander came to: Rather than burying his teams in server and software maintenance, they could increasingly build on public cloud services from AWS.

Again, this carries risks. One, as Red Hat engineer Timothy St. Clair highlights, is "hidden costs as those [cloud] services morph." An enterprise that builds on someone else's foundation remains dependent on it.

Developers don't seem to mind. Perhaps this is because they're cutting their teeth with new infrastructure as open source code first, then going cloud as AWS, Microsoft, and Google operationalize it for them. In this way, open source remains highly relevant to initial use by early adopters, but it's the cloud that makes it real for the mainstream.