If you have ambition, open source at scale is essential

“No proprietary software can solve all the problems of companies that operate at the scale of Didi,” says Li Luo, technical director of big data at Didi Chuxing, the Uber of China

Become An Insider

Sign up now and get FREE access to hundreds of Insider articles, guides, reviews, interviews, blogs, and other premium content. Learn more.

When your job is to provide the cloud infrastructure to run analytics and workloads across three datacenters that are more than 100 miles apart, sucking 100-plus petabytes from each daily, it’s no longer an even remotely credible option to buy it from Megavendor X. These days, the only place to find such software is on an open source repository somewhere.

Which is exactly what Didi Chuxing, the Uber of China, did.

“No proprietary software can solve all the problems of companies that operate at the scale of Didi,” said Li Luo, technical director of big data at Didi. “We need access to the source code so we can update it and contribute back frequently to meet changing requirements. For companies like ours, open source software is the right choice.”

In reality, it’s the only choice.

Five years ago, Cloudera cofounder Mike Olson wrote, “No dominant platform-level software infrastructure has emerged in the last ten years in closed-source, proprietary form.” In significant measure, this stems from the realities of operating at web-scale: The financial costs, never mind the technical costs, of trying to scale proprietary hardware and software systems are simply too high. Companies like Google and Facebook keep gifting genius creations to the open source community, driving innovation faster, well beyond the realm of proprietary firms’ ability to compete in data infrastructure.

To continue reading this article register now