Having developed its own devops tools to deal with the management of hundreds of thousands of servers, Facebook is now making these tools available to others via open source.
The company already has been offering two of its Chef "cookbooks," which work with the Chef configuration management platform, and plans to make more available over time, said Phil Dibowitz, Facebook production engineer, in an interview on Tuesday.
"We use [Chef] for managing the OS and configuration for everything from middleware all the way down the system for our entire fleet," Dibowitz said. This includes configuration of the operating system and middleware layers and package installation.
Accessible on GitHub, the two cookbooks released are for running scheduled jobs and for system controls, including setting up defaults, adding new system controls, and changing existing ones. Chef shops running a lot of servers can scale their infrastructure by using Facebook's tools. "The upside is that it sets you up to scale incredibly well, but the downside is that it's a different model from the way a lot of people use Chef," Dibowitz said.
Facebook also has released its Taste Tester testing framework and re-released its Grocery Delivery tool, which watches a source code repo, such as Git, and keeps the local Chef server in sync. Other cookbooks are expected to be released over time for such functions as controlling which DNS server is used. “Certainly, it is our goal to release as much as possible," said Dibowitz.
Facebook's Chef platform serves as a devops mechanism, with devops geared to shared responsibility between systems personnel and code writers. Engineers were required to write cookbooks describing their systems, which provided for shared knowledge, Dibowitz said.
With its switch to Chef, Facebook was looking for simplicity in making changes to systems. "What we really wanted to do three years ago was build a system that didn't require one central team to understand every possible combination of configurations that could exist throughout the entirety of Facebook because that doesn't scale very well when it comes to people," Dibowitz said.
Instead, Facebook wanted a system where a small, central team could build a set of APIs that allowed engineers who owned any system to describe what they wanted, such as a DBA running MySQL server who wants more shared memory, he explained. A one-line variable should be all that was necessary to make that happen.