Facebook open sources its Watchman tool for speeding up app builds

The tool saves developers time by kicking off automated scripts in response to filesystem changes

Further bolstering its reputation as a good open source citizen, Facebook has open-sourced Watchman, a tool developed in-house and designed to help Web developers speed up build times by automatically tracking file changes and kicking off automated responses.

Watchman represents one of several open source projects Facebook has shared with the world, including HipHop VM, an open source virtual machine designed for executing programs written in PHP; PHPembed, a library with a simplified API built on top of the PHP SAPI; and perhaps most famously, the Open Compute Project, the company's initiative to bring open source software collaboration to the world of data center hardware.

Facebook's engineering team developed Watchman as part of an internal process to speed up build times. "Like with many modern and complex Web applications, much of the code behind Facebook goes through a series of build steps to transform it from PHP source code and assets into the running application," wrote Wez Furlong, software engineer at Facebook. "These steps include identifying where source code classes live, mapping Web controller paths, spriting images, using CSS and JavaScript to package it up, and more."

With an app the size of Facebook, "walking over the entire tree to figure out what needs to be built is a non-trivial exercise," according to Furlong.

Thus, the engineering team devised an incremental build-step process, both to simplify and speed up the process. As part of the process, developers created Watchman, which monitors files within on one or more directory trees and records when they change or are deleted. It also maintains its own view of a filesystem and can launch queries and trigger scripts, such as rebuilding assets, in response to observed changes.

"We looked around to see if there were other projects that met our requirements, and while there are numerous projects that can trigger scripts when files change, we didn't find anything that had the right set of properties," Furlong explained.

The tool, which is provided as a single executable binary that acts as both service and client, waits for a watched root to settle down before it will start to trigger notifications or command executions, according to the developers. It's designed to be conservative in its actions in that "it considers the files to be freshly changed when you start to watch them."

Facebook's engineering team used Watchman, which runs on Linux, OS X, FreeBSD, and Solaris, to kick off builds while engineers are editing files. "Between this and more intelligent incremental processing, we've been able to reduce our interactive-user p50 build times by 60 percent," Furlong wrote. "It's not a silver bullet for making builds go fast, but it has helped us to make noticeable improvements both here and in some other internal projects."

This story, "Facebook open sources its Watchman tool for speeding up app builds," was originally published at InfoWorld.com. Get the first word on what the important tech news really means with the InfoWorld Tech Watch blog. For the latest developments in business technology news, follow InfoWorld.com on Twitter.

From CIO: 8 Free Online Courses to Grow Your Tech Skills
Join the discussion
Be the first to comment on this article. Our Commenting Policies