Review: 3 Web stack monitors in the cloud
Monitoring services from Boundary, Circonus, and Librato combine simple setup and richly different capabilitiesFollow @peterwayner
Librato: Simple metrics or resource control
The Librato world comes in two parts. The first part, called Metrics, is an online database for whatever you measure. It's connected to graphing and alerting engines. The second, Silverline, offers the instrumentation tools that dig up the numbers you might want to watch on your server. You can use one without the other or use them both, but they're relatively independent tools that don't use each other. If you log into one, you won't automatically be logged into the second.
The Metrics system is a Web-based graphing package for any kind of data you want to collect. You gather together your information in a JSON package and POST it to the Metrics URL. The next thing you know, your graphs are changing as the new data arrives.
Metrics is a flexible system because most software has the ability to send off an HTTP message today. JSON-based logging is becoming common, even for C-level programming. It took me only a few seconds to start sending messages via the Curl command line. The biggest glitch seemed to be caused by the fact that I wasn't including decimal points in my integers.
The Metrics look is quite pretty and very simple by design. There are few buttons and few options, echoing the design style of companies like 37signals. Much of the complexity is hidden beneath the surface, and the Metrics folks clearly want to build something that "just works." If you POST a new measurement in your JSON package, Metrics lists it with the others. There's no need for updating schema or adding another line to a configuration file -- whew! If the data shows up, Metrics assumes you want to track it.
This approach is ideal if all you want to do is keep track of basic statistics and watch them scroll across your screen. You get two types of data: gauges and counters. The graphs have a few basic date ranges. This confused me at first because I was starting with sample scripts with old dates that weren't appearing. While Metrics is smart enough to recognize new data fields, it loses data with timestamps outside of the current window.
In addition to JSON/Curl and a number of open source collection agents, Librato points to several more libraries and packages for assisting in reporting. The collection is heavily skewed toward modern, trendy stacks such as Node.js, Clojure, and Erlang. But in a tip of the hat to those who graduated more than 10 minutes ago, Python and PHP are also supported. Java? You can be the first to contribute this code, but there's some for handling JMX.
The Silverline half of Librato is aimed at managing servers at the OS level. It burrows into the OS and sets itself up to watch over the applications. While other tools focus on the system as a whole, Silverline wants to drill down into the application.
The simplest thing that Silverline does is report information about resources to their central server. It tracks basic numbers such as CPU load, network bandwidth, and disk bandwidth to help you get a handle on what the machines in your cluster are doing.
This is just the beginning -- Silverline also sneaks its way into the scheduling algorithms controlling the server. Once you put an application in a Silverline "container," Silverline can limit the resources it can consume, a process Librato calls "workload virtualization." Background tasks such as disk defragmentation can be given minimal headroom. High-computation tasks like converting photos or videos can also be limited so that a burst of uploads won't swamp your machines. You can control these spikes more easily at the OS level.
Librato Silverline monitors the essential metrics of the system. This graph shows the rise in network write.