With virtualization driving the meteoric rise in server instance counts, it's no wonder that so many configuration automation tools are hitting their stride. Where once we either did without configuration management or created our own custom scripts, we now have many options to manage changes to large numbers of servers.
One of the best options is Puppet Enterprise, the commercial version of the Puppet open source project. Version 3.0, just arrived, passes along many advances borne from the community edition, as well as new features found only in the enterprise version.
[ Puppet or Chef: The configuration management dilemma | 12 clever cloud tools for devs and ops | Subscribe to InfoWorld's Data Center newsletter to stay on top of the latest developments. ]
If you're already well versed in Puppet, then this version will bring with it welcome changes. The first is the speed. Although I wasn't able to fully corroborate Puppet Labs' claims of a 60 percent faster catalog core compilation, 2.7x speed in agent runtimes, or the heralded "200 percent performance boost," the reality is definitely close to that. For longtime Puppet users with large-scale deployments, the speed improvements are a very big deal, as they will reduce the load on both Puppet servers (that is, Masters) and managed nodes, as well as the time required to roll out changes across the board.
Puppet Enterprise 3.0 also introduces a new orchestration engine that allows for more granular and more stable large-scale changes. As compared to earlier versions, the new orchestration features include finer-grained filtering to select subsets of systems to modify or directly control, as well as batching functions to permit a gradual rollout of changes rather than performing them all at once.
Working with Puppet requires learning Puppet's configuration language and knowing your way around Puppet's classes and modules, which are organized in configuration files in a specific hierarchy under /etc/puppetlabs/puppet. Without going into the heavy details, you can consider all modules to be includes in a big script, with classes defined and linked to specific server nodes or groups of nodes. This provides a very straightforward method of dealing with the complex and detailed requirements of managing multiple server platforms and distributions, not to mention switching and storage applications.
The Web UI included with Puppet Enterprise is well appointed and straightforward, allowing for the creation and management of nodes, groups, and classes, as well as report generation and viewing. It is a useful tool for manipulating node organization and quickly determining issues with certain parts of the framework. It also provides limited Live Management of nodes or groups of nodes. However, for many implementations it may be easier to think of the UI as read-only, and to work mostly through the command line. The deeper into Puppet you get, the more likely this will become, as you may discover that you can only apply certain configurations and modules to nodes and groups through the CLI, and these changes may not be reflected in the Web UI.
An example might be a module downloaded through the Puppet Forge module-sharing site that cannot use a class due to the nature of what it's configuring. In this case, you will need to apply the module to nodes or groups from within the Puppet configuration files, as you can't define a class that maps to a class definition within the Web UI. This configuration element will thus not appear in the Web UI.
I also encountered a few bugs in the Web UI, such as the fact that you can add a group with a name containing a period, but you cannot thereafter edit or delete that group through the UI. This would seem to be an input validation bug, because it appears that a period is an illegal character in group names.
Standing up the Puppet
Installing Puppet is conceptually very simple. Choose a supported platform for your Puppet Master server, and run the install script. This will prompt for a variety of packages and options to be installed: the Puppet Master, PostgreSQL, Cloud Provisioner, and so forth. The install script will also prompt for a username and password for the first administrator in the Web UI.
These quibbles and pitfalls aside, the initial configuration is very straightforward, requiring DNS for all nodes and (obviously) clear network connectivity. Once the Puppet Master is installed, each node is configured essentially the same way, by running an installer for that node's platform, which generates a certificate and contacts the master server for approval. When the node is approved, it becomes configurable through Puppet and can be added to groups or bound to classes through the Web UI or CLI.
Then comes the harder part: configuring Puppet to make changes to those systems. This is done through modules, either custom-coded or downloaded from the Puppet Forge site and customized.
Puppet in action
As a very simple example of how Puppet works, let's look at how we would use Puppet to make sure that the NTP (Network Time Protocol) service is properly configured and running on a selection of nodes.
Assuming we already have the nodes approved, we need to download a module, such as the third-party
erwbgy-ntp module, written by Keith Burdis. We do this by installing the module from the command line:
puppet module install erwbgy-ntp
To apply this module to a node, we would then define a new class in the Web UI called
ntp, edit a node group or a node itself, and add that class to the group or node. This will cause Puppet to apply that configuration to the node through the Puppet agent installed on the node.
By default, the agent checks in with the Puppet Master every 30 minutes, but of course that is a configurable parameter. The node can also be forced to update through the Live Management tab in the Web UI. Live Management allows an admin to enable, disable, run, and monitor the Puppet agent on a node or group of nodes. We can select a node to refresh, then force the refresh. If we select the node we configured with our
ntp module, it will check in with the server, download the module information, install the
ntp package, modify the configuration file, and start the service.
If somewhere down the line, the node checks in with the master and discovers the configuration has been manually changed or the service has been stopped or uninstalled, the agent will again apply the changes and start the service back up.
This is a very simple example of what Puppet can do. There are hundreds of modules available from Puppet Labs and contributors that can be used to manage a wide variety of configuration elements on Linux and Windows systems, across a wide variety of distributions and versions.
If we were to do the same thing for Windows servers, we might use the
winntp module by Adam Denning. This is configured similarly on the Puppet Master:
Reports, inventory, and Live Management
The Web UI also provides reporting and inventory features. Through a tabbed UI, you can view not only at-a-glance information on how Puppet itself is running on all nodes, but also detailed info on users, groups, installed packages, and running services on any given selection of nodes.
You can also perform certain tasks across nodes or groups of nodes from Live Management. Aside from enabling, disabling, and running the Puppet agent on demand, you can cause a node or group of nodes to install a package, start a service, or stop a service, or you can use
yum to update the server. There are also facilities to perform queries of the agent on a node or various
rpcutil tasks, such as retrieving individual configuration items.
As far as the Puppet reporting goes, you get a tabular display of nodes and their run status. You might see a few nodes with failed updates, for example, and you can quickly dig into the logs to see what failed to apply. You can also get detailed reports on each agent run that provide in-depth info on what did or did not happen during that run. In addition, you can see how long it took to make the run, how many modifications were made, and so forth.
You can also search through inventory items across nodes. If you wanted to display a list of all nodes running kernel version 2.6.32, for instance, you would simply query for inventory item
kernelversion = 2.6.32. By default Puppet collects a ton of inventory data, from BIOS version to system uptime to swap space to SELinux policy enforcement.
Powerful means complex
Puppet is not for the faint of heart, nor should it be. Shepherding the configurations of a wide range of services and elements, managing package installation and removal, and collecting inventory data from dozens, hundreds, or thousands of servers is an inherently difficult task. Add to that the ability to heavily customize existing modules and to develop your own modules according to your own specifications, and you have a recipe that requires a great deal of complexity. Trying to obscure that complexity with overly simplistic, high-level configuration interfaces might make easy things easier, but it would make hard things impossible.