Introducing continuous integration

How continuous is your integration and what could your team be doing to improve it? A JavaWorld excerpt from Continuous Integration: Improving Software Quality and Reducing Risk (Addison-Wesley Professional, June 2007)

1 2 3 Page 3
Page 3 of 3

How does CI complement other development practices?

The practice of CI complements other software-development practices, such as developer testing, adherence to coding standards, refactoring, and small releases. It doesn't matter if you are using RUP, XP, RUP with XP, SCRUM, Crystal or any other methodology. The following list identifies how the practice of CI works with and improves these practices.

  • Developer testing: Developers who write tests most often use some xUnit-based framework such as JUnit or NUnit. These tests can be automatically executed from the build scripts. Since the practice of CI advocates that builds be run any time a change is made to the software, and that the automated tests are a part of these builds, CI enables automated regression tests to be run on the entire code base whenever a change is applied to the software.

  • Coding standard adherence: A coding standard is the set of guidelines that developers must adhere to on a project. On many projects, ensuring adherence is largely a manual process that is performed by a code review. CI can run a build script to report on adherence to the coding standards by running a suite of automated static analysis tools that inspect the source code against the established standard whenever a change is applied.

  • Refactoring: As Fowler states, refactoring is "the process of changing the software system in such a way that it does not alter the external behavior of the code yet improves its internal structure."5 Among other benefits, this makes the code easier to maintain. CI can assist with refactoring by running inspection tools that identify potential problem areas at every build.

  • Small releases: This practice allows testers and users to get working software to use and review as often as required. CI works very well with this practice, because software integration is occurring many times a day and a release is available at virtually any time. Once a CI system is in place, a release can be generated with minimal effort.

  • Collective ownership: Any developer can work on any part of the software system. This prevents "knowledge silos," where there is only one person who has knowledge of a particular area of the system. The practice of CI can help with collective ownership by ensuring adherence to coding standards and the running of regression tests on a continual basis.

How long does CI take to set up?

Implementing a basic CI system along with simple build scripts for a new project may take you a few hours to set up and configure (more if you don't have any existing build scripts). As you expand your knowledge of the CI system, it will grow with the addition of inspection tools, deployments that are more complex, more thorough testing, and many other processes. These additional features tend to be added a little at a time.

For a project already in progress, it can take days, weeks, or even months to set up a CI system. It also depends upon whether people have been dedicated to work on the project. Usually you must complete many tasks when moving to a continuous, automated, and headless system such as when using a CI server. In some cases, you may be moving from batch or shell scripts to a build scripting tool such as Ant or managing all of the project's binary dependencies. In other cases, you may have previously used your IDE for "integration" and deployment. Either way, the road map to full CI adoption could be quite a bit longer.

CI and you

In order for CI to work effectively on a project, developers must change their typical day-to-day software-development habits. Developers must commit code more frequently, make it a priority to fix broken builds, write automated builds with tests that pass 100% of the time, and not get or commit broken code from/to the version control repository.

The practices we recommend take some discipline, yet provide the benefits stated throughout this chapter. The best situation is one where most project members agree that there is an exponential payback to the time and attention they pay to the practices of CI.

There are seven practices that we've found work well for individuals and teams running CI on a project.

  • Commit code frequently

  • Don't commit broken code

  • Fix broken builds immediately

  • Write automated developer tests

  • All tests and inspections must pass

  • Run private builds

  • Avoid getting broken code

The following sections cover each practice in greater detail.

Commit code frequently

One of the central tenets of CI is integrating early and often. Developers must commit code frequently in order to realize the benefits of CI. Waiting more than a day or so to commit code to the version control repository makes integration time-consuming and may prevent developers from being able to use the latest changes. Try one or both of these techniques to commit code more frequently.

  • Make small changes: Try not to change many components all at once. Instead, choose a small task, write the tests and source code, run your tests and then commit your code to the version-control repository.

  • Commit after each task: Assuming tasks/work items have been broken up so that they can be finished in a few hours, some development shops require developers to commit their code as they complete each task.

Try to avoid having everyone commit at the same time every day. You'll find that there are usually many more build errors to manage because of the collisions between changes. This is especially troublesome at the end of the day, when people are ready to leave. The longer you wait to integrate with others, the more difficult your integration will prove to be.

I just can't commit: A friend runs a 25-developer project and he'd like to incorporate many CI practices, but he is experiencing challenges in getting the developers to commit code frequently. I've found that the main reason that changes are not committed frequently is because of the project culture. Sometimes developers do not want to commit their code until it is "perfect." This usually happens because their changes affect too many components. Committing code frequently to the version control repository is the only effective way to implement CI, and this means that all developers need to embrace this development practice by grabbing smaller chunks of code and breaking up their tasks into smaller work items.

Don't commit broken code

A dangerous assumption on a project is that everyone knows not to commit code that doesn't work to the version control repository. The ultimate mitigation for this risk is having a well-factored build script that compiles and tests the code in a repeatable manner. Make it part of the team's accepted development practice to always run a private build (which closely resembles the integration build process) before committing code to the version control repository. See the later section, Run Private Builds, for additional recommendations before committing your code.

Fix broken builds immediately

A broken build is anything that prevents the build from reporting success. This may be a compilation error, a failed test or inspection, a problem with the database, or a failed deployment. When operating in a CI environment, these problems must be fixed immediately; fortunately, in a CI environment, each error is discovered incrementally and therefore is likely very small. Some projects have a penalty for breaking the build, such as throwing some money in a jar or placing the picture of the last developer to break the build on the company's large-screen monitor (just kidding; hopefully no one is doing this). The project culture should convey that fixing a broken build is a top project priority. That way, not just some but every team member can then get back to what they were doing.

Write automated developer tests

A build should be fully automated. In order to run tests for a CI system, the tests must be automated. Writing your tests in an xUnit framework such as NUnit or JUnit will provide the capability of running these tests in an automated fashion. Chapter 6 provides details on writing automated tests.

All tests and inspections must pass

In a CI environment, 100% of a project's automated tests must pass for your build to pass (this is a technical criterion, not an expectation that all workers or all work should be perfect). Automated tests are as important as the compilation. Everyone accepts that code that does not compile will not work; therefore, code that has test errors will not work either. Accepting code that does not pass the tests can lead to lower-quality software.

An unscrupulous developer may simply comment out the failing test. Of course, this defeats the purpose. Coverage tools assist in pinpointing source code that does not have a corresponding test. You can run a code coverage tool as part of an integration build.

The same goes for running automated software inspectors. Use a general rule set of coding and design standards that all code must pass. More advanced inspections may be added that don't fail the build, but identify areas of the code that should be investigated.

Run private builds

To prevent broken builds, developers should emulate an integration build on their local workstation IDE after completing their unit tests. This build allows you to integrate your new working software with the working software from all the other developers,6 obtaining the changes from the version control repository and successfully building locally with the recent changes. Thus, the code each developer commits has contributed to the greater good, with code that is less likely to fail on the integration build server.

Keep builds in the "green": I find that there are two measures of using CI effectively: number of commits and build status. Each developer (or pair) should have at least one commit to the repository per day, and the number of checkins usually demonstrates the size of the changes (more commits usually means smaller changes-and this is good). Your build status should be "green" (pass) a large percentage of the day; set this value for the team. We all get a "red" build status sometimes, but what's important is that it's changed back to green as soon as possible. Never let your team get used to waiting in the red status until this or that other project task is done. The willingness to leave the status at red for other criteria defeats much of the strength of CI.

Avoid getting broken code

When the build is broken, don't check out the latest code from the version-control repository. Otherwise, you must spend time developing a workaround to the error known to have failed the build, just so you can compile and test your code. Ultimately, it's the responsibility of the team, but the developers responsible for breaking the build should already be working on fixing their code and committing it back to the version control repository. Sometimes a developer may not have seen the e-mail on the broken build. This is when a passive feedback mechanism such as a light or sound can be useful for collocated developers. We consider it critical that all developers know the state of the code in the version control repository. For more information on continuous feedback mechanisms, see Chapter 9. An alternative, but not preferable, approach to avoiding a checkout is to use the version control system to roll back any changes since the most recent commit.


Now you have the ammunition to go talk to others about CI. This chapter covered some of the basics of CI, discussed how to get to a continuous process, and pointed out all the other areas that get explored in detail in subsequent chapters. Table 2-1 summarizes seven practices to follow when using CI. The next chapter delves into the software risks that CI can help mitigate to improve quality.

TABLE 2-1 CI practices discussed in this chapter



Commit code frequently

Commit code to your version-control repository at least once a day.

Don't commit broken code

Don't commit code that does not compile with other code or fails a test.

Fix broken builds immediately

Although it's the team's responsibility, the developer who recently committed code must be involved in fixing the failed build.

Write automated developer tests

Verify that your software works using automated developer tests. Run these tests with your automated build and run them often with CI.

All tests and inspections must pass

Not 90% or 95% of tests, but all tests must pass prior to committing code to the version-control repository.

Run private builds

To prevent integration failures, get changes from other developers by getting the latest changes from the repository and run a full integration build locally, known as a private system build.

Avoid getting broken code

If the build has failed, you will lose time if you get code from the repository. Wait for the change or help the developer(s) fix the build failure and then get the latest code.


Practicing CI is more than installing and configuring some tools. How many of the following items are you consistently performing on your project? How many of the other CI practices can improve your development capabilities?

About the authors

Paul M. Duvall is the CTO of Stelligent Incorporated, a consulting firm and thought leader in helping development teams reliably and rapidly produce better software by optimizing software production. He has worked in virtually every role on a software development project, from developer and tester to architect and project manager. Paul has consulted for clients in various industries including finance, housing, government, health care, and large independent software vendors. He is a featured speaker at many leading software conferences. He authors a series for IBM developerWorks called Automation for the People, is a coauthor of the NFJS 2007 Anthology (Pragmatic Programmers, 2007), and is a contributing author of UML 2 Toolkit (Wiley, 2003). He is a co-inventor of a clinical research data management system and method that is patent pending. He actively blogs on and

Stephen M. Matyas III is the vice president of AutomateIT, a service branch of 5AM Solutions, Inc., which helps organizations improve software development through automation. Steve has a varied background in applied software engineering, including experience with both commercial and government clients. Steve has performed a wide variety of roles, from business analyst and project manager to developer, designer, and architect. He is a contributing author of UML 2 Toolkit (Wiley, 2003). He is a practitioner of many iterative and incremental methodologies including Agile and Rational Unified Process (RUP). Much of his professional, hands-on experience has been in the Java/J2EE custom software development and services industry with a specialization in methodologies, software quality, and process improvement. He holds a bachelor of science degree in computer science from Virginia Polytechnic Institute and State University (Virginia Tech).

Andrew Glover is the president of Stelligent Incorporated, a consulting firm and thought leader in helping development teams reliably and rapidly produce better software by optimizing software production. Andy is a frequent speaker at various conferences throughout North America as well as a speaker for the No Fluff Just Stuff Software Symposium group; moreover, he is the coauthor of Groovy in Action (Manning, 2007), Java Testing Patterns (Wiley, 2004), and the NFJS 2006 Anthology (Pragmatic Programmers, 2006). He also is the author of multiple online publications including IBM’s developerWorks and O’Reilly’s ONJava, ONLamp, and Dev2Dev portals. He actively blogs about software quality at and



[2]Based on Software Configuration Management Patterns by Stephen Berczuk and Brad Appleton.



[5]Fowler, et al. Refactoring: Improving the Design of Existing Code (Reading, MA: Addison-Wesley, 1999).

[6]Some configuration management tools, such as ClearCase, have an option to automatically update your local environment with the changes from the version control repository (called "dynamic views" in ClearCase).

This content is an excerpt from the new book titled Continuous Integration: Improving Software Quality and Reducing Risk, authored by Paul Duvall, Steve Matyas, and Andrew Glover, Copyright 2007 Pearson Education, Inc., published by Addison-Wesley Professional, June 2007, ISBN 0321336380. For additional information, please visit:

Copyright (c) 2007 Pearson Education. All rights reserved.

Learn more about this topic

This story, "Introducing continuous integration" was originally published by JavaWorld.

Copyright © 2007 IDG Communications, Inc.

1 2 3 Page 3
Page 3 of 3