CI/CD as a service: 10 tools for continuous integration and delivery in the cloud

From Bitbucket and GitLab to AWS CodePipeline and Travis CI, look to these 10 cloud services to automate your software builds, tests, and deployments

CI/CD as a service: Continuous integration and delivery in the cloud
Aleksei Derin / Getty Images

The cloud and continuous integration (CI) are a natural match. While the cloud frees us from the pain of installing and maintaining physical servers, continuous integration automates away much of the pain of building, testing, and deploying our code. If both aim to take work off the shoulders of development teams, it only makes sense to combine them and eliminate even more drudgery with one step.

There are many continuous integration services and they all do much the same thing, at least in an abstract sense. They begin with a list of the tasks like compiling or testing that must be accomplished before the world can appreciate the genius in your new software. When you commit your lines of code, the tools start working through the checklist until they run into a roadblock. If there are no roadblocks, everyone is happy.

Anyone can use continuous integration for any software development project, but the biggest advantages are enjoyed by teams, preferably large teams that are working on the same, interlocking blocks of code. The most thorough implementations of continuous integration build and rebuild the code before testing and retesting it, all in search of new errors and incompatibilities that may have been created as different team members check in their code. Continuous integration servers synchronize the work of all of the programmers and help the team detect any problems. 

Some lists of tasks for the CI server end with the tests, but lately more and more teams are extending the lists to include deployment of the new code, a process sometimes dubbed “continuous deployment.” Completely automated deployment makes some people nervous and they will often add some manual pauses in the process. Adding a bit of accountability and human assurance lets them relax a bit. They’ll call this hybrid approach “continuous delivery” because it delivers the code to some staging or testing cluster where it will wait for a human to make the final push to production. 

If continuous integration is great in the server room down the hall, it can be even better in the cloud where there are great opportunities for faster delivery and greater efficiency. In the best cases, the clouds can split up the work and run the tasks in parallel. The services start with a large pool of hardware and then share it among many teams. As long as everyone isn’t pushing their code at the same time, the builds and tests will run much faster. Purchasing the same huge rack of hardware for just the moments when the developers want to run all of the tests is prohibitive, but if the teams share the rack they can all enjoy the bursts of speed.

There are dangers and worries, though, and the biggest can be loss of control. All of the cloud services require handing over your code to a third party, a choice that may feel liberating to some but frightening to others. All of the cloud services try hard to emphasize security, but somehow it feels different when the code is under your own roof.

In addition to broad support for all of the major languages, these services cover a surprising number of the minor ones and more than a few truly strange and unusual ones. This is more the result of good architectural decisions at the beginning than any heroic effort by the developers. The lists of tasks are almost always encoded as commands for some shell or command line, so the continuous integration tools pretty much keep issuing the commands until the list is exhausted or some insurmountable roadblock appears. Some of the languages like Java offer more sophisticated options but for the most part the tools can accomplish anything that you can do with a command line. 

Here are 10 different options for doing continuous integration in the cloud.


CloudBees Core started with Jenkins, the well-known open source project for continuous integration, then added testing, support, and some assurance that the code will just run. The company winnowed out all of the experimental plugins, added a few of their own, and then polished the right ones so they work as expected when you need them.

CloudBees still employs 80 percent of the Jenkins development team and they frequently contribute code to the open source project, so you can be sure they’ve got a good understanding of this dominant platform. To speed things up, CloudBees also added extensive parallelization as well as instrumentation to track your development process.

CloudBees offers a variety of price points that range from free tiers to “starter kits” for a full year of the service. The company also breaks out support for Jenkins for anyone who needs help with the tool but doesn’t need or want the cloud computing.

AWS CodePipeline

Amazon’s tool for continuous integration and deployment, AWS CodePipeline, is optimized for delivering code to an AWS server while still being open to more elaborate pathways for your code and data. The basic tool offers a nice selection of pre-configured build environments for the major languages (Java, Python, Node.js, Ruby, Go, Android, .Net Core for Linux) and then dumps the result in an S3 bucket before sending it off to a server to start running.

There are a surprisingly large number of layers with slightly different names. CodeBuild grabs your latest genius from CodeCommit when it’s triggered by CodePipeline and then hands off the result to CodeDeploy. If that’s too many Code things for you to configure, you can jump right to CodeStar, which offers another layer of automation. If only there were a CodeBugEraserStar to wipe away all of our mistakes automagically too. It’s worth noting that you don’t technically pay for any of these Code layers. Amazon bills you just for the compute and storage resources used along the way. It’s not exactly free, although it does feel like it.

Bitbucket Pipelines

Atlassian, the developers of the popular job tracking board, Jira, and code repository, Bitbucket, decided to leverage their hold on our workflow by creating Bitbucket Pipelines, a continuous integration tool in the Bitbucket cloud. The secret sauce is more integration, in this case in the form of connections between the build mechanism and Atlassian’s other tools. At least cosmetically, Pipelines isn’t even a separate thing. It’s just another menu option for each project in Bitbucket. Another menu option points to deployments, letting you select where the builds end up.

The connections are a blessing and a limitation. If you choose one of the templates already defined for the major languages (Java, JavaScript, Python, PHP, .Net, etc.), you can be building and deploying your code in a few clicks. But if you drift away from the standards, you’ll start to find that the options aren’t there. Atlassian does encourage a marketplace of apps that seem to be a mixture of charts and webhooks into other services. The top app on the chart as I write this will connect Bitbucket with Jenkins, presumably to do something that can’t be done quickly inside the walls.

The principle advantage of Pipelines is speed. Atlassian has pre-engineered most of the major pathways from code to running deployments and you can follow in the company’s footsteps for just a few dollars. It’s hard to compare the cost of using Bitbucket because the builds are priced in minutes, like most serverless models, but teams often dedicate a cluster of instances to handle Jenkins builds. Even if you shut them down on nights and weekends, the hours add up.

GitLab CI/CD

One of the biggest competitors to Atlassian is GitLab, another company that wants to handle every step of the process between your fingers and running deployment. GitLab’s build, test, and deployment mechanisms are likewise connected directly to its Git repositories so they can be triggered on commitment. The process is largely built around Docker containers and this caching can greatly simplify some of the configuration work that must be done around Jenkins builds.

The build tasks can target any language but must be triggered by GitLab Runner, an autoscaling tool written in Go that’s ready for most platforms. This flexibility means you can trigger any random job on other machines, something that might be useful with elaborate architectures that do more than just deliver microservices.

The pricing is bundled in with the various tiers to approximate need. Gold tier groups, for instance, get all of the best features like security dashboards and 50,000 minutes of building on the shared cluster of machines. There’s no charge for using your own machines for part of the process or separate instances in some other cloud.


Many of the continuous integration tools focus on code that can be built in the Linux environment. CircleCI builds and delivers in the Linux world, but it also offers a product that will build Android apps and anything that comes out of Apple’s Xcode (for iOS, MacOS, tvOS, or watchOS). If you’re working on a team that is producing apps for these platforms, you can commit your code and let CircleCI enforce some testing discipline on all of the divergent genius of your team.
The lists of tasks are spelled out in YAML files. CircleCI uses Docker, in all of its multi-layered glory, to configure the test environments for the code. The builds start with fresh containers and so do all of the tests. The Mac work runs in virtual machines that have a similarly short lifespan. This avoids some of the problems with configuration because the clean environments don’t have any leftover bits lying around. (So if your problems are caused by lingering digital flotsam, well, that’s your fault.)

The pricing is focused on how much CPU your builds suck down. The number of users and number of repositories are capped at infinity. The number of build minutes and containers doing this building, though, are metered. The first container is free and you can run one build in it. If you want more parallelism or more throughput, CircleCI gets to make some money. The Mac users don’t get the same free deal, but there are introductory plans for anyone testing out the service.

Travis CI

If your builds produce code that needs to be tested on Windows boxes, then Travis CI offers you a single stop. The company has offered MacOS and Linux options for some time but has just rolled out the Windows option, making it simpler to produce code that runs in even more places.

The task lists are also spelled out in YAML and the jobs are run in clean virtual machines with fairly standard configuration. Linux code gets some basic versions of Ubuntu, the Mac code runs in one of a dozen combinations of OS X and Xcode and JDK. The Windows code can only end up in one version of Windows Server (1803) for now. Travis CI offers a long list of 30 languages and build rules that are pre-configured and pretty much ready to run.

The pricing is based on how many concurrent jobs can execute at once but there are no formal limits on the number of minutes that these builds can take up. It’s like you get a fixed number of dedicated instances for your work and they’re ready all of the time. There are no free options for proprietary work, but open source projects are “always free”—so that may be the simplest way to try Travis CI out.

Azure Pipelines

If you wonder if the modern Microsoft has a “Not invented here” attitude, look no further than Azure Pipelines. The sales literature says, “Any language, any platform.” While this is almost certainly a bit of hyperbole and Azure probably doesn’t have much to offer the ENIAC programmers, it does prominently offer Microsoft, Linux, and MacOS paths for your code. The Apple corner targets only MacOS builds, not iOS or tvOS or watchOS, but let’s not get picky. This is a glass that’s much more than half full.

In the abstract, the system is similar to the others. There are agents that execute builds to produce artifacts. Some of these can be self-hosted if that option helps. The stack fully embraces Docker containers and Azure’s hardware is ready to run them for you. All of these details can be clicked together with a visual designer built into a webpage, or specified with YAML if you prefer living in the command-line world.

The pricing comes with a free “parallel job” with 1800 minutes of build time. If you want more parallelism or more built time, you start to pay. The plan includes a generous free tier for open source projects, again emphasizing Microsoft’s desire to take part in the general open source community. But if Microsoft is going to spend $7.5 billion to buy a seat at the table by acquiring GitHub, well, it makes plenty of sense. Where will all of this code run? Azure Pipelines will be happy to move it smoothly to Azure hardware.

1 2 Page 1
Page 1 of 2