As “software eats the world,” to use Marc Andreessen's phrase, versioning becomes critical to business success because corporate value is often contained in the software assets created by the company. As such, companies need to balance speed, flexibility, and collaboration with the need to lock down and protect that intellectual property.
Yet, as companies embrace devops, they find friction between the teams responsible for development and those responsible for delivery. In short, the demands of devops require deeper functionality and more collaborative capabilities than traditional version control systems deliver. To minimize this friction, Perforce Helix provides a range of capabilities that support consistent, collaborative development and delivery. In this article, we’ll take a look at how Perforce Helix helps companies implement successful devops by harmonizing workflow among teams.
Back in the day
Remember the build team? In the past they lived in a world of Cron, Bash, Perl, and nightly builds. It’s hard to believe in today’s world that 24-hour turnaround was once considered standard practice. A whole day’s worth of changes, across all developers, would be compiled and built overnight. In the best case, QA would test the build the following morning. Often the build would break in the middle of the night, in which case it was back to square one and another 24-hour cycle before testing even began.
In that world, developers threw code “over the wall” to the build team, and assuming the build didn’t break, the executable code got passed along to QA to see if it actually delivered what was promised -- not necessarily what the customer requested. Eventually, assuming it had jumped through all of the necessary hoops, the application would turn up on a production server for the operations team to manage and maintain.
When agile methods displaced waterfall, developers sought to complete coding tasks quickly, releasing in small increments and validating changes immediately via unit tests. Quicker releases increased pressure on the operations team, whose processes were intended for six-, 12-, or even 24-month release cycles. How could they keep up with such rapid change without compromising service quality or uptime?
Automation became a fundamental requirement to handle the new pace of change. Perhaps counterintuitively, increasing the pace actually helped improve customer satisfaction and quality. Removing risky manual processes, making incremental changes, and being able to adjust product plans in response to changing market and customer demands improved customer satisfaction.
Key to making this transition successful for many organizations has been the adoption of a devops approach. Bringing developers and operations experts together as one team benefits both sides. Developers gain an understanding of the whole operating environment stack, so they build applications that can be easily deployed and managed. At the same time, operations staff gains more visibility into the changes being made and can prepare to deploy and manage them as quickly as possible.
Versioning for devs, ops, and extended teams
Having a shared view of the systems being worked on is key to the tight integration between all the contributors in the devops process, which includes other key players such as designers, testers, documenters, and project managers. This shared view is what agile proponents call a “single source of truth.” The temptation is for each of these teams to select tools and storage repositories they prefer without worrying about being able to share with others. Unfortunately, the tools they use may actually inhibit or prevent that sharing in many cases. For example, a source code management tool such as Git was never intended to manage large chip designs or build artifacts.
Perforce Helix was built to be that single store that can manage all types of digital assets, regardless of type, size, or how they were created. Besides the actual assets or files pushed into the devops pipeline or created by the build system, the automation scripts and environment configuration should be versioned as well. Helix was designed to store all of these critical assets.
Version control is the bedrock of the entire delivery process, and as such it can serve as a single source of truth for the other components of the delivery toolchain. Helix provides capabilities that go beyond branching, helping your team manage the versioned assets flowing into a release. This results in support for both the development and the delivery stages of the process. The features enable effective and secure collaboration and sharing all the way down the delivery pipeline.
Managing the flow of change
Unlike typical source code management tools, Helix has intrinsic awareness of an asset’s place in the overall lifecycle. A key feature of Helix is streams. Streams are “branches with brains” in the source code repository. Helix Streams arranges branches in terms of stability: The most stable shown at the top of a graph and the least stable shown at the bottom. The flow of change needs to support this model by merging down and copying up.
Typically, when you work with streams, you define and populate the main stream first, then create development streams and release streams as children thereof. A stream’s type and its relationship to other streams determines what sort of changes can be made and in what order changes flow from one stream to another.
The Perforce Visual Client, aka P4V, represents related streams as shown below:
A typical streams diagram showing the flow of changes.
The children of main are shown both above and below main. Release streams are above, being more stable, while development and task streams are below, being less stable. Again, the idea is that changes flow from the bottom to the top, from least stable to most stable. The direction and color of arrows linking streams indicate both the direction and order of flow.
Combined with Helix’s support for Git, which we’ll dive into next, streams allow development teams to remain in sync with teams responsible for delivery.
Helix and Git: Best of both worlds
I have discussed some of the limitations of Git for devops in another article. The short version is that Git’s practical limitations lead to “Git sprawl,” a condition in which development teams create lots of small repositories to maintain acceptable performance. Such sprawl helps to compensate for Git’s poor support for large files, its lack of “narrow cloning,” and its total lack of access control. Perforce addresses these issues with Helix GitSwarm.
GitSwarm is an all-in-one, project-based Git management system that delivers the functionality of tools such as GitLab or GitHub, but as an integrated component of the Helix platform. The GitSwarm master Git repo is automatically mirrored into the Helix repository and used by all other contributors as well, not only developers using Git.
GitSwarm includes a range of developer productivity tools needed for efficient project management workflows. These tools include project and group definitions, merge-request workflow, a helpful wiki, code snippets, issue tracking, Web hooks, analytics, and integration with enterprise authentication systems. It offers the usual code review system integrated with merge-request workflow that developers using Git have come to prefer.
GitSwarm and narrow cloning
“Narrow cloning” refers to the ability to clone a subset of an existing repository. It is useful when large distributed teams are working on large projects combining code with other digital assets. For distributed teams, narrow cloning is a huge benefit. Users of Git, Mercurial, and other distributed version control systems have longed for narrow cloning for years, largely due to problems with handling many files and large files. Native Git users remain out of luck. Mercurial users can look to some community extensions and hacks, but they’re not much better off.
In Helix, it is straightforward to interact with only a single file, such as a huge CAD file. You can work on your massive file without having to download everything else, which is great for engineers and designers. Likewise, developers may clone only a subset of files that they are working on. Helix’s “repo-remapping” capability allows developers using Git to do the same: select only the folders from the master monorepo that they need for their current work to create their local repo. When they push back to the GitSwarm master repo, the changes are all put back in the right place in the Helix depot. This enables your team to organize the content to suit their needs without losing the big picture of what’s happening across all contributors and projects.
Collaborative development and delivery
Devops is first and foremost about communication. That’s why it’s easier to pull off devops in smaller organizations where the communication overhead can be held to a minimum. Larger organizations require collaborative capabilities that go beyond those provided by typical Git services.
Perforce Helix facilitates collaboration for any organization by providing a shared repository that both development and operations teams can use with protections that prevent inappropriate access or change. This allows developers to control what gets rolled into production, whereas operations can see what changes are coming. The repo shows ops all of the changes that have been made, along with versioning all of the automation scripts, environment configurations, and documentation the ops team needs to deploy and manage the application or service.
Collaborative development with Helix Swarm
For contributors not using GitSwarm, Helix Swarm provides a browser-based collaboration tool with a focus on ensuring that high-quality changes are committed to the project mainline. The primary function of Swarm is to manage code reviews and conversations around changes being made. As part of a continuous integration pipeline, code reviews can be requested when changes are made.
The reviews may be performed by multiple people (typically the more experienced members of the team) to ensure that code complies with standards and meets requirements for efficiency and easy maintenance. Conversations can be held within Swarm (“Why was the change made this way?” or “Perhaps you could try …”) and stored in the Helix repository, creating an audit trail and sharing code knowledge. Swarm’s immutable shared history is an ideal tool when trying to understand why a piece of code was written in a particular way long after the fact. Integration with continuous integration tools means that build and smoke tests can be executed before anyone has to review the code.
Activities and code review conversations are held in the Helix Swarm repository.
Keeping it all secure
Most managers don’t want to admit it, but continuous delivery has the potential to open up lots of security holes. The more users of the system who need to access assets at multiple points in the delivery pipeline, the harder to implement “least privilege” access control.
When things go wrong, developers and operations teams might share credentials to get something running or to spin up new environments. Devops users typically wind up with more access rights than they need. As noted earlier, Git always requires developers to download the whole repository -- necessarily with full access rights to create and delete.
Perforce Helix provides solutions for these problems. Helix supports fine-grained access control over the assets in a repository. Information can be maintained on a need-to-know basis through the entire asset base. Protections determine which Helix commands can be run, on which files, by whom, and from which client. In short, Helix not only provides extremely fine-grained control, but also maintains an immutable history of who did what and when. The latter provides high-integrity audit logs that can be imperative, particularly in regulated industries.
Give Helix a try
Companies need to create a seamless environment where speed, flexibility, and security don’t get in the way of employee collaboration. The capabilities that Helix provides both resolve that conflict and support the collaborative development and delivery needed for a frictionless devops world. You can download an evaluation version of Helix or evaluate Helix in the cloud with no local server installation required. For detailed information on how to use Perforce Helix, you can visit the Helix documentation.
John Williston is a veteran software developer for Windows, .Net, and the Web. He is currently a product marketing manager at Perforce Software.
New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.