Ever since Fred Brooks described the failure of traditional management practices applied to software development in his world-changing book, “The Mythical Man-Month,” we have aspired to deliver software faster and cheaper while also achieving higher quality. The agile methodology introduced simple, easy-to-follow ideas that revolutionized how teams approach software delivery.
Those ideas, which focused on combining good software discipline and high-performance team practices, not only enabled software to be delivered faster, but also made the whole process more enjoyable than traditional, sequential, task-oriented approaches. With the success of agile at the team level, organizations started looking at how they could extend agile to more teams and combine it with other software delivery disciplines. But scaling agile is difficult because you cannot create only one big team. Rather, you have to apply agile across teams, disciplines, and sometimes organizations.
But agile was designed for teams, not for organizations. How do you enable flow, collaboration, and governance across team and organizational boundaries without compromising the underlying methods that make agile agile? Here is a step-by-step guide to enabling a scaled agile software delivery lifecycle.
Hybrid agile process models
Agile has become the standard way for software delivery teams to organize and manage their work. But as a rule, agile starts and ends at the team. The agile team is invariably surrounded by processes and groups that are not only un-agile themselves, but require the agile team to adopt un-agile processes. The reality for most organizations is water-scrum-fall, with agile processes fitting into a broader, more sequential process that plans and delivers software infrequently. This mismatch results not only in agile teams becoming frustrated, but in reducing overall efficiency, effectiveness, and business value.
But the world of water-scrum-fall is slowly changing with the introduction of new process models and approaches that either enable agile to integrate more effectively with the needs of the traditional enterprise or add new techniques that scale agile practices for practitioners outside of the agile team.
Perhaps the best known among these new process models are SAFe (Scaled Agile Framework) and DAD (Disciplined Agile Delivery). But these two high-profile models are not the whole story behind agile at scale. Recently Ken Schwaber and Scrum.org introduced Nexus, a scrum-based approach to scaling agility. Scaling seems to be the most important problem to solve for the agile community, and all of these approaches include great ideas necessary for solving this problem. However, process knowledge is not enough for a number of reasons:
- Organizations are often a chaotic collection of practices, people, and culture that makes any change difficult to manage and implement. For all of their fancy offices, processes, logos, and management consultants, organizations are simply a confused collection of people trying to deliver a product or service to another confused collection of people -- who can be counted on to change their requirements and needs. Agility at scale requires some level of change, but it's difficult without a level of stability in the groups that are changing.
- Practices are codified in the tools used by the organization. Those tools might be complex project portfolio management (PPM) and application lifecycle management (ALM) tools or simple spreadsheets and email templates, but each tool has been implemented in the context of a process. Sometimes that process is explicit in the tool with clear artifact and process model naming. But in the case of many tools, the process has been shoehorned into a tool that was meant for something else. This leads to inconsistent data and models that require some level of folklore to understand and operate.
- Work doesn’t stop for change. As I write this article I am sitting on a Boeing 737 traveling at almost 500 miles per hour at 35,000 feet. I would get very worried if a service engineer walked by with a tool box and some plans! Preferably, all servicing and enhancements would be undertaken when the plane is safely on the ground and comprehensively tested prior to me getting back onboard. Organizations, however, do not have the luxury of stopping. It is not possible to put up a sign that says, “Not working due to moving to agility.” In fact, for many organizations, the economics of a move to agility are already factored into budget for the next year -- meaning the business already expects the product group (or groups) to deliver software that is faster, cheaper, and of a higher business value.
A focus on lifecycle
Contrary to the agile manifesto, scaling agility requires teams to look to their processes and tools. After all, when scaling outside of the team the only way to ensure repeatable endeavors is via process and tools. Much to the annoyance of many agile practitioners, as you scale you have to look at processes, practices, and tools to better enable enterprise effectiveness. That overall lifecycle should focus on providing an understanding of the following:
- Flow: How artifacts move through the lifecycle, how ownership changes, how information is added and changed throughout the development process.
- Collaboration: How team members and supporting organizations work together in the context of the work. Email and meetings are traditional ways in which information flows, but with the addition of modern development collaboration tools such as Atlassian’s HipChat, email is slowly being replaced with in-context activity streams.
- Reporting: How teams are being measured. Measurement enables teams to effectively know what they are striving for and how effective they are in that motion.
- Traceability and compliance: How software development lifecycles fit into larger business requirements. No software lifecycle operates in a vacuum, and it is important to understand how the process connects into a broader set of standards and rules. These rules might be externally set, such as in the case of medical or financial compliance, or internal audit rules that are in place to ensure money is being spent wisely and risk is being managed.
The software development lifecycle is a daunting and often confusing mess of tools and processes. Without the backing of a large management consulting practice or many months of research, analysis, and meetings, the lifecycle would impossible to document and even harder to change. Instead, when scaling your agile practices, focus not on the process but on the artifacts that comprise it. Most software delivery processes don’t have a massive array of artifacts that either drive work or are worked on. And some of those artifacts do not cross team or departmental boundaries. The first step is to list the key artifacts, describe what happens to them today, and note where agility is causing problems. Defect, ticket, or plan items might be a good place to start.
Artifacts are not all the same
At the simplest level, two primary types of artifacts are used in a software delivery project: Artifacts that drive work (story, task, defect, plan item) and artifacts that are the assets worked on specification (code, binary, test case). This distinction is important because work items describe the lifecycle, but the assets are the value the lifecycle works on. Broadly speaking, work fits into the disciplines of planning (investment, product, team), design, development, test, release, operations, and support.
Ultimately, agility provides the ability to change, but change cannot and should not operate only at the team level. Teams need to broadcast change to enable the right decision making. In addition, scale normally means an increased number of dependencies across the organization, which needs to be made aware of any change. That means that agile at scale results in an increased reliance on artifacts. For artifacts that cross disciplines, this means either consolidating those artifacts into one tool, or investing in integration technology that enables those artifacts to present consistent yet different information for each discipline. Lifecycle integration is an emerging practice, with many organizations starting to treat their software delivery process as a key business process and, as a result, look to automate its integration.
Adopting a process model like SAFe or DAD requires fundamental changes to your planning, delivery, and management practices. Those changes take time and require people to support them. Also, for many scaling practices, a big-bang change in which a company discards out-of-date practices in favor of a new agile lifecycle and process framework may seem to be the only effective approach. But big bangs are dangerous.
By focusing on the artifacts and looking to standardization or integration to enable those artifacts to drive flow, collaboration, reporting, and governance, you can scale agile software delivery with less risk, while achieving better results.
Dave West, chief product officer at Tasktop Technologies, is one of the foremost industry experts on software development and deployment. He has helped advance many modern software development processes, including agile and lean methodologies, process improvement, and project and requirements management. West is a frequent keynote speaker at major industry conferences, a widely published author of articles and research reports, and author of an acclaimed book, "Head First Object-Oriented Analysis and Design," which helped define new software modeling and application development processes.
New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to firstname.lastname@example.org.