In 1843, Henry Ellsworth, the U.S. patent commissioner stated to Congress that “the advancement of the arts from year to year taxes our credulity and seems to presage the arrival of that period when human improvement must end.” In other words, anything worth inventing will have soon been thought of. While that was clearly not the case, it is true that many older ideas seem new when applied in a new context.
Fast forward to a 1968 article in the Harvard Business Review, wherein authors Arthur Walker and Jay Lorsch examined the tradeoffs between two fundamentally different organizational structures: one organized around function, and the other organized around product. To do this, they studied two large consumer products companies making the same product using the same technologies. The plant organized around function is “Plant F,” the one organized around product is “Plant P.”
In the Plant F, people were more focused on the activities within their functions. A quote from a production supervisor: “I hope that they get that line going soon. Right now, however, my hands are tied. Maintenance has the job. I can only wait. My people have to wait, too.” Job responsibilities were very clearly defined, but there was a high reliance on rules and procedures. And because each functional unit prioritized its own goals, there was greater focus on short-term results.
In Plant P, goals were more “diffuse” with individuals focusing on their own tasks but also on those of the entire plant. A quote from a maintenance manager: “We’re all interested in the same thing. If I can help, I’m willing. If I have a mechanical problem, there is no member of the operating department who wouldn’t go out of his way to solve it.” Job responsibilities in Plant P were less clearly defined, with lower reliance on rules and procedures and equal focus on near-term and longer-term goals.
The authors reference studies that show an inverse relationship between differentiation and integration: the more specialization there is between different functions, the harder it is to integrate across them. However, the research also showed that both differentiation and integration are necessary for an organization to be effective. They argue that the optimal organizational choice will depend on external factors such as the market and technologies as well as the goals themselves.
There were also myriad differences in communication styles across both plants, but perhaps most notably a stark difference around surfacing and managing conflict. At Plant F, there was a tendency to simply smooth over conflict without truly addressing the issue. Essentially, relying on organizational hierarchy to resolve conflict swamped managers with too many decisions, leaving issues unresolved. At Plant P, managers resolved conflict themselves and thus they achieved better integration because conflicts were resolved closer to where they happened.
The results? Although both plants had the same objectives, Plant F achieved greater efficiency and lower cost than Plant P. However, Plant P outperformed Plant F when it came to improving the capabilities of the plant itself. The punchline: Productivity at Plant P increased 23 percent over three years, while it increased only 3 percent at Plant F.
You may be wondering at this point how any of this relates to running a technology organization. Consider this excerpt in the conclusion of the 1968 article:
“…the functional organization seems to lead to better results in a situation where stable performance of a routine task is desired, while the product organization leads to better results in situations where the task is less predictable and requires innovative problem solving.”
This is part of the article where all the pieces came together in my head…the rate of technology-driven change, digital disruption, and the need for organizational and cultural transformation to be able to keep up. For many years, the focus of technology in the enterprise was around maximizing operational efficiency, a good fit for a functional organizational approach. Waterfall approaches provided slow-moving predictability as work flowed from one department to another. Improving the software factory’s capabilities themselves was secondary.
Then along came the Internet revolution followed by the explosive rise of the mobile app economy. The name of the game was velocity and ongoing adaption and improvement. The ability to continuously improve the modern software factory’s capabilities – to be able to sense and respond to changing customer needs – became the critical need. Functional organizations with their large silos are ill-equipped for rapid innovation and problem-solving. Technology departments – let alone entire companies – have been organized like Plant F, and what they needed is to be Plant P with focus on product rather than function.
The fundamental change in technology’s role in the modern digital business has given rise to devops and agile methodologies. At the heart of both of these is cross-functional coordination, communication, collaboration and continuous problem-solving with a focus on outcomes. They both aim to find the right balance between functional expertise and integration, and both agile and devops approaches encourage pushing decision-making and conflict resolution down to the team level.
Finally, scaled agile aims to apply the agile development concept to be a company-wide methodology. But regardless of newer terminology – devops and agile – and a set of associated rituals like scrum teams and standups, the foundational elements behind today’s “modern” approaches can be found in a paper published almost 50 years ago. And perhaps that’s not surprising, because while technology has changed, the dynamics between people within organizations have not.