From June 13 until June 15, I went to the Enterprise Architecture Conference Europe 2016. I was there to deliver the EA keynote (about Chess and the Art of Enterprise Architecture), but that also gave be the opportunity to go to presentations and workshops myself. One of these was by John Zachman, widely seen as the inventor of Enterprise Architecture and the author of the famous Zachman Framework®.
When I said I was going to visit John's workshop (because it was a nice opportunity to actually hear from him in person), I was told by one of the other speakers: "Well, his ideas are not that relevant anymore these days."
Now, I must admit that did not sound too strange to me. I had only superficially looked at his framework in the past. It had been presented to me a long time ago in a way that made me conclude it was a big hopeless waterfall that was totally unusable in practice and utopian in its reach. Maybe nice for designing and building commercial airliners or Channel tunnels, where designing is an initial stage that costs many millions, takes several years and produces a design of a product that has a life expectancy of several decades or more. Or nice in the deep past of enterprise architecture, when the world was still somewhat manageable with such approaches.
But I was wrong. If I took one thing from the workshop it was that Zachman's framework is one thing above all else: misunderstood.
The Zachman Framework
So, for those who don't know it, what is the Zachman Framework? It's a way to divide everything you have to document about design in a certain way. It is -- as John these days also explicitly tells people -- an ontology. According to my dictionary, an ontology is "the branch of metaphysics dealing with the nature of being," which doesn't immediately sound like something practical. But, being a bit interested in philosophy, I know the shorthand for this, which is that an ontology describes what there is. In this case, John's ontology describes what should be in a design and for that he has created a matrix that combines two divisions that are both as old as the way to Rome.
© John A. Zachman, reused with permission John A. Zachman's Enterprise (design) Ontology
One axis, normally spread about the horizontal of the matrix, consists of the following fundamental elements of human analysis:
- What -- Inventory
- How -- Process
- Where -- Distribution (Location)
- Who -- Responsibility (Roles)
- When -- Timing (Order of behavior)
- Why -- Motivation
In the workshop, John stressed that he did not invent this and he does not take credit for it. He said he just re-uses what has been a basis of human analysis for as long as humans can remember. He noted that these categories have always been used for any subject of design, be it buildings or airplanes or anything else and that they thus also should apply to designing enterprises. His use is not a proposal for an EA mechanism, it is just saying that design is design, or in John's words: "Architecture is architecture is architecture," and thus that this generic ontology is by definition also applicable to enterprise architecture. It could not be not the case.
The second axis, normally positioned along the vertical, consists of several transitions from the more 'abstract' ideas to the more concrete descriptions (and finally implementation), which he said is as old as the fundamental insights of the Greek philosophers of 2500 years ago. These layers of intent are linked to certain stakeholders in his scheme:
- Executive -- (Business) Context, or Identification
- Management -- (Business) Definition
- Architect -- System Logic, or Representation
- Engineer -- Technology Physics, or Specification
- Technician -- Configuration
Below this in the ontology is the actual instantiation of the design. John stressed this at the start of his workshop: the building is not architecture, it is an instantiation of an architecture (design). It is the transformation of design to something real.
John also explicitly stated a truism, namely that leaving out details in one area leaves it to interpretation when transforming one intention to the more concrete ("down" in the ontology) is just a matter of taking risks. He told me afterwards that "any facts about the enterprise that you do not make explicit are implicit, that is, you (and anyone or everyone) are making assumptions about those implicit facts. If the assumptions are correct, it is positive because you save time and money.
On the other hand, if the assumptions are incorrect, it is negative in that incorrect assumptions are the sources of defects which could range from inconvenient to cataclysmic. So, my Framework does not say you have to formalize, populate, make explicit any Enterprise facts at all. However, any facts that you do not formalize are potential sources of risk, risk of defects." Note specifically the pragmatic nature of John's approach to filling the squares. It is quite telling that John's neutral description of what happens when you leave details out has been transformed by many of his disciples into a requirement to fill in all the details (which is impossible anyway).
Both axes have a different character. The horizontal axis is a division into (semi-)independent aspects of design. The vertical axis is a transformational axis, it has a waterfall somewhat built into it, where we go from top to bottom when going (transforming) 'down' from idea to actual implementations. According to John the waterfall is only that "any one Row must support the intent of the Row above it".
John's message is that both these axes are generic dimensions of any design activity. So, they must hold for designing enterprises too.
The surprises
Several issues surprised me during the workshop with John Zachman.
The first is that John is 82, and he has a stamina and speed when trying to explain it all that is beyond many 40 year-olds. He's a man with a mission, the mission being to let people understand his story. Incidentally, during that whole morning he lost his train of thought once, then told us "you're watching a senior moment here," after which he quickly returned to the story. Earlier that week, when I gave my own keynote, I think in the space of an hour I had at least three such senior moments, and I'm by far not 82.
The second surprise was that John explicitly said that -- though the vertical axis is from more ideal to more concrete when going down -- it is not about from less detail to more detail. Each of his squares can be detailed, or at least to the extent that details are necessary. Enterprise Architects often use the word "abstraction" to mean "less detail" because they use their abstraction as a way to make the complexity manageable for themselves and their audience, and John's ontology has often been used in such a process. But John's story is more subtle and the idea that going from abstract rough ideas to concrete details is not part of it.
The third surprise was that John in his workshop explicitly talked about going forward on the basis of incomplete descriptions. He explicitly stated that it is not the idea to go into 4-year super-waterfall and end up with the perfect design which is then implemented. No, John said that even in his favourite analogical world, that of designing and building large airplanes, it may happen that work is already progressing and that a change higher up may lead to scrap and rework. Now, as he explained it that way, it was easy to see the current world of Agile IT in front of you. Yes, you go ahead in your little part of the world, you do the best you can, but be prepared to do scrap and rework when needed. If you're not prepared to do scrap and rework in agile, technological debt will build up.
And if there is too much scrap and rework, then you take a step back and say, "Wait, I need to halt my production and wait until certain fundamental questions have been answered". Not because we need to wait until every design is done in excruciating detail, but because it becomes a pragmatic economic choice to do so.
So, I got in -- thinking to get a presentation about the ultimate unusable waterfall, from high level abstractions to low level details-- and I found out it's use according to John himself has agile and pragmatism built in to the core.
The analysis
So, now having looked at it better and have it explained to me by its inventor, what do I think about John's Enterprise Ontology?
Well, with respect to the horizontal axis, I think John re-uses a very pragmatic and usable shopping list in "What, How, Where, Who, When, Why". It is important to make sure you have paid enough attention to each of these when creating designs/architectures.
For me, though, the Why is not part of the design and I have strong doubts about the feasibility of using modelling (hence structure) for the more abstract Why. The doubts about modelling the why are also related to philosophy, in this case for instance Wittgenstein. Take for instance all the ethical issues that may be part of the why. We have no ruleset for ethics, it is hardly ever strictly separable in good and bad (or in discrete and disjunct elements -- which modelling requires), and so forth. So, if your task is designing a solution in the context of patient records, it is not doable to model the why at all in enough detail such that you won't get trouble when transforming to the more concrete. This, by the way, is part of what in my own keynote I called the cultural seduction, the assumptions that lead us architects to assume that certain approaches will work, while in reality they don't.
But I do agree that the why is important, so there is not much harm done when leaving it in. Besides, John's Why is only abstract when in a vertical-sense we are still in the upper idea area of his ontology. Going down, and the why might become very strict and doable requirements engineering.
The discussion on modelling the upper why brings us also to the vertical axis: the route from ‘abstract' ideas to concrete implementations. As mentioned before, John is neutral about the level of detail when going from idea to implementation and there I agree. There is for me a relation with the golden rule for outsourcing that I mention somewhere in my book. Sometimes, the requirement for a solution may be orders of magnitude less complex than the solution. There, outsourcing and waterfall work well, because sharply defining the requirements works well. If I want a secure data link with less than one in ten to the umpteenth bit errors, the requirement is small and exact, the actual implementation may be extremely complex, depending on the value of umpteen. However, I can also have a situation where the requirements are very complex, here and there fuzzy and even sometimes contradictory (think patient records or most strategic change initiatives) where the actual technical implementation may even be relatively simple, even if the requirements are complicated. Sometimes, going 'down' even means less complexity, not more.
So, I guess, outsourcings work well, and waterfall is efficient and effective, when ‘from abstract to concrete implementation' coincides with ‘from abstract model (less detail) to more detailed description'. If this is not the case, then (fixed price) outsourcing is problematic and the waterfall is costly.
Returning to the issue of requirements engineering, it might seem that when going from the abstract (ideas) to the concrete (implementation) it gets easier to be precise, but I doubt that too, mainly because the actual requirements at many levels may be more complex, such as the fact that a choice made now in the technical domain may remain in effect for a long time, which means that the requirements for that choice cannot be as clear cut as the waterfall would suggest. Airplanes (John's favourite analogy) such as Boeing 747s are complex, yes, but as an instrument they're rather robust in their role, which is roughly the same as 40 years ago: carrying people and cargo safely from A to B. IT systems in information-heavy enterprises have to contend with much larger variability in strategies and goals. That doesn't mean that John is wrong about the importance of knowing what you want (designing it), just that it might be less feasible to be as detailed as one would like it to be.
Besides, I'm totally with John that being explicit as much as is feasible in a practical sense is a good idea. Just like Agile may just be a poor excuse to not do what you should do (design properly), or just like the enterprise architect's shying away from detail into simplistic pictures can be a flight reaction, the problem of the complexity and unpredictability of our world doesn't mean that we should completely forget about modelling and being as precise as we can (or need to be). We might not be able to do it as it is done in the airplane industry, but we certainly can often do a better job than we generally do. That is why I like modelling, e.g. in the enterprise architecture language ArchiMate (even if it is not perfect and not really open). And yes, that sometimes means we have to accept that there will be some ‘scrap and rework' on our part.