Architecture as balance #1: elephants vs building blocks

(This is the first of an occasional series on finding the balance in technology architecture, based on a few successes and a lot of mistakes over the course of my career.)

Q: How do you eat an elephant?

A: One bite at a time.

This is an old joke, and one that technology architects are very familiar with. We spend a lot of our time dealing with big, open ended questions and big problems. How do we make this legacy system respond to digital needs? How do we integrate this acquisition and realise efficiency gains without breaking the business? How do we turn this disruptive force into a competitive advantage?

One of our most basic techniques is to break the big problem down into smaller pieces: to eat the elephant one bite at a time. We progressively refactor the legacy system into micro-services. We figure out the processes, functions, systems and data of the acquired entity and map them to the integrated state. We get to grips with the fundamentals of the disruptive force, and work out the best places to start putting it to work in our current enterprise.

Yet, powerful as it is, the technique of breaking things down into smaller pieces has its own problems: we can confuse ourselves and others with complexity and detail. Modern, flexible architectures comprise ever smaller components with ever more dynamic life cycles. Companies deliver services through shifting ecosystems of partnerships connected through APIs. Business processes and customer journeys comprise many moments. Technology architects have always run the risk of drowning in detail, and these days it seems as if the tide of detail is rising ever faster.

Of course, breaking things into pieces is not our only technique. We also frequently do the opposite: we combine fragmented concepts and components into abstractions, and give these labels such as architectural building blocks.

Abstracting complexity into bigger concepts is also a powerful tool. It helps us manipulate ideas and compose architectures without having to get to grips with all of the underlying detail. It enables us to draw boundaries between areas of our architecture, to define distinct roles and accountabilities, to discover where our interfaces sit, and to define good APIs.

Yet abstraction has its own dangers. Drawing boundaries and defining capabilities can lead people to believe that these things are fixed and immutable, that they have no prospect of change. This is a particular danger when we use terms such as ‘building blocks’. We know that this is an analogy, a term of art. Yet it unavoidably creates an image of static, unbreakable impermanence. I have been using the term ‘building block’ for years, and know that I am using it to stand in for a network of systems, data and processes - but it still conjures up a picture of a big granite slab in my mind.

So, if we risk drowning in detail by breaking down problems, and we risk creating false barriers and boundaries by through abstraction, what is a technology architect to do? Naturally, it is to do what we always try to do: to deploy our technical skills, communication skills and leadership skills (the trio of skills we call Zang Jing Ge in HSBC) to figure out what the most useful level of abstraction is for the work that needs to be done. If we are defining the deployment and service management architecture for a micro-services application, then we need to get right down into the detail. If we are defining the role of that application across customer journeys, then we need to operate at a level of abstraction. If we are trying to define how the behaviour of the application needs to change to meet the needs to particular customer journeys then we need to navigate between the two.

And we must always remember that when we pick the level of detail or abstraction we are going to operate at, we are influencing the mental landscape of the project, team or product we are working with. If we are doing our jobs properly, we are leading the thinking of others. If we get lost in detail, we lose the team. If we stay in the stratosphere, we fail to solve ground level problems.

Architecture is balance. Sometimes this balance does not just show up in our design: it shows up in the way we fit communication to context.