What does 'architectural significance' mean in the age of AI?
Photo credit: Valery Fedotov via Unsplash
When is a decision architecturally significant?
This question has vexed all of the technology architecture teams and governance structures which I have attempted to set up in my career. The thinking goes something like this: technology is complex and difficult, and we seem to have made some bad choices in the past; it would be a good idea if we were more thoughtful about our choices, and spent time trying to get them right; we could do that through some sort of governance process; however, if we subject every single decision to that governance process, we will erode autonomy and slow everything down; let’s focus our governance process on decisions which are architecturally significant, and let all other decisions be taken locally.
This sounds great until we get to the question of whether a decision is architecturally significant or not. Technology architects being technology architects, there is a natural inclination to try to define a set of criteria for significance. And organisations being organisations, it seems sensible to use criteria such as cost, risk, effort and impact.
The problem with all such criteria is two-fold: first, they can easily be evaded, and second, they don’t catch everything that really matters. No-one loves governance, so if project budgets can be organised in such a way that the cost slips below the threshold, then the project manager is probably going to be happy (especially if this move evades other sorts of governance as well). Conversely, a decision may legitimately not qualify as significant according to formal criteria, yet turn out later to be highly significant: a simple API, for example, might cost little to implement, but if it becomes a lynchpin of the architecture, it matters.
For several years, my preferred criterion has been the ‘you know it when you see it’ rule: that is, to ask architects who are working on projects or who lead architecture for business units to identify when they are making choices that the rest of the organisation has a right to care about. To fans of strict rules, this may seem unforgivably lax, but rests on trust, skill and professional judgement – which I believe are stronger and more adaptable than a set of governance criteria.
Although this approach has worked for me (and increases the importance of building a strong architectural team) I don’t claim to have solved the problem. I also think that this problem is about to get a whole lot more pressing.
Over the last few weeks, I have been experimenting, along with many other people in the computing field, with the phenomenon of ‘vibe coding’: the practice of using AI assistants to build applications without using anything other than natural language prompts. The goal of this experimentation is learning and discovery, testing the ‘reductio ad absurdum’ question: what if we could code without coding? I have no intent to put any of this code into production (and no-one has paid me to write production-worthy code for at least a couple of decades anyway).
The experience is instructive. After the initial shock at just how able and versatile the current generation of AI coding assistants seems to be (along with relief that I conducted this experiment on an old machine which I am happy to rebuild later), comes the more grounded experience of what AI generated code actually looks like.
My experience was that it was impressive . . . but weird. In routine tasks, such as building an html page to display a form, it was much faster than I could manage by hand, and took away boring, repetitive work. In more fundamental tasks, though, such as defining the database or organising classes, it made some strange choices which I knew that I would regret later.
Moreover, unlike fixing basic bugs in forms, which required no more than cutting and pasting error message, or even just typing ‘fix errors’, coming up with the prompts to change those fundamental structures was difficult, and led to a kind of ‘prompt wrestling’ - the experience of iteratively persuading an AI tool to deliver a certain output, and finding that the results get worse and weirder over time.
Overall, though, the experience reminded me that designing, building and running IT systems involves an endless succession of choices. Some of these are trivial and have no more than local consequences (the conventions I use to lay out a particular HTML page). Some of them are much more consequential and fundamental (the structure of a database). These are not always obvious at decision time: something as basic as choosing the name of a variable or function may not seem to matter, but can influence the entire conceptual structure of your code.
Just like architecture governance, though, it is hard to write rules which will always work (and therefore hard to build into models or prompts) – and I continue to have more faith in trust, skill and professional judgement than in rules. As we move into an era where we delegate more decisions to AI – at least in the field of software development – then trust, skill and professional judgement matter more than ever.
I have long been an advocate of learning to code, not because I believe that everyone should be a professional software engineer, but because I believe that it provides a deeper understanding of the world that we are building out of software. I now believe that learning to code alongside AI assistants is becoming equally important, so that we get better at choosing the decisions we want to take and the decisions we can safely delegate.
