- AI
- ambiguity
- APIs
- architecture
- augmented reality
- books
- bureaucracy
- career
- change
- Christmas
- cloud
- collaboration
- communication
- complexity
- computer history
- corporate life
- data
- decisions
- delivery
- devops
- end user tools
- ethics
- failure
- fear
- fundamentals
- gaming
- government
- halloween
- history
- humans
- hype
- identity
- infrastructure
- innovation
- language
- leadership
- learning
- legacy
- management
- measurement
- mental health
- money
- networking
- New Year
- operations
- partnership
- philosophy
- physics
- platforms
- prediction
- process
- procurement
- programming
- quantum
- reliability
- resilience
- risk
- science
- science fiction
- security
- shadow IT
- space
- standards
- strategy
- teaching
- teams
- technical debt
- technology advocacy
- testing
- thinking
- transformation
- TV
- virtues
- vision
- writing
Let’s avoid the equation ai_fragility * software_fragility = very_bad_things
Anyone who has built classical, deterministic software systems can tell you how fragile they are. Such systems perfectly follow precise, explicit instructions, and are incapable of operating outside those instructions. That means that, if you have written instructions which do not specify the behaviour you want, which are inconsistent with one another, or do not cater for unexpected values, your system will fail. If you are lucky, that failure will be loud: a crash with an ugly error message. If you are unlucky, it will be quiet: steady, ongoing corruption of data values which you won’t discover until later.
Anyone who has built newer, AI driven, probabilistic software systems can also tell you how fragile they are. Such systems treat inputs (prompts, context, training data and so on) as factors which affect the probability of getting different outputs. That means that, if your inputs are insufficient to tilt the balance of probabilities in favour of your desired outputs, your system will fail. If you are lucky, that failure will be obvious: results which are factually incorrect or logically incoherent. If you are unlucky, it will be obscure: results with subtle biases which you won’t discover until later.
Locally rational; strategically irrational
Enterprises frequently find themselves sliding down the slope from strategic rationality to strategic irrationality, while acting in ways which seem locally rational. To put it another way: if everyone’s so busy, how come we’re not going anywhere?
Working for a large enterprise can feel like being on a ship where the boiler is being stoked, the engines are churning, the propeller is turning, the kitchen is turning out meals, the brasswork is being polished, but, when you look out the window, you are still at the dock.
Actually, that might be too optimistic: many enterprises feel like a ship where the coal is being delivered to the kitchen, the stewards are working in the boiler room and the navigator is holding the charts upside down.
And if you work in technology, it often feels like the engines are overdue for a service, steam is escaping from the valves, and the pumps are struggling to keep the water at bay, but despite the constant, frantic activity, basic maintenance does not seem to be anyone’s job.
Rethinking phase three: your AI adoption programme is stalled, but your organisation is adopting AI anyway
I used to tell a story about AI adoption which went something like this . . .
AI adoption in traditional organisations has taken place in three phases.
In phase one, prior to the public release of ChatGPT, AI was a specialist pursuit for specialist people, solving specialist problems with specialist datasets. AI was used within banks to detect fraud and financial crime, or in retail to manage stock levels and distribution. But it was not knowingly used by most people most of the time.
After the release of ChatGPT, AI was available to anybody who could frame an English sentence and type it into a browser. It seemed to become a general purpose technology which could be applied to any business process which needed to handle language or make decisions.
Return of the spec
How much thinking do you need to do before you start coding?
When I got my first professional programming job, back in the late 1980s, the answer seemed to be all the thinking. We were encouraged - no, obliged - to use a method known as Structured Systems Analysis and Design Method, or SSADM, developed by the UK government. It was the very definition of a Big Upfront Design method, where you had to write multiple layers of specifications, accompanied by a whole array of models which described data structures, data flows and the lift history of data items (it was a very data-oriented method), before you even thought about writing a line of code.
What does 'architectural significance' mean in the age of AI?
When is a decision architecturally significant?
This question has vexed all of the technology architecture teams and governance structures which I have attempted to set up in my career. The thinking goes something like this: technology is complex and difficult, and we seem to have made some bad choices in the past; it would be a good idea if we were more thoughtful about our choices, and spent time trying to get them right; we could do that through some sort of governance process; however, if we subject every single decision to that governance process, we will erode autonomy and slow everything down; let’s focus our governance process on decisions which are architecturally significant, and let all other decisions be taken locally.
Technology architects aren't vampires: they don't need to be invited in
Vampires are puzzling. They have all sorts of rules which help make dramatic stories and films, but which don’t make much sense if you stop to think about them. For example, consider the rule that vampires can’t enter a house unless they have been invited in. Does it matter if the inviter lives at the house or not? What if the vampire receives a party invitation in the post? Is it okay if they are the +1 rather than invitee? Does a ‘Welcome’ mat work? It’s probably best for us not to ask too many questions, but just to accept the rules and enjoy the story. Oh no! The protagonist has blithely invited the monster in for a cup of tea!
However, while this seems strange in fiction, we often encounter its equivalent in real life, especially if we work in the technology department.
There's always a bigger goat: don't let big problems stop you solving smaller problems
In the story of the three billy goats gruff, the goats want to cross a bridge guarded by a troll. They manage this by each telling the troll that there is a bigger goat just behind them until (spoiler alert!) the biggest goat comes along and butts the troll into the sky.
Sometimes, when we are trying to make the case for enterprise technology capabilities, it feels like we are the trolls, and that we are so scared of the biggest billy goat that we won’t tackle the smaller goats. When we look across our technology landscapes, we see mess, waste and mayhem, and wish that we had some of the foundational capabilities that would help clean things up. Yet we hesitate, because we know that every time we build something we will uncover another problem, and another problem, and another problem, until we get to problems that are so big that we cannot imagine how to solve them.
It’s more complicated on the inside than it is on the outside
We don’t need time machines to create paradoxes in technology: they are built into the way we work. One of these paradoxes is that the simpler technology appears on the outside, the more complicated it is on the inside.
I was reminded of this recently when talking to someone who confidently told me that the more sophisticated AI models get, the easier they will be to use, for technologists as well as end users. AI would solve its own skills problem. I was surprised by this because, to me (and, I expect to most other technologists), while we understand how natural language interfaces can radically simplify the experience for end users, the introduction of the current wave of AI into our architecture makes it more complicated.
Build systems like you hire teams
Despite advances in AI, software systems are not people. But perhaps we should sometimes treat them like people - at least when it comes to budgets and financing.
A couple of weeks ago, I claimed that building software is not like building a bridge, and that the analogy of construction which we use - which I use - so frequently, may be harmful rather than helpful. Rather less usefully, I also suggested that there aren’t any perfect analogies, and that the best way to think about building software is to understand the business of building software. I’m now going to contradict myself and suggest another analogy: that we should think about building software in the same way that we think about hiring teams.
Good standards are like a good starting word in Wordle
What’s your starting word in Wordle?
I use STEAM. It has a good combination of common vowels and common consonants, and for some reason I find it pleasing (something to do with steam engines?). There are words which are mathematically proven to be better, but I like this one.
In case you’re one of the few people who didn’t become familiar with Wordle during the pandemic, I should explain. It’s an online game where you have six attempts to guess a five letter word, and each letter in the guess is marked green, for a correct letter in the correct place, yellow, for a correct letter in the wrong place, or grey for a letter which is not in the word at all. It’s just about hard enough to give you a sense of achievement, but easy enough that you get the right word most of the time. It also has the virtue that everyone who plays the game worldwide gets the same word every day, making it feel as if you are all playing together - you can also share your pattern of green, yellow and grey without giving the game away.
Freedom is not free: managing your cognitive and operational economies
Freedom is not free.
Let’s unpack that apparently paradoxical phrase into something a bit more unwieldy, but a little clearer.
In the context of enterprise technology, freedom of choice comes with a cognitive and operational price which we must be sure we want to pay.
The question of freedom versus control in enterprise technology has been debated with varying degrees of heat and passion for as long as the field has existed. Centralised corporate IT functions usually like control. They like to be able to manage costs and risks, to limit the complexity of their architecture and their supply chain, to strive for confidence and predictability. By contrast, development teams and product teams typically like freedom. They like to be able to pick the technologies, frameworks and ways of working which suit them best, to innovate freely, to use their skills and express their professional expertise.
Complexity and simplicity are friends
How many applications is too many? 100? 200? 1,000? 7,000?
I have worked for many different organisations with different numbers of applications. Whatever the numbers, though, they all had one thing in common: somebody, somewhere thought that we had too many applications.
I have always found this belief puzzling. We build applications because we believe that they have value (otherwise we wouldn’t put so much effort into writing those business cases). We celebrate our success when we implement them. And then we record their existence in our great big list of applications, lament that we have so many, and wish that we had fewer. Is there any other field of endeavour where what we have built moves so quickly from asset to liability (whatever our accounting rules say)?
I believe that people worry about the number of applications in their estate because of an inherent fear of complexity, and because the number of applications is a visible signal of complexity. They perceive that their technology is becoming too costly, too slow and too risky, and they also see that these factors rise in sync with the number of applications. The more stuff they have, the harder it seems to get