- AI
- ambiguity
- APIs
- architecture
- augmented reality
- books
- bureaucracy
- career
- change
- Christmas
- cloud
- collaboration
- communication
- complexity
- computer history
- corporate life
- data
- decisions
- delivery
- devops
- end user tools
- ethics
- failure
- fear
- fundamentals
- gaming
- government
- halloween
- history
- humans
- hype
- identity
- infrastructure
- innovation
- language
- leadership
- learning
- legacy
- management
- measurement
- mental health
- money
- networking
- New Year
- operations
- philosophy
- physics
- platforms
- prediction
- process
- procurement
- programming
- quantum
- reliability
- resilience
- risk
- robotics
- science
- science fiction
- security
- shadow IT
- space
- standards
- strategy
- talent
- teams
- technical debt
- technology advocacy
- testing
- thinking
- transformation
- TV
- virtues
- vision
- writing
The software paradox: it’s rational to make plans; it’s irrational to expect them to work
Software projects suffer from a paradox. They are typically expensive endeavours which are of great importance to the organisation which is executing them. It is rational to want to be assured that they will be successful, that they will not cost more than the organisation can afford, and that they will not take longer than the organisation can wait. At the same time, they are fundamentally unpredictable. Any software project worth doing involves work which no-one has ever done before, creating new logic and new capabilities, and integrating existing components in complicated and interesting ways. This is difficult work which involves intellectual challenge and problem solving. Things which look hard often turn out to be easy, while (rather more frequently) things which look easy turn out to be hard.
The change *is* the system
What do you think of when you think of computer systems? Do you think of a fixed set of features and functions, designed to meet a particular purpose, which changes steadily over time as a result of planned projects or essential maintenance? Or do you think of a dynamic process of continuous change, which adapts to meet a set of needs and opportunities revealed through practice and experience?
Many people who have worked in systems development for a while would favour the second definition: they recognise that a computer system is never ‘done’ and that its capacity for change is one of its most essential features. However, most organisations continue to construct investment plans, portfolios and even balance sheets as if the first definition was correct: IT systems are fixed assets with defined purposes which are built through upfront investment and then depreciated over time.
Business sponsor translation service
It’s tough being the business sponsor of a technology initiative. You want to achieve an outcome; you are responsible for achieving that outcome; it’s your budget that is being spent; and you will be judged on the result. But you are dependent on people you don’t know, concepts that you don’t understand, products that you may never even see, and suppliers that you have never met.
Given how tough the job is, it seems like a good idea not to make it any tougher. However, because business sponsors are rarely experts in technology, they often accidentally make their jobs tougher without realising – sometimes just by saying a few words. Because business sponsors are senior leaders, the things they say have consequences: they prompt the people around them to take action. And, if the sponsor says the wrong things, those actions will be counter-productive.
Coping with volatility: don't panic; seek truth; release frequently
If you’re in the last stages of a multi-year digital delivery programme, then you probably feel frazzled. That’s the normal condition of late stage programme teams. If your programme has coincided with the last five years (five year digital delivery programmes are still a thing) then you must feel frazzled to a historic degree.
All I want for Christmas is speed and reliability
What were your Christmas lists like as a child? Were they modest requests for improving books and educational toys? Or were they, like most Christmas lists, an extravaganza of wishes, containing toys and sweets and games . . . and possibly a dragon and a unicorn?
It sometimes feels like our requirements for software development are like a Christmas list written by a very small child that wants everything at once. We’d like the ERP package to have world class embedded processes, but we’d also like it to customise it to meet our every need. We’d like the cloud platform to give us capacity on demand and pay as we go, but we’d also like it to be on-premise because we’re worried about security. We’d like an AI system that predicts our customer’s needs, but we’d like to do it without using our data.
And being a project manager or a product owner can feel like a harried parent who can’t possibly afford everything on the list, and is worried that Christmas morning is going to be a disappointment. If I give them all the customisations they want, then we can never take the upgrade. I can give them security on cloud, but they’ve got to understand the shared responsibility model. I can build them an AI model, but not unless someone’s prepared to share the data.
Are you overfitting your delivery decisions - and to the wrong dataset?
Overfitting is a term from the world of machine learning. It describes the problem that arises when you train a model on a dataset, and use that model to make accurate predictions - as long as they are limited to that dataset. Unfortunately, the model starts to go haywire when you apply it to the wider world.
Imagine a fictional example: you wish to use data about students to predict their academic performance. You train your model on a class where there are several students called ‘Geoff’, and they all happen to be doing very well in their studies - there’s no reason for this: it’s pure coincidence. But the data doesn’t know that. When you make predictions about this class only, your model seems to work: it finds the high performing Geoffs. When you use it to make wider predictions, however, in a world where there are rather fewer Geoffs, and where being a Geoff is not a reliable indicator of academic performance, the model stops working.
Build systems like you hire teams
Despite advances in AI, software systems are not people. But perhaps we should sometimes treat them like people - at least when it comes to budgets and financing.
A couple of weeks ago, I claimed that building software is not like building a bridge, and that the analogy of construction which we use - which I use - so frequently, may be harmful rather than helpful. Rather less usefully, I also suggested that there aren’t any perfect analogies, and that the best way to think about building software is to understand the business of building software. I’m now going to contradict myself and suggest another analogy: that we should think about building software in the same way that we think about hiring teams.
Building software is not like building a bridge . . . except when it is
Software professionals live in a world of analogies. This is because most people don’t understand how software gets built. This isn’t their fault: the number of people who use software greatly exceeds the number of people who build software for a living, so we have to find creative ways of explaining what is going on. Analogies seem to help.
The most pervasive and persistent analogy is that of construction: we talk about building software as if it like building a bridge. We talk of engineering and we talk of architecture. We talk about foundations, and about building blocks. We might even talk about town planning.
It’s easy to understand why we use this analogy: I have used it thousands of times. We live in buildings. Millions of us live in cities, where buildings dominate our environment. We see them being erected and we see them being demolished. For most of us, they are the most visible and familiar example of something being created. It seems reasonable and helpful to talk to people in terms that they know.
Speed bumps in reality: overcoming failed attempts at change
‘We’ve already tried that. It didn’t work.’
How many times have you heard these words? They’re a familiar feature of change in large organisations. If you have been the leader of a change programme, or an eager consultant, or a manager inspired by the prospect of doing things differently, then these words have probably left you daunted and deflated. You knew that this was going to be difficult, but you didn’t anticipate apathy and resignation.
Before we condemn such a reaction, though, and write off the people who react this way as resistant cynics, let’s consider whether such a reaction is reasonable.
Build your way out of the ivory tower
How do you escape the ivory tower?
No technology architect wants to be accused of living in an ivory tower. Such an accusation means that there is at least one person who thinks that you are detached from practical reality, that you add no value, and that you do nothing but slow people down. It is never good to be characterised as an irrelevance.
Yet it is possible as a technology architect to find yourself living in an ivory tower without realising it. This is much less likely if you are a solution architect, working as part of a team, solving problems and making decisions every day. It is even less likely if you are a hands-on technologist, actively contributing to the solution that you work with.
The secret to great technology architecture is . . . timing
It’s been fascinating and nerve wracking to watch the journey, unfolding and testing of the James Webb Space Telescope since its launch at the end of 2021. Fascinating because, to a non-expert like me, every time I read about it, the mission seems more complicated than I thought it was. Nerve wracking because the telescope is a very, very long way away (at the Lagrange point L2, about one and a half million kilometres from Earth). This means that, if anything goes wrong, there’s no realistic prospect of sending someone (or something) to fix it (unlike Hubble, which has had five shuttle missions for servicing, upgrades and repairs). Anybody who has been involved in any sort of launch, even of software that has never left a machine, let alone left the Earth, knows that ‘just one last thing’ feeling before the point of no return is crossed.
I find the JWST interesting because it is an extreme example of knowing when to make choices about architecture and design. I must admit that I don’t know the full production lifecycle of the JWST, the manufacturing lead times, or the critical path. But I do know that, once the Ariane 5 rocket was lit, there was no going back.
Trust and accountability can guide us out of the process maze
If you want to know the history of a company’s IT failures, take a look at their release management process. If they’ve got more than one process, look at the gnarliest, longest, most frustrating process you can find. Typically, you’ll find a set of checks, inspections and approvals that comprise an archaeology of previous mistakes and failures. Twenty years ago several systems ran out of capacity shortly after launch, so a capacity plan is now an essential part of the process. Ten years ago, there was an audit finding questioning the validity of DR planning, so a complete DR plan is required for every release. Five years ago, there was a licensing compliance breach, so evidencing license coverage is now mandatory.
Taken individually, these steps all seem rational. Of course we want to be sure that there is enough capacity to run the system. Of course we want to know that our DR plans work. Of course we want to be compliant with our licensing agreements. But collectively, these steps turn into a complex maze, where it seems that every release must atone for the errors of the past.