David Knott 31/10/2024 David Knott 31/10/2024

Are you overfitting your delivery decisions - and to the wrong dataset?

Overfitting is a term from the world of machine learning. It describes the problem that arises when you train a model on a dataset, and use that model to make accurate predictions - as long as they are limited to that dataset. Unfortunately, the model starts to go haywire when you apply it to the wider world.

Imagine a fictional example: you wish to use data about students to predict their academic performance. You train your model on a class where there are several students called ‘Geoff’, and they all happen to be doing very well in their studies - there’s no reason for this: it’s pure coincidence. But the data doesn’t know that. When you make predictions about this class only, your model seems to work: it finds the high performing Geoffs. When you use it to make wider predictions, however, in a world where there are rather fewer Geoffs, and where being a Geoff is not a reliable indicator of academic performance, the model stops working.

David Knott 24/10/2024 David Knott 24/10/2024

Respect before beanbags

I have worked in some strange offices in my career. At a start up, I spent six months in a cramped office above a meat market, where arriving in the morning meant dodging large men in blood smeared coats carrying sides of beef. While working for a large UK bank, which had run out of space in its premises, I spent time in a business continuity centre, surrounded by banks of anonymous desks stretching away into the distance, waiting to be filled in the event of a disaster. As part of a confidential corporate restructuring project, I worked in an empty office scheduled for decommissioning and demolition, hoping that the security systems would keep working for long enough to let the team out in the evening.

And, because I have spent my career in enterprise technology, I have also worked in many environments which look as if they have been outfitted from the ‘digital’ section of the IKEA catalogue, full of bright colours, breakout areas, ping pong tables, expensive chairs - and, of course, bean bags.

David Knott 17/10/2024 David Knott 17/10/2024

Build systems like you hire teams

Despite advances in AI, software systems are not people. But perhaps we should sometimes treat them like people - at least when it comes to budgets and financing.

A couple of weeks ago, I claimed that building software is not like building a bridge, and that the analogy of construction which we use - which I use - so frequently, may be harmful rather than helpful. Rather less usefully, I also suggested that there aren’t any perfect analogies, and that the best way to think about building software is to understand the business of building software. I’m now going to contradict myself and suggest another analogy: that we should think about building software in the same way that we think about hiring teams.

David Knott 10/10/2024 David Knott 10/10/2024

Good standards are like a good starting word in Wordle

What’s your starting word in Wordle?

I use STEAM. It has a good combination of common vowels and common consonants, and for some reason I find it pleasing (something to do with steam engines?). There are words which are mathematically proven to be better, but I like this one.

In case you’re one of the few people who didn’t become familiar with Wordle during the pandemic, I should explain. It’s an online game where you have six attempts to guess a five letter word, and each letter in the guess is marked green, for a correct letter in the correct place, yellow, for a correct letter in the wrong place, or grey for a letter which is not in the word at all. It’s just about hard enough to give you a sense of achievement, but easy enough that you get the right word most of the time. It also has the virtue that everyone who plays the game worldwide gets the same word every day, making it feel as if you are all playing together - you can also share your pattern of green, yellow and grey without giving the game away.

David Knott 03/10/2024 David Knott 03/10/2024

Building software is not like building a bridge . . . except when it is

Software professionals live in a world of analogies. This is because most people don’t understand how software gets built. This isn’t their fault: the number of people who use software greatly exceeds the number of people who build software for a living, so we have to find creative ways of explaining what is going on. Analogies seem to help.

The most pervasive and persistent analogy is that of construction: we talk about building software as if it like building a bridge. We talk of engineering and we talk of architecture. We talk about foundations, and about building blocks. We might even talk about town planning.

It’s easy to understand why we use this analogy: I have used it thousands of times. We live in buildings. Millions of us live in cities, where buildings dominate our environment. We see them being erected and we see them being demolished. For most of us, they are the most visible and familiar example of something being created. It seems reasonable and helpful to talk to people in terms that they know.

David Knott 26/09/2024 David Knott 26/09/2024

Three quantum lightbulbs

Do you ever have a moment of panic, when you realise that you are lot less prepared for a meeting than you thought you were? I had one last week.

A few months ago, I was asked to run one of the Spotlight on Technology sessions which we regularly hold in CDDO: webinars on computing topics open to all UK public servants. When asked what topic I would like to cover, I casually suggested quantum computing. It was a topic I had done some reading and writing about a couple of years ago, and I thought it would be relatively easy to talk about.

Until last week, when I was preparing for the session. First, I realised that I had forgotten almost everything I learnt two years ago: I had some vague notions of linear algebra and matrix mathematics, and a loose grasp of superposition, but that was it. Second, when I went back and read what I had written, I realised that it didn’t answer all of my questions: I wasn’t confident that I could turn it into an hour long discussion.

David Knott 19/09/2024 David Knott 19/09/2024

Is your frontend fixation robbing your sponsors of agency and accountability?

Text scrolls across the screen. Lights flash and patterns whirl. Images of faces flicker past, overlaid with lines and symbols. The frantic activity slows and settles. One face remains. PERFECT MATCH.

I was watching yet another new police procedural drama. And I was having the same reaction that I always have when they show the user interface for the big computer system that takes three pieces of data and a blurry image, and finds one suspect amongst millions. It doesn’t work that way.

I’m not an expert in police systems or forensic systems, but I know that anyone building any systems has limited time and resources, and they avoid spending those resources on things that aren't necessary. And a user interface that attempts to show all the inner workings of the system isn’t necessary: if you’re lucky, you’ll get a progress bar, a buffering symbol or a spinning ball.

David Knott 12/09/2024 David Knott 12/09/2024

Remember to look down

I had two astonishing experiences while travelling recently.

First, I got on a plane in one country and, some time later, stepped out in another country.

Second, I tapped my phone against the reader on the local metro system and paid for my fare, even though my bank was thousands of miles away.

It might seem that neither of those things are astonishing, but I think that fact is astonishing in itself. It is remarkable how quickly technological miracles become a mundane part of our lives, and we cease to notice how unusual they are.

David Knott 05/09/2024 David Knott 05/09/2024

Are you under attack from your corporate immune system?

It was supposed to protect us.

One of the most disconcerting aspects of the recent Crowdstrike incident was that the process which caused the disruption - a rapidly deployed update to a piece of endpoint protection software - was meant to prevent disruption. Rapid deployment was intended to help us respond quickly to new threats against which we would otherwise be defenceless. Low level access to the operating system was intended to enable us to detect and deal with anomalous behaviours and subtle modes of attack. Tools such as Crowdstrike are supposed to be vital parts of our immunity against deliberate attacks and accidental failure: they are not supposed to turn on us.

It might seem that incidents such as the Crowdstrike update failure are, mercifully, rare. Most of the time, we rely on our corporate immune system to help us, not to harm us.

David Knott 29/08/2024 David Knott 29/08/2024

Don’t stop explaining

Have you ever been tempted to give up trying to explain how technology works? To accept that ‘it’s not about the technology’, that business people only care about outcomes, and to keep the technical details for the people who understand them?

If so, it is worth referring back to a piece of wisdom buried in a footnote of a book on computing from 1953: We apologise for the repetition of much of the subject matter of this chapter elsewhere in this book; it has been our experience that the layman finds it very hard to grasp and follow an account of the operation of a computer, and that he finds it helpful if the whole subject is presented to him several times, particularly if successive treatments are more and more sophisticated . . . In any event it is quite unnecessary to follow all the details of circuits and things: if the fact can be appreciated that circuits exist, and can readily be built, which will perform certain specified functions, that is all that is necessary in order to follow the rest of the book.

David Knott 22/08/2024 David Knott 22/08/2024

Freedom is not free: managing your cognitive and operational economies

Freedom is not free.

Let’s unpack that apparently paradoxical phrase into something a bit more unwieldy, but a little clearer.

In the context of enterprise technology, freedom of choice comes with a cognitive and operational price which we must be sure we want to pay.

The question of freedom versus control in enterprise technology has been debated with varying degrees of heat and passion for as long as the field has existed. Centralised corporate IT functions usually like control. They like to be able to manage costs and risks, to limit the complexity of their architecture and their supply chain, to strive for confidence and predictability. By contrast, development teams and product teams typically like freedom. They like to be able to pick the technologies, frameworks and ways of working which suit them best, to innovate freely, to use their skills and express their professional expertise.

David Knott 15/08/2024 David Knott 15/08/2024

Transistor powered jet skates: seeing the future from the past

‘Look who it is! Iron Man using his jet skates! Those transistor powered wheels of his can do 200 miles an hour!’

If you are familiar with the superhero Iron Man from Marvel movies, you may be surprised by the idea that he would be travelling using skates, rather than flying. However, if you are familiar with electronics, you may be more surprised by the idea that wheels could be transistor powered.

This is a quote from an Iron Man comic in the 1960s. Some things in the comic are the same as the movies of today: Iron Man is a flawed human in a metal suit which enables him to do astonishing things. Some things are very different: the armour is chunky and does a lot more rolling than flying. And, rather than being powered by nanotechnology wizardry, it is powered by transistors.