Failing fast is fun until you fail at scale

Finding bugs in your code is like being betrayed by your own brain.

Sometimes the betrayal is blunt and obvious. The system generates errors or crashes outright. When you look at the code, you realise that you have mis-spelt a variable name (again), or left out punctuation (again), or called the wrong function (again).

Sometimes the betrayal is subtle and insidious. The system appears to be working properly, until you realise that the data in the database is drifting away from reality. When you look at the code, it looks fine, until you wrap your mind around the logic and discover that it is decidedly wonky. And this is logic that you created.

This is one of the many reasons that I advocate learning to code, even in the age of AI (we’ll get to AI in a minute). If you have never written code, then you underestimate the ability of your brain to betray itself when attempting to construct precise logic. If you have written code, then no mistake seems implausible, and you learn never to trust code which does not have comprehensive tests. (And then you learn to mis-trust the tests.)

This difference in attitude is apparent in the exhortation, usually by business leaders, to be ready to fail fast. I’ve written before about how non-technologists often think that they have to encourage technologists to be willing to take risks and accept failure, while technologists treat risk and failure as constant companions. This time I want to consider whether people who advocate failing fast really mean what they say.

The meaning of the term ‘fail fast’ has mutated a few times. It was originally a programming term, describing systems which were intended to give early warning of potential error conditions. It was then adopted as a business term, describing the approach of treating your business strategy as an experiment, pivoting your approach in the face of objective data. In current corporate life, it is used much more loosely, and typically denotes a) a general desire to go faster, b) a wish not to be bogged down with irritating details such as thinking things through, or boring steps such as testing and security checks, and c) a promise that no-one is going to get punished or fired when everything goes wrong (apart from the people who get punished or fired). A term denoting software safety has become an excuse to ignore (among other things) software safety.

I think that one reason that this corrupted meaning of failing fast has become so prominent in corporate life is that (absent learning to code), non-technical business leaders tend only to think of certain types of failure in software projects. They anticipate that the project might deliver late or spend too much. This is entirely survivable: projects go wrong in this way all the time. They anticipate that the project might not deliver its projected benefits. This is also survivable: most organisations are so relieved when projects are finished that they don’t bother measuring benefits anyway. And they anticipate the blunt and obvious forms of failure caused by bugs: screens that don’t look right and systems that crash. These are definitely survivable for business leaders: they can point at their suppliers or technology teams.

Meanwhile, the technology teams (who expect their brains to betray them) know that the rush to deliver that comes with the directive to ‘fail fast’ can result in other, more insidious forms of failure. They worry about the errors in the code that, for example, store the transaction date in the date of birth field. Or write the current customer’s details on the previous customer’s record. Or apply a 5% discount instead of a 0.5% discount. And they know that, because of the speed of computing, this type of fast failure can mean millions of mistakes are made before anybody notices. When computers fail fast, they sometimes fail at scale, and in ways that may be impossible to recover from.

It is 2026, so it is natural to suppose that the answer may lie in AI. AI can generate code, so perhaps people don’t need to learn how to code after all. AI can generate tests, so perhaps we can finally have comprehensive test coverage. AI sounds confident about the quality of everything it produces, so perhaps we should trust it and take it at its word.

Of course, life is more complicated than that. If you’re going to delegate your reasoning to a tool (and coding is a highly structured and unforgiving form of reasoning), then you’d better understand what that tool is doing. You’d better recognise that tests are code too, and ask whether that code works as expected. And you’d better recognise how misleading the apparent confidence of AI output can be, and the dangers of thoughtless trust.

Furthermore, AI introduces new ways of going wrong. The non-deterministic nature of models means that, unlike traditional code, you can run the same test twice and get different results. And, even if you can build reliable tests, they can never exhaust the boundaries of the model’s behaviour.

Writing tests for software has always been a task that requires an ability to think up edge cases that the designers and developers have not anticipated. Developing evaluations for AI models demands even more imagination and tenacity. And the tendency of AI solutions to return results which are incorrect but convincing means that their failures are more likely to be subtle and insidious than blunt and obvious.

This does not mean that we should not use AI for development or build it into our solutions. Learning to code teaches us that software is fragile and that we can be betrayed by our own brains. It also teaches us the techniques we need to make software tough and to overcome our own betrayals. AI exposes us to the betrayals of other people’s brains: if we are to use it well, that means understanding its behaviour and outputs at a detailed level, and learning the equivalent techniques of safety and control. These techniques and tools are still evolving, but the only way to develop them is through practice.

The notion of failing fast is a valuable concept, as long as we are clear about what it means. If you find yourself saying that a project should be allowed to ‘fail fast’, it’s worth asking: exactly what sort of failure are you prepared for, and how fast are you prepared to let it happen?