A few phrases to help resist AI illusions

Photo credit: Yumu via Unsplash

‘It can’t hurt you: it’s not real!’

You might hear those words from the hero of a horror, science fiction or fantasy film. They could be walking through a dream world, subject to a hallucinogenic drug, or under the spell of a sorcerer. They know that the things that they are seeing are not real, and that all they have to do is to try to ignore what they think they can see and hear. Telling themselves that what they are experiencing is not real is a guard against fear, against stepping off the path, or, worst of all, the temptation to talk back to the illusions.

Dealing with current forms of AI can feel like this. Not just because AI is surrounded by hype, marketing, inflated expectations and a big dose of FOMO. And not just because AI can be used to produce fake videos, fake images and fake words.

The reason that dealing with modern AI feels like dealing with an illusion is because the material it generates - language, images, videos, sound - invites us to believe that it is real, whether that is the intent of the human that instigated the generation or not. We are more engaged and entranced by it than previous iterations of computer interfaces.

In traditional computing, when a machine moved colourful pixels around a screen, we were charmed and entertained, but we rarely felt the need to ascribe it authority and personality. When a machine displayed a web page or a screen of data, we might have been inclined to afford it credibility, but only because we recognised the author, or the reality which the data aimed to represent. But when a machine talks the same language as us, we are naturally disposed to see something that resembles a person.

I am not qualified to address the social or psychological implications of this tendency. I am more concerned that those of us who design and build technology systems approach AI without illusions: that we treat it like any other technical component, with a full awareness of its properties, limitations and operational characteristics.

To do this, it might help to have a few phrases which do the same job as the words, ‘It can’t hurt you: it’s not real!’ If we keep repeating them to ourselves, we might be able to stay grounded - and remember how AI components differ from the other components that we use.

‘Models are models’

The term artificial intelligence has always been vague, woolly and disputed. In current discourse it has become even more vague and woolly - and used so often that disputes about its legitimacy are hard to hear.

In the world of enterprise technology, I think that precision matters: it helps to remind us what we are dealing with, whatever the label on the box or marketing material. And in the field of AI, we have precise terms that we can use: specifically, we can talk about machine learning models, or transformer models, or foundation models, or large language models. There are many other more precise and narrow terms which most of us don’t have the expertise to use confidently (I certainly don’t).

But we can anchor on one common term: model. This term helps us to understand what lies at the heart of the thing we call AI: a mathematical representation of a set of data. The means of producing this model may be different to other models: it involves randomisation, trial and error rather than conscious construction on the basis of theory and analysis. But it is nevertheless a model: a great big equation.

When we think of AI as a fancy wrapper for lots of maths, which is itself just an abstracted representation of lots of data, it seems rather less powerful, scary or capable of carrying a personality.

‘Models are inconsistent’

One of the fundamental expectations of traditional computing systems is that they are consistent: if you give a system the same inputs, you should expect the same outputs. But if you give a modern AI system the same inputs, you might get very different outputs. (If you haven’t experienced this, try it now: open up your AI tool of choice, and try asking it the same question in two different chats. I just did this with a simple query about my own published material - and got two results which were not only different in detail and facts, but significantly different in style and structure.)

This is part of the attraction of AI systems: the variation in responses makes them feel more natural, more alive, less robotic. It might make them appropriate for contexts such as customer service, where a natural tone and style matter.

But it makes them terrible for the types of deterministic systems for which we use computers today. If we want to know our bank balance or blood pressure, if we want to check a price or a passport, then we need to be confident that we will get the same answers as last time we asked - or that the answers only change because reality has changed, not because our prompt activated a different branch of the model.

When we remember that models are inherently inconsistent, we can figure out where that variation is valuable, and where we need confidence and certainty.

‘No-one knows what a model can do’

For most of the history of the software industry, software products have had well defined behaviours, and any deviation from those behaviours was a bug. The right response was to raise a ticket.

For current forms of AI, behaviour is very broadly defined (to caricature: ‘put some words in; get some words out; hope they’re useful and relevant’). When we encounter something we don’t expect, the correct response might be excitement (‘I didn’t know it could do that!) or dismay (‘I didn’t know it would do that.’).

For people buying and deploying AI systems, this unbounded behaviour can be exciting: the enterprise now has access to capabilities which it can’t even imagine, and hopes that its creative and ingenious team members will find ways to unearth them. What other product (other than a software compiler or interpreter) offers such unlimited potential?

However, for those same people this lack of boundaries can also be terrifying: anyone who can access the AI system may be able to make it do things which it is not intended to do. Despite several years of guardrails, testing and governance frameworks, no-one has yet found a way to guarantee that the most imaginative attacks and abuses will not make it through.

When we remember that models are entities with unknown and unbounded behaviour, it helps us decide where they work best - and where open-ended capability may mean open-ended danger.

AI (however vague, woolly and disputed the term) is an important technology, and like all technology, we can put it to work to do useful things. I am confident that we will continue to surprise ourselves with the things that we build. We stand a better chance of those things being safe and reliable if we keep repeating to ourselves: ‘models are models; models are inconsistent; no-one knows what a model can do.’