Software is an illusion: AI doubly so

Photo credit: Randy Jacob via Unsplash

The term ‘digital native’ was invented in 2001 by Marc Prensky, in a paper which argued that teaching methods should be adapted to suit students who had grown up exposed to digital technology, particularly ubiquitous devices such as smart phones. Since then, like many such terms, the concept of ‘digital natives’ has been used, misused and contested. Perhaps its most simplistic characterisation is that in which young people are simultaneously capable of holding multiple conversations with their friends, streaming a film, making a video (and, these days, consulting with their favourite chatbot), while their parents or older colleagues are struggling to set the timer on the microwave.

I’m not qualified to judge whether the original use of the term ‘digital native’ was valid, or what the implications are for education (although I will observe that every generation seems to commission research which proposes that there is something weird and different about this set of young people, while describing differences that may be attributable to older people forgetting what it means to be young, and younger people lacking the experience of being older). However, I do think that there is a dangerous assumption in the term, that people who are highly familiar with a set of consumer technologies are somehow more ‘tech savvy’, and have a deeper understanding of the digital world.

This assumption is not confined to differences between generations. Every office contains at least one expert who knows the finance system, or the expenses system, or the CRM system, or the core banking system, and that everybody turns to when they have a question (often in preference to calling the service desk, and probably in preference to talking to that support chatbot you are building). The know-how that these people have is valuable, just like that of the people who know how to navigate social media platforms or get the best out of their phone. Given our dependence on technology, learning how to control the technology we use every day has become, well, an everyday skill.

However, this know-how is a useful illusion. The menus, options and interfaces which these people have learnt is an abstraction: a set of visual and tactile metaphors and analogies which do not resemble the actual operation of the machine. We can try to get past this abstraction by looking at the code, only to realise that this is yet another illusion: a set of symbols which must be turned into binary digits, which then become electrical signals being pushed through circuits.

This illusion is not a bad thing. Indeed, it is essential: the modern digital world would not exist if every one of us had to work out the bit level logic required to make computers work. Furthermore, the illusion is explicit. When we say that a pattern of pixels on a glass screen is a ‘button’, we know that it is not really a physical button, even if we interact with it by pressing it. When we say that a cluster of bytes in a chip or on a disc is a ‘file’, we do not imagine that there is a pile of paper somewhere in the machine, even if we organise the file by dragging it between ‘folders’.

I think that the illusion with current forms of generative AI is different, though, and that we have to work harder to remember that it is an illusion. This is because the interface of many is language, and we are configured to respond to language. If we are literate, and we see a written sign, it is almost impossible to see shapes rather than words. If we hear someone speaking in a language in which we are fluent, it is almost impossible to see sounds rather than words. And when we understand those words, it is hard not to respond. (Consider hearing someone say, ‘Excuse me,’ or ‘Oi, you!’ in a crowded train.)

This means that the interface of generative AI (language) is different from the interface of more traditional software (buttons, windows and menus). The metaphorical nature of the traditional user interface reminds us that it is an illusion and an abstraction. The linguistic nature of the AI user interface can seem just like the real thing.

And it seems that we are taken in by the illusion. Even if we leave aside more speculative questions such as whether AI is conscious (other than stating my position: it isn’t), it is not hard to find examples of people ascribing authority to AI. You might have encountered this in conversations or arguments which include the words, ‘Let’s ask AI . . .’ or ‘I fed it to AI and it says . . .’ or ‘Have you tried asking AI . . ?’. I’ve been surprised by the number of times I’ve encountered these words spoken by serious people in serious contexts, with the implication that they were going to take the AI generated responses seriously.

We can get useful output from AI, just as we can from other forms of software. If we are to trust that output and extend it authority, though, we need to know what is real and what is illusion, which illusions are helpful and which illusions are harmful. With traditional software, the illusion is explicit and apparent; with AI it is much harder to see. That should not prevent us trying.

(The paraphrased quote in the title of this article comes, of course, from The Hitch Hiker’s Guide to the Galaxy by Douglas Adams, where Ford Prefect says, ‘Time is an illusion. Lunchtime doubly so.’ When you find a line like that on page 22 of a book, you know that you are in safe hands for the rest of the way.)