When talking to AI, be careful what you ask for

You’ve got to ask the right questions.

According to Herodotus, when Croesus went to the Oracle at Delphi to ask whether he should go to war, the Oracle replied that, ‘If you make war on the Persians, you shall destroy a great empire’. Encouraged, Croesus launched his war, only to find that he was defeated, and it was his empire that was destroyed.

This lesson seemed particularly important when I was attempting my second week of learning in public about generative AI. As with my exploration of quantum computing, I began this second week by opening my browser, entering some search terms and reading the top few news articles that were returned.

However, I found that I got quite different results depending on what question I asked. If I searched for ‘ChatGPT’, the AI chatbot which is making headlines right now, I got lots of very excited responses about astonishing things that the technology could do. If I searched for ‘generative AI’ I got some more thoughtful articles, typically from more specialist publications. I suppose that ‘generative AI’ is a slightly more technical term than ‘ChatGPT’, and it reminded me that I must get beyond the brand to the category.

If I believed the first articles that I found when I searched for ‘ChatGPT’, then I would conclude variously that it will; put journalists out of work; help students to cheat on exams; stand in for humans when negotiating with utility providers; and replace search engines. And, naturally, change the world, either for the worse or the better. What I struggled to find was any explanation of how ChatGPT could do all these things, other than that it was a Large Language Model (LLM) which had been fed with a huge corpus of pre-existing text.

When I search for ‘generative AI’ I learnt much more. I learnt that ChatGPT and other generative AI models are (in a very rough sense which I will attempt to refine in subsequent articles) predictive models, where they are trying to predict what should come next in a given context. It sounds similar to the experience of launching into a sentence without quite knowing what you’re going to say next: you take the sentence one word at a time, hoping that you will construct something coherent and eventually find your way to a full stop.

This is interesting and starts to help me understand how the technology is actually working. I know enough about other forms of AI to know that they are particularly suited to prediction and categorisation problems, and I can just about start to see how those capabilities could lay a predictive path through a new piece of text, creating it and navigating it at the same time.

The articles I read which talked about image models such as DALL-E, where even more interesting, as they described how the process of building an image was like tuning it out of static. For those of us who grew up twiddling the dials of old-fashioned TVs or radios, that sounds like the process of trying to find a signal which only exists because you are looking for it. There are many ghost stories waiting to be written here.

All of the articles I read, whether the prompt was ‘ChatGPT’ or ‘generative AI’, had an uneasy balance of optimism and pessimism, whether implicit or explicit. They all seemed excited about the potential of the technology, but concerned about the impact on creative professions, and puzzled about how the word ‘creative’ should be used in this context. The more thoughtful articles also noted that these models are effectively parasitic on the existing corpus of human knowledge, and raised questions about intellectual property and consent. If you’re an artist or a writer, it’s worrying to be told that AI is coming for your jobs - and even more concerning to hear that it learnt everything it knows from you and your fellow creators. 

To return to our lesson from Herodotus: it matters what questions you ask. As users of ChatGPT and other generative AI models have found, the results you get are highly dependent on the prompt that you enter. This is the case to such a degree that some people are even talking about ‘prompt engineering’. I don’t particularly like this term (except as a reminder that most tools that attempt to take the engineering out of engineering eventually lead us back to . . . engineering) but it does guide us to be thoughtful about how we approach a topic as new and uncertain as this one.

I plan to keep on asking questions, and to go a bit deeper next week into specialist literature.