David Knott 26/01/2023 David Knott 26/01/2023

How do we get what we want from AI systems - and human systems?

I am sure that you have heard of the paperclip problem. Just in case you haven’t, it is the idea that, if you ask an AI system to make paperclips, then it may go on making paperclips, until the whole world is nothing but paperclips. There’s even a fun game based on this concept.

The paperclip problem illustrates the problem of setting goals for AI systems which represent what we truly want. Unlike us, AI systems do not come ready equipped with goals and desires: we have to provide them, in the form of what is often known as a reward function.

And crafting this function can be more difficult than it first appears. When I wrote some recent articles on generative AI, it was suggested that I read the book Human Compatible by Stuart Russell. It’s a great book, and triggered lots of other reading: it took me down a rabbit hole of articles and papers about optimisation, particularly a phenomenon known as specification gaming.

David Knott 19/01/2023 David Knott 19/01/2023

The technology learning curve has never been steeper, or more important to climb

The first time I tried to learn about computing was difficult, because I didn’t actually have access to a computer. I just knew that this was a field I was interested in. My school had a Commodore PET which I wasn’t allowed to touch, but there was something eerie and fascinating about the glowing letters on its screen. I wanted to be able to make those letters do what I wanted them to do.

So, I got a book from the library and tried to read it. Fortunately, it was a book on BASIC, which most microcomputers ran at that time. I wasn’t able to run programs, but I was able to learn the commands and the syntax, and to write some simple programs on paper. When I finally got my hands on my own computer, it helped me make a quick start.

And, years later, when I got my first paid programming job, writing COBOL on an ICL mainframe, I was able to transfer many of the skills I had learnt on my home computer into the work environment. Even though I learnt quickly that the GOTO I had relied on for many of my amateur programs was frowned upon, most of the other logic constructs worked and were useful. And that has been my experience throughout my programming life: these days I mostly write Python, but many of the basic constructs in Python do the same job as, well, BASIC.

David Knott 12/01/2023 David Knott 12/01/2023

In the wonderland of new technology, let’s be curiouser and curiouser

What can Aristotle, a philosopher from the 4th century BCE, teach us about technology that was released in 2022?

Well, he wrote about techne, practical skill, the term that gives technology its name. However, I think that we can learn more today from what he wrote about arete, or virtue.

Aristotle’s conception of virtue experienced a renaissance in the 20th and 21st centuries, and many better people than me have written many words on the topic. For our purposes, though, we can understand virtue simply as a habit of thought and behaviour of which we approve, which forms part of a flourishing life, and which is self-reinforcing: the practice of virtue makes us virtuous. Virtues include attributes such as courage, honesty and generosity, and are opposed by vices such as cowardice, dishonesty and miserliness. Aristotle also suggested that vices are extremes, and that virtues are the mean that navigates between them. For example, generosity is the mean between being a miser and being a spendthrift.

David Knott 05/01/2023 David Knott 05/01/2023

Generative AI and the duty to understand

Generative AI worries me.

This is an unusual position for someone like me, who has worked and played with technology for most of their life. It’s impossible not to be intrigued and excited by the sudden appearance of software which is capable of generating convincing text, or creating images far better than I could ever hope to produce..

But, for the last few weeks, as I have been attempting to learn in public about generative AI, I have grown increasingly concerned, not just about its ethical implications, but about the ability of the companies that will put generative AI to work to grasp and respond to those ethical implications.

In those weeks, I have learnt enough to have a rough mental model of how generative AI, particularly Large Language Models (LLMs) work: put simply, they are large scale statistical models, trained on huge quantities of data, implemented through multi layered neural network architectures, which predict an acceptable visual or linguistic response to a text input (a prompt).