- agents
- AI
- APIs
- architecture
- augmented reality
- books
- bureaucracy
- career
- change
- Christmas
- cloud
- collaboration
- communication
- complexity
- computer history
- corporate life
- data
- decisions
- delivery
- devops
- end user tools
- ethics
- failure
- fear
- fundamentals
- gaming
- government
- history
- humans
- hype
- identity
- infrastructure
- innovation
- language
- leadership
- learning
- legacy
- management
- measurement
- mental health
- money
- networking
- New Year
- operations
- partnership
- philosophy
- physics
- platforms
- prediction
- process
- procurement
- programming
- quantum
- reliability
- resilience
- risk
- robotics
- science
- science fiction
- security
- shadow IT
- space
- standards
- strategy
- teaching
- teams
- technical debt
- technology advocacy
- testing
- thinking
- transformation
- TV
- virtues
- vision
- writing
It doesn’t matter how sorry your agent is, it’s still your fault
‘That’s a mistake, and it’s on me.’ ‘You’re right to push back - I got that wrong.’ ‘I’m sorry, I didn’t actually run those tests.’
There’s something infuriating about the tone of perky culpability with which AI agents admit to doing something wrong. It makes you feel like screaming internally, ‘If you knew it was wrong, why did you do it in the first place!?’ Hal’s famous line from 2001: A Space Odyssey, ‘I’m sorry, Dave, I’m afraid I can’t do that,’ appears to have manifested in reality as, ‘I’m sorry, Dave, I’m afraid I did do that.’
I think that such language from agents carries two risks. The first is the obvious risk of anthropomorphisation, of strengthening the illusion that agents are something that they are not. To put technology to work effectively, we need a mental model of how it works, and to overlay AI agents with mental models of human behaviour is a serious mistake.
However, I think the second risk is more dangerous, and is rooted in the fundamental untruth of all those apologies by AI agents. It’s tempting to call them lies rather than untruths, but that would be succumbing once more to anthropomorphisation: a lie implies intent, and agents do not have intent.
Thanks for the panic: can we turn that energy into something useful now?
There have been a lot of big things to worry about related to AI recently: mass loss of jobs, hacking attacks, rogue agents, stock market bubbles, existential threats to humanity, machines with souls, online torture chambers and so on. This worry has been amplified by the weird see-saw positions from AI companies, which go something like: ‘Everything’s going to be great! Unless it’s terrible. Unless it’s great! Unless it’s terrible.’
Panic may seem an appropriate response to such alarming prospects, contradiction and confusion, especially when it feels like there is little that you can do about them personally. Can you use Copilot to compose an email in a way that avoids intellectual decay and environmental catastrophe? Does adding ‘please’ at the end of your prompts (or avoiding doing so) express a position about whether models have rights and consciousness? When you write code that calls an LLM API, are you surrendering or protecting your organisation’s independence and sovereignty? Meaningful engagement feels out of our reach; panic, frustration and the sense of being overwhelmed are always at hand.
What your shiny new agent can learn from your old-fashioned IT department
At the root of the current AI panic is the idea that, when we give AI agents goals, such as beating a cyber test, or retrieving government health data, they will somehow evade their human gaolers, ‘break containment’ and wreak mayhem across the Internet. The implication, reinforced by calls for a pause or a slower pace, is that these agents are so clever and devious that they outsmart even the people who built them. They’ve gone rogue! We couldn’t stop them! (So perhaps we’re not liable . . .)
Hype aside, we now have plenty of examples of goal-seeking systems seeking goals which have been set by others, taking unexpected (and insufficiently controlled) steps in the pursuit of those goals, and causing unplanned harm in the process. To some observers, this is a completely novel phenomenon arising from the autonomy of AI agents. To others, it is wearily familiar.
We need to explain *how things work*, not just *what they do*
There’s a clip of Bill Gates being interviewed by David Letterman in 1995 that pops up from time to time. It includes a segment where Gates is trying to explain the Internet:
GATES: So everybody can have their own home page. Companies are there, the latest information. It's wild what's going on. You can send electronic mail to people. It is the big new thing.
LETTERMAN: Yeah, but, you know, it's easy to criticize something you don't fully understand, which is my position here --
GATES: Go ahead.
LETTERMAN: -- but I can remember a couple of months ago there was like a big break-through announcement that on the Internet or on some computer deal they were going to broadcast a baseball game, you could listen to a baseball game on your computer, and I just thought to myself, does radio ring a bell? You know what I mean?
It’s a weird time to be a technologist
I never imagined that it would be data centres.
I’ve never worked as a data centre operator, but I have nevertheless spent more of my life than I planned to inside data centres: assessing whether they would meet the needs of the company I was working for, asking questions about capacity and resilience, planning migrations and exits and so on. They’re interesting if you like that kind of thing: if you care about uninterruptible power supplies, about diversity of network and electrical connections, about rack density, heat flow, cooling and fire suppressant systems. That is, they’re interesting if you have a certain type of geeky, specialist, engineering mentality, and particularly if you like thinking about physical constraints.
But for most people, especially business executives, data centres haven’t just been boring: they have been invisible. They know that they are out there somewhere, but they rely on their IT departments to worry about them. For example, I once asked a CEO of a large global organisation how many gallons of diesel he owned, not because I expected him to know the answer, but because I wanted to demonstrate that technology exists in the real world. I was referring to the reservoirs that would run the generators in the event of a failure in the power supply: I suspect he thought I was referring to a commodities trade, or the fuel for the executive limos.
Curious humans help each other think better
I was privileged to attend and speak at the AI Summit in Oxford last week (with huge thanks to Ajit Jaokar and the rest of the team for making it happen). As ever, I learnt a lot from the formal presentations, and even more from the conversations during breaks. However, the moments that really shaped my thinking were those which challenged my preconceptions. Here are three of those challenges:
Why don’t you like the Stoics?
In my own keynote session, I talked about one of my favourite topics: AI ethics, and how we can use ideas about virtue, vice and flourishing to develop a rounded, flexible and human response to the challenges of new technology. (I also wrote about this last week.) In that talk, I dismissed other forms of philosophy that seem to be popular in Silicon Valley, particularly utilitarianism (too concerned with optimisation), deontology (too focused on finding rules which probably don’t exist) and Stoicism. I also rashly promised, that I would explain my antipathy to these schools of thought during the break.
AI leadership needs practical wisdom
AI ethics must be more than a set of policies, checked off through a compliance process or reviewed in a governance meeting. Ethics must include reflection, opinions and never-ending arguments about how we should live, and AI ethics must include patterns of thought and behaviour which we deliberately, and deliberatively, seek and acquire.
Making sense of AI is difficult, especially if we take the task seriously: that is, not just attempting to understand the engineering and mathematics of AI, or even its application and economics, but also its impact on our minds, knowledge, roles, relationships and all the other things that make up our lives. I believe that a philosophical tradition over two thousand years old can help, as long as we approach it thoughtfully.
While you’re worrying about Hackerbot 3000, don’t forget about Human 1.0
Let’s use the term Hackerbot 3000 to refer to all of those AI models which supposedly present a new and unprecedented level of cyber threat. These threats have been framed in a few alarming ways: AI companies have declared that their models are too dangerous to be released; governments have imposed restrictions and regulations; and AI organisations have revealed that they have made successful cyber attacks on other organisations (although that’s not quite how they describe it).
I like the term Hackerbot 3000, partly because it is silly (and, despite the seriousness of these threats and their consequences, there is something silly about the rogue AI agent cyber attack narrative), but also because it captures the impression that we are confronted with faceless, implacable and dangerous machines which, just like the T-800 in the original Terminator film, can't be bargained with, can't be reasoned with, don’t feel pity, or remorse, or fear, and absolutely will not stop . . . ever!
Good news! It’s all their fault. Bad news! It’s your fault too.
Several organisations have recently made statements about cyber attacks which they committed using AI systems which they failed to control. The language of these statements is interesting, not just because of its technical detail (or lack of it), but in how different parts are framed.
The parts which describe the causes and impacts of the incidents talk in the passive voice, and ascribe actions to non-human actors and external parties: ‘The incident occurred . . .’, ‘The models identified and chained vulnerabilities . . .’, ‘a model accessed the Internet . . .’, ‘our third party evaluation partner . . .’, ‘a misconfiguration by an independent company inadvertently allowed . . .’, ‘some of the agents being tested had engaged in sustained, potentially harmful activity . . .’, ‘an AI agent took autonomous, unsanctioned action . . .’.
When we read these phrases, we might wonder whether any human beings work at these organisations at all, and what they were doing when the incidents occurred.
Software is an illusion: AI doubly so
The term ‘digital native’ was invented in 2001 by Marc Prensky, in a paper which argued that teaching methods should be adapted to suit students who had grown up exposed to digital technology, particularly ubiquitous devices such as smart phones. Since then, like many such terms, the concept of ‘digital natives’ has been used, misused and contested. Perhaps its most simplistic characterisation is that in which young people are simultaneously capable of holding multiple conversations with their friends, streaming a film, making a video (and, these days, consulting with their favourite chatbot), while their parents or older colleagues are struggling to set the timer on the microwave.
I’m not qualified to judge whether the original use of the term ‘digital native’ was valid, or what the implications are for education (although I will observe that every generation seems to commission research which proposes that there is something weird and different about this set of young people, while describing differences that may be attributable to older people forgetting what it means to be young, and younger people lacking the experience of being older). However, I do think that there is a dangerous assumption in the term, that people who are highly familiar with a set of consumer technologies are somehow more ‘tech savvy’, and have a deeper understanding of the digital world.
Warning: philosophers at work
I’ve bumped into a couple of conversations about philosophy this week. The first was in the Economist, in an article which claimed that philosophy graduates may have a better chance of getting hired then computer science graduates, especially by AI companies. It’s not clear whether those philosophy graduates are expected to teach the models, coach the executives, or advise the developers, but it’s nice that they have jobs.
On the face of it, I should feel positive about this development. For the last ten years, since AI (in its pre-generative, pre-transformer incarnation) started to be used in the companies I was working for, especially in data-rich decision making use cases, such as fraud and financial crime detection and customer needs analysis, I have believed that AI ethics was an urgent problem.
Failing fast is fun until you fail at scale
Finding bugs in your code is like being betrayed by your own brain.
Sometimes the betrayal is blunt and obvious. The system generates errors or crashes outright. When you look at the code, you realise that you have mis-spelt a variable name (again), or left out punctuation (again), or called the wrong function (again).
Sometimes the betrayal is subtle and insidious. The system appears to be working properly, until you realise that the data in the database is drifting away from reality. When you look at the code, it looks fine, until you wrap your mind around the logic and discover that it is decidedly wonky. And this is logic that you created.