Archive
Subscribe on LinkedIn
AI leadership needs practical wisdom
David Knott David Knott

AI leadership needs practical wisdom

AI ethics must be more than a set of policies, checked off through a compliance process or reviewed in a governance meeting. Ethics must include reflection, opinions and never-ending arguments about how we should live, and AI ethics must include patterns of thought and behaviour which we deliberately, and deliberatively, seek and acquire.

Making sense of AI is difficult, especially if we take the task seriously: that is, not just attempting to understand the engineering and mathematics of AI, or even its application and economics, but also its impact on our minds, knowledge, roles, relationships and all the other things that make up our lives. I believe that a philosophical tradition over two thousand years old can help, as long as we approach it thoughtfully.

Read More
While you’re worrying about Hackerbot 3000, don’t forget about Human 1.0
David Knott David Knott

While you’re worrying about Hackerbot 3000, don’t forget about Human 1.0

Let’s use the term Hackerbot 3000 to refer to all of those AI models which supposedly present a new and unprecedented level of cyber threat. These threats have been framed in a few alarming ways: AI companies have declared that their models are too dangerous to be released; governments have imposed restrictions and regulations; and AI organisations have revealed that they have made successful cyber attacks on other organisations (although that’s not quite how they describe it).

I like the term Hackerbot 3000, partly because it is silly (and, despite the seriousness of these threats and their consequences, there is something silly about the rogue AI agent cyber attack narrative), but also because it captures the impression that we are confronted with faceless, implacable and dangerous machines which, just like the T-800 in the original Terminator film, can't be bargained with, can't be reasoned with, don’t feel pity, or remorse, or fear, and absolutely will not stop . . . ever!

Read More
Let’s avoid the equation ai_fragility * software_fragility = very_bad_things
David Knott David Knott

Let’s avoid the equation ai_fragility * software_fragility = very_bad_things

Anyone who has built classical, deterministic software systems can tell you how fragile they are. Such systems perfectly follow precise, explicit instructions, and are incapable of operating outside those instructions. That means that, if you have written instructions which do not specify the behaviour you want, which are inconsistent with one another, or do not cater for unexpected values, your system will fail. If you are lucky, that failure will be loud: a crash with an ugly error message. If you are unlucky, it will be quiet: steady, ongoing corruption of data values which you won’t discover until later.

Anyone who has built newer, AI driven, probabilistic software systems can also tell you how fragile they are. Such systems treat inputs (prompts, context, training data and so on) as factors which affect the probability of getting different outputs. That means that, if your inputs are insufficient to tilt the balance of probabilities in favour of your desired outputs, your system will fail. If you are lucky, that failure will be obvious: results which are factually incorrect or logically incoherent. If you are unlucky, it will be obscure: results with subtle biases which you won’t discover until later.

Read More
Good news! It’s all their fault. Bad news! It’s your fault too.
David Knott David Knott

Good news! It’s all their fault. Bad news! It’s your fault too.

Several organisations have recently made statements about cyber attacks which they committed using AI systems which they failed to control. The language of these statements is interesting, not just because of its technical detail (or lack of it), but in how different parts are framed.

The parts which describe the causes and impacts of the incidents talk in the passive voice, and ascribe actions to non-human actors and external parties: ‘The incident occurred . . .’, ‘The models identified and chained vulnerabilities . . .’, ‘a model accessed the Internet . . .’, ‘our third party evaluation partner . . .’, ‘a misconfiguration by an independent company inadvertently allowed . . .’, ‘some of the agents being tested had engaged in sustained, potentially harmful activity . . .’, ‘an AI agent took autonomous, unsanctioned action . . .’.

When we read these phrases, we might wonder whether any human beings work at these organisations at all, and what they were doing when the incidents occurred.

Read More
Build a movement, not an enclave
David Knott David Knott

Build a movement, not an enclave

The current wave of AI products have been around long enough for enterprises to launch ‘AI transformation’ initiatives, to wonder why those initiatives aren’t yielding the results they hoped for, and to reach for new ways of working to try to jolt them into life.

For people who have worked in technology for more than a few years, this cycle is familiar: a new technology becomes generally available (cloud, mobile, web, Big Data and so on); bold claims are made for the impact of that technology (cut costs, improve customer engagement, make better decisions and so on); initial projects fail to make that impact; big transformation programmes also fail to make that impact; enterprises make organisational changes to try to get the benefits they were promised.

If you have seen this cycle many times, you have probably developed several survival tactics: try to get behind the hype and learn the fundamentals of the new technology; learn what it can do and what it can’t do; calm unrealistic expectations; salvage value from failures; and, ultimately, work with your peers to develop sustainable professional practice. 

Read More
Locally rational; strategically irrational
David Knott David Knott

Locally rational; strategically irrational

Enterprises frequently find themselves sliding down the slope from strategic rationality to strategic irrationality, while acting in ways which seem locally rational. To put it another way: if everyone’s so busy, how come we’re not going anywhere?

Working for a large enterprise can feel like being on a ship where the boiler is being stoked, the engines are churning, the propeller is turning, the kitchen is turning out meals, the brasswork is being polished, but, when you look out the window, you are still at the dock.

Actually, that might be too optimistic: many enterprises feel like a ship where the coal is being delivered to the kitchen, the stewards are working in the boiler room and the navigator is holding the charts upside down.

And if you work in technology, it often feels like the engines are overdue for a service, steam is escaping from the valves, and the pumps are struggling to keep the water at bay, but despite the constant, frantic activity, basic maintenance does not seem to be anyone’s job.

Read More
Software is an illusion: AI doubly so
David Knott David Knott

Software is an illusion: AI doubly so

The term ‘digital native’ was invented in 2001 by Marc Prensky, in a paper which argued that teaching methods should be adapted to suit students who had grown up exposed to digital technology, particularly ubiquitous devices such as smart phones. Since then, like many such terms, the concept of ‘digital natives’ has been used, misused and contested. Perhaps its most simplistic characterisation is that in which young people are simultaneously capable of holding multiple conversations with their friends, streaming a film, making a video (and, these days, consulting with their favourite chatbot), while their parents or older colleagues are struggling to set the timer on the microwave.

I’m not qualified to judge whether the original use of the term ‘digital native’ was valid, or what the implications are for education (although I will observe that every generation seems to commission research which proposes that there is something weird and different about this set of young people, while describing differences that may be attributable to older people forgetting what it means to be young, and younger people lacking the experience of being older). However, I do think that there is a dangerous assumption in the term, that people who are highly familiar with a set of consumer technologies are somehow more ‘tech savvy’, and have a deeper understanding of the digital world.

Read More
Warning: philosophers at work
David Knott David Knott

Warning: philosophers at work

I’ve bumped into a couple of conversations about philosophy this week. The first was in the Economist, in an article which claimed that philosophy graduates may have a better chance of getting hired then computer science graduates, especially by AI companies. It’s not clear whether those philosophy graduates are expected to teach the models, coach the executives, or advise the developers, but it’s nice that they have jobs.

On the face of it, I should feel positive about this development. For the last ten years, since AI (in its pre-generative, pre-transformer incarnation) started to be used in the companies I was working for, especially in data-rich decision making use cases, such as fraud and financial crime detection and customer needs analysis, I have believed that AI ethics was an urgent problem.

Read More
What was I thinking?
David Knott David Knott

What was I thinking?

A few months ago, I decided to set up this blog site to hold all of the articles I have published on LinkedIn, whether in the A Lot to Learn newsletter or standalone articles from before LinkedIn used newsletters.

That might seem like a self-indulgent exercise, but I thought, first, that it might be useful, and, second, that, if it was useful, I might need a content lifeboat. Unfortunately, social media platforms tend to decay, and I anticipated that there might come a point in the future where I didn’t want to be on this platform any more. (I won’t comment in depth here about the quality of LinkedIn, other than to say that I have a mental threshold for the number of toxic, divisive posts I’m prepared to see, and, sadly, the content here increasingly approaches that threshold.)

Whatever the motivation, I found the process of transferring articles across to the new site interesting. I kept thinking that this is exactly the sort of busywork that I should be using AI to help with, but found that I enjoyed the process of reviewing what I had previously written and trying to figure out what I thought about it.

Read More
Failing fast is fun until you fail at scale
David Knott David Knott

Failing fast is fun until you fail at scale

Finding bugs in your code is like being betrayed by your own brain.

Sometimes the betrayal is blunt and obvious. The system generates errors or crashes outright. When you look at the code, you realise that you have mis-spelt a variable name (again), or left out punctuation (again), or called the wrong function (again).

Sometimes the betrayal is subtle and insidious. The system appears to be working properly, until you realise that the data in the database is drifting away from reality. When you look at the code, it looks fine, until you wrap your mind around the logic and discover that it is decidedly wonky. And this is logic that you created.

Read More
Rethinking phase three: your AI adoption programme is stalled, but your organisation is adopting AI anyway
David Knott David Knott

Rethinking phase three: your AI adoption programme is stalled, but your organisation is adopting AI anyway

I used to tell a story about AI adoption which went something like this . . . 

AI adoption in traditional organisations has taken place in three phases.

In phase one, prior to the public release of ChatGPT, AI was a specialist pursuit for specialist people, solving specialist problems with specialist datasets. AI was used within banks to detect fraud and financial crime, or in retail to manage stock levels and distribution. But it was not knowingly used by most people most of the time.

After the release of ChatGPT, AI was available to anybody who could frame an English sentence and type it into a browser. It seemed to become a general purpose technology which could be applied to any business process which needed to handle language or make decisions.

Read More
Performative reasoning
David Knott David Knott

Performative reasoning

How many of your reasons are comforting illusions? How many of the decisions which your organisation makes are based on well-ordered reasoning, and how many are simply surrounded with the trappings of reasoning in order to make you feel better?

Organisations make a lot of decisions. What products to buy, what products to launch, who to hire, where to invest, which projects to support and which initiatives to cancel. These decisions are particularly apparent in the field of enterprise technology, where we make choices about how to design, build and operate systems, how to organise resources and how to adopt and integrate new capabilities. The need to tell machines precisely what to do seems to require precision in our own thinking.

Because these decisions seem important, we feel that we should take them seriously, and be seen to take them seriously. When we are making purchasing decisions, we construct elaborate scoring criteria, invite bids and conduct extensive evaluations. When we are planning investments, we build detailed business cases, evaluate ROI and risk factors, and construct portfolios of change. When we are running delivery programmes, we create dashboards, produce reports and run steering meetings, so that we can respond to circumstances and keep everything on track.

Read More