Innovation and application

Photo credit: Roberto Sorin on Unsplash

A couple of months ago, I wrote about the book The Elements of Computing Systems, by Noam Nisan and Shimon Schocken, and how it was helping me traverse the layers of abstraction out of which computing is constructed. After many hours of reading, thinking and programming, I am nearing the end, and was struck by a few sentences in the penultimate chapter which are worth quoting in full:

Modern high-level programming languages are rich and powerful. They allow defining and using elaborate abstractions like functions and objects, expressing algorithms using elegant statements, and building data structures of unlimited complexity. In contrast, the hardware platforms on which the programs ultimately run are spartan and minimal.

At a time when Moore’s law has been running for decades, when semiconductor manufacture is an industry of global significance, and when ever more specialised chips are being produced to optimise the performance of artificial intelligence models, it’s easy to forget that, at root, all this silicon is, as Nisan and Schocken say, ‘spartan and minimal’. However we arrange them, the atomic units of computing remain ones and zeroes. (Let’s leave quantum computing to one side for a moment.)

We should further remember that these arrangements of ones and zeroes, and their use in computation, were defined all the way back in 1936, by Alan Turing in his paper On Computable Numbers, with an Application to the Entscheidungsproblem. Famously, Turing approached the problem of determining whether an algorithm could be solved by computation by defining a hypothetical machine with an infinite tape, which could be moved backwards and forwards, its marks erased and written. The difference between this automatic-machine, or a-machine, as Turing called it (or Turing machine as we now call it) and our modern computers is that our computers actually exist - but the fundamentals are still the same.

There is a sense in which all of modern computing is implicit in Turing’s theory, and that all we have done in the near-century since is to work out some of these implications.

I think it is worth, in the midst of all our projects and upgrades and deployments, in the middle of our digitisation and automation and transformation, to remember where we came from and how we got here, and to realise that, however elaborate and sophisticated our digital world appears to be, we are only starting to explore some of the infinite space of computing potential.

I also think that there is a more prosaic lesson for those of us who work with enterprise technology. We work in a field of innovation, where new products, techniques and capabilities emerge almost daily. We naturally become focused on the next wave of change, on the version after this version. This is apparent in the coverage of large language models, which is often dominated by stats such as the number of parameters in the model, the cost of training, or the size of the datasets.

These advances are important, but if we pause to think that everything we are doing is still working through the implications of concepts developed in the 1930s, then we remember how much potential remains to be realised from the technologies that we already have. If no new large language models appeared for the next ten years, we would still be figuring out new ways to make the most of the capabilities that exist today. Furthermore, if we examine the way in which most of our enterprises operate and are organised, we quickly realise that we have not yet realised the full potential of basic coding and automation technologies.

This is not a suggestion that we should cease innovation: continuous creativity is one of the things that makes computing exciting and rewarding. However, it is a suggestion that we should also deploy our creativity in application, and revere it just as much: we still have a lot of work to do.