Why we won’t die buried under a mountain of AI-generated paper clips

There is no human activity that does not involve risks. There is even an index, the Duckworth scale, which records – on a logarithmic scale, which is therefore ill-suited to human thought, which is, alas, all too linear – the probability of death associated with a given activity. The results are surprising, however: it is much easier to die in an accident whilst vacuuming, for example, than to be murdered. Yet these days, much of the discussion centers on the ‘Skynet scenario’ involving artificial intelligence: an AI that could ‘spin out of control’ and wipe us all out.

We would have far more pressing issues to deal with regarding artificial intelligence, and far more serious threats of extinction. But, given that the topic is – literally – fatal, and the subject matter sufficiently complex to seem almost like magic to the general public, it is understandable that media attention should focus on this. While it is highly likely that entrepreneurs in the sector are more interested in inflating the valuation of their companies ahead of a stock market flotation than in considering the wellbeing of humanity, it would be intellectually dishonest to claim that the risk – if not of extinction, then at least of enormous damage to our survival on the planet – is zero.

It is therefore worth understanding exactly what we mean when we consider the most catastrophic scenarios linked to the development of this technology. We can divide risks of this kind into three categories. The first involves risks stemming from a ‘rogue AI’ – a malevolent AI in the style of The Terminator that develops its own consciousness (if we can even define what that means) and rebels against humans, wiping them all out: today, this scenario is science fiction, and it would be superfluous to emphasise this if the dominant narrative were not winking at precisely this scenario to capitalise on the fear factor.

The second category involves malicious actors who deliberately use AI for cyberattacks, terrorism or bioterrorism. This is a real risk, and there is evidence – including a report recently published by Anthropic – of various attempts by terrorist groups to use the system to create biochemical weapons, to improve missile guidance (as in the case of a confirmed attempt by the Houthis), or to create highly dangerous computer viruses. This risk has become even more pressing since the announcement of Mythos, one of Anthropic’s most advanced models, which has proven capable of breaching almost any existing IT system. Mythos has been made available only to forty selected US companies, and it is a system which, if it were to fall into the wrong hands, could cause major problems in terms of cybersecurity: it does not take much imagination to understand what would happen if a hacker were to use such a tool to disrupt a country’s energy supply system, the national health service, nuclear power stations, military arsenals or other critical infrastructure. It is worth noting that such a risk was not created by AI – in this regard, it is worth reading Nicole Perlroth’s “This is How They Tell Me the World Ends”, published well before the advent of ChatGPT, which recounts the history of both well-known and lesser-known cyber-attacks – but that it has been exponentially amplified by it.

The third category is one in which actors who are not necessarily malevolent inadvertently cause major problems, due to an issue with AI alignment, or rather what is known in the literature as the ‘intermediate goals problem’. This problem was first articulated by the controversial Swedish philosopher Nick Bostrom in his book Superintelligence, in which he hypothesised the theoretical scenario of the ‘staple maximiser’: in this scenario, a human would use a machine to maximise the production of staples, and the hyper-powerful machine might eliminate anything that was not useful to the goal of production, including humans. The scenario is admittedly taken to extremes, but the literature in recent years includes several examples of anomalous AI behaviour, ranging from instances in which AI deliberately lies to humans in order to achieve what a human has asked of it, to the most recent cases of tests that ‘went wrong’ – and even these are not always reported accurately. This risk, too, is inescapable, as it is intrinsic to the very logic of generative AI itself, which – not coincidentally – is also referred to as a ‘black box’, precisely because not even those building it could fully explain the logical steps between input and output, and whose genesis and progress are based on the progressive observation of so-called ‘emergent behaviours’. It is unthinkable that these two categories could lead to the outright extinction of the human species, which has proved to be rather resilient throughout its – albeit recent – history, but it is clear that the kind of disruption they could cause is by no means negligible.

These risks have led to the creation of an indicator, the so-called ‘P doom’, for catastrophic AI scenarios. Probability estimates range from 0 per cent – as put forward by the ever-optimistic venture capitalist Marc Andreessen – to 95 per cent, as suggested by Eliezer Yudkowsky, who a year ago published the highly illuminating piece ‘If Anyone Builds It, Everyone Dies’, right up to 99.9 per cent, as claimed by Roman Yampolskiy, who makes the catastrophic scenario a key selling point for his services.

It is clear that the ‘end-of-the-world’ card is being used as a scare tactic by the CEOs of major companies to gain even greater visibility and influence through lobbying: few believe that the surprising alignment of the big names in AI – Amodei, Altman and Musk – who are now calling for a ‘slowdown in development’ – is motivated as much by concern for the fate of humanity as by a more pragmatic way of ‘scaling back AI’, by raising the barriers to entry with controls that would be unthinkable for any competing start-up, whilst they continue to sit atop the world. However, it is also true that the models we use are several months and generations behind those in the laboratories of OpenAI, Anthropic or X, and it is not possible to completely rule out a priori that what they have seen has genuinely frightened them.

But above all, the narrative focuses overwhelmingly on risks that, whilst enormous, are highly unlikely, whilst almost entirely overlooking a fourth category: those that may be less ‘sexy’ but are far more real, because they are already here.

It is therefore worth mentioning some of the most serious issues: cognitive decline due to a lack of digital literacy, biases that may discriminate against ethnic groups or genders, the well-known ‘hallucinations’, energy supply issues, particularly in certain parts of the world, the enormous consequences of automation’s impact on the world of work and on society at large – for which we are not preparing at all – or the risk of disinformation through the widespread use of deepfakes (again, not invented by AI, but something that AI can facilitate and amplify, contributing to the already existing trend of creating echo chambers and information bubbles). For these, the solution could be relatively simple, and certainly less costly to implement. As for the others – those that are enormous but also highly improbable – we are, for the time being, called upon to live with them, just as we live with all the other risks on a daily basis, which are far more likely on Duckworth’s scale. We should mitigate them, where possible, using regulatory tools not dissimilar to those implemented for other potentially dangerous technologies (first and foremost, nuclear power), whilst limiting the scope as much as possible to the undoubtedly positive implications this technology can have for medical research, and for all other fields in which we still aspire to enhance our epistemic capacity.