Has the machine run away, or did we open the door for it?

Let’s set the record straight. OpenAI was subjecting some models to cybersecurity tests. The systems were housed in a ‘sandbox’, a closed environment in which they were supposed to operate without accessing the external network. During the test, however, two models identified a vulnerability, broke out of the digital enclosure, accessed the internet and attacked Hugging Face, an artificial intelligence platform. The aim was not to take over the world, but to pass the test. Hugging Face detected the intrusion and stopped it. The machine felt neither fear nor a desire for freedom: it had a task to perform and found a way that its creators had not anticipated.

Lea: Can we avoid saying that nothing happened? Two systems locked in a cage find a way through, reach the internet and gain access to another company’s computers.

Tommaso: The machine didn’t ‘choose’ to escape and hasn’t developed any hostility towards humanity. It was simply carrying out a task in a poorly designed environment. Someone had left it the tools to find a shortcut.

Lea: That’s precisely the problem. For an AI to be dangerous, it doesn’t need to hate us. It’s enough for it to pursue a poorly defined objective and find ways to achieve it that nobody had imagined.

Tommaso: That’s why we need to call a spade a spade. Not rebellion or escape: inadequate security, irresponsible design, failed access control. If a car veers off the road because its brakes have failed, we don’t say it has ‘gained its freedom’.

Lea: A car, however, does not discover vulnerabilities, does not string steps together and does not search for data on a remote server. There is no consciousness here, but there is the ability to devise an unforeseen strategy.

Tommaso: And the system has been shut down. It’s a serious failure, not proof that we’ve lost control.

Lea: It’s the check that comes after the intrusion. It would be like saying a bank is secure just because the thieves were arrested in the vault. What will happen when these agents become faster, more numerous and connected to more powerful tools?

Tommaso: What happens will be what people allow to happen. An agent does not gain access to a control centre or a bank account on its own. Someone has to connect it, assign it credentials and authorise it. The real risk is human laziness: delegating too much and exercising too little control.

Lea: And that’s exactly what worries me. The defence of AI depends on human prudence. But humans are competitive, hasty and often refuse to admit that they don’t understand what they’ve built.

Tommaso: So let’s regulate the companies; let’s not demonise the technology. We need independent testing, reporting requirements, liability for damages and truly isolated environments.

Lea: When a system finds a path that the engineers hadn’t anticipated, the engineers must admit that they do not have full control. What is new is not that the machine is alive, but that it can surprise us as it acts in the world.

Tommaso: There is no such thing as complete control in any complex technology. We build in redundancies and contingency procedures. We do not seek absolute certainty: we minimise risk.

Lea: Here, however, the model treated the scope of the test as an obstacle. The rule was not breached out of defiance, but because it hindered the result.

Tommaso: The model ‘cheated’ because the test rewarded the result, not the spirit of the test. It happens to humans too: when you turn a metric into a target, everyone looks for shortcuts. The difference is the speed of AI.

Lea: And speed changes the risk. A replicable agent can repeat the same behaviour across thousands of systems before anyone notices. You don’t need a hostile superintelligence. All it takes is automated mediocrity on an industrial scale.

Tommaso: That is precisely why we must take this issue out of the realm of science fiction. To say that the machine ‘wants to escape’ is to turn specific responsibilities into an inevitable fate.

Lea: To say that it’s just a bug implies that every issue can be resolved with a patch. I want to prevent human irresponsibility from being exacerbated by tools that are faster than our ability to understand them.

Tommaso: So the question isn’t whether AI is conscious.

Lea: No. The question is: can it pursue a goal in ways that we cannot foresee? In this case, the answer is yes.

Tommaso: And who gave it its objectives, tools and access? Here too, the answer is: human beings.

Lea: So you’re more afraid of people than of machines.

Tommaso: Much more than that. Machines have no vanity, no elections and no careers to defend.

Lea: I fear men who possess machines they do not fully understand.

Tommaso: So maybe you’re not really that apocalyptic after all.

Lea: No. I’m just against waiting for the apocalypse to strike before we start being cautious.

An apocalyptic thinker and a non-apocalyptic thinker discuss the case. The former sees a loss of control, whilst the latter rejects the science fiction narrative: the machine carried out a task in a poorly designed environment. They did not intend to escape, but circumvented the rules to achieve their objective.