The Trap of Artificial Autonomy and the Necessary Return of the Human

Source: Il Sole 24 Ore – “The Mathematical Trap of AI Autonomy and the Need for the Human”, Paolo Benanti, December 2025

The illusion of the autonomous agent

Paolo Benanti’s article addresses one of the most delicate issues in the current phase of artificial intelligence: the promise of autonomy. AI agents are presented as systems capable of perceiving their environment, making decisions and acting without continuous human intervention. It is a powerful narrative, one that fuels the idea of an ever-expanding delegation to machines.Yet behind this promise lies a structural misunderstanding: autonomy does not coincide with reliability. And, above all, it does not eliminate the responsibility of those who design, deploy and use these systems.

The mathematics of cumulative error

At the core of the analysis is not philosophy, but mathematics. Benanti recalls a simple principle that is often overlooked: in complex processes, even minimal errors tend to accumulate. An agent operating through long decision-making chains may maintain high local accuracy while failing dramatically in the final outcome.This is the so-called “compound error trap”: as task complexity increases, the probability of a correct overall result drops sharply. Autonomy, in this sense, is not a linear achievement, but a zone of growing risk.

When AI exceeds its natural domain

The article shows that AI agents perform best in narrow, repetitive and well-defined contexts. Data extraction, summarisation, classification, support for bounded decisions: here automation generates value.The problem arises when these systems are pushed beyond their “point of equilibrium,” and tasked with governing long, open-ended processes characterised by high uncertainty. In such cases, AI does not become more efficient—it becomes fragile. Error ceases to be an exception and becomes a structural possibility.

The false alternative between human and machine

Benanti rejects the simplistic opposition between human and artificial intelligence. The real frontier is not replacing humans, but redesigning their role.AI should not become an opaque decision-maker, but a tool that amplifies human judgment, keeping supervision, context and responsibility in human hands. Total autonomy is not progress; it is de-responsibilisation disguised as efficiency.

Big Tech and the temptation of total delegation

This reasoning takes on political weight when applied to the Big Tech model. The autonomy of AI agents is not merely a technical choice, but an economic lever: it reduces costs, accelerates processes and shifts responsibility.Yet the greater the autonomy, the greater the asymmetry between those who control the algorithm and those who bear its consequences. Without genuine human oversight, algorithmic error becomes systemic and invisible, difficult to challenge for citizens, workers and institutions alike.

Algopolio’s perspective

Benanti’s article reinforces a core conviction of Algopolio: AI is not neutral, and autonomy is not a value in itself. Every automated system embeds choices, limits and interests.Algopolio operates precisely in this critical space, where technological innovation risks turning into a loss of democratic control. Defending the role of the human means defending transparency, accountability and the possibility of contestation.

In an era in which artificial autonomy is sold as inevitable, recalling its limits is not technophobia—it is political clarity.And without such clarity, AI does not make us freer. It only makes us more dependent.