Yuval Noah Harari — a ghost hunter of technototalitarianism

I was recently reviewing my self-study plans and decided once again to begin by thoroughly studying the ‘early’ Harari — starting with his books *Sapiens: A Brief History of Humankind* A Brief History of Humankind’ and *Homo Deus: A Brief History of Tomorrow*.

However, the very first random video I came across on YouTube highlighted a completely different theme from Harari’s later work.

And a couple of days later, a transcript of the interview was published. I listened to it again and read through it a couple of times.

The narrative is explosive. Intellectual, democratic, humanistic, yet potentially explosive. Perhaps because, as things stand, it is probably the clearest and most thorough critique of the processes currently unfolding on the planet and affecting humanity. Essentially, it’s a critique of any form of accelerationism and a warning against a new form of totalitarianism – one that not even science fiction writers have managed to conceive of.

NOT THE TERMINATOR

The first thing that springs to mind when we hear the words ‘machine uprising’ is *The Terminator*—a cinematic cliché that has been rather overused by now.

Its archetype features a supercomputer controlling hordes of domestic robots driven mad by class hatred.

Harari paints a different picture, one that differs sharply from the stereotype. In his view, the threat will be less centralised and almost entirely non-physical — at least, not physical in the conventional sense.

The main risk lies in AI beginning to replicate humanity’s core functions — the ability to speak, to produce information and to bureaucratise life as a whole.

This marks a significant shift in the debate on technology. Ten years ago, the main fear among futurologists was the loss of factory jobs. Today, the question is no longer whether an algorithm will replace a lorry driver or a shop assistant. There is no point in discussing the obvious.

The question Harari poses is much broader: who will make decisions on court verdicts, bank loans, public procurement, budget allocation and the reliability of news?

In this way, he raises the question of the likelihood of a loss of political sovereignty (!).

Perhaps this is why Harari’s symbol of the future is not the ‘Terminator’ – a robot soldier – but the ‘clerk’ – a robot bureaucrat.

On the one hand, such an image seems less dramatic. At the very least, it is not associated with the destruction of civilisation in the flames of nuclear war. However, Harari observes the key point with the insight of a biblical prophet: ever since Sumer, it is bureaucracy that has historically remained the main instrument of power.

No empire has ever been sustained by its army alone. Above all, it existed thanks to the people who collected taxes, maintained land registers, registered property and concluded transactions. This is probably why, in the 20th century, the sociologist Max Weber called bureaucracy ‘the most efficient administrative machine in history’.

Harari, however, argues that in the 21st century this machine will be able to function without people. Science fiction? Yes. But so realistic that it could become a reality within our own generation.

It is worth noting here that there is, in fact, nothing fantastical about his interview. There is no mention of ‘robots becoming self-aware’ or their struggles with moral dilemmas.

The only thing that truly interests Harari is the source of power. More precisely, the changing nature of power and its shift beyond the human realm.

No, this is not some new-age Luddism.

Yes, previous technological revolutions have given us steam engines, electricity and the internet. But all of this pales in comparison to artificial intelligence, which operates using the fundamental tool of our species and of civilisation as a whole – language.

LANGUAGE IS POWER

Yes, the steam engine replaced muscle power, and the computer standardised production processes. But the striking difference with AI is that it intervenes not in the production of goods, but in the decision-making algorithm.

Perhaps this is precisely why Harari considers AI to be the first technology capable of acting as an independent political entity. Previously, algorithms lacked agency. Now that they are increasingly writing, recommending, evaluating and so on, power is gradually shifting from humans to them. And this is all because algorithms are becoming faster, cheaper and, most importantly, more objective than any state apparatus.

It is probably at this very moment that the ‘specre of technototalitarianism’ is born. A phenomenon that requires no charismatic leader, no mass terror and not even censorship, since the algorithm merely ensures that the rules are followed.

In essence, it is the civil servant we dream of: honest, intelligent and, above all, incorruptible.

Its totalitarian nature is like a ripple on the water — brief, sporadic and practically invisible. It arises in the gaps left by the automation of human labour. This is where we hand over decision-making to algorithms. And over time, as Harari believes, we cease to understand their internal logic for ourselves.

Harari’s concerns are rooted in his view of history. Back in his book *Sapiens*, he put forward the thesis that our species conquered the planet thanks to our ability to collectively believe in fictional entities. These include such familiar concepts as states, money, corporations, religions and human rights. Strictly speaking, all of these exist because millions of people have agreed to regard them as reality.

Consequently, if civilisation is built on language, then technology that masters language better than we do begins to create its own stories, persuade people and shape new meanings.

Here, Harari’s views resonate with the philosophy of Jürgen Habermas. For Habermas, democracy is underpinned by the public sphere. This is a space in which citizens can freely and rationally debate and shape a shared political vision. Their decisions are legitimate as long as they arise from dialogue.

Harari transposes this idea into the 21st century. He believes that if algorithms generate information faster than we can verify it, the public sphere ceases to be human.

Yes, democratic institutions will still function, elections will still be held, and parliament will still sit. But! The social fabric will increasingly be woven by algorithms.

This web will begin to ensnare the remnants of informational sovereignty. The main challenge facing humanity will be the question: ‘Who controls the flow of information?’ Already today, suspicion falls on the tech giants, who are, step by step, building a rather asymmetrical form of ‘platform capitalism’.

AGAINST ACCELERATIONISM

As a liberal and a humanist, Harari is close to the classical critics of totalitarianism. However, unlike them, his version has no specific face. There are no Stalinist or Hitler-style moustaches in it. On the contrary, the defining feature of techno-totalitarianism is the absolute transparency of society to the system of governance. And the complete opacity of the system to the people.

Let us recall Hannah Arendt. According to her views, the essence of totalitarianism lies in the desire to subjugate all spheres of life. The party bureaucracy served as the instrument for this. In contrast, algorithmic bureaucracy will not resort to repression. But it will monopolise real power thanks to its efficiency, incorruptibility and honesty. On the one hand, this is the very embodiment of the ‘benevolent ruler’ archetype. On the other, it is clear that Harari is engaging in a veiled polemic with accelerationism, an influential intellectual movement in Silicon Valley. The most interesting thing here is that Harari and the accelerationists share the same view of the actual state of affairs. They differ only in their assessment of it.

Right-wing accelerationism – in particular, the works of Nick Land – is based on the premise that humankind is no longer the principal agent of history. Its place has been taken by capital, networks and machine intelligence. Therefore, according to Land, democracy is too slow and inefficient. Consequently, its demise is not a tragedy but a natural evolutionary process that must be accelerated.

Harari agrees with this description of current events. He, too, observes that algorithms are taking over governance functions, whilst political institutions are failing to keep pace with technology. But whereas for Land this represents a welcome progression, for Harari it is a catastrophe and a loss of human sovereignty.

In this sense, Harari’s critique of the well-known slogan of techno-optimists is quite telling: ‘We will create a god, but he will be our slave’. In Harari’s view, if AI surpasses humans in thinking and planning, it will cease to be a tool. A slave cannot remain a god, nor can a god remain a slave.

At the same time, strange as it may seem, Harari also disagrees with left-wing accelerationism — for example, with the authors of the ‘Inventing the Future’ manifesto, Alex Srnicek and Nick Williams. Left-wing accelerationists believe that technology can be harnessed to serve social needs through measures such as ‘democratic control’ and ‘planned regulation’. Harari shares none of this optimism. In his view, states are already losing out to corporations. At least, they do so when they become infrastructure-dependent on foreign technologies, satellites and cloud servers.

THE PREMISES OF SKEPTICISM

Harari’s arguments draw on the philosophical tradition of the twentieth century. Since around that time – perhaps beginning with Asimov – the finest minds have been trying to understand what will happen when technology ceases to be merely a tool.

• Jacques Ellul, for example, argued as early as 1954 in his work *La Technique* that the world is governed by the logic of technical efficiency. If something can be done faster and more cheaply, society will do it without giving the consequences a second thought.
• Michel Foucault analysed disciplinary power. In his view, it does not so much punish as measure, redefine and normalise. Whoever possesses knowledge of the population is the one who governs it. The potential of AI makes it possible to render this logic all-encompassing.
• In her book *The Age of Surveillance Capitalism*, Shoshana Zuboff described how digital platforms have turned people’s behaviour into a commodity. Harari takes her idea further: what will happen when this system begins to determine not only our purchases, but also our political decisions?
• Evgeny Morozov has for years criticised ‘technological solutionism’ — the belief that any complex problem can be ‘engineered’, in other words, solved with the help of an improved algorithm. According to this approach, social and political conflicts are treated as technical problems. As a result, justice gives way to optimisation.
• In his book *Seeing Like a State*, James Scott demonstrated how the state makes society ‘readable’ through censuses, tax registers and standardisation. AI, in turn, takes this readability to the absolute extreme. Whereas the state once observed society once every ten years during a census, today an algorithm observes it every second.

In short, Harari is not afraid that AI will become self-aware. He fears the emergence of a system that will know more about society than society knows about itself.

THE BLURRING OF RESPONSIBILITY

Harari is not a Luddite, nor is he a technophobe; he does not call for the destruction of computers or a return to some period in the past. In essence, all he does is question yet another wave of modernisation, yet another form of historical determinism.

Communists, for example, believed in the withering away of classes. Techno-fascists insist on the inevitability of the death of democracy, which will be forced to give way to more efficient systems of governance.

However, Harari is convinced that administrative efficiency is not the only criterion. At least, not in politics, the aim of which is to resolve conflicts between people with different values.

Yes, democracy is ‘slow’. But it is precisely this limitation on speed—all these debates, procedures and investigations—that acts as a safeguard against the monopolisation of power by anyone.

In this respect, Harari almost echoes Karl Popper and his critique of ‘historicism’—the belief in the existence of a single correct course of history. An illusion that led to the communist and fascist experiments of the 20th century – the bloodiest century in the history of our species.

Harari extrapolates this argument to the 21st century. The difference is that in the 20th century, the bearer of ‘historical truth’ was a party, whereas today it is increasingly becoming an algorithm. Furthermore, this ‘new totalitarianism’ will not require an ideology. Its decisions will be – or will be perceived as – mathematical, and therefore objective and well-calibrated.

And most importantly: who will be held accountable for all this? Who should be blamed for a mistake? The developer, the company, the state, or the algorithm itself? Previously, the author bore the responsibility. But what author does an algorithm have, apart from the owner who, for example, bought or leased it?

In Orwell’s work, totalitarianism takes the form of Big Brother. In Huxley’s, pleasure serves as the prerequisite for control. For Harari, it is a network of algorithms in which people do not, it would seem, perceive any obvious coercion. But gradually, step by step, they are distanced from their own decisions and delegate them.

A WORLD WITHOUT POLITICS

The main weakness of Harari’s concept, like that of many similar humanistic constructs, is the lack of a plan of action. Yes, he is a brilliant diagnostician, but he has no idea how to treat the tumour discovered in the body of human civilisation.

His remedies are weak and resemble homeopathy. All these international regulations and codes of ethics are like wishing someone a safe journey. Especially in a world experiencing serious geopolitical upheavals and gradually sliding into a new era of deglobalisation.

However, he shifts the focus of the discussion to a problem that may become the central one of the 21st century: will humans remain political actors in a world where decisions are made by algorithms that they are unable to fully understand?

If not, then technototalitarianism is our future. It will have no need for camps or mass terror. It will take the form of a very convenient, efficient and benevolent advisory system, much like a grandmother.

A ‘grandmother’ who will deprive people of the luxury of thinking. First, the need; then, the ability. And it will steer civilisation in the direction described as far back as 1958 by Isaac Asimov in one of his short stories. In it, a programmer ‘invents’ a way to multiply numbers.

Harari is not afraid of the emergence of an artificial god, which is commonly seen as the technological singularity.

He fears that people will first delegate power, and then relinquish even the exercise of that power.

And then technototalitarianism will set in — an era capable of bringing an end to our species’ 300,000-year history.

A species that will cease to be curious, and thus cease to evolve.