The Critical Theory attributed to the Frankfurt School is primarily the work of thinkers and philosophers such as M. Horkheimer, T. Adorno, H. Marcuse, E. Fromm, W. Benjamin and J. Habermas, who from various angles, investigate the structures of the modern capitalist societies, including forms of domination, the role of ideology and the possibility of emancipation.
Here, we argue that many of the issues raised by Frankfurt School are highly relevant for the AGI debate, existential risk and the dramatic changes to human life in the presence of future superintelligent machines.
The core ideas of the Frankfurt school reveal how modernity with all its promises of emancipation and betterment of human life collapses into new forms of domination. Furthermore, they argue that capitalist domination is sustained through culture, ideology and psychology. The function of Critical Theory is to expose the hidden power structures at work and to help contribute to emancipatory change. In other words, “critique”, according to them, is proposed as “the path” towards emancipation.
The relevance of the Frankfurt school to the AGI debate is that it rejects the idea of “value neutral” science. This includes in particular AGI as its creation is done in the intersection of technology, science, psychology, sociology and political theory, just to name a few, most of which, even if science were excluded from the list, cannot be considered particularly value-neutral.
To begin with, the futility of compiling a set of universally objective human values is quite obvious. But even if such a set of values existed, the subjectivity of the values, their interpretation would be a major problem. It is also plausible that no set of values will be able to incorporate the plurality of human thought and that a particular subset will have to be chosen, which in turn can/will give rise to a dominant discourse putting the AGI in the hands of a dominant group that possesses those particular values.
But even if we were to base AGI on a thought experiment where humans somehow agree upon a diverse and inclusive set of human values accepted by everyone, it can be argued that a perfectly aligned machine poses a much bigger threat to humans. And that is the instrumentalisation of a superintelligent, hence unquestionable, machine that will determine and dominate human life from science to psychology, sociology etc. Historically, we have witnessed how humans to whom natural human values are ascribed can commit atrocities within systems of power (Nuremberg trials).
This is in fact the Frankfurt school’s critique of instrumental reason. Their argument is that rationality can (enlightenment did) devolve to a tool for domination, which is exerted through institutional bureaucracy (oppression) and technology as a means of controlling human life. In one of the major texts of the school, Dialectic of Enlightenment, Adorno and Horkheimer describe the rise of fascism and stalinism as a failure of enlightenment: Reason gave birth to fascism by reducing everything to measurable utility. In other words, an AGI, no matter how aligned, is projected to give rise to a much more dystopian world of unquestionable actions from which humans are entirely removed and can exist only as subjects.
An interesting new ingredient to the AGI debate and the emergence of superintelligent systems seems to point in multiple directions compared to the one observed by the Frankfurt school that considers fascism as a product of capitalism and the social conditions imposed by it. Adorno and Fromm point out the authoritarian family structures giving rise to fascism through rigid obedience-prone personalities.
First of all, a superintelligent system that takes all sensible, if not all, decisions for humans, will replace traditional authoritarianism with an ultimate technological authoritarian whose decisions cannot be scrutinized (due to its superiority), let alone be understood as it would be beyond our understanding, or critiqued as it would be the ultimate reasoning authority. While capitalism turned humans into disciplined labourers whose work and free time were dominated by the economic system as creators and consumers (Adorno views leisure in capitalist systems as a continuation of labor), the impact of AGI will be far more profound as it will dominate every single aspect of human life, indeed creating what reality an particular individual can see, what she does, why and when. This is a total invasion of the human psyche and a complete removal of personal space, all at the service of an omnipotent machine that is naturally created to serve the goals of the dominant classes.
But humans live with humans and so the question is what, if any, type of morality or values will be necessary in human societies. Will AGi turn humans to passive subjects rejecting their own morality or will it turn them into passive subjects who abide by the morality of the machines?
Besides the commodity fetishism (art becoming decor) and consumer capitalism, which we already observe, a complete destruction of meaning can be foreseen in the sense that art, creativity, criticism if even possible, will be outsourced to the algorithmic procedures designed to serve the omnipotent designers of machines who may or many not even have control over their own creations.
A closely related subject, also pointed out by the Frankfurt School are the notions of Total administration and One-dimensional man. According to Marcuse, advanced capitalism creates one-dimensional man who mistakes consumption for freedom and loses capacity to imagine otherwise. As no realistic and viable alternatives to the current capitalist systems are being discussed or proposed, the question is how the introduction of AGI will alter these conditions.
Let’s forget for now that no countries where AGI or some primitive versions of it present themselves as realistic alternatives, have plans to take care of the social conditions in the wake of future/current mass unemployment. The question is whether the addition of superintelligent systems that will affect every single aspect of human life will contribute to any form of emancipation where human values, morality and a range of pluralistic points of view will play any role at all. Social decisions, indeed the organisation of the society itself based on minimizing a crude loss function of some sort, and decisions that will likely be beyond scrutiny risk to reduce human life/role to less then what we have already observed under the current economic system. Adding to this, Adorno’s idea that society (even dissent) is a totally administered world under capitalism, it is reasonable to imagine that the future dominated by AGI presents itself as a much more grim alternative.
There can simply not be any meaningful emancipation if humans are declared superfluous to the formation of decisions and actions dictated by a perfectly aligned AGI (designed by tech feudalists). Consequently, it seems that the human condition under AGI and Tech feudalism will be much more dystopian than ever was under capitalism.
The AGI presents also a clear breakdown of traditional dialectics of Hegel/Marx. Imagining a superintelligent machine as a “thesis”, there is simply no “antithesis” and so no synthesis can be formed. Here, the relevance of the Frankfurt school becomes clearer than ever. True, Adorno presents the notion of Negative Dialectics as a resolution of the capitalist era syntheses that, in his opinion, masks oppression. For instance, freedom and authority synthesize to law which in many ways suppresses contradictions, forcing reality into false forms which is then used by capitalism to mask oppression. Adorno’s negative dialectics is proposed as a way out of this situation. It is a radical critiquing framework that is meant to shatter systems of domination.
It can be argued that AGI poses an even bigger threat to any possible criticism of dominating structures. Furthermore, it is also a breakdown of Adorno’s negative dialectics: A supposedly superintelligent machine that does not need human oversight, a machine whose decisions cannot be questioned or even understood, a situation where the only choice would be a total submission to an algorithmically synthesized product in a black box, is not only contradicts the traditional dialectics but also the one proposed by Adorno. How can we even criticize something we do not understand or follow?
Similarly, AGI presents itself as a totally new framework where Habermas’ Communicative Reality becomes irrelevant. Habermas argued that emancipation comes from open communication free from power distortions. This was proposed as a counterweight to Instrumental reason which, as explained above, is viewed as the root of fascism in capitalist structures. However, we clearly see that the current systems neither offer, nor need, nor encourage free communication. And even if they did, what freedom is there in surveillance capitalism where everything, thoughts, words etc have become a datapoint in the service of someone who can make a profit of it?