Midjourney concept art, by Michael Howe-Ely

When the moment of reckoning finally arrives, when the tidal wave of job loss from artificial intelligence has submerged every household in the country, when, no matter how hard we squint, we can barely discern a future for our children anymore, we will turn, in panic, to the government for relief. All the policy ideas for managing the crisis that sound so radical today — Universal Basic Income, treating AI like a public utility — will appear, in that moment, hopelessly meek. Facing annihilation, we will seek acts of heroism from the state commensurate to the scale of the emergency.

But by then, it will be too late. Whatever is left of what we know now as “government” will no longer respond to our needs. In a world in which humans play no productive role in the economy, our interests will have no relevance to the state. The only interests that will register to it will be those of the oligarchs.

We have never before seen a society devoid of human labor, but we have seen many in which such labor was cheap. And we know that democracy is impossible in such societies.

In the late nineteenth century, the French sociologist Émile Durkheim argued that the fundamental difference between “primitive” and “modern” societies was the degree of complexity of their respective divisions of labor. In societies with simple divisions of labor — say, a community composed largely of subsistence farmers — there is no immanent social glue that ties one self-sufficient household to another. Like a colony of ants, if a typhoon wiped out a quarter of the population of such a society, little would necessarily have to change in the way of its larger social structures. Religion, Durkheim argued, emerged to create, through ritual, that missing social glue.

In modern societies with highly complex divisions of labor, by contrast, we are all materially dependent upon one another. A few thousand TSA agents don’t show up to work and air travel across the country becomes a nightmare. Ships are blocked through the Strait of Hormuz and the global food supply is threatened. The difference between pre-modern and modern societies, in Durkheim’s view, is like the difference between a plant and a complex animal. If you cut off a few branches of a tree, they’ll just grow back, with no impact to the health of the overall organism. If you cut a few limbs off of a human being, we will be permanently disabled if we survive at all.

The relative value of the individual to the whole of society is thus greatly enhanced by a more complex division of labor. Durkheim believed that this is why, in such societies, individual rights are accorded an esteem that verges on the sacred: it’s a reflection of the greater community’s material reliance on each of us. The degree to which the state responds to the needs of its citizens or subjects, in other words, isn’t merely a matter of morality or ideology. It is a function of the material balance of power between a governed population and its government.

But what happens in a world in which human beings no longer play that vital role? Can individual rights survive in a society in which humans continue to consume resources but produce no economic value?

Modern examples would suggest not. A paper titled The Intelligence Curse (which I’ve discussed before) compares the future before us to the circumstances of people living in countries with abundant natural resources. Take the Democratic Republic of the Congo, for example. The DRC is the richest country in the world measured in its trillions of dollars of extractable minerals, yet one of its poorest in terms of human welfare. The government of the DRC is flush with wealth from foreign companies — now primarily Chinese — that purchase leases to mine the country’s cobalt, a critical mineral for rechargeable batteries. There is no incentive for the government to invest in education, urban infrastructure, or decent housing for the vast majority of its citizens, because the state’s fiscal health does not depend on taxing a prosperous citizenry, and the poverty of the Congolese provides foreign investors with cheap labor. The results are predictable: mass poverty, rule by oligarchs, tyranny. Economists describe countries like the DRC as suffering from a “resource curse.”

The paper’s authors expect the same to happen in an A.I.-dominated economy in which wealth is overwhelmingly produced by machines and not by humans. Why would the state, in such a world, continue to recognize such things as the civil liberties and personal property rights of a human population with no value to it? Why would it make concessions to those dispossessed of all bargaining power? Why would it regard us any differently than we regard the fish of the ocean?

Short of stopping A.I. research altogether — a prospect so unlikely as to be basically inconceivable — the main solution proposed by those in the field who worry about such things as democracy and human survival is AI alignment. “Alignment” refers to the process of bringing the ethical values of AI into harmony with those of humankind. It’s a technologically challenging task, potentially an impossible one.

It is also a wholly inadequate technical solution to this profound political problem. You can’t hard code democratic values into a large language model; you can only train it and hope for the best. But even if you could achieve that end, there’s no reason to believe that the frontier AI corporate leaders and politicians of the future will choose to do so. When people are handed power, they rarely decide voluntarily to forfeit it to the public. In a society in which the mass of humanity has little power, we can expect those who rule us to concede nothing.

It’s easy to see the outlines of what such a future governing regime will look like, because we’ve seen it before. After the barbarians sacked Rome, the kingdoms they established lacked the former Empire’s bureaucratic capacity to tax their subject populations. In place of taxation emerged a system of political patronage, in which favors, mainly in the form of land rights, were distributed to local strongmen with the military power to extract agricultural surpluses from the peasantry. Feudalism was the formalization of this mafia-style extortion economy.

We can expect something similar to arise in a world run by AI. Power and resources will accrue to the tech titans who still possess economic value to the new regime. The rest of us will compete for the favor of those elites by finding some way to serve their interests in a way that machines cannot yet do: perhaps by providing them human company, or menial servitude, or some other form of social subordination. Those who cannot compete in that arena will simply fail to survive. With time, those elites will find themselves fewer and fewer in number, as they are supplanted, like the rest of us, by machine intelligence.

Democracy is not just a set of values. It’s a social structure that rests on certain economic conditions — conditions that are not long for this world. Training ethical values into LLMs will do nothing to address this transformation. We will not prosper in a world of benevolent dictators, whether machine or human. As the economic value of human labor declines, our social value, in the eyes of the state, will decline accordingly. We are creating a successor society in which we have no future.