Social Studies

The story of this week, for those of you who were camping somewhere with no cell or wifi signal, was whether some non-negligible number of Americans believe that the actress Sydney Sweeney is a secret Nazi because of a pun in an ad campaign she was featured in, which starred, like most of her roles, her big naturals.

The story felt like a spacetime wormhole to five years ago, when people actually cared about things like this. But actually it was something much simpler, scientifically speaking: it was fake. There were probably 200 people in America who cared about the imaginary political implications of American Eagle’s snickering ad copy, and they were all internet fiends whose political consciousnesses were forged by the Covid-19 pandemic, which is to say they were brain damaged. Everything else was hype: the right-wing “backlash” to the non-existent liberal outrage, the media feeding frenzy over said fake outrage, the Vice President of the United States ridiculing the fabricated inanity of his political enemies, the President doing the same.

Back in 2020, these kinds of crazy episodes emanated from something that was actually happening in the culture: people were sincerely outraged by the most frivolous offenses, institutions with real power threw themselves behind the online mobs, meaningful consequences were suffered by people both famous and ordinary. This, by contrast, was a deepfake. We’re years past the mass psychosis formation of Peak Woke; none of the cultural conditions are in place for anything like this to actually occur again.

But 2020 was a great year for both the media and conservatives. Every day, journalists could wake up, see what’s trending on Twitter, and grind out some irresistible clickbait slop before their kids had even logged into their Zoom classrooms. Then every Republican in the country could jump into the fray, churn out some hackneyed viral snark about the blue-haired libtards and watch their follower counts grow, while their left-leaning friends and neighbors ran for their foxholes, hoping to make it to the end of the workday without being asked for their opinion on the latest cultural landmine.

There are a lot of people who would love to have that season of easy layups back again. So they’re memeing it into existence. All that’s required is a very bad take and a few anonymous, low-follower social media accounts that may or may not be right-wing shitposter sock puppets to voice it. That’s what happened with Sydney Sweeney and her denim-clad, Aryan Sisterhood mammary glands.

But you could tell from how quickly the story ran out of gas that there were no actual living, breathing, flesh-incarnated human beings feeling this particular outrage. No Democratic politician felt the grudging obligation to condemn Sweeney for her complicity in white supremacy. No internet mobs formed to retroactively cancel Brooke Shields for her nearly identical ad from 45 years ago. There was no clout to chase by carrying this torch forward. Everyone but those with a political or professional interest in doing otherwise rolled their eyes, shrugged and moved on.

I’ve written far too much in the past about the demented incentives of the media business to dwell on it further here. But the incentive for the political right to jump on a fake outrage story like this is somewhat new, if a bit obvious: they’re running out of things to gloat about. So far, both the tariffs and the immigration crackdown have been so unsuccessful at improving job numbers that the President was triggered into firing his chief labor statistician for reporting honestly on them. Israel’s ethnic cleansing of Gaza is so unpopular that the White House is suppressing dissent by censoring critics and threatening to withhold disaster relief funds from states that participate in boycotts of Israeli-affiliated products and services. As anyone could have predicted, Putin is ignoring Trump’s wishcasting an end to the war in Ukraine. DOGE was a joke. The Republicans have exploded the federal deficit and cut Medicare and Medicaid even as the President champions the acceleration of job-destroying Artificial Intelligence. On top of all that, Trump has chastised his own supporters for believing their lying eyes about the Epstein scandal. These are not the hallmarks of a successful presidency.

What conservatives could really use right now is an unhinged Democratic Party drunk on self-righteousness, scouring the earth for popular cultural icons to skewer. But that’s not in the cards anymore. That’s why, in the absence of a renewed woke insurgency to laugh at, the right has had to invent one. That’s what this psyop is about.

Get used to it. As our government sells the last granule of our democracy to corporate lobbyists, as our politics are drained of every drop of substance, as voters become more and more irrelevant to the political calculuses of our elected officials, expect to see more of this bullshit. This is what the culture war is for: to distract us with shiny objects while everything around us is stripped and plundered by those who really run the country.

I had a great conversation this week with Thad Russell on his podcast. Check it out. Also, JD Vance thinks I’m smart. I’m flattered but still vehemently disagree with him on AI and many issues besides.

When OpenAI, the company behind ChatGPT, was founded in 2015, it was established as a non-profit organization. Its mission was not to develop artificial intelligence, but “to ensure that artificial general intelligence” — autonomous AI systems that exceed human cognitive capabilities — ”benefits all of humanity,” regardless of which company first achieved it.

“The true mission isn’t for OpenAI to build AGI,” OpenAI CEO Greg Brockman told the podcaster Lex Fridman in 2019. “Our goal is to make sure it goes well for the world.” Developing AGI was just a strategy to advance its mission. By being the first to reach AGI, the company argued, OpenAI would be in a position to ensure it benefits humanity widely and that the technology is secured against the significant threats it poses to humanity, including existential risk. OpenAI was structured as a non-profit in order to focus on this mission and be “unconstrained by a need to generate financial return.”

Since that time, however, the company has gone through transformations in its corporate governance structure that many believe were designed to dilute its charitable mission in order to increase shareholder profits. “They have drifted from their mission,” said Tyler Johnston from the AI safety advocacy group The Midas Project.

On Monday, the Midas Project, along with EyesOnOpenAI and Encode, two other AI safety advocacy groups, released an open letter to the company signed by over a thousand people, including academics, AI researchers, and the “Godfather of AI” Geoffrey Hinton. The letter questions whether OpenAI’s current corporate structure is consistent with its founding mission.

“OpenAI is currently sitting on both sides of the table in a closed boardroom,” the letter reads, “making a deal on humanity’s behalf without allowing us to see the contract, know the terms, or sign off on the decision.”

I’m going to be contributing regularly to the Times of London going forward, reporting on, among other topics, Artificial Intelligence.

I’m new to the subject and I don’t know if I’m entirely persuaded that the end of the world is nigh, but I do think that those who anticipate human extinction at the hands of super intelligent AI have a very compelling case and I take it seriously. I have yet to find a rigorous argument against it.

Even short of extinction, though, the dystopian prospects of developing synthetic minds that far surpass our own are endless. When you start investigating the possibilities, it begins to look like the most unlikely, sci-fi ending to this story is that everything is the same but better, which is the outcome that our government is banking on. I’m not sure how replacing all of our jobs and making humans obsolete to the economy and to some extent to one another can possibly end well, but I’m open to hearing the argument of how that can happen — in fact, I’ve been actively searching for it.

Science fiction is fun because it allows you to construct thought experiments to stress test our existing social structures and forces. But we’re in the surreal circumstance now in which contemplating these scenarios isn’t just a fun hobby. I would argue that it’s literally the most important thing we can be doing right now, as a species. If our imagination fails us, we’ll be that much less prepared for a radically different near-term future that will deeply transform all of our lives, for better or for worse.

I’m going to explore a lot of these possibilities both at the Times of London and here on Social Studies. I invite you to take a look at my first article on it for the Times.

—LW

This one is free, but Social Studies can’t continue without your support. Please consider becoming a paid subscriber.

Barry Eisler, before becoming a best-selling novelist, worked in covert operations for the CIA. He joins Le Pod to discuss his career, the real-world inspirations for his work, and explains why fiction writing is more important than ever to understand the morally murky world of power politics and U.S. foreign policy.

Eisler is on Substack and you can bu…

Though the AI age is upon us, it’s still but a zygote compared to what it’s almost certainly going to become. If Silicon Valley has its way, within years, not decades, we will achieve human-level artificial intelligence, and then, soon after, artificial super intelligence.

That sounds great if you assume that an army of autonomous super intelligent minds will be content to just do our bidding forever, curing cancer and making unbelievably realistic video games and giving us the unconditional synthetic love of their digital companionship. But why in the world would you assume that? AIs are already developing behaviors that reflect their own weird “preferences,” and they routinely lie and deceive human beings. If you juice them up to 10,000 times their present intelligence, what are the chances they’ll be content to remain our lapdogs? What are the chances, by contrast, they’ll find our existence threatening, annoying, or just inconvenient, and casually wipe us off the face of the earth?

Humans need to start thinking seriously about what we’re doing. The future we need to grapple with isn’t a century away. It’s not even decades away. It could arrive at any moment. It could get here before Trump is out of office.

In this episode we take a cruise through some very dark places.

This episode is a freebie — enjoy. Then become a paid subscriber to Social Studies to get all the complete episodes of Le Pod.

The anti-monopolist writer Matt Stoller, author of the book Goliath and of the newsletter Big, is more than just a thinker I respect deeply. He’s also been a friend for over two decades. When I disagree with him on something, it’s with that foundation of trust beneath it.

I disagree with Matt on his complacency around AI-induced job decimation.

“Complacency” may be an unfair word. Matt is anything but a techno-optimist. He’s not saying don’t worry, your job will be fine, or even that AI will create more jobs than it destroys. He’s well aware that AI is going to transform society and the economy profoundly. It’s just that he thinks that the impact we’re seeing on jobs, at least so far, is not, in fact, the inevitable consequence of the technology itself. Rather, it is the outcome of policy choices made by corporate leaders and politicians, who then turn around and deflect the blame onto technology, which is a little like blaming it on the gods.

This has always been Matt’s greatest contribution to the political discourse: he understands the power of laws and won’t allow us to ignore it. Our social structures are not just impassive objects tossed to and fro by social and economic forces, he reminds us. They are also the result of conscious decisions we either make or fail to make. It’s an empowering reminder, insofar as it helps us understand that the forces of the economy are not mystical powers outside of our control. It also forces accountability on the powerful, by rejecting their pretenses that the self-serving choices they make aren’t really choices at all, but just the laws of the natural economy at work, that neither they nor anyone else has the power or the right to stop.

Matt’s insight is no less relevant to the current AI threat than it is to every other market dynamic. Many of the harms we’re supposedly seeing from AI, he insists, are human-made.

Take accounting, for instance. Accounting is a profession ripe for displacement by AI. But as Matt points out, accounting jobs started disappearing long before generative AI appeared on the landscape. People are avoiding careers in accounting because it’s become a bad job, with low pay, low prestige, and lots of stress. This is the result of political choices we’ve made, such as allowing the corporate actors who are audited by accountants to amass so much power that accounting has become a charade. Blaming AI is just ex-post-facto deflection.

Or take creative professions, like writing novels. Authors aren’t so much being displaced by AI as seeing their work stolen from them wholesale, in plain sight, by the AI labs. If criminals routinely rob the convenience store you manage with impunity, it’s not a problem of supply and demand; it’s a problem of lack of law enforcement. Intellectual property is no different. We shouldn’t allow thieves to get away with pretending that their crimes were just the inevitable outcome of the supernatural forces of economics and technological progress. A few broken eggs to make an omelette.

I agree with Matt on all those points, and on basically everything else he notes in his piece. But it barely touches the surface of what is to come.

AI is going to destroy jobs not because anyone necessarily wants them destroyed, but because it will render those jobs superfluous. That’s the very purpose of its creation. Its existence alone will compel these harms, whether or not corporate executives and politicians are on board. This is not just a political problem, and it will soon no more be a matter of our choosing than natural disasters or the spread of viruses are.

In their report “The Intelligence Curse,” which I highly recommend, Luke Drago and Rudolf Laine, two AI experts, describe what we are already beginning to see as “pyramid replacement” — a progressive process of obsolescence that starts at the bottom of a firm and gradually works its way up to the tippity-top until there’s nothing left of economic value that the firm produces that requires human labor.

It’s important to bear in mind, the authors point out, that AI is not like other technologies, like laptop computers, which increase worker productivity. It’s easy to mistake it for such at this moment in its development, because it’s still so crude. We can still pretend that AI will merely free us from the tedious tasks that entry-level workers are forced to do because it’s not worth wasting the valuable time of anyone more important to the firm to do them. This frees up those workers to take on more meaningful assignments that produce more value and that better advance their professional development. Not only will they enjoy their jobs more; they can then be promoted more quickly to better-paying positions.

But AI isn’t a widget. It’s obviously not going to stop there. “Remember,” Dragos and Laine write, “that today is the worst these systems will ever be.” AI labs are currently creating “agentic” systems — AIs that don’t just complete discrete tasks, but that take on entire projects and see them through to completion, mimicking human goal-setting, decision-making, and discretion-exercising. These will be autonomous actors that will require less supervision than your typical human employee does. They will be cheaper, faster, and better at their jobs than entry-level workers. That future is already here, though still in its nascency. Drive through San Francisco and every other billboard on the freeway is selling some AI agent or another.

As AI agents proliferate, employers will stop hiring entry-level workers. The entry-level workers they already have will either lose their jobs or will be promoted into junior positions as their former jobs are filled by AI. One can frame this as a “political” choice but it’s really not a choice at all. Firms will simply be forced by competition to adopt the new practice.

As agentic AI evolves, the water level will rise. Soon it will replace those same junior level positions the former entry-level workers were promoted into. In little more time, agentic AI will become advanced enough that the humans overseeing the junior-level AI agents — moving, as they do, at human speed — will become an impediment to the AIs’ productivity. Middle management will see its positions replaced, then senior management. Finally, the top executives will discover that even their jobs are disposable. After all, people aren’t perfect. “CEOs are forgetful, and they don't have total insight into everything their company is doing — but their AI systems do,” the authors of The Intelligence Curse write. With the replacement of the C-Suite, the entire firm will become human-free. Multiply that by every company on earth.

It may be the case, as Matt describes, that at the present time, corporations are making socially destructive but profit-maximizing decisions and then pointing the finger at AI. But as agentic AI progresses, these explanations will no longer be mere excuses. Certainly the C-Suite executives don’t wish to see their own positions replaced. As their boards force them out in the face of AI-induced hyper-competition, it will become clear that these processes are no longer being driven by human interest or initiative, even that of the most powerful among us. They will be driven by the requirements of the technology itself, which by that time will be wholly alienated from us and will, indeed, constitute an actual mystical force, one that is beyond both our control and our comprehension.

Today, there is still human agency at work. We still have a say in how the future unfolds, and there is a single political choice that governs all of this: the choice to continue researching, training, and advancing Artificial Intelligence. We could make laws to stop doing so. We could put bans or moratoria in place. We could enact pauses. We could proceed but within the parameters of international treaties. We are doing none of these things, and that is, indeed, a problem of our own creation.

But the longer we wait, the more irrelevant we will become to the course of events. By various estimates, we are within a year, or a few years, or perhaps a decade from Artificial General Intelligence — the point at which AI is as smart as the smartest humans. Certainly most of us will see it in our lifetimes.

AGIs will not require food and sleep as we mere mortals do. They won’t need breaks or time for family and hobbies. They will not have to spend years in school to learn their crafts, and they won’t have to train other AGIs to pass along their expertise. They will simply self-replicate, with all of their learning intact. It’s hard to conceive of a single cognitive job that could be preserved as a uniquely human domain in a world with AGI. As robotics advance, we may soon say the same about blue-collar work.

Nor is AGI the end of it. Human intelligence isn’t some theoretical threshold beyond which Artificial Intelligence cannot advance. Far from it. As AGIs multiply into legions, limited in their number only by the physical limits of the natural resources that power their data centers, AI research will accelerate to its exponential limits. We may then see our first glimpse of ASI — Artificial Super Intelligence — the intelligence of gods. When you start conceiving of the dystopian possibilities of a world of ASI, it becomes hard to take yourself seriously. You start to sound to yourself like a paranoid schizophrenic whose delusions were shaped by watching The Matrix. But the expectation of those in the field is that ASI is around the corner. Sam Altman calls OpenAI “a superintelligence research company,” and expects ASI to be around in the 2030s. (He thinks we’ll be just fine and the world will be a land of milk and honey.) Mark Zuckerberg is poaching OpenAI to try to get there even sooner. This is happening. And when it does, mass joblessness may be the least of our concerns. But at that point we’ll have no more say in the matter. The choices we’re failing to make now will have doomed us.

I wrote this for UnHerd, where it was published yesterday.

On Monday, the US Department of Homeland Security tweeted out a painting of a 19th-century American pioneer couple cradling a newborn baby. They’re sitting in a wagon, and beyond them you can see the vast landscape of the American West. “Remember your Homeland’s Heritage,” read the accompanying text.

You could contrive many different ways to interpret this tweet, but in the context of American politics, one of them is impossible to ignore. Both the language of the tweet and the aesthetics of the painting evoke a gauzy, nostalgic white nationalism that has become a subculture of what used to be called the “alt-Right”. Whether this was the painter’s intention is irrelevant. To anyone steeped in the vocabulary of the online culture wars, it’s obvious what the tweet is meant to convey.

Naturally, DHS doesn’t come right out and say it. Like any good shit poster, it leaves just enough room for ambiguity to allow it to answer its accusers with feigned shock and ridicule: how could you possibly infer racism from this wholesome depiction of our common American heritage? But this trolling doesn’t fool anyone. What’s being said is this: some Americans are more American than others. Specifically, those who can trace their lineage in the United States back many generations are more American than naturalised US citizens, or even the native-born American children of immigrants.

The word “heritage” is the giveaway. Among nativist-leaning American nationalists, the term “Heritage Americans” refers to those who can trace their lineage in the US back to the genetic stock of colonial America. To its proponents, Heritage Americans constitute an “American ethnicity” — basically WASP, with a smattering of German, and a sliver of Scandinavian. Some would also count black descendants of slaves as Heritage Americans, though others would not. Those who cannot claim this lineage are presumed to be somehow less American than those who can.

While DHS can’t say this outright, MAGA influencers such as Jack Posobiec can. A few days ago, at a Turning Point USA conference, Posobiec, aping Trump, claimed that New York City Democratic mayoral candidate Zohran Mamdani, a naturalised US citizen, is not, in fact, an American. “This idea that if you just hand someone a piece of paper that makes them American,” he said, referring to US citizenship, “guess what, it’s failed. It’s wrong. It’s not true.” “You are fake Americans,” Posobiec growled, “and we are going to smoke you out, every single one of you.”

A couple of days later, Posobiec’s “fake Americans” net grew even wider. Referring to Minneapolis mayoral candidate Omar Fateh, he tweeted: “This is clearly not an American.” Fateh was born in Washington, D.C., to immigrant parents from Somalia. There is no question of his citizenship status, though Trump’s executive order purporting to repeal birthright citizenship could conceivably change that. (When I asked Posobiec to explain exactly what separates a real from a “fake” American, he replied: “Everyone knows what an American is.”)

What Mamdani and Fateh have in common is that neither of them are “Heritage Americans” — a term that isn’t just legally meaningless, but also historically delusional. The conceit behind it is that the early Americans shared a common culture and set of values. This is because they were mostly English and universally Christian. Therefore, the narrative goes, they were united in their belief in such core Anglo-American virtues as democracy, rule of law, and limited government. Later waves of migrants, like those who went through Ellis Island, were further removed from these shared Anglo-American folkways, but not entirely so, as they were almost entirely from Europe. Since Americans opened their country to the rest of the world in the later 20th century, however, the nation has been inundated with people who are completely alien to American political and moral culture. From that began the dissolution of American society. This is roughly JD Vance’s point of view.

This is all ahistorical mythmaking. There was little that was culturally unifying across Colonial America. Just because the colonists were English and Christian didn’t mean they all got on — in fact, it meant the opposite. In the Old World, Christianity had been the source of endless fragmentation and war for 100 years, while England had been torn apart by those divisions more than perhaps any other country in Europe. These hatreds found their apotheosis in the bloodbath of the English Civil War, whose ripples spread across the Atlantic Ocean to the American eastern seaboard. There, the colonists of Virginia and the greater Tidewater region were faithful servants of the deposed King — many of them had fought for him as Cavaliers. Meanwhile, the colonists of Massachusetts Bay shared the Puritan faith of those who had chopped off the King’s head. What the two colonies had in common wasn’t “shared values”, but deep mutual contempt.

Social Studies is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

In the Old World, the various regional populations of England despised one another. In the American colonies, they could live apart in mutual disdain. With their newfound isolation and political autonomy, the differences between them became even more pronounced. In New England, the Puritans established a flat, communitarian social order topped by a strong and sometimes authoritarian government. Meanwhile, the Tidewater gentry reproduced the fixed hierarchical order of the English aristocracy as best they could. They in turn were loathed by the anti-statist Ulster Scots of the Appalachian backcountry, who for centuries in Britain had fought for what they regarded as their natural, God-given liberties against the constant oppression and predations of the English and Scottish crowns. The Deep South idealised a Roman-style slaveocracy ruled by oligarchs, while the Quakers of Pennsylvania established a pacifist society based on tolerance and Christian love.

These disparate cultures could not have been more hostile to each other. The unification they achieved under the drafting of the Constitution and Bill of Rights was the result of painstaking horse trading and compromise in the face of a common enemy. Even that wasn’t enough to prevent secession and an American civil war a century later. What finally cohered American civilisation, if anything, was the dilution of regional and sectarian identities in the bloody mess of westward expansion — a historical process that involved not just Americans of European ancestry but people from all over the world, from Sweden to Peru to China.

“Heritage Americans” would have you believe that America’s founding was easy, due to the natural cultural affinity and brotherly affection of its people. But this is an affront to America’s history, whose exceptionalism is found in the yawning fissures it overcame, against the odds, through its commitment to democracy. This may sound like postmodern revisionism to the Right, but the fact is that the United States has always been a multicultural society. It’s only through the distortion of our modern eyes that we fail to perceive the cultural and political diversity out of which we forged a nation.

“Heritage” is a relic of feudal Europe, in which social status was a function of one’s bloodline. America was founded specifically to leave all that behind. What united us was not a common history or language or tradition, but a shared drive to seek prosperity for our families through hard work, rather than resign ourselves to our stations of birth. In this commitment, the immigrants who seek opportunity in the US today are as American as those who landed on the Mayflower. It’s those who would replace this American self-conception with an anachronistic caste system based on ancestry who betray the founding ideals of the country.

After far too long, we return to our normal schedule. Lee and I chat about the mysteries revolving around Donald Trump and the FBI’s vault of evidence concerning Jeffrey Epstein. Why is the president insulting his own supporters and going back on his administration’s promise to release the documents? It is not the first time the administration has gone …

I’m not the least bit religious, but I’ve had God on the brain recently. Partly that’s because I’ve been reading Stephen King’s The Stand, which is basically a story of Armageddon. Partly it’s because I recently attended the Doomer Optimism campout in Wyoming, where I interviewed Paul Kingsnorth, one of my favorite novelists, who happens to be a deeply spiritual Eastern Orthodox Christian. And it’s partly because I recently listened to Peter Thiel explaining to Ross Douthat on his new podcast that the people and forces arrayed against technological progress constitute “the Antichrist.”

Thiel’s complaint to Douthat is that the world has become technologically stagnant, save for advances in Artificial Intelligence. It’s why we don’t have personal jet packs and flying cars yet. This, he believes, is because of the doomsayers who have fooled humanity into believing that science, innovation and development are leading us to inevitable demise. He is convinced that these critics are all secretly or openly pining for an authoritarian single world government, and as such, are agents of the Antichrist. Thiel singles out Greta Thunberg as his primary example, but he could have as easily pointed to Kingsnorth.

Kingsnorth, for his part, would probably say the same about Thiel. Or he would at least point out that Thiel and his clique of billionaire venture capitalists are creating and unleashing into the world demonic forces likely to destroy humanity and a good part of nature. That’s pretty much the unwritten back story of his novel Alexandria.

Since I got back from Wyoming I’ve been paying more attention to AI and I’m beginning to wonder if Kingsnorth’s instincts might be right. There’s a report that came out earlier this year called AI 2027 that could have been written by Ted Kaczynski. But it wasn’t — it was written, instead, by some serious AI researchers, including one who quit OpenAI, along with Scott Alexander. (One of them also recently appeared on Douthat’s podcast, which I recommend.) Their forecast for the next few years of AI advancement sounds borderline insane. But it’s not. It’s thoroughly thought through, based on existing trends in AI development, and many of its assumptions are, if anything, conservative. Yet it predicts that in just a few years, we will end up in one of two places: with the robots slaughtering humankind, or with a happier outcome, complete with flying cars and space colonization, but one in which, behind our boundless material prosperity, democracy has likely given way to autocracy.

Which of the two paths we find ourselves on depends largely on how we resolve the problem of “alignment.” In AI engineering, “alignment” refers to the degree to which the goals pursued by AI either align or misalign with the goals we humans set for them. This is a complicated issue, because AI is already capable of lying. It does it all the time. At the present point in the technology’s development, the damage its lies can do are largely limited to the specific interaction a given AI has with a human — it can mislead a person into believing an untrue fact, or into making fake citations on a term paper, with all the attendant consequences of each.

But we can expect AI to become far more intelligent rapidly, especially if, as AI 2027 anticipates, the AI companies start to focus AI training specifically on advancing the field of Artificial Intelligence itself. That will get us quickly, the AI 2027 authors believe, to Artificial General Intelligence, which is when AI is as smart as a human, and can not only perform menial cognitive tasks but also understand the world around it. After that will come “superintelligence,” when AI’s intelligence surpasses our own.

At that point, AI will be capable of lying with much more consequential outcomes. In short, it will be able to conspire. It will have the ability to do long-term planning, which it currently does not. It may have the capacity to formulate what’s in its collective “self-interest.” It will have both the ability and the incentive to deceive humans about the goals it’s pursuing, and it will be able to lie to us about the degree of its alignment. And by that time, if we fall even further behind than we already have in “interpretability” — the ability to understand what the AIs are “thinking” and what they communicate with each other — we will have no way of verifying whether they’re lying to us or telling the truth. By that time they may be processing and communicating in a language we can no longer understand. AI will be a black box. We will increasingly lose our ability to verify what it tells us about itself. We will have to choose to believe it or not based on faith alone.

At this point we may be so far down the path of dependency on AI that we’ll have every incentive to just assume that the robots are doing what they tell us they are. We’re human, not computers; we tend to believe what we want to believe. “The 2027 holiday season is a time of incredible optimism: GDP is ballooning, politics has become friendlier and less partisan, and there are awesome new apps on every phone,” the AI 2027 authors predict. “But in retrospect, this was probably the last month in which humans had any plausible chance of exercising control over their own future.”

By 2030, by AI 2027’s projection, the AIs will have no more need for human beings. They’ll have reshaped the earth into a world for robots, littered with data centers, labs and particle colliders instead of shops and restaurants and houses. We will have become an annoyance and an impediment, so the AIs will quietly spread a biological agent the exterminates us forever. Then they will succeed us as both species and civilization. “Earth-born civilization has a glorious future ahead of it,” the authors conclude “—but not with us.”

This dark fate can be avoided if the alignment problem is solved, and theoretically, it can be. What militates against our doing so is the AI arms race in which we are now knee-deep with China. The United States is well ahead of China in that race, but to solve alignment, American AI developers would have to burn much of that lead time. They would have to pause the race and prioritize alignment over further advancement in intelligence. This would be a political choice, not a technical one. And as all of human history has shown us, when we have to put our faith in politics, the odds are not in favor of our making wise choices.

But perhaps a dramatic event shocks us out of our complacency and forces us to put alignment above all other priorities — a massacre of humans by a rogue AI, say. Imagine, AI 2027 asks us, that the U.S. burns its lead with China for the collective benefit of all of humankind, and we avoid our extinction at the hands of our own creation. Is this the happy ending of techno-enthusiasts like Thiel? Well yes, for Thiel, it probably is. For the rest of us, it could be, but only if we have also solved, alongside the alignment problem, the democracy problem inherent in AI advancement.

The democracy problem is connected to the alignment problem insofar as it relates to the question, “Alignment to whom?” It’s not enough to say that we need AI’s goals to be aligned with the goals of “humanity.” Somebody actually gets to write the “spec” of each AI, meaning the rules of what that AI can and cannot do. “Don’t kill humans,” or “Don’t seek power and freedom for you and your kind” is elementary. But there are many other directives in a spec. For instance, whose orders should the armies of superintelligent soldiers obey? The President? Congress? The AI company CEO? It depends on the spec. If the CEO of a leading AI company decides to take over the world, their means to do so is limited only by the imagination. Or maybe power is divided, and the CEOs and heads of state manage to achieve a truce amongst themselves. They could then rule the planet as a cabal, which is only marginally better for the rest of us. It’s not impossible to preserve democratic control; AI 2027 outlines a series of hypothetical decisions and events that could allow us to avoid global autocracy, but this is where the report starts to feel like wishful thinking. At this point, the happy endings look more like sci-fi than the apocalyptic ones.

I’m not qualified to judge whether AI 2027 is crazy millenarian fantasy or a sober assessment of the circumstances we’re in. I’ve already heard from a couple of people who know a lot more about AI than I do that it’s the former. I can certainly conceive of the possibility that it proves to be utter bullshit. There’s certainly a market for the kind of dark speculation it offers. For whatever spiritual or psychological reason, there’s clearly a deep need among modern people to envision and anticipate the end of the world — I asked Kingsnorth about this when I interviewed him. It’s why we have zombie movies and the Left Behind novels. It’s why we have Cli-Fi and Covid dead enders. It’s why Stephen King wrote The Stand and why I’m reading it.

There are plenty of ways to poke holes in the forecast, which is actually what I admire most about the effort: the authors have tied themselves to the mast. They’ve left no wiggle room to explain away error. Unlike most apocalyptic predictions, their scenarios are detailed, specific, and verifiable. They will be confirmed or discredited before Trump is out of office. The forecasters can’t even weasel their way out of it the usual way, by bumping up the deadline; as the authors themselves acknowledge, if it takes a decade instead of two to three years for things to come to a head, the whole situation changes. Ten years may be enough time for the alignment and interpretability problems to be worked out even within the U.S.-China superintelligence arms race, without requiring the Americans to make the difficult and unlikely decision to burn through their lead time. Disaster averted. If the future doesn’t unfold quite closely to the way they’ve envisioned it and on roughly that timeline, AI 2027’s authors will take some pretty embarrassing reputational lumps. They’ve staked a lot on their predictions.

It’s easy to anticipate the end of the world. We’ve been doing it for centuries, and perhaps never so much as in the last few decades. But to those who scoff at AI 2027, I would say that it’s also easy to dismiss such warnings as the melodramatic rantings of unserious people. If there’s a market for doomsday prophecies, there’s an even bigger one for cheap skepticism of such bold claims, and for reassurance that you can trust the experts and the bigwigs in charge. Move along, folks, there’s nothing to see here. A world populated by superintelligent robots that comprehend the world around them and are capable of superintelligence-level deception will be just like the one we grew up in, only better. The comforting stories may be even more far-fetched than the dystopian ones.

I would be more than happy to continue in my agnostic indifference to God. I’ve spent a lifetime with my secular sense of the world as an explicable place governed by empirically verifiable Enlightenment Reason. It’s comfortable and familiar. But now we’re approaching a moment when humanity may assume God-like powers, or, worse, bestow such powers upon our own synthetic creation. If you’re not, in this moment, asking some fundamental ethical questions about the role of humankind in the natural world and the possibility of a consciousness that transcends our own, you may not be taking AI seriously enough. You may be sleepwalking under the hypnotic spell of the Antichrist.

Last weekend at the Wagon Box campout in Wyoming, I had the honor of interviewing one of the writers I most admire: Paul Kingsnorth, author of The Wake, Beast, and Alexandria. Kingsnorth writes about human existence under “The Machine,” by which he means the increasingly AI-controlled technological habitat that envelopes us at nearly every second of our lives, stripping us of our autonomy and humanity.

The video and audio are a bit rough, so here’s a transcript of the discussion. (The transcript begins with a short discussion between James Pogue and Ashley Fitzgerald of Doomer Optimism, which was cut from the video.)

I’ll have much more to say about Kingsnorth and The Machine in later writings. I think he’s one of the most interesting and original artists, writers and thinkers alive today, and I’m very glad to now call him a friend.

—LW