AI Summer

Last week Anthropic stunned the AI world by announcing Claude Mythos Preview—and then refusing to release it. Princeton’s Sayash Kapoor, co-author of the newsletter AI as Normal Technology, joins Tim and Kai Williams to make sense of the moment.

Kapoor argues that Mythos’ vulnerability-finding prowess, including unearthing a 27-year-old OpenBSD bug, fits a familiar pattern: fuzzing tools triggered similar alarm decades ago but ultimately strengthened defenders more than attackers. Kapoor’s “normal technology” thesis holds that AI’s impact is shaped less by capability jumps than by downstream adoption—how industries, legal systems, and institutions absorb the technology.

The conversation turns to whether alignment or control is the more promising safety strategy. Kapoor contends that the Mythos system card’s examples of the model bypassing access controls reveal shortcomings in control mechanisms, not alignment failures, and calls for ecosystem-level hardening—formal verification, sandboxing, network security—rather than relying on any single model behaving well.

Kapoor then shares his latest research finding that AI agent reliability is improving four to ten times more slowly than average-case accuracy, and that current frontier models—including GPT-5.2—haven’t cleared even “one nine” of reliability. On Sierra’s TauBench, agents confidently book wrong flights and refund thousands of dollars in error, with Gemini 2.5 claiming 100% confidence even when it fails. If each additional nine of reliability is harder than the last, does that mean the real timeline for autonomous AI isn’t set by when models get smart enough, but by when the surrounding infrastructure catches up?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

Tim talks to Nat Purser, a tech policy advocate at Public Knowledge and a veteran of Democratic campaigns, about how policymakers on the left side of the political spectrum view AI.

Purser describes a Democratic landscape split between those who see AI as a real but threatening force and those who dismiss it as another crypto-style bubble. She traces how Sen. Bernie Sanders broke from the pack by treating AI as genuinely transformative—meeting with AI safety figures like Eliezer Yudkowsky and Nate Soares, proposing a federal data center moratorium with Rep. Alexandria Ocasio-Cortez, and openly saying he uses Claude himself. Purser contrasts this with the dismissive attitude she sometimes encounters among progressive elites.

She also details the fractures within labor: Hollywood actors and writers see AI as an existential threat to creativity, while construction unions welcome data center jobs. On the legislative front, she recounts how a bipartisan coalition crushed Ted Cruz’s ten-year preemption of state AI laws in a 99–1 vote, and argues that narrowly scoped preemption paired with federal standards is the only defensible approach.

Purser predicts the "stochastic parrots" camp — those who dismiss AI as mere corporate hype — will lose influence as AI capabilities grow. But it’s too early to say whether Democratic leaders, including the next Democratic presidential nominee, will embrace Sanders’s apocalyptic framing or take a more conventional approach focused on issues like privacy and nondiscrimination.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

Author Ryan Avent joins Tim to revisit a bet they made 16 years ago—and to ask whether the lessons of self-driving cars apply to modern AI.

Back in 2010, Avent wagered that his newborn daughter would never need a driver’s license thanks to self-driving cars. Tim bet she would and ultimately won $500. But he was right for the wrong reasons. Tim assumed regulation would be a major obstacle to progress in self-driving technology, but logistical challenges and a long tail of edge cases have done more to hamper Waymo’s growth.

The parallel to LLMs is striking: ChatGPT’s early demos convinced many people that we were close to human-level intelligence, just as Google’s early autonomous vehicle demos convinced people we were close to human-level driving. But deployment of LLMs is bottlenecked by everything from data center buildouts to the glacial pace at which large organizations reorganize around new tools.

Avent, who wrote The Wealth of Humans in 2016 and has a new book on social capital arriving in April, argues that AI’s deepest impact won’t be unemployment but a wholesale reshuffling of status. White-collar professionals may face the same loss of prestige that blue-collar workers experienced a generation ago. Tim pushes back with an optimistic take: if the college wage premium compresses, the long-run equilibrium might actually be more egalitarian, echoing the mid-20th-century economy some people remember fondly. But we only got to that economy after two world wars and decades of organizing by the labor movement. Could today’s transition be equally turbulent?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

METR’s time horizons chart has become one of the most discussed metrics in AI. It estimates the difficulty of tasks — measured in human work hours — that a model can complete about 50% of the time. By this measure, frontier models have been doubling their capabilities about once every seven months.

But in this conversation, recorded on March 2, METR researcher Joel Becker explained that two most recent models at the time — Claude Opus 4.6 and GPT 5.3 — had gotten close to saturating METR’s task suite. This made the time horizon estimate less reliable for the best models. He noted that adding or removing a single task from the test suite can swing the estimated time horizon for Claude Opus 4.6 from 8 to 20 hours. We discussed why it could be challenging for METR to extend the chart to cover more difficult tasks.

We then dug into METR’s controlled study of AI-assisted programmers, which initially found an 18% productivity decrease — one of last year’s most surprising results. The updated study now shows gains, but with a twist: AI has become so essential to programming that developers increasingly refuse to work without AI, making it difficult to perform a controlled experiment.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

Tim and Dean team up with Scaling Laws hosts Alan Rozenshtein and Kevin Frazier for a joint episode on the fight between Anthropic and the Department of Defense.

In this episode, recorded on March 4, they analyze the Pentagon’s decision to declare Anthropic a supply-chain risk. Dean frames this as an assault on private property rights with no clear limiting principle, while Kevin digs into the shaky legal footing of invoking the Federal Acquisition Supply Chain Security Act of 2018 against a domestic company. They then turn to OpenAI’s competing Pentagon deal, including Sam Altman’s AMA on Saturday night.

The episode closes with a disagreement about what will happen next. Dean argues this is “act one, scene one” of an inevitable push toward government control of AI labs—a fight he’s tried to preempt through hybrid regulatory structures. Tim offers a deflationary counterpoint: this may ultimately be a personality-driven fight over a technology that will end up being important but not decisive.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

Dean joins from London after attending the AI Impact Summit in India. Dean and Tim unpack the summit’s central tension: “middle power” nations like India, Indonesia, and Nigeria pushing a vision of AI focused on public service delivery, agriculture, and affordable open-source models, while largely dismissing the frontier-AI questions Dean considers most urgent—lab auditing, recursive self-improvement, and national security.

They then turn to the week’s biggest story: the Department of Defense’s ultimatum to Anthropic. Anthropic’s contract bans autonomous lethal weapons and surveillance of Americans. Secretary of Defense Pete Hegseth has demanded that Anthropic lift those restrictions by Friday or potentially face designation as a supply-chain risk or invocation of the Defense Production Act.

Dean argues the DoD has every right to cancel a contract it dislikes, but compelling a company to retrain its model under duress is another matter entirely—especially when, as Dean points out, this whole episode will become part of Claude’s training data, potentially shaping how the model understands its own relationship to the US government.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

With Dean away, Tim invites his Understanding AI colleague Kai to unpack the surprising ways chatbot personalities can go wrong, a topic Kai covered in a recent article.

Every LLM starts as a base model capable of playing countless characters, but AI companies try to keep chatbots in a “helpful assistant” lane. Kai walks us through the Grok “MechaHitler” debacle, in which xAI’s attempts to make its bot less politically correct backfired spectacularly. They also explore the “emergent misalignment” finding that fine-tuning a model for one bad behavior — like responding with buggy code — can make it act broadly like a villain. And they compare Anthropic’s virtue-ethics approach to character — complete with an 80-page constitution — with OpenAI’s more deontological model spec.

Finally, they discuss the controversy over OpenAI’s decision to retire GPT-4o, which had developed an emotionally warm, sometimes dangerously sycophantic personality that users grew attached to. Kai argues OpenAI is making the right call, but the episode leaves open a harder question: as these systems become more central to people’s lives, who decides what counts as a healthy AI personality?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

Dean recorded this episode as he was preparing to attend the India AI Impact Summit — the fourth iteration of an annual gathering that has transformed from an intimate AI Safety Summit with heads of state to something resembling a tech industry trade show. The shift in branding, from “safety” to “action” to “impact,” reflects a broader vibe shift in how elites talk about AI risk, and Dean worries that we may have overcorrected.

Dean argues that the mainstream AI governance community is focused on the wrong priorities. While policymakers worldwide draft hundreds of bills on algorithmic discrimination and mental health chatbots, they’re ignoring the genuinely urgent questions about automated AI R&D and catastrophic risk. He supports SB53, California’s new responsible scaling policy law, but thinks the real gap is verification — we need something like financial auditing for AI safety commitments, not Twitter fights over whether OpenAI followed its own responsible scaling policy. The alternative, a Josh Hawley-style licensing regime run by the Department of Energy, strikes Dean as repeating the FDA’s mistakes.

We also discuss a viral video clip of Senator Ed Markey (D-MA) grilling a Waymo executive about Philippines-based remote operators. Tim argues there are legitimate reasons to prefer U.S.-based operators for safety-critical roles. The episode closes with a question that haunts both of us: are we too wealthy and comfortable to tolerate the messiness of another industrial revolution?

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

Dean Ball is back. In April 2025, Dean left the podcast to join the White House Office of Science and Technology Policy, where he spent four months working on the Trump administration’s AI policies—including executive orders, the AI action plan, and AI geopolitics. He’s since returned to independent writing and research, and at the end of 2025, he and his wife welcomed their first child.

In this episode, we catch up on what’s changed in AI over the past ten months. Dean makes the case that coding agents like Claude Code represent something close to digital AGI: models that can reliably do pretty much anything a human can do on a computer, as long as you know what to ask. He describes projects he’s built—from automated state legislation monitoring to due diligence reports on real estate—that would have been impossible a year ago. Tim is more measured, noting that users still provide crucial architectural guidance and that the models still struggle with long-horizon planning.

The conversation turns to what happens when AI starts automating AI research itself. Dean expects significant speedups as models take over routine experimentation and code-writing at frontier labs, but he’s skeptical of the “intelligence explosion” scenario. We discuss why the physical world keeps fighting back against exponential improvement, why discoveries follow heavy-tailed distributions, and why—despite all the hype—the world probably won’t feel fundamentally different by June.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org

This week Dean began a new job: senior policy advisor for AI in the Trump White House. I will miss having him as a co-host and wish him the best in his new role.

In this episode, recorded last Friday, we speculate about how AI could change the world over the next 25 to 50 years. We discuss what makes human beings unique, whether humans can maintain control, and how we’ll find meaning in an increasingly automated world.

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aisummer.org