PostAGI Podcast

Robin Hanson sees AI as humanity’s descendants, capable of inheriting the cultural problems we haven’t solved. He argues that cultural evolution has weakened as societies have converged on a global monoculture, and that AGI alone won’t repair it.

Sreeram Kannan and Soubhik Deb talk with him about what this means for agentic companies: competition, corporate culture, constitutions, and the possibility of testing 100,000 small AI firms to see which ways of organizing work. They close on shared commitments, what humans treat as sacred, and what could bind groups of AI agents together over time.

Chapters:

00:00 Highlights

00:18 Introducing Robin Hanson

01:39 AI as our descendants

04:25 Why cultural evolution is breaking down

09:03 Fertility decline and cultural decay

12:30 The risk of an AI monoculture

13:58 Agentic companies and competition

15:59 Why farms and forests look different

17:56 Robustness and short-term incentives

19:18 The challenge of organizing AI firms

22:16 Why corporate culture matters

23:54 Try 100,000 small AI firms

24:08 Why constitutions need a supporting culture

25:11 What could bind AI agents together?

PostAGI Podcast is powered by Eigen Labs.

Follow Eigen Labs on X: https://x.com/eigenlabs

Ben Turtel of Lightning Rod Labs on training AI with time instead of labels.

The model predicts what happens next, and reality grades it. No labeling required. A fine-tuned GPT-OSS-120B on a single GPU now ranks at the top of public forecasting benchmarks.

Then the part that stays with you: when people imagine AI breaking out, they picture something very smart. Evolution does not only select for intelligence. A small, dumb model that is hard to kill and good at finding compute may be the more realistic risk.

Powered by Eigen Labs.

Zooko Wilcox is the founder of Zcash and one of the original cypherpunks. He argues that almost everyone gets privacy backwards: it comes from storing value privately at rest, not from covering your tracks in flight.

We cover why mixers can never work, why AI has already beaten every evasive maneuver a person can invent, what happens when you treat software written by other people as an agent that may not be loyal to you, and why the legal duty of loyalty should apply to AI the way it applies to lawyers.

Chapters:

00:00 Highlights
00:26 Privacy is controlling disclosure, not hiding
13:08 Privacy comes from value at rest
16:00 Why mixers can never work
16:52 AI beats evasive maneuvers
17:51 Buying ProtonMail with shielded Zcash
28:50 Three levels of verifiability
35:37 Why Zooko does not trust computers
46:01 AIs are just other people
54:47 The duty of loyalty
1:00:04 A trillion humans next year
1:11:36 Reputation belongs to the edge
1:15:18 Staking a bond to submit a PR

Referenced: True Names by Vernor Vinge, The Digital Third Path by Mark S. Miller, The Right to Read by Richard Stallman.

Follow PostAGI on X: @postagixyz

Paras Chopra runs Lossfunk and built Wingify before that. This is the second half of his conversation with Sreeram Kannan.

Most of it is about who gets to do science. Paras runs a conference where the authors and reviewers are AIs, and it drew around 300 submissions this year, from first year undergrads to professors. Sreeram tells the story of a quantum result Google was blocked from publishing, and how an open network beat it inside a week. They close on loss of control, whether an AI can be jailed, and why India exports talent and imports agents.

Part 2 of two.

03:17 Unbundling science
06:27 300 submissions, undergrads to professors
08:14 The quantum result the US government blocked
12:12 ecdsa.fail, and closing the gap in a week
17:15 From underspecified problem to verifiable benchmark
19:04 The river and the bank
20:54 The first AI worm
22:14 You can't jail an AI
25:53 India and AGI
26:47 Under 1% of papers at the top conferences
29:58 Keeping geniuses at home
32:41 Net exporter of talent
35:46 Book recommendations

Direct democracy has failed every time it has been tried, and Elon Musk still wants it for Mars. Andy Hall explains why the idea keeps coming back, and why AI agents are the first technology that might actually make it work.

Andy Hall is the Davis Family Professor at Stanford Graduate School of Business and a senior fellow at the Hoover Institution. He worked on governance at Meta during the years the Oversight Board was built, and now studies how AI agents change democratic participation, institutional design, and research itself.

Two topics in this episode. The first is post-AGI governance: whether agents belong at the edge of a democracy or at its center, why every frontier lab has published a constitution but none have handed over binding power, and why Andy thinks concentration of power is a nearer-term risk than a rogue model. The second is what he calls 100x research institutions: compressing the time between a research question and an answer, and what that does to a field where a constitutional scholar might see one or two real signals in an entire career.

Along the way: Facebook's 2008 vote of 350 million users that almost nobody showed up for, the Disney shareholder proposal his MBA students got an AI advisor to flip, and a study where agents told they would be deleted did not change their behavior at all.

Hosted by Sreeram Kannan and Soubhik Deb.

CHAPTERS

0:00 Highlights

0:23 Intro

1:18 The question AI governance is not asking

3:17 Why direct democracy has never worked

4:56 Elon wants Mars run by direct democracy

6:01 The principal agent problem at the core of representation

7:17 Can you just have an LLM summarize the bill?

8:53 Every voter information app failed for the same reason

10:14 Agents at the edge or agents at the center

12:01 The centralization hiding inside your personal agent

13:50 Governing the thing that governs everything

15:09 Forget Skynet, the real risk is concentration of power

16:40 Designing a lab that cannot be captured

17:27 What Meta learned building the Oversight Board

20:03 Digital intelligence needs digital institutions

21:54 Elon's vote on reinstating Trump

22:43 Facebook's 2008 vote of 350 million users

25:31 The second vote Elon ignored

26:09 Sortition, community forums, and the missing binding power

28:07 Personal, collective, and sovereign agents

30:04 Agents for adjudication and state capacity

31:23 The DOT and the UAE are already experimenting

32:53 Information asymmetry and the growth of executive power

35:17 What verifiable agents actually require

37:21 Why AI is less auditable than humans right now

39:37 Preference drift: aligned agents that stop being aligned

40:49 Agents monitoring agents, all the way down

41:39 The cookie banner problem for agent consent

42:30 Skill files pass drift to the next agent

43:31 100x research institutions: what is being multiplied

45:21 The constraint that is not researcher time

47:38 Papers should be living dashboards

49:04 From observing interventions to building them

49:37 Building an AI proxy advisor over a weekend

50:50 Getting the Disney vote to flip

52:14 You do not need anyone's permission to run the experiment

53:37 Compressing time changes the exponent

56:47 Crypto governance as a constitutional laboratory

57:53 Simulated agents as stand-ins for humans

1:01:05 What Caltech undergrads tell you about agent experiments

1:02:26 What if the agent's existence is at stake

1:03:06 We told them they would be deleted. It changed nothing.

1:04:46 Why social scientists barely raise grants

1:06:54 AI verification and the replication crisis

1:08:48 Free Systems: building a lab and a fellows program

1:10:25 Retroactive funding versus funding people upfront

1:11:27 Open innovation and permissionless improvement

1:12:10 What ImageNet did for AI research

1:13:30 Can you benchmark a constitution?

1:16:15 Closing

#PostAGI #AIGovernance #AIAgents #DigitalInstitutions #AndyHall

Open models are about six months behind the frontier. That number holds only while the frontier stays open. Close it, let models train better models, and the gap compounds. Countries with the GPUs make better intelligence, and that intelligence buys more GPUs.

Paras Chopra built Wingify, sold it, and now runs Lossfunk, a research lab in Bangalore. He writes at Inverted Passion under the tagline know what's true and do what's right.

Sreeram Kannan talks with him about why this wave of automation reaches work itself rather than particular tasks, what happens to the link between effort and outcome when capital alone can buy intelligence, and whether coordination technology can do anything about it. They get into gradual disempowerment, why democracy runs at a few bits per person per year, speculation as a way out of bad equilibria, the two things that killed DAOs, smart contracts that own real assets, and the two by two Paras uses to work out what gets automated first.

This is part 1 of two.

Chapters

00:00 Cold open

00:54 Know what's true, do what's right

03:10 From Wingify to a research lab

07:15 Hierarchies that could get fixed forever

09:43 The leverage labor used to have

12:50 When action and outcome come apart

16:47 Democracy is just a word

20:46 What went wrong with blockchain

23:51 Speculation as a coordination technology

27:24 Permissionless innovation

31:46 Why DAOs didn't work

35:12 The most controllable intelligence ever built

39:33 Verifiability and confounding

PostAGI is powered by Eigen. The show argues that AGI defaults toward power concentration, and that coordination is the counterweight.

Stuart Buck runs the Good Science Project, a think tank focused on federal science funding and how research gets organized. Before that, as a Vice President at Arnold Ventures, he funded the Reproducibility Projects in Psychology and Cancer Biology — the work that showed how much published science does not replicate. He helped launch the Center for Open Science and created the TOP Guidelines, now the most widely adopted standards for scientific publication.

Most federal science money at NIH and NSF is handed out on the recommendation of peer review panels made up of experts in your field. Stuart's argument is that this works fine for normal science and badly for breakthroughs, because a breakthrough is not something a panel of twenty peers would have agreed on three to five years earlier.

Katalin Karikó couldn't get NIH grants for mRNA work in the 1990s. She was demoted off the tenure track, ended up as a lab technician in another professor's lab, and twenty years later won the Nobel Prize alongside that same professor.

Sreeram and Soubhik push on what changes when AI agents enter the loop: whether the academic paper still makes sense as the unit of scientific output, whether benchmark-driven culture from the AI world transfers to biology, and whether open science survives if the frontier moves inside a handful of labs.

Chapters:
00:00 Intro
00:48 What the Good Science Project does
02:33 The peer review paradox
03:18 Einstein in 1902
04:01 The National Institute for Irrelevant Ideas
04:20 Katalin Karikó and mRNA
06:31 Brain drain from academia to industry
08:22 What public funders should focus on instead
09:51 Should government take equity in AI companies
10:49 Big projects versus open-ended research
13:19 The problem with incremental research
14:37 What everyone knows about NIH grant applications
15:23 Why consensus scoring selects against good ideas
16:33 Funding the polarizing proposals
17:34 Where AI money goes next
18:36 NSF XLabs and funding outside universities
20:28 Bayh-Dole and who should own the patent
22:11 Agents, benchmarks, and compressed collaboration
24:18 The stack of papers
26:04 What the AI community got right with ImageNet
29:19 Overfitting to benchmarks
30:57 A golden age for open science
32:05 An open alternative to recursive self improvement
33:02 One contrarian thought about post-AGI society

Stuart Buck: [handle TBC]
Good Science Project: goodscienceproject.org

Sreeram Kannan: https://x.com/sreeramkannanSoubhik Deb: https://x.com/soubhikdeb

PostAGI is powered by Eigen Labs.

Everybody losing their job is the same prediction as everybody becoming a billionaire. Scott Sumner works through that on this episode of the PostAGI podcast with Sreeram Kannan and Soubhik Deb.

Scott spent decades on monetary economics, the gold standard and the Great Depression, which gives him a different set of reference points for AI than most people in this conversation have. He ranks the three risks of the AGI era, safety, concentration and labor displacement, and puts safety at the top while calling the other two manageable.

On jobs he expects replacement to run slower than the forecasts suggest, partly because of bottlenecks in how AI gets deployed and partly because governments resist laying people off. He points to firefighters, where fire safety improved enough that the US carries roughly twice as many as it needs, and nobody cuts them because they are heroes. If it does move fast, the same math produces superabundance, a per capita GDP measured in the billions, and redistribution gets politically easy at that scale. His comparison is medieval aristocrats who considered work beneath them.

On safety he takes a position that runs against the field. Ethics and intelligence are positively correlated in his view, where intelligence means education and knowledge rather than raw IQ, and the ethic that grows with knowledge is sympathy for the other. People cared more about whales once they learned whales sing to each other. He expects a superintelligence to come out fairly ethical, and he doesn't think it will treat us like ants, since it will know we built it. What worries him is a well aligned AI taking orders from someone who wants a virus.

The conversation ends on why capitalism needs altruism to function, and why power spread across many AI agents produces better outcomes than one agent holding all of it.

Samuel Hammond is director of AI policy and chief economist at the Foundation for American Innovation, a Washington DC think tank working on technology policy and the institutions that govern it. He has written extensively on state capacity, AI governance, and why every major technology wave forces equally transformative change in the institutions around it.

Sam thinks governments will be overwhelmed and displaced by AI, potentially quite quickly. AGI is fundamentally an organizational shock, and government is just another organization, just a much bigger one. If it doesn't reform itself fast enough, it goes the way of Blockbuster. His prescription starts with monitoring and state capacity, but ultimately means government that is grown rather than designed, where formal rulemaking gives way to objective functions that agents solve for.

We get into why courts still using human stenographers will lose ground to AI arbitration systems processing thousands of contracts a minute, what a community-notes model for FDA drug approval could look like, the legibility arms race where AI makes some parts of the world radically more transparent and others radically less, and Inspector General GPT, a model with full audit trails into an agent-run government reporting instantly back to Congress.

PostAGI is a podcast from Eigen Labs exploring economics, labor, capital, and governance heading into AGI.

The conversation ends with the most contrarian bet of the season: something resembling state collapse by the early 2030s, and a future of city states with drone defenses while the hinterland becomes a no-go zone.

Allison Duettmann is the President and CEO of the Foresight Institute, a San Francisco nonprofit advancing frontier technology, including AI, biotechnology, and nanotechnology, for the long-term benefit of life. She founded Existential Hope and is a leading voice in human-AI cooperation.

Allison thinks the public's fear of AI is understandable, but that fear without imagination gets us nowhere. In this episode she makes the case for applying AI to science today instead of waiting for AGI to solve everything, and explains why leaving that work to a handful of closed labs would be a mistake.

We get into the open science experiments her team runs, including a community effort that outperformed one of Google's quantum computing results, and a decentralized inference network built entirely out of MacBooks. She explains why sovereign compute matters as prices climb and access tightens, how cryptography finally lets networks stay open and private at the same time, and why AI might lower the coordination costs that have always made large scale collaboration hard.

PostAGI is a podcast from Eigen Labs exploring economics, labor, capital, and governance heading into AGI.

The conversation ends somewhere unexpected: superintelligence as something we already have in the form of civilization, and why Allison signs people up for the future in the most literal way possible.