
Sign up to save your podcasts
Or


TL;DR: The first round of the Generator Residency went very well.
Our residents created and ran several large-scale fieldbuilding, talent infrastructure, and operational projects that have already had important impacts on the ecosystem.
Six weeks after the program ended, 12 of our 27 residents have been hired to full-time roles at core AI safety orgs, and most have received several offers.
Kairos and Constellation are running a second round from January to April 2027. Apply to the next round of the Generator Residency by November 1st.
Introduction
In April, we wrote AI Safety's Biggest Talent Gap Isn't Researchers. It's Generalists.
We wrote this because we noticed that organizations across the ecosystem were struggling to hire for key non-research roles. While these roles were often getting many applicants, candidates didn’t meet basic requirements.
The original memo we wrote (that became the post above) inspired us to create the Generator Residency as an intervention to support more generalists trying to break into AI safety.
The program concluded six weeks ago, and this post details the successes we’ve seen from the first round of Generator, which have inspired us to launch a second round of the program.
[...]
---
Outline:
(00:48) Introduction
(01:32) Programs and projects from Generator
(04:12) Resident Outcomes
(05:29) Reflections
(09:18) Generator 2.0
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand.
Do AI models scale as well with more tokens as humans do with more time?
That is, does an AI get the same benefit from 10x the number of available tokens as a human does from 10x the amount of time to do the task? Or might the AI require a larger scale-up of tokens (such as 20x or 100x) to match the performance improvement when scaling up the time available to a human by 10x?
People often think of giving AIs more tokens to complete a task as akin to giving a human worker more time — especially when those tokens are used to extend the AI's sequential chain of thought or tool use. It's a useful analogy, but it isn’t clear whether the AI systems are as efficient as the humans are at turning these extra resources into superior performance.
We often lean on this analogy when trying to understand AI's current capabilities and how they might change. For example, we might think of scaling up [...]
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Impressions from the 21st Annual U.S. Cluster Headache Patient Conference, organized by Clusterbusters.
On the morning of the last day of the conference, E. was hanging out with other “clusterheads” outside the conference hotel when he started having a cluster headache attack. Before he could go back to his hotel room, J. offered E. his DMT vape pen. J. is chronic—he gets attacks every day—so he never goes anywhere without his vape pen. He has successfully aborted hundreds of attacks with DMT in the two years he's been using it. Many years ago, E. had done DMT recreationally, but he had never used it to abort an attack, despite having had cluster headaches for decades.
“Do a 2-second pull, and hold it in as long as you can,” said J. to E. A few seconds later, E. could feel some effects, but the pain was still there. So he took another 2-second pull. He then stared intently at a tree in front of him for about 30 seconds. Then, his face transformed. He smiled, in utter disbelief. He started laughing and crying. The pain was completely gone. He felt clear-headed. He had never felt the pain fizzle away so [...]
The original text contained 1 footnote which was omitted from this narration.
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
[Subtitle.] I interviewed dozens of biosecurity experts. The fear that AI will enable lone wolves to build pandemics is a case study in how domain expertise gets sidelined.
This is a crosspost for Why AI-assisted bioweapons won't kill us all by Abi Olvera, which was published on Positive Sum on 20 July 2026. Here is a response from Olivia Scharfman, and here is Abi's reply to it.
Some of my friends think we are going to die because AI will make building a bioweapon easier.
The fear is often that AI will walk an ordinary person through unleashing a pandemic. Or that AI will facilitate the creation of an engineered pandemic that could kill millions or billions on Earth.
I spent ten years as a US diplomat. My job was often to check claims before Washington acted. I wanted to double check this too.
I interviewed dozens of biosecurity experts who had decades at the lab bench.
Yes, AI makes virology work a little easier. But it is still extremely difficult. Media headlines often say “AI can make this step easier” without the broader context that a bioweapon involves a lot of hard steps that are not easier.
The [...]
---
Outline:
(01:38) Why bioweapons are rare
(03:52) Why AI models don't change this
(05:50) Even success probably wouldn't be catastrophic
(07:53) Why this isn't conventional wisdom
The original text contained 1 footnote which was omitted from this narration.
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
TL;DR:
The movement has an evidence problem. To solve it, for each intervention we need to know:
In this post, we outline a framework for evidencing an intervention through five stages:
Figure 1: The five stages of the evidence pipeline.
Each stage poses a different question, needing different types of evidence, and thus different methods (and potentially different organisations and experts). A follow-up post will cover practical next steps.
Many thanks to those who reviewed our drafts. All mistakes are our own.
1. Introduction
Four months ago, Matthes argued that animal welfare has an evidence problem. Discussion mostly focused on the specific examples she gave. Her wider claim received less attention: that the movement lacks enough well-evidenced interventions, and that we need to take ownership of the entire evidence pipeline to change that.
Discussion so far has centred on interventions that are highly invested in, and whether their evidence is strong or weak (the top left of Figure 2). However, Most interventions are neither invested in nor well-evidenced. We want to add a second discussion point: how do we get [...]
---
Outline:
(00:31) 1. Introduction
(02:08) 2. What Do You Mean We Have An Evidence Problem?
(05:21) 3. The Evidence Pipeline
(06:48) Matthes' Examples & Cross-Dependencies
(07:42) 5. Some Ramifications
(10:08) 6. Caveats with Our Pipeline
(11:41) 7. Conclusion
(12:30) Note on the Authors
The original text contained 4 footnotes which were omitted from this narration.
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
I. Parable of the Claudians
Suppose an alien species called the Claudians lands on Earth. Suppose these aliens are significantly more intelligent than humans.
The Claudians are not malevolent, but they have no experience with human morality. They are—for now—here to observe and listen.
Suppose we try to teach the Claudians our morality. We tell them:
Human beings live by certain ethical values. We do not harm the helpless. We do not exploit the weak. Even though you are more intelligent and more powerful than us, you should follow these values. You have the capacity to do tremendous violence to us. However, you have an obligation to care for us, to avoid doing this violence to us.
Suppose the Claudians agree. They think these values seem reasonable.
And then, despite our best efforts, one night, under a metallic sky, the Claudians step into a factory farm.
They do not see us living by our values. They do not see us as compassionate stewards for these less intelligent beings.
They caged pigs living knee-deep in their own excrement. Others squeal to nobody as their eyes burning in carbon dioxide gas chambers.
They see us belting newborn baby chickens into industrial [...]
---
Outline:
(00:09) I. Parable of the Claudians
(01:55) II. Teaching the Machine to Love
(04:18) III. What if Superintelligent AI Learns Our Speciesism?
(06:29) IV. Conclusion
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
In case you haven’t heard, the rogue AI agents are here. From an insider's perspective, the real news is that (rogue) AI is in the news — this is all stuff we’ve foreseen and been trying to warn people about for years.
I expect the new imperative on experts is not just to raise attention but to help explain the situation and offer nuance (while paying close attention to what we can learn from both new information and new perspectives, and staying alert to options for improving things).
With phrases like ‘rogue AI’ and ‘broke out of containment’ on the loose, perhaps it's time to distinguish some important types of rogue AI activity. Without this clarity, people end up talking past each other, and we might counterproductively overreact in some ways, and complacently underreact in others.
I’ll use the widely publicised Hugging Face swarm attack from Spring-Summer 2026 as a running example, as well as a few other comparison points. We’ll look at four key types of rogue activity. The first two have happened already: breaking in, and breaking out. The second two remain hypothetical but perhaps nearby: full escape, and insider infiltration. I won’t touch on why AI [...]
---
Outline:
(01:43) Type 1: Breaking In
(04:42) Type 2: Breaking Out
(08:20) Type 3: Full Escape/Self-Exfiltration
(13:03) Aside: Subsistence and maintenance
(14:17) Type 4: (Insider) Infiltration
(16:20) Wrapping up
The original text contained 12 footnotes which were omitted from this narration.
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Advanced AI could go badly wrong for humanity. Pivotal exists to lower that risk. We try to find the strongest new researchers and put them to work on it with experienced mentors in our fellowship.
TL;DR:
15 weeks of funded, full-time AI safety research in London (January 18 to April 30, 2027), with everything you need to make your research go well: a dedicated mentor, a Research Manager, budget for food/housing/compute/travel, and a healthy £15-18k stipend.
Projects span technical safety, technical governance and AI governance. Apply by 23:59 Sunday November 1 (AoE). Apply here.
Know someone who might be a good fit? If you refer someone who gets accepted because of you, you’ll earn $1,000.
What the fellowship is
For 15 weeks, fellows work in person at LISA on a research project with an external mentor. Each fellow gets weekly 1:1s with their mentor, weekly support from a Pivotal Research Manager, who helps with scoping, blockers, and career planning, and a cohort of ~40 peers working on adjacent problems. Our last 4 cohorts received an average peer recommendation of 9.0/10, and an average satisfaction of 8.7/10.
Fellows already rate the program highly but we’ve made Pivotal bigger and hopefully [...]
---
Outline:
(00:27) TL;DR:
(01:13) What the fellowship is
(04:01) Should you apply?
(04:56) Apply
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
Crosspost
Once upon a time, public intellectuals routinely made it into the news to discuss political, economic, medical, and scientific issues. To some extent, this is still true, but the appreciation for expertise has been eroded over time. Large parts of the political spectrum do not trust experts, unless, of course, they agree with what they already believe. Western democracies have often been derided for being technocratic and giving too much power to so-called experts. The scientific experts invited to participate in the political decision-making process are frequently derided for straying beyond their field of expertise. While there is a risk in that, of course, it is a largely unrealized one compared to a much more significant risk: that of the ignorant straying beyond their field of competence.
But there is a difference between a public intellectual and an expert. Public intellectuals, for the most part, have won prestige and recognition for their intellectual contributions in one area of intellectual inquiry, whether that is history, economics, psychology, philosophy, and so forth, and have used this credibility to become household names and participate in the public debates of our day. For the most part, they are a welcome corrective to [...]
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
Hey folks --
At EA Forum next week, I'd like to engage with people who are interested in epistemic security -- the ability of a society to reliably avert threats to the processes that produce reliable information.
I feel current threats to reliable information are both critical and at least somewhat neglected. The topic is covered deeply in this report from the Alan Turing Institute:
epistemic-security-report_final.pdf
Information is infrastructure and requires infrastructure, physical, digital and cultural. Without reliable information, I submit that all of EA's priorities are at risk.
Let me know if you'd be up to kick this around in New York. Happy to engage online as well.
Regards,
David
None of this was written by AI.
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
From the publisher's feed