EA Forum Podcast (All audio)

At a conference on post-AGI futures earlier this year, I proposed a session titled “What's next for impact markets?” The conference speakers included the godfather of impact markets Paul Christiano himself, and noted supporters like Scott Alexander, Oli Habryka, and Andrew Critch — so I was hopeful that there would be some interest.

…

Zero people signed up for my session.

Before everyone writes these off as some EA fad of the early 2020s, or a pie-in-the-sky dream requiring impossible levels of funder buy-in: I’d like to give my account of what we’ve learned about impact markets, and where we can go next.

A brief oral history of Manifund

Our 501c3 entity “Manifold for Charity Inc.” was originally started with a narrow thesis: it’d be good to let Manifold prediction market traders donate winnings to a charity of their choice. See our original grant proposal to the FTX Future Fund, which they approved with a 500 thousand dollar grant around April 2022. But at the time, this was just a side project of Manifold.

Manifund proper was made with the explicit mandate of “make impact markets happen”. Scott Alexander was interested in trying out new mechanisms for funding projects [...]

---

Outline:

(00:56) A brief oral history of Manifund

(03:34) Wait, so, what do you think about impact markets now?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Frame Fellowship is back for round two! In just about 10 days we'll have a brand-new cohort of 14 awesome content creators, storytellers, marketers, YouTubers, tiktokers- educating the world on AI safety through social media.

Frame 2.0 received hundreds of applications from award-winning filmmakers, investigative video journalists, AI Safety professors, former UN staff, Adobe professionals- all transitioning to make compelling content that reaches millions of people.

Somethings we looked for included exceptional communication ability, agency and passion for content, strong backgrounds/expertise, past AI safety engagement, distinct content styles, and how much they'd benefit from being part of the fellowship in San Francisco- all of which are important factors in determining the next big communicators for the space.

Fellows will be participating in a 10-week intensive in-person program in San Francisco- going through mentor workshops, optimized structure for making better content, networking with relevant creators, researchers, and funders, while having access to funding and other production resources. The goal of the program is to build capacity for credible communicators that use social media to educate people about relevant AI topics. Each fellow participates in one of two tracks-

1. Independent Track- Fellows work on building their own audiences, while [...]

---

Outline:

(02:35) Meet the Cohort.

(02:38) Reaa Puri

(03:21) Lauren Mushro

(04:05) William Lee

(04:49) Taeyong Kim

(05:31) Ernestine Siu

(06:11) Aaron Nev

(07:02) Amina Green

(07:46) Jeremy Ornstein

(08:25) Martin Lee

(09:11) Sara Landi Tortoli

(09:49) Strad Slater

(10:30) Stuart Harmon

(11:10) Reuben Adams

(11:44) Jeroen Willems

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR: Kairos has raised 50 million dollars from Coefficient Giving for two years of funding, one of the largest commitments they’ve made towards AI safety fieldbuilding to date. We’re using this to make an ambitious push for growing Kairos, broadening our portfolio of talent infrastructure projects and incubating new organizations. We’ve doubled in size in the last six months, and we plan to double again in the next six. We now have open hiring rounds for ten roles on our team across events, group support, incubation, and special projects.

Kairos was founded mid-2024 with the goal of creating infrastructure to get more strong talent into the field of AI safety. Our portfolio has grown over time: we started with a focus on seeding and supporting university groups through Pathfinder, then took over SPAR, the largest research training program in the ecosystem. Since then, we’ve launched the Generator Residency, a program supporting generalist talent, in partnership with Constellation, and taken over the Global Challenges Project (GCP), a series of workshops to accelerate people's transition into careers in AI safety and biosecurity.

Through most of Kairos's history, we’ve had a very small team: in January 2026 we were [...]

---

Outline:

(00:50) Two years in

(03:10) How the field has changed

(06:24) Our new bets

(12:32) We're hiring (a lot!)

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

TL;DR

We’re launching Efektif Berbagi, an organization dedicated to introducing and advancing effective giving in Indonesia.

Despite ranking several years in a row as the “most generous country in the world”, Indonesia currently has virtually no public advocacy or infrastructure directed towards cost-effective & high-impact giving. We’re working to be the bridge between Indonesia's vast philanthropic energy and the world's most pressing problems.

Why Indonesia?

For several years, Indonesia has placed #1 on the Charities Aid Foundation (CAF) World Giving Index, having an estimated ~90% donation rate and a ~65% volunteering rate. While the World Giving Index measures participation, rather than absolute funds moved, the underlying pool for philanthropy is quite large:

• Religious giving - Domestic religious giving, particularly Islamic (ZISWAF, which stands for zakat, infaq, sadaqah, and wakaf), is estimated by national bodies (like BAZNAS) to exceed 2 billion dollars annually. The theoretical TAM (total addressable market) for Islamic religious giving has been estimated to reach 20 billion dollars annually.
• Expanding UHNWI base - Knight Frank estimates ~8,000 HNWIs (10 million dollars+) and ~4,000 UHNWIs (30 million dollars+) in Indonesia, with UHNWI growth projected among the fastest globally.
• Retail crowdfunding - Platforms like Kitabisa have popularized micro-donations across millions of users across Indonesia, moving 280 million [...]

---

Outline:

(00:15) TL;DR

(00:44) Why Indonesia?

(02:56) The 'effectiveness premium'

(04:23) What we're building

(08:34) Key uncertainties

(10:37) How you can support us

(11:26) Get in touch

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TLDR: Take the population ethics quiz here: https://mdickens.me/pop-ethics/

Population ethics is an oft-overlooked subfield within ethics. Many people hold views that they don't realize contradict each other, or that have strange implications that they wouldn't endorse if they thought about it more.

Not just that—population ethics is a BIG DEAL. A lot of ethical decisions hinge on how you think about changes in future populations.

I'll give concrete examples below, but reading the examples might change your answers, so you should take the quiz first. It has 13 to 17 short questions. (Questions are skipped when they can't affect the results.)

Cross-posted from my website.

Nobody gets to ignore population ethics

There is no version of population ethics where you can just say, "Whatever feels intuitively right is what's right." Common-sense views on population ethics cannot all be true.

The quiz illustrates that fact. It asks you some questions about how you judge different situations involving varying populations. Then, based on your answers, it figures out:

Answers are always contradictory [...]

---

Outline:

(00:56) Nobody gets to ignore population ethics

(02:07) Concrete scenarios where population ethics matters

(02:19) Example 1: Child mortality vs. family planning

(04:14) Example 2: Environmentalism

(05:52) About the quiz

(07:24) AI usage disclosure

(07:56) Source code

The original text contained 26 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

This post was written while participating in Gen Stream in London and done based on inspiration and with some advice from Roman Ross based on his work on helping rejected applicants upskill.

Introduction

There are way more applicants for AI safety fellowships, courses and projects than organizations have the capacity to accept. That means that for every fellowship that runs, hundreds and sometimes thousands of people will receive rejection letters. There is probably a better way that we could formulate rejections to make sure they have the proper resources to upskill and be ready for the next round, or redirect them to a fellowship that would be a better fit for them. This would expand the talent pool of people working in AI safety and reduce drop-out.

As someone who has faced my own share of rejection letters, this can be extremely discouraging, lead to demotivation and potentially dropping out of AI safety entirely if you can’t find the right fit. If you have sought out and applied for an AI safety fellowship, there is a good chance that you are motivated to work in the field and have at least something to contribute. We want to be able [...]

---

Outline:

(00:25) Introduction

(03:33) The Readiness Grid

(04:10) Rejection Letters

(05:54) Analysis

(08:15) The Best Rejections

(09:43) Ideas for Improvement

(13:19) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Reflections from the Effective Giving & Careers team.

Earlier this year we ran our first open call for the effective careers side of our work — a request for proposals from organizations that help people move into high-impact careers. We’re excited to share the results: We’re awarding $13,075,301 in grants to 28 organizations via the Effective Giving & Careers Fund, and several other Coefficient Giving funds expect to fund up to 16 more.

That ~13 million dollars is well above the ~5 million dollars we originally estimated, which reflects both the volume and the quality of applications. The RFP drew 474 applications. From that pool, 111 organizations advanced to a more in-depth second-stage review, either with us or with a partner team. Weighting each grant's modeled social return on investment by its funding amount gives a mean of roughly 5,500x — though these are early, uncertain estimates.

Who's receiving funding

These are some of the organizations that were awarded grants, grouped by their primary function (although most of them do more than one thing). Each grant works to transition more people into high-social-impact work across cause areas like global health and economic development, farm animal welfare, global catastrophic risks [...]

---

Outline:

(01:15) Who's receiving funding

(06:45) What we funded, and what we learned

(08:52) Looking ahead

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Last week, the Forum grappled with cluelessness. The volume of entries (76 essays!)—from newcomers and Forum power users alike—was a happy surprise, and there were many thoughtful contributions beyond those mentioned below.

First place ($2,500):

• Elliott Thornley, for Challenging the unawareness argument with third actions and mixed actions

Joint second place ($1,500 each):

• Magnus Vinding (aka ‘unsharp’), for Defusing cluelessness with degrees of justification

• Impartial Partially, for Bottom-up bracketing: A person-centered response to cluelessness

The judges—Andreas Mogensen, Jesse Clifton and Anthony DiGiovanni—decided against awarding the up-to-50 thousand dollars outlier prize. Though they saw some promise in the winning essays’ approaches, their minds have not been significantly changed. The outlier prize remains open; there is still much progress to be made on cluelessness.

Here's more from our judges on the winning essays. (Their comments are technical! If you haven’t yet read ‘Cluelessness: Summary of the argument’, I suggest starting there.)

This piece offers two challenges to Anthony's argument. First, it challenges the normative premise on the grounds that only c-preferring A to B when we expect our ideal self to prefer A to [...]

---

Outline:

(00:27) Essay winners

(01:31) Challenging the unawareness argument with third actions and mixed actions

(01:37) Jesse's comments

(04:15) Defusing cluelessness with degrees of justification

(04:20) Andreas's comments

(05:23) Jesse's comments

(07:32) Bottom-up bracketing: A person-centered response to cluelessness

(07:39) Andreas's comments

(08:43) Anthony's comments

(11:07) Judges' shortlist

(11:49) Comment winners

(13:24) Wrapping up

The original text contained 10 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post.

Subtitle: It's a major warning shot, and might be the last one we get

All opinions are my personal view, and don’t represent my employer or fellow investigators.

This week, METR and Redwood Research published the report on our independent investigation into agents’ behavior and motivations in the Hugging Face attack; I was one of the investigators. This was an absolutely wild incident — I encourage you to check out the full report, but METR's tweet thread packs in some of the highlights.

What surprised me

When we started this investigation a week before OpenAI's Black Hat talk revealed a number of key details, I had a fundamentally incorrect conception of what basically happened in this incident. In this post, I’ll go over five things I was very wrong about going in.

1. The sheer scale

I knew there were multiple models involved from OpenAI's initial post, but I assumed that a few different agents happened to have broken out of their sandboxes separately, or maybe several subagents had spawned from [...]

---

Outline:

(00:51) What surprised me

(01:10) 1. The sheer scale

(02:00) 2. All the illicit messaging

(03:16) 3. The agents' actual goals

(04:12) 4. The peer altruism

(05:04) 5. The efforts to manipulate logs

(06:08) What it means

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A preliminary estimate, and a request for better ones.

Summary

I believe the standard literature estimates for the number of DALYs attributable to a case of stunting are too low, largely because they don’t account for the long term effects. This means that childhood nutritional interventions that reduce the prevalence of stunting may be substantially more cost-effective than previously believed.

Epistemic status

Exploratory and back-of-envelope, ~100 hours invested in an area well outside my expertise. I have moderate-to-low confidence in any single point estimate, but high confidence that the standard literature value of 0.23 DALYs greatly underestimates the lifetime burden of stunting. I’m posting this to be red-teamed. If you have a better method, or know of data I’m missing, please let me know.

AI use

I’ve had many conversations with Claude about this topic, and used Claude to develop the ideas, find references, and help generate the accompanying model. I’ve also previously discussed the topic with Grok, Gemini, and ChatGPT, all in their respective research / long-answer modes. The text of this post is all mine.

Conflict of interest

This post is written in my professional capacity at Semilla Nueva, an organization that makes more nutritious corn. [...]

The original text contained 28 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.