EA Forum Podcast (All audio)

Convergence 2024 Impact Review home page.

2024 marked the first full year with the new Convergence Analysis 9-person team. This year we published 20 articles on understanding and governing transformative AI. Our research impacted regulatory frameworks internationally. In the US we provided consultation to the Bureau of Industry and Security that directly informed their proposed rule on reporting requirements for dual-use AI, while in the EU we saw specific recommendations incorporated into the EU AI Act GPAI Code of Practice. We led expert field-building around AI's economic impacts through the Threshold 2030 conference, and AI scenario modeling via the AI Scenarios Network. Our work reached mainstream media, universities, and over 184,000 viewers on social platforms.
We organized our activities into three programs: AI Clarity, AI Governance, and AI Awareness.

1. AI Clarity: performing AI Scenario Planning

• Published over 170 pages of AI Clarity research across 10 [...]

---

Outline:

(00:16) Impact overview

(04:09) Convergence's mission

(05:22) List of outputs

(07:21) Outcomes and impacts in more detail

(07:27) AI Clarity

(08:14) AI Scenario Planning

(10:13) Theories of Victory

(11:58) AGI Economics

(13:07) AI Timelines

(14:01) AI Governance

(14:20) Technical Controls and Infrastructure

(16:40) National Policy Frameworks

(18:40) Strategic Governance Research

(20:03) AI Awareness

(22:02) Operations

(22:41) 2024 budget

(23:45) 2025 budget

(24:18) 2025: January and February outcomes

(26:44) 2025: Ongoing initiatives

(30:39) Funding Gaps and opportunities

(31:59) Three funding scenarios

(32:17) Base: $880,000

(32:32) Moderate Growth: $1,850,000

(33:10) Strategic Growth: $4,150,000

(33:49) The case for funding Convergence

(34:44) Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I recently decided to check how much I've donated to GiveWell's Top Charity Fund over the last three years. I only got properly bought into EA in 2021/22, and only started donating on a monthly basis in Jan 2022. I also don't earn a huge amount, so haven't been donating much monthly in real terms, although 18 months ago I stepped up my donation amount.

I kinda forgot about my donations, as one does, until recently I was like "oh, I've been doing this for like 3 years now, I wonder how much I've actually donated?"

I couldn't easily find a way of seeing my total donations at a glance on my Giving What We Can dashboard (which might well be me not looking at the right thing) so I just got my calculator out and added up all the individual payments.

I was kinda shocked to see that [...]

---

Outline:

(01:10) But what does that actually mean?

(02:29) How it feels to (maybe) save a life

(02:56) Abstract vs concrete impact

(05:54) Does this make me a saint?

(07:54) Finding a balance I can live with

---

First published:

Source:

Narrated by TYPE III AUDIO.

Summary

Understanding how people first encounter EA and what helps them get involved in EA is important for understanding how to optimize recruitment. With the Meta Coordination Forum survey suggesting that it would be ideal for the EA community to grow rapidly, this is of high importance. To meet the recommended targets, some of these routes into EA would need to scale very dramatically over the next year.

• Personal contacts (17.9%) and 80,000 Hours (15.8%) remain the largest sources of people first hearing about EA
  • 80,000 Hours continues to increase in importance
  • Personal contacts were particularly important for women first hearing about EA (25.2% vs. 15.2% for men)
  • 80,000 Hours was particularly important for non-White people first hearing about EA (22.4% vs. 13.4%)
  • Highly engaged EAs disproportionately first learned about EA through personal contacts (19.9% vs. 15.6%)) and EA groups (10.1% vs 6.4%)
• 80,000 Hours and personal contacts [...]

---

Outline:

(02:52) Introduction

(04:19) Where do people first hear about EA?

(09:16) Personal contact

(10:07) Books, articles, and blog posts

(10:12) Authors

(11:29) Books

(12:41) Articles

(13:10) Blogs

(13:33) Podcasts

(14:15) Group differences

(14:19) Gender

(16:17) Racial/ethnic identity

(17:09) Engagement

(18:20) The effect of outreach

(20:32) Factors Important for Getting Involved

(25:25) Group differences

(25:29) Gender

(28:18) Racial/ethnic identity

(29:44) Engagement

(31:15) Referrer Robustness Checks

(34:21) Appendix

(34:25) Appendix 1: Alternative year-over-year comparison of where respondents first heard of EA

(34:50) Appendix 2: Composition of engagement levels within each source of where respondents first heard of EA

(35:07) Appendix 3: Alternative year-over-year comparison of factors important for getting involved in EA

(35:36) Appendix 4: Factors important for getting involved in EA by cohort (as a proportion of the number of respondents in each cohort)

(36:02) Appendix 5: Referral exclusions

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Today marks the launch of a new federal election campaign from Australians for AI Safety — bringing together leading AI experts to advocate for crucial safety policies ahead of the federal election. Good Ancestors has been leading the coordination of this initiative over the past few months and we need your help now!

What the campaign includes:

• An expert-led open letter calling on the next government to establish an Australian AI Safety Institute and progress an Australian AI Act with mandatory guardrails for high-risk AI
• An election scorecard showing where candidates in each electorate stand on these AI safety policies
• Tools to contact local candidates who haven't shared their positions

Why this matters:

Australia committed to creating an AI Safety Institute as part of the Seoul Declaration but remains the only signatory country yet to deliver. With potentially transformative AI developments possible within the next few years, securing policy commitments during this election [...]

---

Outline:

(00:31) What the campaign includes:

(00:55) Why this matters:

(01:51) How to help (even if youre not in Australia):

(02:33) Why are people signing?

---

First published:

Source:

Narrated by TYPE III AUDIO.

One of the most reasonable ethical aims from a variety of perspectives is to focus on s-risk reduction, namely on steering the future away from paths that would entail vastly more suffering than Earth so far. The research field of s-risk reduction faces many challenges, such as narrow associations to particular ethical views, perceived tensions with other ethical aims, and a deep mismatch with the kinds of goals that most naturally motivate us. Additionally, even if one strongly endorses the goal of s-risk reduction in theory, there is often great uncertainty about what pursuing this goal might entail in practice.

To address these challenges, here I aim to briefly:

1. Highlight how s-risk reduction can be highly valuable from a wide range of perspectives, not just suffering-focused ones. (§2)
2. Address perceived tensions between s-risk reduction and other aims, such as reducing extinction risk or near-term suffering. While [...]

---

Outline:

(02:19) 2. Common ground

(03:56) 2.1. Better lotteries

(05:33) 2.2. Shared risk factors

(08:31) 2.3. Staying safe

(11:12) 3. Need for proxies

(13:20) 3.1. Positive for motivation

(14:59) 3.2. Positive for coordination and shared aims

(15:51) 3.3. Positive for measurability

(16:59) 4. Promising proxies

(19:29) 4.1. General proxies

(23:18) 4.2. Personal proxies

(31:25) Acknowledgments

(31:38) Appendix: Longer motivation table

The original text contained 30 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Robert Evans has made an in depth four part series on the Zizians, which includes a lot of background on Rationality in the bay area, CFAR, and a lot of the ideas leading up to the murders (first three episodes).

Part One: The Zizians: How Harry Potter Fanfic Inspired a Death Cult

Part Two: The Zizians: Birth of a Cult Leader

Part Three: How The Zizians Went Full On Death Cult

Part Four: The Zizian Murder Spree (or Exactly How Harry Potter Fanfic Killed A Border Patrol Agent)

In my opinion this gives a pretty good perspective of how outside people perceive the Rationalist and EA communities.

Key points I think are important:

---

Outline:

(00:54) Key points I think are important:

(01:26) Opinion part

---

First published:

Source:

Narrated by TYPE III AUDIO.

TL;DR:

• Promise: Far-UVC demonstrates remarkable air-cleaning capabilities. In one of the most promising studies, its effectiveness was equivalent to “changing the air completely over in the room 184 times every hour,” compared to the CDC's recommendation of 5+ changes—and even hospital operating rooms only require 20 (Wood 2020).
• Neglect: Despite its promise, far-UVC is largely unknown outside academic and niche research circles.
• Tractability: Real-world safety and efficacy data are still pending; no FDA oversight exists, and current research is fragmented.
• Blueprint Biosecurity: They’ve dedicated two years to a 266 page pre-print report—the far-UVC pre-print Blueprint—which outlines research priorities and a path toward public trust, while also seeking to coordinate progress in this space.
• Call for feedback: Before they publish the final version of the far-UVC Blueprint, they are gathering feedback.
• Blueprint Biosecurity is Hiring! Blueprint Biosecurity is hiring for a Project AIR (Airborne Infection Resilience) Program Director and a Government Affairs Director.

A Fresh Look at Far-UVC [...]

---

Outline:

(02:29) The Promise of Far-UVC

(03:18) Why Isn't Far-UVC Everywhere?

(04:30) Blueprint Biosecurity's Role

(05:32) Opportunities and Next Steps

---

First published:

Source:

Narrated by TYPE III AUDIO.

Hi everyone,

So far, Vetted Causes has published two charity reviews, and people have questioned our intentions and criticized us for being anonymous. Based on this I (the founder of Vetted Causes) have decided it's best to reveal my identity. Previously, I’ve referred to myself as Isaac on this forum, but that was a pseudonym. My real name is Adam Hebert.

When I started Vetted Causes, I wanted to remain anonymous in order to:

• Prevent bias: I want to avoid developing any personal relationships with charities to prevent bias. Being anonymous makes this easier. Additionally, I want evaluations to be judged based on their content, rather than the backgrounds of the people who wrote them.
• Make my intentions clear: If Vetted Causes becomes a well-respected organization, I could gain prestige or financial benefits. However, these are not my goals. My sole motivation behind creating Vetted Causes is [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

TL;DR Having a good research track record is some evidence of good big-picture takes, but it's weak evidence. Strategic thinking is hard, and requires different skills. But people often conflate these skills, leading to excessive deference to researchers in the field, without evidence that that person is good at strategic thinking specifically.

Introduction

I often find myself giving talks or Q&As about mechanistic interpretability research. But inevitably, I'll get questions about the big picture: "What's the theory of change for interpretability?", "Is this really going to help with alignment?", "Does any of this matter if we can’t ensure all labs take alignment seriously?". And I think people take my answers to these way too seriously.

These are great questions, and I'm happy to try answering them. But I've noticed a bit of a pathology: people seem to assume that because I'm (hopefully!) good at the [...]

---

Outline:

(00:34) Introduction

(02:47) Factors of Good Strategic Takes

(05:43) Conclusion

---

First published:

Source:

Linkpost URL:
https://www.neelnanda.io/blog/50-strategic-takes

Narrated by TYPE III AUDIO.

Applications are open for the ML Alignment & Theory Scholars (MATS) Summer 2025 Program, running Jun 16-Aug 22, 2025. First-stage applications are due Apr 18!

MATS is a twice-yearly, 10-week AI safety research fellowship program operating in Berkeley, California, with an optional 6-12 month extension program for select participants. Scholars are supported with a research stipend, shared office space, seminar program, support staff, accommodation, travel reimbursement, and computing resources. Our mentors come from a variety of organizations, including Anthropic, Google DeepMind, OpenAI, Redwood Research, GovAI, UK AI Security Institute, RAND TASP, UC Berkeley CHAI, Apollo Research, AI Futures Project, and more! Our alumni have been hired by top AI safety teams (e.g., at Anthropic, GDM, UK AISI, METR, Redwood, Apollo), founded research groups (e.g., Apollo, Timaeus, CAIP, Leap Labs), and maintain a dedicated support network for new researchers.

If you know anyone who you think would be interested in [...]

---

Outline:

(01:40) Program details

(02:52) Applications (now!)

(03:10) Research phase (Jun 16-Aug 22)

(04:12) Research milestones

(04:43) Community at MATS

(05:39) Extension phase

(06:05) Post-MATS

(07:42) Who should apply?

(08:43) Applying from outside the US

(09:25) How to apply

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.