EA Forum Podcast (All audio)

This post was cross posted from LessWrong

TL;DR:

• If you enjoy LessWrong (and/or do EA/AI safety field building), you should consider starting a rationality reading group at your university. It's fun, cheap, and a good way to meet like-minded friends, and it can reach rationalist-ish people who’ve never encountered EA or AI safety.
• Starting one can be as easy as grabbing a couple friends, picking a time, place, and reading, sending an email blast, and bringing food. The ACX Meetups Everywhere call (organizer form due October 9th) can be a good way to find an initial group.

Context: this post was loosely based on a memo originally written by Noah Birnbaum, and I (Carolanne Jiang) most of the current version. “I” in this post refers to Carolanne, and “we” refers to Carolanne and Noah.

Also, UChicago Rationality is still going! If you are a student at UChicago interested in joining, meetings this quarter are on Wednesdays from 6 :00 – 7:00 PM in the Reg. Noah and I are both no longer at UChicago–for more information, please reach out to Will Kyle at [email protected]

UChicago Rationality was a huge success: it was very fun, a great place to make [...]

---

Outline:

(01:59) The backstory of UChicago Rationality

(02:58) More reasons on why you should consider running a rationality group

(04:46) Q: I am sold. How do I go about starting one?

(06:08) Appendix

(06:11) Counters arguments to why a rationality group might not contribute to field building for AI safety and/or EA

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I have recently decided to leave the Center for Reducing Suffering (CRS) to pursue independent research on the possible implications of UAP and quiet expansionist hypotheses for reducing suffering.

I have some history of writing about these topics (e.g. this essay collection), so this decision amounts to stepping more fully in that direction rather than keeping it perpetually in the background.

I co-founded CRS with Tobias Baumann in 2020, and I greatly appreciate all the support I have received from donors and colleagues while working there. Below, I will briefly explain why I have decided to make this shift.

Why do I prioritize these topics?

Two of the main reasons I prioritize these topics are theory and evidence. When it comes to theory, it seems that people tend to dismiss the possibility of quiet alien expansion based on very strong theoretical assumptions, but I don’t think those assumptions are nearly as solid as people generally assume. In fact, most of the essays I’ve written about this topic (e.g. From AI to distant probes, Silent cosmic rulers, and The cooperative silence hypothesis) argue against the typical theoretical assumptions that supposedly rule out quiet expansion or nearby alien technology. (See also [...]

---

Outline:

(00:48) Why do I prioritize these topics?

(02:32) Why will I do this work outside CRS?

(03:44) What this means for CRS

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

For years I had a blindspot in my thinking. I dismissed policy and advocacy work as “in another [lower-status] category” because I’m obviously a technical person! Even when I felt super pessimistic about my ability to do research that meaningfully contributes to addressing the threat of extinction from AI.

I’m not the type of person who would be “good at comms”. I’m an autistic math guy. Luckily comms has learnable skills, and I have gotten better through practice.

I’m now prioritizing advocacy, which currently seems a lot more promising than research.

I got lucky because, thanks to Coxon, AI extinction became mainstream shortly after I started. Even so, I've been very surprised to have become the biggest anti-extinction expert voice in Finland as a young person. This happened because:

• I have a background in AI safety research and active work affiliation with an AI safety org, showing credible expertise
• Finnish journalists want to hear from Finnish experts
• Finnish academics in media have been mostly dismissive of extinction, and
• other genuinely concerned Finnish experts were hardly speaking up.

So far, most notably, I…

1. Published an expert op-ed in the most prestigious slot in Finland (co-authored with @So8res).
2. Did four [...]

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand.

Cross-posted from my website and Substack.

Verifying an international agreement pausing AI development needs evidence of how compute is used. Cryptography can give protocols that run in software on the chips already installed, without trusted hardware checking each computation. This post is a literature review of one such tool, proof of useful work (PoUW), with which a GPU proves that it did a given amount of work, here typically matrix multiplication. In contrast with inference verification, PoUW has been developed almost exclusively in relation to cryptocurrencies in order to use a more useful computation instead of the useless hashing used in Bitcoin. PoUW could be used to show remotely that chips are kept busy on useless tasks during an emergency shutdown, or, together with a zero-knowledge proof of the workload, that they run only declared inference.

I describe the two existing constructions, by Komargodski and Weinstein for matrices over a finite field, and by Pearl Research for the FP8 format used in AI, verifiable for now [...]

---

Outline:

(02:15) Cryptography for verification

(11:05) How PoUW works and what it is for

(14:11) Why useful work is harder than useless work

(17:26) Komargodski-Weinstein: hiding the instance behind low-rank noise

(24:12) Pearl: quantization removes the shortcut but needs a new assumption

(29:37) Deployments and related protocols

(32:31) Deployment in a treaty

(35:31) Limitations of KW and their current status

(36:01) Open problems for mathematicians

(39:31) Outlook

(40:44) Acknowledgments

(41:30) References

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Our ecosystem is bottlenecked by people who have deep context and relevant skills.

This means that the time of people with context and relevant skills is very valuable: usually more valuable than their salary.

When taking this effect into account, it turns out that:

1. The net cost to the ecosystem of an organisation made up of high-context and relevant-skill people is much larger than the financial costs of that organisation.
2. The net cost to the ecosystem of an organisation which trains many staff might be much lower than the financial costs of that organisation.
3. The return curves of these organisations can get pretty whacky, in a way which will sometimes affect strategic decisions.
4. Giving your staff a good experience in the ecosystem and enabling them to increase their context / skills is important: it's likely a meaningful fraction of your organisation's impact.

Here are some hypothetical figures to illustrate what these effects might look like.

If you’d like, you can play around with these figures here.

The rest of the post is focused on better explaining the above and belabouring implications. I expect that if the above is intuitive to you and you’ve already absorbed the strategic implications [...]

---

Outline:

(01:59) What a hire costs

(03:10) A worked example

(04:27) Why the return curves get whacky

(05:57) Put your own numbers in

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Two days after announcing that Caroline Ellison had joined Manifund, Austin Chen asked CEA's community health team to 'please let me know if there are things you believe I should do moving forward to mitigate this impact.'

I attempt to answer this question, albeit put slightly differently: if you'd been responsible for implementing this decision, what should you do before it's published?

I'll get my view on the decision itself out of the way first, because it'd be odd to pretend I don't have one. I think it was a bad call. Zachary Robinson's comment makes the principled case better than I could, and Julia Wise puts it more succinctly: 'don't hire someone with a fraud conviction for a finance or philanthropy role.' So I'll add only the practical one, which I don't think has been talked about: it's just not worth the hassle. Caroline Ellison is one person. Austin says he hired her because she was an excellent candidate, and I believe he believes that. But is she so exceptional that nobody else could have done the job? And is she worth all the ensuing very public drama and ostracisation? In my eyes, absolutely not.

But I work [...]

---

Outline:

(01:43) Alex, this exercise is silly, why are you bothering?

(03:20) Step 0: Disagree on the record, then set the conditions for reversal

(04:16) Step 1: Keep the trial internal, then announce on day one

(06:18) Step 2: Design the safeguards before you announce

(08:54) Step 3: Tell the people who matter first, and give them weeks, not hours

(11:06) Step 4: Write an announcement that only says what helps

(13:34) Step 5: Get the tone right afterwards

(15:00) Austin's question

(15:47) Acknowledgements

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Opinion piece by Marcus A. Davis, CEO of Rethink Priorities

TL;DR:

1. The problem: Major donors choose between causes like AI safety, global health, and animal welfare without writing down why. In interviews, many couldn’t say what their gifts achieved or how they weighed one cause against another.
2. Why it happens: Skipping the spreadsheet feels simpler and more humble, but every gut call is still a model, just one nobody can see or check.
3. How it goes wrong: The hidden model makes strong claims by accident. An even split says every cause is equally good at the margin. “I’ll adjust a bit for my doubts” could mean very different strategies. Anything you didn’t think about gets counted as zero. A model you only follow when it agrees with you corrects nothing.
4. Why it gets worse with harder questions: Cross-cause decisions are complex and give no feedback, which is exactly when intuition is least reliable, and mistakes are hardest to notice.
5. What it costs: Millions can be given with no all-things-considered comparison behind them, so money is likely being misallocated.
6. What to do: Small donors can rely on a fund or evaluator. Large donors should pay for in-depth analysis [...]

---

Outline:

(03:09) Congratulations, that was a model

(04:52) The critics of models have a point, but it doesn't prove what they think

(06:19) Leaving it out is the same as counting it as zero

(07:33) Now let's try a harder question

(10:00) "Just adjust a bit" is (hilariously) underspecified

(12:27) "But the numbers are made up"

(14:14) Why not use rules of thumb?

(15:01) It's tempting to only follow models when they agree with you

(15:50) But all of this takes too much effort

(17:08) "I can't follow the model, the model says 'we don't know'"

(17:35) What this means for you

(19:06) Flip the default

(20:44) Acknowledgements

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: I’ve puzzled over this for a long time but I’m not sure I came to the right answers. I'm sharing this partly because it might help others work on resource allocation, but mostly because some smarter reader might tell me how to do it better.

TL;DR:

• I helped set up a large foundation and we wanted a consistent way to allocate grants
• We planned to allocate substantially more than our field was able to absorb right now, giving us the opportunity to actively shape our field
• We outlined key principles and considerations
• We quickly learned we could not just copy the for-profit world: so what could we do?
• We could try and optimize perfectly, mathematically, but it gets absurd and impractical
• Instead, we tried out a bunch of different approaches to finding a consistent grant allocation formula, settling on the one that most “felt” right given our starting principles and constraints
• I quite liked this process of setting principles and constraints, and then vibing your way to an allocate formula that feels right vs being super mathematical
• I wanted to share this process as a real example of trying to work out how to allocate funding. [...]

---

Outline:

(01:33) x = height of the opposing player

(02:50) Searching for the perfect grant-allocation formula

(03:30) Our core metrics

(04:32) We couldn't just copy from other grantmakers or the for-profit world

(07:46) Turning our principles into a utility function + optimizing

(09:56) "Vibing" our way to a utility function

(10:35) The "Obvious" Option: Allocate by Expected Cost-Effectiveness

(11:38) The "Risk-Buckets" Approach

(12:43) The Kelly Criterion

(14:22) Sharpe ratio

(14:57) Edge

(16:24) What We Actually Did

(17:11) If we settled on edge, doesn't that mean there was a calculable utility function in the background the whole time?

(20:00) What I learned in all of this

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I’ve been an organizer for my EA uni group (EA Purdue) for nearly two years. Despite my relative experience, I’m often unsure of what exactly I should say when giving advice to younger organizers. Each group is different, its organizers are different, and what advice will make sense to them will also be different. Not every organizer is struggling with the same problems I once did or will benefit from the same solutions.

The one place where I do feel confident that almost all newer organizers are making a mistake is in individual outreach. From what I’ve observed, newer organizers have a pretty high bar for reaching out to someone on an individual basis, and don’t send many messages to specific people. I think newer organizers should be individually messaging their members much, much more. It seems likely to me that the best and most experienced organizers send ten times the amount of texts and messages for outreach, compared with newer organizers. This might look like the experienced organizer texting more than a dozen people each week.

As an organizer, you’re running a group that has various activities and programming. In order to get people to attend this programming [...]

---

Outline:

(06:08) Do I really need to *text* them?

(07:12) Isn't this all kind of weird?

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Applications open: Strategic Animal Funding Circle autumn 2026 round

The Strategic Animal Funding Circle, now run by Senterra Funders, is accepting applications for its autumn 2026 funding round.

We’re seeking promising, cost-effective opportunities to reduce farmed and wild animal suffering, with a particular focus on neglected areas and funding gaps.

Application deadline: 1 November 2026, 11:59 pm PST.

Read the full request for proposals | Apply here

What we’re looking for

Our aim is to complement existing grantmaking by supporting valuable work that might otherwise struggle to attract funding. We’re especially interested in:

• Smaller or newer organisations with promising approaches and room to grow.
• Neglected species, geographies or interventions.
• Early-stage or unusual approaches with a strong theory of change.
• Opportunities outside larger grantmakers’ scope, including smaller funding requests (under 200 thousand dollars), and organisations with annual budgets below US$500,000.

These funding and budget figures describe preferences, rather than strict eligibility limits. Please consult the full RFP for the eligibility criteria, areas of interest and exclusions.

We are working with Coefficient Giving and the EA Animal Welfare Fund to help identify gaps where the circle could add value.

What makes a strong application?

We want to understand:

• How [...]

---

Outline:

(00:13) Applications open: Strategic Animal Funding Circle autumn 2026 round

(00:50) What we're looking for

(01:51) What makes a strong application?

(02:30) How the process works

(03:12) Apply or share

---

First published:

Source:

---

Narrated by TYPE III AUDIO.