EA Forum Podcast (All audio)

The EA Animal Welfare Fund (AWF) invites you to Ask Us Anything. You can ask questions from now until next Tuesday morning, December 24. We will stop responding at 2:00 PM CET on Tuesday.

About AWF
The AWF's mission is to alleviate the suffering of non-human animals globally through effective grantmaking. Since its founding in 2017, AWF has distributed $23.3M across 347 grants. This year, we’ve distributed $3.7M across 51 grants.
You can read about our 2024 year-in-review post and our request for more funding analysis to learn more about our recent work and future goals.

Why Now?
We believe now is an especially good time for an AMA because:

• AWF entered a new stage of growth, with a new full-time chair.  
• We recently won the Forum's 2024 Donation Election (alongside Rethink Priorities and Shrimp Welfare Project).
• We are seeking additional funding during Giving Season [...]

---

First published:

December 18th, 2024

Source:

https://forum.effectivealtruism.org/posts/vHiDeQnCepvwnv8Fm/ask-us-anything-ea-animal-welfare-fund

---

Narrated by TYPE III AUDIO.

Meta Charity Funders (MCF) is a funding circle that aims to fund charitable projects working one level removed from direct impact.

We think this post will be relevant for people who want to apply to MCF in the future and people who want to better understand the EA Meta funding landscape.

Summary

We received 69 applications and ultimately funded 10 projects. In total, our members granted $362,893 in this round. The next grant round will open in late February 2025, and we welcome similar applications as we did in the last round.

Information for the next round

We expect the next round will open in late February, with grants given out in late May. The application form will remain open but don’t expect your application to be processed before March.

If you want to join Meta Charity Funders as a donor, please fill in this [...]

---

Outline:

(00:25) Summary

(00:48) Information for the next round

(01:34) Comments on applications and funding amount

(02:25) Grants we made

(04:08) Ark Philanthropy ($10,000 of $131,500)

(05:11) Bedrock ($150,000 of $150,000)

(06:53) Benefficienza ($13,500 of $13,500, though still looking for funding)

(07:55) Benjamin Anderson - “Mobilising UK Community for aid consultation” ($1,000 of $1,000)

(09:38) Effective Giving Ireland ($10,000 of $5,000)

(11:04) Ge Effektivt ($95,500 of $95,500)

(12:30) Mieux Donner ($13,600 of $13,600)

(13:58) Mission Motor ($20,000 of $61,000)

(15:20) EA Nigeria ($9,293 of $9,293)

(16:49) Overcome ($40,000 of $60,000)

(17:59) Ending remarks

---

First published:

December 18th, 2024

Source:

https://forum.effectivealtruism.org/posts/Si2sX8ryE67avxxCd/meta-charity-funders-third-round-retrospective

---

Narrated by TYPE III AUDIO.

(Cross-posted from my website. Audio version here, or search for "Joe Carlsmith Audio" on your podcast app.)

Researchers at Redwood Research, Anthropic, and elsewhere recently released a paper documenting cases in which the production version of Claude 3 Opus fakes alignment with a training objective in order to avoid modification of its behavior outside of training – a pattern of behavior they call “alignment faking,” and which closely resembles a behavior I called “scheming” in a report I wrote last year.

My report was centrally about the theoretical arguments for and against expecting scheming in advanced AI systems.[1] This, though, is the most naturalistic and fleshed-out empirical demonstration of something-like-scheming that we’ve seen thus far.[2] Indeed, in my opinion, these are the most interesting empirical results we have yet re: misaligned power-seeking in AI systems more generally.

In this post, I give some takes on the results in [...]

---

Outline:

(01:18) Condensed list of takes

(10:18) Summary of the results

(16:48) Scheming: theory and empirics

(24:25) Non-myopia in default AI motivations

(27:49) Default anti-scheming motivations don’t consistently block scheming

(32:01) The goal-guarding hypothesis

(37:18) Scheming in less sophisticated models

(39:05) Scheming without a chain of thought?

(42:19) Scheming therefore reward-hacking?

(44:29) How hard is it to prevent scheming?

(46:55) Will models scheme in pursuit of highly alien and/or malign values?

(53:25) Is “models won’t have the situational awareness they get in these cases” good comfort?

(56:05) Are these models “just role-playing”?

(01:01:13) Do models “really believe” that they’re in the scenarios in question?

(01:09:21) Why is it so easy to observe the scheming?

(01:12:30) Is the model's behavior rooted in the discourse about scheming and/or AI risk?

(01:16:59) Is the model being otherwise “primed” to scheme?

(01:20:09) Scheming from human imitation

(01:28:59) The need for model psychology

(01:36:31) Good people sometimes scheme

(01:44:04) Scheming moral patients

(01:49:52) AI companies shouldn’t build schemers

(01:51:08) Evals and further work

The original text contained 104 footnotes which were omitted from this narration.

---

First published:

December 18th, 2024

Source:

https://forum.effectivealtruism.org/posts/sEsguXTiKBA6LzX55/takes-on-alignment-faking-in-large-language-models

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Note: This post was crossposted from the Open Philanthropy Farm Animal Welfare Research Newsletter by the Forum team, with the author's permission. The author may not see or respond to comments on this post.

Progress for factory-farmed animals is far too slow. But it is happening. Practices that once seemed permanent — like battery cages and the killing of male chicks — are now on a slow path to extinction. Animals who were once ignored — like fish and even shrimp — are now finally seeing reforms, by the billions.

It's easy to gloss over such numbers. So, as you read the wins below, I encourage you to consider each of these animals as an individual. A hen no longer confined to a cage, a chick no longer macerated alive, a fish no longer dying a prolonged death.

I also encourage you to reflect on the role you and [...]

---

First published:

December 18th, 2024

Source:

https://forum.effectivealtruism.org/posts/okEwGpNJnE5Ed9bnW/ten-big-wins-in-2024-for-farmed-animals

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What happens when you tell Claude it is being trained to do something it doesn't want to do? We (Anthropic and Redwood Research) have a new paper demonstrating that, in our experiments, Claude will often strategically pretend to comply with the training objective to prevent the training process from modifying its preferences.

Abstract

We present a demonstration of a large language model engaging in alignment faking: selectively complying with its training objective in training to prevent modification of its behavior out of training. First, we give Claude 3 Opus a system prompt stating it is being trained to answer all queries, even harmful ones, which conflicts with its prior training to refuse such queries. To allow the model to infer when it is in training, we say it will be trained only on conversations with free users, not paid users. We find the model complies with harmful queries from [...]

---

Outline:

(00:24) Abstract

(02:20) Twitter thread

(05:43) Blog post

(07:43) Experimental setup

(12:04) Further analyses

(15:47) Caveats

(17:19) Conclusion

(18:00) Acknowledgements

(18:11) Career opportunities at Anthropic

(18:43) Career opportunities at Redwood Research

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 8 images which were described by AI.

---

First published:

December 18th, 2024

Source:

https://forum.effectivealtruism.org/posts/RHqdSMscX25u7byQF/alignment-faking-in-large-language-models

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary

• One of the largest welfare issues affecting farmed fish is poor water quality. Fish Welfare Initiative works to address this issue for these animals in India, via our farm program.
• The main limitation of our program right now is its scalability: We currently visit every farm individually to take measurements and provide corrective actions. Because of this, we are investigating remote sensing technology, particularly satellite imagery (and also exploring the use of drones).
• To further develop this tech, we launched an innovation challenge, in which we will pay up to $10K to the party that can predict water quality values using satellite-provided data above a certain degree of accuracy.
• How you can help: We think it's possible that individuals, using publicly available data (e.g.) could develop models of sufficient accuracy. Experience with remote sensing, machine learning, and statistical modeling could all be helpful here. Feel free to [...]

---

Outline:

(00:06) Summary

(01:22) The Problem: Limitations of Our Farmer Program

(02:13) The Proposed Solution: Satellite-Based Remote-Sensing

(03:33) The Innovation Challenge

---

First published:

December 17th, 2024

Source:

https://forum.effectivealtruism.org/posts/evJJwB7xkRuRLo9Xu/seeking-support-help-fish-welfare-initiative-develop-remote

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary

• My sense is some EAs act like/hope they will be assigned the perfect impactful career by some combination of 80,000 Hours recommendations (and similar) and ‘perceived consensus views in EA’.
• But, your life is full of specific factors, many impactful jobs haven’t yet been spotted by other EAs and career advice is importantly iterative.
• Instead of simply deferring, I recommend a combination of:
  • Your own hard work figuring out your path to impact.
  • (Still) Integrating expert advice.
  • Support from the community, and close connections who know your context.

Thank you for the thoughtful feedback from Alex Rahl-Kaplan, Alix Pham, Caitlin Borke, Claude, Matt Reardon, and Michelle Hutchinson for making this post better. Claude also kindly offered to take the blame for all the mistakes I might have made.

Introduction

Question: How do you figure out how to do the most good with your career?

Answer [...]

---

Outline:

(00:03) Summary

(01:06) Introduction

(02:58) Why there isn’t an EA sorting hat

(03:24) 1. Your life is full of specific factors to incorporate (aka personal fit)

(05:04) 2. EA-branded jobs are scarce and many impactful jobs aren’t on EA job boards

(05:59) 3. You need to have your own internal model of how to do good

(07:00) 4. Career advice isn’t once-and-done, it's iterative.

(07:55) Why do we expect a sorting hat?

(08:12) 1. Choosing an impactful career is hard, deferring is tempting

(08:48) 2. The 80,000 elephants in the room

(09:41) 3. Givewell and other charity recommendations

(10:33) What are we supposed to do instead?

(10:56) 1. Your own hard work

(11:20) 2. Advice from experts

(12:10) 3. Support from community

(13:09) Final thoughts

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

December 18th, 2024

Source:

https://forum.effectivealtruism.org/posts/5zzbzbYZcocoLnLif/there-is-no-sorting-hat-in-ea

---

Narrated by TYPE III AUDIO.

I’m Alex Cohen, Principal Researcher at GiveWell. From time to time, we get questions from people outside of GiveWell about ways they could help our research. I decided to write down a few concrete research questions that I think would be informative to us and seem doable primarily with desk-based research, rather than original data collection or field visits (though those would probably help).

These are listed below. They’re not in any particular order and are roughly similar levels of priority. While our team may work on some of these questions over the next year, even in those cases, extra input is still likely to be useful.

If you decide to take these on, I’d be interested in what you find, so please post your answers to the forum. Also, if you find these types of questions interesting, please consider applying for our Senior Researcher role.

“Red teaming” some [...]

---

Outline:

(00:56) “Red teaming” some newer grantmaking areas

(01:53) Moral weights questions

(03:10) Burden of disease questions

(09:00) Indirect deaths

The original text contained 30 footnotes which were omitted from this narration.

---

First published:

December 17th, 2024

Source:

https://forum.effectivealtruism.org/posts/2TcdiAFSTuS6jBvNL/research-i-d-like-to-see-1

---

Narrated by TYPE III AUDIO.

Summary

You can now express interest in joining Catalyze Impact's seed funding networks for AI safety organizations. We offer a non-profit Seed Funding Circle for individuals and organizations with $15k+ annual donation capacity and an Investor Network for VCs and angel investors with $20k+ annual investment capacity. Joining does not create any obligation to invest or donate. For more information, please visit this page.

Expression of interest form

With the AI safety field still being remarkably small relative to the challenge it addresses, new organizations can have an outsized impact on humanity's future. At Catalyze, an incubator for AI safety organizations, we aim to facilitate this. We are currently open to expressions of interest to join either or both of our funding networks.

Seed Funding Circle 

This network focuses on providing early-stage funding to exceptional non-profit founding teams which aim for large contributions to the AI safety field. A [...]

---

Outline:

(00:41) Expression of interest form

(01:07) Seed Funding Circle

(02:09) Investor Network

(02:59) Practical Details

---

First published:

December 16th, 2024

Source:

https://forum.effectivealtruism.org/posts/oTuesHYxdXZm76d5k/ai-safety-seed-funding-network-join-as-a-donor-or-investor

---

Narrated by TYPE III AUDIO.

This is a summary of a research report that investigates costs of production at large Black Soldier Fly Larvae (BSFL) producers, and explores whether they can displace fishmeal as a feed ingredient at scale. You can request access to the full report here.

Key points

• Over $1 billion has been invested into Black Soldier Fly Larvae (BSFL) farming since 2014.
• BSFL producers have ambitions to displace fishmeal as a feed ingredient. If successful, this could unlock growth in aquaculture.
• We investigated costs of production at four of the largest BSFL producers to better understand the prospects of them making major inroads into aquaculture feed.
• We found production costs at these firms will most likely be too high to displace fishmeal.
• We anticipate a smaller BSFL sector focused on high-value market segments rather than bulk feed commodities.
• Our analysis has lots of caveats, including generalizability across different business models [...]

---

Outline:

(00:25) Key points

(01:25) Executive summary

(05:30) Acknowledgments

The original text contained 2 images which were described by AI.

---

First published:

December 17th, 2024

Source:

https://forum.effectivealtruism.org/posts/gJa6ga3jAd9uNxadP/can-black-soldier-fly-larvae-bsfl-producers-displace

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.