EA Forum Podcast (All audio)

Crosspost of this on my blog.

1 My basic view

'Cause everybody cries
Everybody hurts sometimes

Sometimes everything is wrong

—Rem “Everybody hurts”

You didn’t think the era of absurdly long posts was over, did you?

Prior to the 2024 election, there were a series of swing states that pundits forecasted could have gone either way. All of them ended up going to Trump by a fairly wide margin, as predicted by some people (e.g. Glenn, Keith Rabois). None of the states that were solidly blue went to Trump, but all the states that pundits and prediction markets weren’t sure about went red.

I have a similar bet regarding consciousness.

I suspect that all of the species that are currently of significantly debated consciousness—call them swing state species—are conscious. This would include crabs, lobsters, fish, and most insects, but it wouldn’t include oysters or AI. There's fairly [...]

---

Outline:

(00:09) 1 My basic view

(06:01) 2 Intensely? What about neuron count?

(17:34) 3 Fish

(28:46) 4 Shrimp shrimpy shrimpy shrimp (and other crustaceans)

(40:52) 5 Insects

(01:01:52) 6 Concluding remarks

The original text contained 6 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Rethink Priorities Worldview Investigation Team is working on a model of consciousness in artificial systems. This article describes some complications to thinking of current LLM AI systems as coherent persons, despite the appearances to the contrary.

Introduction

The standard way that most people interact with cutting-edge AI systems is through chatbot interfaces like the ones pictured above. These interfaces are designed to look like the traditional messaging platforms that people use to talk with friends and family. The familiarity of this framework and its apparent analogy to interpersonal conversations encourages us to understand these systems in roughly the way we understand human conversation partners. However, this conception of AI systems is inaccurate. Moreover, appreciating its inaccuracies can help us think more carefully about whether AIs are persons, and what might be suitable indicators for AI consciousness and moral patienthood. We should take care to ensure [...]

---

Outline:

(00:29) Introduction

(02:44) Background

(05:20) Complications

(05:24) 1. The user is not interacting with a single dedicated system.

(09:18) 2. An LLM model doesn’t clearly distinguish the text it produces from the text the user feeds it.

(14:04) 3. An LLM's output need not reflect its commitments or represent its own perspective.

(27:04) 4. An LLM's output gives us fairly narrow insight into internal activity.

(37:46) Conclusion

The original text contained 54 footnotes which were omitted from this narration.

The original text contained 5 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

On the emergence of history's reins

One general law, leading to the advancement of all organic beings, namely, multiply, vary, let the strongest live and the weakest die.

― Charles Darwin

We are a way for the universe to know itself.

— Carl Sagan

If you don’t know where you are going, you’ll end up someplace else.

— Yogi Berra

Imagine you were trying to explain how the world worked 10 billion years ago. Back then, the best explanation would be in terms of physics: galaxies forming and supernovas producing heavy metals. Ten million years ago, though, you’d talk about evolution: plants, mammals, and early hominids. Ten thousand years ago, when agriculture was being established, you might talk about culture and the spread of ideas.

Each of these forces — physics, evolution, and culture — gave rise to the next, producing [...]

---

Outline:

(00:07) On the emergence of historys reins

(04:15) The forces shaping history

(05:56) The blind forces

(07:48) The anthropocene

(12:49) The ascent of choice

(14:45) Is a Choice Transition inevitable?

(16:10) What the Choice Transition is not

(16:40) It doesn’t mean omnipotence

(17:18) It needn’t mean a single chooser

(18:12) It needn’t mean the end of new forces

(19:04) AI and the Choice Transition

(19:44) 1) Better foresight capabilities and understanding enable steering

(20:21) 2) Better coordination capabilities could allow for more coherent steering

(22:20) 3) Agentic AI systems could be (among) the entities steering

(23:20) 4) Advances in AI could lead to centralization of power

(25:03) 5) AI might empower forces that squeeze out deliberate steering

(27:22) 6) Automation of research means it might all happen quickly

(28:35) The space of possible Choice Transitions

(30:27) Acknowledgements

(31:05) Appendix: comparison to existing frames

(31:11) Comparison to normal AI x-risk frames

(33:30) Related concepts

(33:34) Bostroms notion of a Singleton

(33:54) Yudkowskys notion of a Pivotal Act

(34:09) Finnveden, Riedel, and Shulman's notion of lock-in

(34:33) Ords notion of the Long Reflection

(34:54) Drexler's notion of Paretotopia

(35:10) Carlsmiths notion of yang

(35:38) Buterin's notion of d/acc

(35:59) Alexander's notion of Moloch

The original text contained 22 footnotes which were omitted from this narration.

The original text contained 4 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A place to explain your preferences, discuss them, and maybe change your mind.

Some comments on this thread are cross-posted from a text box which appears when you reach the end of the voting process, but everyone is welcome to post here whenever.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Join us in the Bay Area on March 1-2 (shortly after EAG Bay Area!) for two days of innovation, collaboration, and inspiration in the developing field of AI and animal welfare.

🎟️ Get earlybird tickets by Dec 1

🎓Apply to speak by Dec 15

Topics will include the future of animal agriculture, interspecies communication, sentience research, increasing animal consideration in frontier AI models, and more.

This year will be more multidisciplinary than in previous years and include a wide range of stakeholders from animal welfare scientists to AI researchers to animal advocates. Sessions will cover practical applications of AI and other advanced technologies for animals and focus on furthering this emerging field through an opportunities fair, networking sessions, and workshops to develop stakeholder maps and repositories of interventions that will serve as useful resources for anyone looking to do work in this field.

We [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Executive Summary

Welcome to ALLFED's 2024 Highlights, our annual update on what Alliance to Feed the Earth in Disasters has been up to this year. From producing fifteen scientific papers on global catastrophes to forming the first blueprint for an Extreme Pandemic Resilience Plan for Vital Industry, to being featured in over 90 media articles—it's been a busy, productive year of impact.

There is much more we could do to help build a resilient future. Thanks to a dedicated anonymous donor couple, up until Giving Tuesday, December 3, 2024, we can match 10% of all individual gifts made up to $5,000, with a special campaign target of $50,000. Please consider donating to us via our website this giving season and see your impact boosted as part of this time-limited offer.

Within These 2024 Highlights:

• Our prolific year in research has seen several papers published in top food science [...]

---

Outline:

(00:08) Executive Summary

(01:02) Within These 2024 Highlights:

(04:19) Strategy

(05:25) Our Core Values

(06:42) Research

(07:50) Our Published, Peer-Reviewed Research

(17:38) Research Pending Peer Review

(24:07) Policy

(27:34) Policy Mechanisms

(27:42) Australia Engagements

(30:40) Collaborations and Partnerships

(33:22) Communications

(34:31) Media Mentions

(34:38) Digital Media - Reach and Engagement

(35:04) Operations

(36:46) Events, Workshops and Presentations

(37:41) External Events

(37:45) EAG London 2024

(38:06) Cambridge Conference on Catastrophic Risks (CCCR) 2024

(39:18) World Food Forum (WFF) Flagship Event

(39:58) ALLFED's Workshops

(40:03) Argentina Government Workshop

(41:10) EAGx Nordics 2024

(42:06) EAG Bay Area 2024

(42:40) Our Team

(43:20) New Board Members

(44:00) Diversity and Inclusion

(44:51) Key Team Changes

(45:46) Volunteers and Internships

(47:22) ALLFED's 2024 Team Retreat

(47:53) Amplify Your Giving

(48:55) Our Thanks

The original text contained 17 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A U.S. congressional commission on Tuesday proposed a Manhattan Project-style initiative to fund the development of AI systems that will be as smart or smarter than humans, amid intensifying competition with China over advanced technologies.

The bipartisan U.S.-China Economic and Security Review Commission's (USCC) stressed that public-private partnerships are key in advancing artificial general intelligence (AGI), but did not give any specific investment strategies as it released its annual report.

Reuters: https://www.reuters.com/technology/artificial-intelligence/us-government-commission-pushes-manhattan-project-style-ai-initiative-2024-11-19/

Full report: https://www.uscc.gov/sites/default/files/2024-11/2024_Annual_Report_to_Congress.pdf

---

First published:

Source:

Narrated by TYPE III AUDIO.

Center on Long-Term Risk (CLR) is accepting applications to the next iteration of our introductory fellowship. We are now planning to run two related programs: the CLR Foundations Course and the CLR S-Risk Seminars.[1]

The CLR Foundations Course will introduce people to CLR's research on s-risk reduction. Apply here by Wednesday November 27th, 23:59 Pacific Time.

The CLR S-Risk Seminars will dive deeper into different priority cause areas within s-risk reduction and is intended for people who have completed the CLR Foundations Course or have a similar level of context. Register your interest here.

What topics does the CLR Foundations Course discuss?

The Foundations Course is designed to introduce participants to CLR's research on how transformative AI (TAI) might be involved in the creation of large amounts of suffering (s-risks). Our priority areas for addressing these risks include work on multiagent AI safety, AI governance, epistemology, risks from [...]

---

Outline:

(00:49) What topics does the CLR Foundations Course discuss?

(01:40) Who are these programs a good fit for?

(02:36) Foundations Course

(02:51) S-Risk Seminars

(03:20) When and how long are these programs?

(03:24) Foundations Course

(04:08) S-Risk Seminars

(04:40) How do I apply to these programs?

(05:33) I have questions!

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Fuck caveats. Fuck hedging. I'm going to write precisely what I believe.

About us

We use volunteers to treat mental illnesses and otherwise help people become happier and more productive. We mainly use CBT and ACT. Here's an overview of our results:

Data on General Population

On average, people are 2 points out of 10 happier after their first six sessions. Each session costs just $5 to deliver (ALL costs inclusive - marketing, training of coach, supervision, management, etc).

Here are the preliminary results of an RCT we're doing to treat insomnia. I'm showing this to demonstrate our results against a control group. This RCT only cost us $32k (all costs inclusive) to run because almost all the relevant staff were volunteers.

The biggest marginal costs for most charities, by far, are counsellor salaries and client acquisition. We [...]

---

Outline:

(00:12) About us

(00:27) Data on General Population

(03:45) Testimonials from Founders

(04:44) To demonstrate that Im not cherry-picking

(06:03) Funding

(07:05) Heres what we could do with different sums of money

(07:10) Successfully treat someone who otherwise couldnt afford it: $45

(08:44) Successfully treat one EA founder: $350

(09:15) Double our applicant pool every month for a year: $4.5k

(09:37) Let me work on Overcome full-time: $15.7k

(09:58) Let us take a shot at becoming fully self-sufficient: $35k

(10:32) Cover our entire operating costs for a year: $150k

(11:03) Other ways to help

The original text contained 5 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary

It's been a while since I last put serious thought into where to donate. Well I'm putting thought into it this year and I'm changing my mind on some things.

I now put more priority on existential risk (especially AI risk), and less on animal welfare and global priorities research. I believe I previously gave too little consideration to x-risk for emotional reasons, and I've managed to reason myself out of those emotions.

Within x-risk:

In the rest of this post, I will explain:

---

Outline:

(00:04) Summary

(01:30) I dont like donating to x-risk

(03:56) Cause prioritization

(04:00) S-risk research and animal-focused longtermism

(05:52) X-risk vs. global priorities research

(07:01) Prioritization within x-risk

(08:08) AI safety technical research vs. policy

(11:36) Quantitative model on research vs. policy

(14:20) Man versus man conflicts within AI policy

(15:13) Parallel safety/capabilities vs. slowing AI

(22:56) Freedom vs. regulation

(24:24) Slow nuanced regulation vs. fast coarse regulation

(27:02) Working with vs. against AI companies

(32:49) Political diplomacy vs. advocacy

(33:38) Conflicts that arent man vs. man but nonetheless require an answer

(33:55) Pause vs. Responsible Scaling Policy (RSP)

(35:28) Policy research vs. policy advocacy

(36:42) Advocacy directed at policy-makers vs. the general public

(37:32) Organizations

(39:36) Important disclaimers

(40:56) AI Policy Institute

(42:03) AI Safety and Governance Fund

(43:29) AI Standards Lab

(43:59) Campaign for AI Safety

(44:30) Centre for Enabling EA Learning and Research (CEEALAR)

(45:13) Center for AI Policy

(47:27) Center for AI Safety

(49:06) Center for Human-Compatible AI

(49:32) Center for Long-Term Resilience

(55:52) Center for Security and Emerging Technology (CSET)

(57:33) Centre for Long-Term Policy

(58:12) Centre for the Governance of AI

(59:07) CivAI

(01:00:05) Control AI

(01:02:08) Existential Risk Observatory

(01:03:33) Future of Life Institute (FLI)

(01:03:50) Future Society

(01:06:27) Horizon Institute for Public Service

(01:09:36) Institute for AI Policy and Strategy

(01:11:00) Lightcone Infrastructure

(01:12:30) Machine Intelligence Research Institute (MIRI)

(01:15:22) Manifund

(01:16:28) Model Evaluation and Threat Research (METR)

(01:17:45) Palisade Research

(01:19:10) PauseAI Global

(01:21:59) PauseAI US

(01:23:09) Sentinel rapid emergency response team

(01:24:52) Simon Institute for Longterm Governance

(01:25:44) Stop AI

(01:27:42) Where Im donating

(01:28:57) Prioritization within my top five

(01:32:17) Where Im donating (this is the section in which I actually say where Im donating)

The original text contained 58 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.