This part 1 in a 3-part sequence summarizes my book, The Darwinian Trap. The book aims to popularize the concept of multipolar traps and establish them as a broader cause area. If you find this series intriguing and want to spread the word and learn more:

1. Share this post with others on X or other social media platforms. 
2. Pre-order the book here.
3. Sign up for my mailing list here before September 24 for a 20% chance to win a free hardcover copy of the book (it takes 5 seconds). 
4. Contact me at [email protected] if you have any input or ideas. 

Global coordination stands as arguably the most critical challenge facing humanity today, functioning both as a necessary component for solving existential risks and as a significant barrier to effective mitigation. From nuclear proliferation to artificial intelligence development and climate change, our inability to collaborate effectively on a global [...]

---

Outline:

(03:24) The Universal Algorithm of Natural Selection

(05:15) Darwinian Demons and Multipolar Traps

(10:57) Why are the demons so hard to escape?

(12:58) Is Selfishness Really Such a Bad Thing?

(16:36) Natural Selection: the Generator Function of Existential Risks

(19:42) We Need Civilizational Alignment To Solve Technical Alignment

The original text contained 6 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

After the release of Ben Pace's extended interview with me about my views on religion, I felt inspired to publish more of my thinking about religion in a format that's more detailed, compact, and organized. This post is the second publication in my series of intended posts about religion.

Thanks to Ben Pace, Chris Lakin, Richard Ngo, Damon Pourtahmaseb-Sasi, Marcello Herreshoff, Renshin Lauren Lee, Mark Miller, Roger Thisdell, and Imam Ammar Amonette for their feedback on this post, and thanks to Kaj Sotala, Tomáš Gavenčiak, Paul Colognese, and David Spivak for reviewing earlier versions of this post. Thanks especially to Renshin Lauren Lee, Roger Thisdell, and Imam Ammar Amonette for their input on my claims about perennialism, and Mark Miller for vetting my claims about predictive processing.

In my previous post, I introduced the idea that there are broad convergences among the mystical traditions of the major [...]

---

Outline:

(02:22) The Goodness of Reality

(08:45) We reap what we sow

(12:27) Immoral behavior as confusion

(16:30) The “afterlife” as the highest-order bit for how to live

(24:12) Perennialism and moral philosophy

The original text contained 19 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Inspired by all the blog posts I've seen promoting unusual diets, nootropic drugs, unusual sleep cycles, and so on, I've decided to make my own post suggesting some radical lifestyle choice.

My suggestion here is, as the title says: you should probably eat oatmeal sometimes. Yes, I know, oats are <2% of global grain production, so this is a pretty crazy-sounding idea, but bear with me. Eating oatmeal sometimes will literally change your life.*

*slightly

oatmeal vs rice

White rice is one of the most popular foods in the world.

Compared to white rice:

about [...]

---

Outline:

(00:41) oatmeal vs rice

(01:07) about whole-wheat flour

(04:00) oatmeal is versatile

(05:05) why isnt oatmeal more popular?

(05:47) how to microwave oatmeal

---

First published:

Source:

Narrated by TYPE III AUDIO.

One can definitely make referendums more complex by mixing in unrelated stuff, unnecessary restrictions, and complexities. But that's not the case with Swiss referendums. Quite the contrary, they seem to be designed for simplicity:

First: There is no quorum. Even if the turnout is low, the referendum is valid. If three people cast their votes in a national ballot and two of them vote yes, the proposal is accepted. It is then written into the constitution, so even the parliament cannot overrule it. No quorum means no strategic voting or, more precisely, no strategic withholding of votes to sabotage the referendum by making it invalid due to not meeting the quorum, as often happens elsewhere.

Second: The referendum process takes many years, often five or more, so it can't be used for short-term, tribal politics, like calling for premature elections. By the time the referendum reaches voters, five or [...]

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic Status: Old Man Blogs at Cloud

Lately there's been a wave of people on LessWrong (and maybe the whole internet) starting off their essays with some Dall-E3 art.

I don't object (and think it can be cool if done nicely) to get some good ML art to add some visual interest to your posts. But, the default thing I'm seeing is mediocre and making the web feel tacky.

I don't want to just yell at people and make them feel self-conscious about their fun. I have an art background, which means I both have a lot of experience making/evaluating art and also probably have random-ass snooty connoisseurship syndrome. Also I was involved with the fleshing out of the current LessWrong Watercolor Aesthetic, and random clip-art looking images undercut it. (But, to be fair nobody actually voted for or opted into the LessWrong Watercolor Aesthetic, and maybe that's [...]

---

Outline:

(01:07) 1. Generally, make landscape (widescreen) art.

(01:45) 2. Good AI art still takes a fair number of generations and effort.

(02:32) 3. Think about the mood you want to convey, not just the posts intellectual content.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Perplexity is the first generalized AI chatbot I’ve found useful enough to integrate into any part of my daily workflow, much less across multiple domains. It speeds me up enough that I’m planning an increase in my freelancing rate.

Perplexity has three key advantages:

The citations are a bigger deal than they sound at first. I could of course google anything ChatGPT tells me and check the results, but that's costly. I have to spin up a new mental thread (and browser tab), sort through Google's increasingly shitty results, remember what claim I’m trying to check… the process eats up a lot of working memory at a time [...]

---

Outline:

(01:49) Test Questions

(02:25) How do fractals relate to chaos theory?

(03:11) Perplexity (link to result including citations)

(03:24) Self-Similarity and Complex Patterns

(03:43) Visualization of Chaotic Processes

(04:11) Characteristics of Complex Systems

(04:27) Applications in Financial Markets

(05:01) Relationship to Complexity Theory

(05:54) ChatGPT 4o

(07:58) Claude

(09:24) Google Search AI Summary

(10:56) How long after antibiotics is bordatello not contagious in cats?

(12:02) Perplexity (link to results including citations)

(13:39) ChatGPT-4o

(15:21) Claude 3.5

(16:45) Google Search AI summary

(16:54) Is the Aegis Shield Playful?

(17:19) Perplexity

(17:48) ChatGPT 4o

(18:31) Claude 3.5

(19:32) Google Search AI summary

(19:38) Conclusion

(19:55) Acknowledgements and (lack of) COI

---

First published:

Source:

Narrated by TYPE III AUDIO.

Owain Evans is an AI Alignment researcher, research associate at the Center of Human Compatible AI at UC Berkeley, and now leading a new AI safety research group.

In this episode we discuss two of his recent papers, “Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs” (LW) and “Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data” (LW), alongside some Twitter questions.

Below are some highlighted quotes from our conversation (available on Youtube, Spotify, Apple Podcast). For the full context for each of these quotes, you can find the accompanying transcript.

Situational Awareness

Definition

"What is situational awareness? The idea is the model's kind of self-awareness, that is its knowledge of its own identity, and then [...]

---

Outline:

(00:54) Situational Awareness

(01:09) Definition

(01:56) Motivation

(02:33) On Claude 3 Opus Insightful Answers

(03:46) What Would Saturating The Situational Awareness Benchmark Imply For Safety And Governance

(04:41) Out-of-context reasoning

(04:55) Definition

(05:20) Experimental Setup

(05:51) Difference With In-Context Learning

(06:29) Safety implications

(07:00) The Results Were Surprising

(07:23) Alignment Research Advice

(07:27) Owains research process

(07:59) Interplay between theory and practice

(08:33) Research style and background

(09:08) On Research Rigor

(09:36) On Accelerating AI capabilities

(10:07) On balancing safety benefits with potential risks

(10:38) On the reception of his work

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Summary

This post gives my personal take on “AI for epistemics” and how important it might be to work on.

Some background context:

• AI capabilities are advancing rapidly and I think it's important to think ahead and prepare for the possible development of AI that could automate almost all economically relevant tasks that humans can do.[1]

• That kind of AI would have a huge impact on key epistemic processes in our society. (I.e.: It would have a huge impact on how new facts get found, how new research gets done, how new forecasts get made, and how all kinds of information spread through society.)
• I think it's very important for our society to have excellent epistemic processes. (I.e.: For important decisions in our society to be made by people or AI systems who have informed and unbiased beliefs that take into account as much of the available evidence [...]

---

Outline:

(00:04) Summary

(04:36) Structure of the post

(05:33) Previous work

(07:29) Why work on AI for epistemics?

(07:34) Summary

(09:38) Importance

(12:15) Path-dependence

(14:45) To be more concrete

(15:37) Good norms and practices for AI-as-knowledge-producers

(17:12) Good norms and practices for AI-as-communicators

(19:30) Differentially high epistemic capabilities

(27:12) Heuristics for good interventions

(28:01) Direct vs. indirect strategies

(29:24) Indirect value generation

(33:30) On long-lasting differential capability improvements

(36:11) Painting a picture of the future

(41:04) Concrete projects for differentially advancing epistemic capabilities

(43:47) Evals/benchmarks for forecasting (or other ambitious epistemic assistance)

(44:27) Automate forecasting question-generation and -resolution

(45:59) Logistics of past-casting

(50:22) Start efforts inside of AI companies for AI forecasting or other ambitious epistemic assistance

(51:34) Scalable oversight / weak-to-strong-generalization / ELK

(53:53) Experiments on what type of arguments and AI interactions tend to lead humans toward truth vs. mislead them

(56:12) Concluding thoughts

The original text contained 22 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Bart, Michael and Patrick are joint first authors. Research conducted as part of MATS 6.0 in Lee Sharkey and Neel Nanda's streams. Thanks to Mckenna Fitzgerald and Robert Krzyzanowski for their feedback!

TL;DR:

• Sparse Autoencoder (SAE) latents have been shown to typically be monosemantic (i.e. correspond to an interpretable property of the input). It is sometimes implicitly assumed that they are therefore atomic, i.e. simple, irreducible units that make up the model's computation.
• We provide evidence against this assumption by finding sparse, interpretable decompositions of SAE decoder directions into seemingly more atomic latents, e.g. Einstein -> science + famous + German + astronomy + energy + starts with E-
• We do this by training meta-SAEs, an SAE trained to reconstruct the decoder directions of a normal SAE. 
• We argue that, conceptually, there's no reason to expect [...]

---

Outline:

(02:36) Introduction

(07:41) Defining Meta-SAEs

(09:46) Meta-latents form interpretable decompositions of SAE latents

(13:04) Are Meta-Latents different from SAE Latents?

(19:56) Using Meta-SAEs to Interpret Split Features

(24:02) Causally Intervening and Making Targeted Edits with Meta-Latents

(29:03) Discussion

The original text contained 2 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

---

First published:

Source:

Narrated by TYPE III AUDIO.