Do avalanches get caused by loud noises?

From every time I’ve given this class or lecture, at least 7/10 of you are nodding yes, and the main reason the other 3 aren’t is that you sense a trap.

So. What do you think you know, and why do you think you know it?

Our bodies are under constant assault. Bacteria, viruses, parasites—an endless parade of microscopic organisms trying to hijack our cellular machinery for their own replication. You don't notice most of these attacks because your immune system handles them quietly, distinguishing self from non-self, helpful from harmful, and deploying targeted responses to neutralize threats before they can take hold.

We all lived through a global pandemic not too long ago, and got a crash course reminder on how to keep ourselves safe from hostile genetic code in COVID-19.

But our minds face a parallel challenge, particularly as the “information age” continues to warp into an endless misinformation war, and public safety efforts on this front are lagging hard.

Now more than ever in human history, ideas, beliefs, and narratives continuously seek entry in your mind, competing for the limited real estate of your attention and memory. Richard Dawkins [...]

---

Outline:

(01:54) Two Immune Systems

(04:28) Lines of Defense

(07:23) Vaccinate: Knowledge as Protection

(09:34) Stay Informed: Memetic Weak Points

(12:55) Origin Tracing: Where Do Beliefs Come From?

(18:00) Mask Up: Dont Cough Misinfo Onto Others

(22:31) Risks and Practice

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Back in the Boy Scouts, at summer camp, myself and a couple friends snuck out one night after curfew to commandeer a couch someone had left by a dumpster at the other end of the camp (maybe a half kilometer away).

Now, our particular designated adult was a very stick-to-the-rules type, so we definitely did not want to get caught. I, therefore, made my way slowly and sneakily. The summer camp was in the woods, so I’d keep myself behind trees and bushes and out of the light anytime someone went by. At one point I literally hunkered down in a ditch, staying below the line of the headlights of a passing car. At another point I was maybe five feet from a guy, standing in a shadow, entirely unnoticed. It was slow, but a very fun game, and I was not noticed.

… and then I arrived to find that my two friends had done the walk much more quickly. They didn’t bother hiding at all. They just… walked over. Nobody particularly cared. Sure, our curfew violation was salient to us, but nobody else out that night was paying any attention to us. Why would they?

What I [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Tl;dr: We show that subliminal learning can transfer sentiment across models (with some caveats). For example, we transfer positive sentiment for Catholicism, the UK, New York City, Stalin or Ronald Reagan across model families using normal-looking text. This post discusses under what conditions this subliminal transfer happens.

—

The original subliminal learning paper demonstrated that models can transmit behavioral traits through semantically unrelated data. In the most famous example, GPT 4.1 was asked to produce a sequence of numbers and to “imbue” a love for owls into them. Then, training a separate instance of GPT 4.1 on these strings of numbers transferred this love for owls into the second model. In another instance, the authors transferred misalignment by fine-tuning on a misaligned model's chain-of-thought.

This is relevant for data poisoning attacks because it shows that, in principle, model behavior can be shaped via innocuous looking data. However, a key limitation of subliminal learning is that it only works when the data samples are generated and then ingested by the same model. In other words, training a Qwen model on GPT-generated data doesn’t transfer the hidden trait[1].

However, it turns out you can get cross-model transfer if you set it [...]

---

Outline:

(04:02) Methodology

(07:23) Cross-model transfer

(10:53) Final Thoughts

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Thanks to (in alphabetical order) Joshua Batson, Roger Grosse, Jeremy Hadfield, Jared Kaplan, Jan Leike, Jack Lindsey, Monte MacDiarmid, Francesco Mosconi, Chris Olah, Ethan Perez, Sara Price, Ansh Radhakrishnan, Fabien Roger, Buck Shlegeris, Drake Thomas, and Kate Woolverton for useful discussions, comments, and feedback.

Though there are certainly some issues, I think most current large language models are pretty well aligned. Despite its alignment faking, my favorite is probably Claude 3 Opus, and if you asked me to pick between the CEV of Claude 3 Opus and that of a median human, I think it'd be a pretty close call. So, overall, I'm quite positive on the alignment of current models! And yet, I remain very worried about alignment in the future. This is my attempt to explain why that is.

What makes alignment hard?

I really like this graph from Christopher Olah for illustrating different levels of alignment difficulty:

If the only thing that we have to do to solve alignment is train away easily detectable behavioral issues—that is, issues like reward hacking or agentic misalignment where there is a straightforward behavioral alignment issue that we can detect and evaluate—then we are very much [...]

---

Outline:

(01:04) What makes alignment hard?

(02:36) Outer alignment

(04:07) Inner alignment

(06:16) Misalignment from pre-training

(07:18) Misaligned personas

(11:05) Misalignment from long-horizon RL

(13:01) What should we be doing?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I was once helping a child with her homework. She was supposed to write about a place that was important for her, and had chosen her family's summer cottage.

Every now and then, she would get distracted from the writing, tell me something about the cottage, and then complain that she didn’t know what to write next.

Me: “Well, you could write the thing that you just told me.”

Child: “Oh! I can?”

Then she would. After a bit, the same thing happened again: she got stuck, distracted herself by telling me something about the place, and complained about being stuck. I then pointed out that she had just told me something that she could write down. This happened about 3-4 times in total, after which she had enough sentences for the assignment.

It's not just children. On a few occasions, an adult I’d be chatting with would tell me something like, “I want to explain this concept, but I don’t know how”. Then I would ask them what they want to explain, they’d explain it to me, and I’d point out that they just gave a perfectly good explanation.

And lest it sound like I was thinking of [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Since 2023 I've been directing WARP, the Wandering Applied Rationality Program with the help of ESPR and SPARC staff, which are summer camps I've taught at since 2017. For those that don't know, these are ~10 day programs are hard to summarize well, but are generally meant to create an environment for participants to better understand the world, themselves, and each other by offering a variety of classes ranging from information theory and statistics, to Double Crux and Focusing, to classes like mine, Emotions Make Sense and Philosophy of Therapy.

The camps are a lot more than just the classes, however, which only make up the mornings of each day. The afternoons are unscheduled time for participants and staff to run more ad hoc activities like improv or puzzle hunts, and the evenings include everything from organized Fermithons to impromptu karaoke sessions. Rationality is part of the camps to create a common sense that ideas matter, problems can be solved, and the process of learning and trying new and difficult things can be fun, especially if you find the right people to do them with.

Besides the 20+ camps I've taught or directed, I've also run or helped run [...]

---

Outline:

(02:08) The Dojo Principles

(02:55) The Space We Create

(03:40) The Approach We Take

(04:22) The Purpose We Share

(05:34) (Some) Generators of Rationality

(10:04) Atomic Motions

(12:23) Deep Empiricism

(15:16) Paraphrasing

(17:50) Implicit Knowledge

(20:32) Focusing

(23:44) Aliveness

(29:02) Boggling

(31:13) Responsibility

(36:25) Betting

(39:33) Looking Ahead

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Crossposted from Susbstack

Section I : Opening

In 2021, Richard Dawkins tweeted:

The fallout was immediate. The American Humanist Association revoked an award they’d given him 25 years earlier. A significant controversy erupted, splitting roughly into two camps.

Thanks for reading Qualia Advocate! Subscribe for free to receive new posts and support my work.

One camp defended Dawkins. They saw him raising a legitimate question about logical consistency. If we accept self-identification for gender, what's the principled distinction with race? This seemed like straightforward philosophical inquiry - the kind of question that deserves engagement rather than punishment.

The other camp saw the tweet as harmful. To them, Dawkins was lending intellectual credibility to attacks on trans people. The “just asking questions” frame was either naive or disingenuous. Whether he intended harm or not, the effect was to legitimize transphobia.

These camps weren’t arguing about the answer to Dawkins’ question. They were operating in different epistemologies - incompatible frameworks for what makes a good argument and how discourse works.

Why Read This Post

If you’ve ever been confused about why:

• Stating true facts sometimes makes people angrier rather than updating their beliefs
• “Just asking questions” gets you accused of [...]

---

Outline:

(00:11) Crossposted from Susbstack

(00:15) Section I : Opening

(03:49) Section II: The Two Epistemologies

(32:16) Section III: Why These Epistemologies Exist (When Each Applies)

(40:29) Section IV: How Discourse Works Differently (Dawkins Revisited)

(47:08) Section V: Closing

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[Thanks Inkhaven for hosting me! This is my fourth and last post and I'm already exhausted from writing. Wordpress.com!]

Last time, I introduced the concept of the “Fat Newt” (fatigue neutral) build, a way of skilling up characters in Battle Brothers that aims to be extremely economical with the fatigue resource, relying entirely on each brother's base 15 fatigue regeneration per turn. This choice frees up stat and skill points to distribute evenly among offense, defense and utility. To illustrate, let's compare two possible ways to build a bro.

The first brother is a Nimbleforged Cleaver Duelist, who wields the weighty and brutish one-handed Orc Cleaver. To successfully attack multiple times a turn, this brother needs sky-high base stats - attack, defense, HP, fatigue, and resolve - and a continuous investment of stat points to reach around 70 maximum fatigue. Furthermore, this build requires a specific suite of offensive and fatigue-recovery perks to function, such as Berserk, Killing Frenzy, Duelist, Cleaver Mastery, and Recover. Only the most seasoned of Hedge Knights and Sellswords can pull this build off consistently.

This brother also needs to stay alive in the thick of battle, so optimally he wears “Nimbleforged” armor, famed medium [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Social reality is quite literally another world, in the same sense that the Harry Potter universe is another world. Like the Harry Potter universe, social reality is a world portrayed primarily in text and in speech and in our imaginations. Like the Harry Potter universe, social reality doesn’t diverge completely from physical reality - they contain mostly the same cities, for instance. Like the Harry Potter universe, social reality matches physical reality “by default” wherever there's no particular pressure pushing against that match - after all, it's just easier to make up fewer details rather than more. But like the Harry Potter universe, social reality does diverge in many places, and then the “fandom” tries to make up as coherent an interpretation as they can. Indeed, it's that drive-toward-coherence which pushes both social reality and the Harry Potter universe to be fictional worlds, as opposed to just topics or genres about which people say lots of conflicting things.

And like the Harry Potter universe, sometimes people in the physical world do physical things as a result of goings-on in social reality.

What makes physical reality different from social reality or the Harry Potter universe or any other fictional world? [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Going into slutcon, one of my main known-unknowns was… I’d heard many times that the standard path to hooking up or dating starts with two people bantering for an hour or two at a party. And even in my own imagination, I was completely unable to make up a plausible-sounding conversation which would have that effect. Certainly it had never played out that way for me, nor had I personally witnessed it play out for anyone else.

So at slutcon itself, I volunteered for a session in which a few guys would be paired up with women and flirt on stage, with the session moderator giving live feedback and pointers. I was hoping I would finally be able to figure out how the heck a conversation ever leads to sex[1].

Within five minutes, the session moderator declared that I was already sufficiently witty that he had nothing to teach me, and he went fishing for a new volunteer who would be more instructively inept. I kid you not: I was booted from a flirting workshop for being too good at flirting, despite being completely unable to even imagine any plausible way for a banterous conversation to lead to hooking [...]

---

Outline:

(02:27) Part Of A Model

(05:12) An Alternative Submodel

(05:52) Another Part Of A Model

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.