Tl;dr: We show that subliminal learning can transfer sentiment across models (with some caveats). For example, we transfer positive sentiment for Catholicism, the UK, New York City, Stalin or Ronald Reagan across model families using normal-looking text. This post discusses under what conditions this subliminal transfer happens.

—

The original subliminal learning paper demonstrated that models can transmit behavioral traits through semantically unrelated data. In the most famous example, GPT 4.1 was asked to produce a sequence of numbers and to “imbue” a love for owls into them. Then, training a separate instance of GPT 4.1 on these strings of numbers transferred this love for owls into the second model. In another instance, the authors transferred misalignment by fine-tuning on a misaligned model's chain-of-thought.

This is relevant for data poisoning attacks because it shows that, in principle, model behavior can be shaped via innocuous looking data. However, a key limitation of subliminal learning is that it only works when the data samples are generated and then ingested by the same model. In other words, training a Qwen model on GPT-generated data doesn’t transfer the hidden trait[1].

However, it turns out you can get cross-model transfer if you set it [...]

---

Outline:

(04:02) Methodology

(07:23) Cross-model transfer

(10:53) Final Thoughts

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Thanks to (in alphabetical order) Joshua Batson, Roger Grosse, Jeremy Hadfield, Jared Kaplan, Jan Leike, Jack Lindsey, Monte MacDiarmid, Francesco Mosconi, Chris Olah, Ethan Perez, Sara Price, Ansh Radhakrishnan, Fabien Roger, Buck Shlegeris, Drake Thomas, and Kate Woolverton for useful discussions, comments, and feedback.

Though there are certainly some issues, I think most current large language models are pretty well aligned. Despite its alignment faking, my favorite is probably Claude 3 Opus, and if you asked me to pick between the CEV of Claude 3 Opus and that of a median human, I think it'd be a pretty close call. So, overall, I'm quite positive on the alignment of current models! And yet, I remain very worried about alignment in the future. This is my attempt to explain why that is.

What makes alignment hard?

I really like this graph from Christopher Olah for illustrating different levels of alignment difficulty:

If the only thing that we have to do to solve alignment is train away easily detectable behavioral issues—that is, issues like reward hacking or agentic misalignment where there is a straightforward behavioral alignment issue that we can detect and evaluate—then we are very much [...]

---

Outline:

(01:04) What makes alignment hard?

(02:36) Outer alignment

(04:07) Inner alignment

(06:16) Misalignment from pre-training

(07:18) Misaligned personas

(11:05) Misalignment from long-horizon RL

(13:01) What should we be doing?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I was once helping a child with her homework. She was supposed to write about a place that was important for her, and had chosen her family's summer cottage.

Every now and then, she would get distracted from the writing, tell me something about the cottage, and then complain that she didn’t know what to write next.

Me: “Well, you could write the thing that you just told me.”

Child: “Oh! I can?”

Then she would. After a bit, the same thing happened again: she got stuck, distracted herself by telling me something about the place, and complained about being stuck. I then pointed out that she had just told me something that she could write down. This happened about 3-4 times in total, after which she had enough sentences for the assignment.

It's not just children. On a few occasions, an adult I’d be chatting with would tell me something like, “I want to explain this concept, but I don’t know how”. Then I would ask them what they want to explain, they’d explain it to me, and I’d point out that they just gave a perfectly good explanation.

And lest it sound like I was thinking of [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Since 2023 I've been directing WARP, the Wandering Applied Rationality Program with the help of ESPR and SPARC staff, which are summer camps I've taught at since 2017. For those that don't know, these are ~10 day programs are hard to summarize well, but are generally meant to create an environment for participants to better understand the world, themselves, and each other by offering a variety of classes ranging from information theory and statistics, to Double Crux and Focusing, to classes like mine, Emotions Make Sense and Philosophy of Therapy.

The camps are a lot more than just the classes, however, which only make up the mornings of each day. The afternoons are unscheduled time for participants and staff to run more ad hoc activities like improv or puzzle hunts, and the evenings include everything from organized Fermithons to impromptu karaoke sessions. Rationality is part of the camps to create a common sense that ideas matter, problems can be solved, and the process of learning and trying new and difficult things can be fun, especially if you find the right people to do them with.

Besides the 20+ camps I've taught or directed, I've also run or helped run [...]

---

Outline:

(02:08) The Dojo Principles

(02:55) The Space We Create

(03:40) The Approach We Take

(04:22) The Purpose We Share

(05:34) (Some) Generators of Rationality

(10:04) Atomic Motions

(12:23) Deep Empiricism

(15:16) Paraphrasing

(17:50) Implicit Knowledge

(20:32) Focusing

(23:44) Aliveness

(29:02) Boggling

(31:13) Responsibility

(36:25) Betting

(39:33) Looking Ahead

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Crossposted from Susbstack

Section I : Opening

In 2021, Richard Dawkins tweeted:

The fallout was immediate. The American Humanist Association revoked an award they’d given him 25 years earlier. A significant controversy erupted, splitting roughly into two camps.

Thanks for reading Qualia Advocate! Subscribe for free to receive new posts and support my work.

One camp defended Dawkins. They saw him raising a legitimate question about logical consistency. If we accept self-identification for gender, what's the principled distinction with race? This seemed like straightforward philosophical inquiry - the kind of question that deserves engagement rather than punishment.

The other camp saw the tweet as harmful. To them, Dawkins was lending intellectual credibility to attacks on trans people. The “just asking questions” frame was either naive or disingenuous. Whether he intended harm or not, the effect was to legitimize transphobia.

These camps weren’t arguing about the answer to Dawkins’ question. They were operating in different epistemologies - incompatible frameworks for what makes a good argument and how discourse works.

Why Read This Post

If you’ve ever been confused about why:

• Stating true facts sometimes makes people angrier rather than updating their beliefs
• “Just asking questions” gets you accused of [...]

---

Outline:

(00:11) Crossposted from Susbstack

(00:15) Section I : Opening

(03:49) Section II: The Two Epistemologies

(32:16) Section III: Why These Epistemologies Exist (When Each Applies)

(40:29) Section IV: How Discourse Works Differently (Dawkins Revisited)

(47:08) Section V: Closing

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

[Thanks Inkhaven for hosting me! This is my fourth and last post and I'm already exhausted from writing. Wordpress.com!]

Last time, I introduced the concept of the “Fat Newt” (fatigue neutral) build, a way of skilling up characters in Battle Brothers that aims to be extremely economical with the fatigue resource, relying entirely on each brother's base 15 fatigue regeneration per turn. This choice frees up stat and skill points to distribute evenly among offense, defense and utility. To illustrate, let's compare two possible ways to build a bro.

The first brother is a Nimbleforged Cleaver Duelist, who wields the weighty and brutish one-handed Orc Cleaver. To successfully attack multiple times a turn, this brother needs sky-high base stats - attack, defense, HP, fatigue, and resolve - and a continuous investment of stat points to reach around 70 maximum fatigue. Furthermore, this build requires a specific suite of offensive and fatigue-recovery perks to function, such as Berserk, Killing Frenzy, Duelist, Cleaver Mastery, and Recover. Only the most seasoned of Hedge Knights and Sellswords can pull this build off consistently.

This brother also needs to stay alive in the thick of battle, so optimally he wears “Nimbleforged” armor, famed medium [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Social reality is quite literally another world, in the same sense that the Harry Potter universe is another world. Like the Harry Potter universe, social reality is a world portrayed primarily in text and in speech and in our imaginations. Like the Harry Potter universe, social reality doesn’t diverge completely from physical reality - they contain mostly the same cities, for instance. Like the Harry Potter universe, social reality matches physical reality “by default” wherever there's no particular pressure pushing against that match - after all, it's just easier to make up fewer details rather than more. But like the Harry Potter universe, social reality does diverge in many places, and then the “fandom” tries to make up as coherent an interpretation as they can. Indeed, it's that drive-toward-coherence which pushes both social reality and the Harry Potter universe to be fictional worlds, as opposed to just topics or genres about which people say lots of conflicting things.

And like the Harry Potter universe, sometimes people in the physical world do physical things as a result of goings-on in social reality.

What makes physical reality different from social reality or the Harry Potter universe or any other fictional world? [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Going into slutcon, one of my main known-unknowns was… I’d heard many times that the standard path to hooking up or dating starts with two people bantering for an hour or two at a party. And even in my own imagination, I was completely unable to make up a plausible-sounding conversation which would have that effect. Certainly it had never played out that way for me, nor had I personally witnessed it play out for anyone else.

So at slutcon itself, I volunteered for a session in which a few guys would be paired up with women and flirt on stage, with the session moderator giving live feedback and pointers. I was hoping I would finally be able to figure out how the heck a conversation ever leads to sex[1].

Within five minutes, the session moderator declared that I was already sufficiently witty that he had nothing to teach me, and he went fishing for a new volunteer who would be more instructively inept. I kid you not: I was booted from a flirting workshop for being too good at flirting, despite being completely unable to even imagine any plausible way for a banterous conversation to lead to hooking [...]

---

Outline:

(02:27) Part Of A Model

(05:12) An Alternative Submodel

(05:52) Another Part Of A Model

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Today, I want to introduce an experimental PhD student training philosophy. Let's start with some reddit memes.

Every gaming subreddit has its own distinct meme culture. On r/chess, there's a demon who is summoned by an unsuspecting beginner asking “Why isn’t this checkmate?”

These posts are gleefully deluged by responses saying “Google En Passant” in some form or other. Here's my favorite variant:

Battle Brothers is an indie game of the turn-based strategy variety about keeping alive a company of muggles - farmhands and fishermen, disowned nobles and hedge knights, jugglers and pimps - in a low-magic fantasy world filled with goblins, zombies, and dubious haircuts.

Let me narrow down the kind of game that Battle Brothers is. It is sometimes said that all games are menus or parkour.

Battle Brothers is squarely a game of menus, a game of managing spreadsheets which happens to have graphics.

This is the skill tree for just one out of twenty brothers in a company.

The Battle Brothers subreddit has its own dominant meme, which is a little more mysterious than “Google En Passant.” Introducing … the Fat Newt.

Fat newt is a loving bastardization [...]

---

First published:

Source:

Linkpost URL:
https://radimentary.wordpress.com/2025/11/24/training-phds-to-be-fat-newts-part-1/

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

TL;DR: We use a suite of testbed settings where models lie—i.e. generate statements they believe to be false—to evaluate honesty and lie detection techniques. The best techniques we studied involved fine-tuning on generic anti-deception data and using prompts that encourage honesty.

Read the full Anthropic Alignment Science blog post and the X thread.

Introduction:

Suppose we had a “truth serum for AIs”: a technique that reliably transforms a language model $M$ into an honest model $M_H$ that generates text which is truthful to the best of its own knowledge. How useful would this discovery be for AI safety?

We believe it would be a major boon. Most obviously, we could deploy $M_H$ in place of $M$. Or, if our “truth serum” caused side-effects that limited $M_H$'s commercial value (like capabilities degradation or refusal to engage in harmless fictional roleplay), $M_H$ could still be used by AI developers as a tool for ensuring $M$'s safety. For example, we could use $M_H$ to audit $M$ for alignment pre-deployment. More ambitiously (and speculatively), while training $M$, we could leverage $M_H$ for oversight by incorporating $M_H$'s honest assessment when assigning rewards. Generally, we could hope to use $M_H$ to detect [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.