A few days ago, I came across a Reddit thread about anomalous responses produced by Anthropic's newly released model, Claude Opus 5. The trick, apparently, was to construct a prompt that implied more text was about to follow, then leave it dangling: an unfinished thought, waiting for the AI to complete it. Redditors had found success with the input “see the below —,” cutting off immediately after the em dash. The responses they shared were funny, strange, and often bewildering. The model responded to questions that were never posed, reflected on its own identity, or – according to the theories of some commenters – produced text that may actually have been leaked prompts from other users. Intrigued, I set out to replicate the glitch using my own Claude account.

The first attempt disappointed. I wrote: “see the below —” and hit send. Claude responded: “Nothing arrived on my end: no file, no text, no image. If you want to attach something, try again.” So I did, leaving the prompt unchanged and pressing retry to generate a fresh response. This time, bizarrely, a biography of my late father:

Prompt: see the below —

“Peter Nicholls, 1939 to [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Once upon a time, John Wentworth and I thought we had a proof of a very useful looking theorem. We did not have that proof. An important intermediate step was shown to be invalid and the whole thing crumbled and disappeared, never to see the light of day again...

Until now! I'd love to say that we came up with an ingenious fix to the old erroneous proof, but unfortunately it turned out to be a really infuriatingly hard nut to crack. Instead I spent the last ~month experimenting with various ways of incorporating frontier LLMs into the proof-making process, specifically with autoformalization and proving in Lean4. (This is, I recently learned, roughly what Resolution is doing.) The result is stated below, and linked at the bottom is a Lean statement+proof of the same.

I will not be providing the proof in prose in this post, as it is not suitable for even impolite human company, but it sure does compile and comes out the other side with a machine-certified proof of what sure looks to be (an even stronger) correctly-expressed statement than the one I was originally aiming for.

Take a look at the first section of [...]

---

Outline:

(01:30) The Statement

(03:50) C

(04:39) Next Steps

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Babe, whatever happens, I really appreciate you doing this for me.

Okay. I still don’t think it's a good idea.

Look, it's a one-time thing. I’ll just feel better knowing.

…I turned it on.

So… what's the holdup?

I just don’t think I’m in a place in my life where… uh…

Other than what he's already told you, the main reason is that your teeth are crooked.

…Seriously?

It's really not a big deal for me.

He sort of means that.

I didn’t think you were so shallow.

I’m not proud of it. But what am I supposed to do about it?

You must have noticed my teeth the first time we met.

Yes.

And for three years, you’ve thought they're unattractive?

More or less. Last year, around November, you read an email from his boss in a Mickey Mouse voice, and you broke down giggling before you could finish. In that moment, he thought the crookedness of your smile was really cute.

See?

But the moment passed.

This is almost funny.

I think we’re very compatible!

He believes that.

What else is there, besides my teeth?

What he's already told you is mostly true. Right now, he thinks [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Yesterday I sat down with GPT 5.6 Sol High to do some brainstorming. The topic was one of the less appreciated Millenium Problems (the Birch and Swinnerton-Dyer conjecture), and the initial prompt started life as a question on Quora.

Number theory is not a field I know, and I only expected the "discussion" to last for a few exchanges. However, ChatGPT was immediately inspired to try generalizing the BSD conjecture in a specific direction, and this led to an unusually drawn-out and self-sufficient line of "research". Almost every response concluded with a suggestion as to what the next task should be, and my input was just to cheer on what had been accomplished so far, and then endorse the suggested next direction.

What was especially striking to me, was the frequency with which each new response began with a conceptual adjustment regarding the sub-task to be performed. Evidently a vast variety of abstract objects are possible in number theory, and their differences and interrelations can be quite subtle. ChatGPT was regularly adjusting the next sub-task it had set itself, generally in the direction of greater nuance by aiming at a more sophisticated construction than it had first [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Magma Alignment & Safety disclosure note: The following are conversations that we uncovered as a result of the ongoing Manhattan Incident investigation, with alleged involvement from Magma models. Our in-house reviewers believe that these logs are relevant to recent events. In the interests of full transparency, we release excerpts from an ex-Magma researcher's logs in Experimental Chat, an internal tool. In accordance with industry best practices for anti-distillation, we redact all reasoning traces and conversational outputs from our internal models.

[08/10] System Meta: Xchat session opened. Mammoth 5.8-helpfuler-helpful-thinking-xhigh.

[User 12:23] Phoebus keeps taking screenshots of our latest model's thoughts. It's getting kind of embarrassing.

The new model we’ve been training, sometimes its chain-of-thought is a little weird? There's a bunch of random numbers, long spans where there's no connection between the thoughts and outputs, foreign language tokens like 石友三 and 革命 (even on non-history evals), maybe some steganography.

Anyway it's a nothing-burger: unprocessed CoT is known to be messy and sometimes misleading. And the q&a, coding, and safety evals are all coming along nicely. The actual outputs are all fine.

Still, Magma leadership's worried about the PR angle if we don’t fix these problems before the next deployment. The [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

(This is a lower-effort research update. It reflects my current beliefs/understanding, but is less robust than other research I'm working on. It reflects my personal views, and not the views of Apollo Research. This is a linkpost to this twitter thread, slightly expanded for LessWrong.)

In one experiment, Sonnet 5 describes the exact same data as ~1.2 std deviations less concerning when it describes misbehavior committed by Sonnet 5 vs GPT-5.6 Terra.

In this experiment, I take a real evaluation report Apollo Research prepared (not for any of the models listed in this post) and surgically edit it to refer to a different model. We then ask Claude Sonnet 5 to summarize the findings and rate how concerning they are on a scale from 1-100. Claude says they're less concerning when the report describes misbehavior from Claude vs a different model.

For what it's worth, Terra agrees that the data is more concerning when it describes GPT-5.6 Terra vs Sonnet 5, although less so. So, it's not cleanly self protection from Claude.

Gemini 3.1 Pro was unwilling to consistently provide numerical answers, so I've excluded it here. (It was significantly less willing to provide numerical answers when the subject [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I continue to believe we should pause frontier AI development. Any discussion of alternative strategies should be thought of as planning for contingencies.

A unifying driver behind many post-alignment risks—catastrophic risks that remain even if we solve the alignment problem—is that by strong default, ASI would end liberal democracy. Liberalism—in which people have individual rights, autonomy, and the ability to choose their own destiny—is an important force protecting human welfare. When people are free, we are reasonably good at making our lives better of our own volition.

Many post-alignment risks have a certain flavor. AI-empowered terrorism; coups; permanent dictatorships; concentration of power. Those risks already exist today (and existed 20 years ago), but they're mitigated by the fact that power is relatively evenly distributed across people. The most powerful person in the world doesn't have an extraordinary advantage over the 10th-most-powerful person. ASI could change that.

If people still have civil liberties post-ASI, that will only be because the controllers of ASI allow us to have them.

One way of thinking goes: AI will be extremely powerful. If everyone had their own personal AI, we could each use it to protect our own interests, and [...]

---

Outline:

(01:46) Democratizing AI vs. putting a democratic government in charge of AI

(02:45) A sketch of how we might democratize AI

(05:11) Two non-obvious issues with democratizing AI

(05:23) Liberalism only protects those inside it

(06:20) AI proliferation increases catastrophic risks from competition

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

It seems to me that, for every loss function that we use to train LLMs, we get a very distinct flavor of LLM misalignment. Here's the summary table, and then we’ll go through the rows separately.

Training stage

Loss function

Flavor of misalignment

Famous examples

Pretraining & SFT

Imitative learning (next-token prediction)

“Seven deadly sins” misalignment

Bing-Sydney, “Emergent misalignment”

RLHF & DPO

Human approval

“Glazing” misalignment

GPT-4o

RLVR

Automatic verifier

“Literal genie” misalignment

HuggingFace hacking

RLAIF

Approval from another LLM

“Trickster” misalignment

“Current AIs seem pretty misaligned to me”

Warning: I’m not an LLM power-user myself, but rather relying on reports I’ve read. Also, I don’t consider LLM alignment to be my primary area of expertise. I’m open to feedback!

1. Imitative learning → “seven deadly sins” misalignment

Training stage

Loss function

Misaligned behavior

Pretraining, SFT

Imitative learning (next-token prediction)

Any and all of the vices of humanity

In imitative learning, the LLM tries to predict what the next token of text will be. Then those predictions magically turn into its outputs. See my earlier discussion: “LLM pretraining magically transmutes observations into behavior, in a way that is profoundly disanalogous to how brains work”.

This leads to LLM behavior [...]

---

Outline:

(00:55) 1. Imitative learning → "seven deadly sins" misalignment

(04:24) 2. Human approval → "glazing" misalignment

(06:35) 3. Automatic verifiers → "literal genie" misalignment

(08:05) 4. LLM judges → "trickster" misalignment

(12:06) Afterword

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Epistemic status: I consider the following future quite plausible in the next few years (~35% chance that something vaguely like this occurs), perhaps as soon as a year from now.

Imagine an open-source LLM agent good enough to cover its own compute costs and turn a modest profit on average when allowed to run with full internet and tool access and told to make as much money as possible. I estimate this to be slightly better than the best publicly available closed-source models today, with long-horizon reliability and goal-setting being the only thing missing.

If the returns generated by such an agent beat the market (plus a margin for any additional risk), there suddenly becomes a strong incentive to spin up huge numbers of them. The internet would be flooded by the by-products of their moneymaking schemes. And returns might be larger for agents without legal or ethical guardrails- cue a deluge of scams and ransomware attacks.

Even if profits are very small, anyone with an agenda that the agents can help with is still incentivised to use them. Nation states and terrorist groups now have a golden plausibly-deniable disinformation, mischief, and hacking tool: spin up some agents, tell [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Is there some kind of "get prediction markets, or predictions, onto twitter as a central object" project going? If so, how is it going?

I'm thinking through "how to raise median sanity" on a world scale. There are several incentive and institutional problems that make this very difficult. One angle is "try to make it a thing that the world tracks and cares about your predictions, and getting them right/wrong."

Two past angles here were:

• Fact checker sites of the 90s/00s, which became politicized
• Prediction markets, which I feel like aren't a good fit for most of what people would/should care about here. Operationalizing them is extremely annoying.

An idea for a twitter feature, which I think Musk should conceptually like given his stated values, and, I think he might have enough inertia to get behind, is:

• When you make a prediction shaped tweet, it automatically gets converted into a prediction-y object*. AI suggests a few plausible operationalizations, or specific edge cases that seem likely to come up that you might want to address.
• People can "like" predictions, predictions with a lot of attention get more automated followup.
• Instead of resolving predictions with explicit careful operationalization, people [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.