Hi everyone,

TL;DR the Safeguards team at Anthopic is hiring ML experts to focus on dangerous capability mitigations - this is a very high impact ML Engineer / Research Scientist opportunity related to serious harm from advanced AI capabilities. We’re hiring multiple roles at varying levels of seniority including both IC and management.

The Safeguards team focuses on present day and near future harms. Since we activated our ASL 3 safety standard for Claude 4 Opus, in accordance with our Responsible Scaling Plan, this means that we need to do a great job of protecting the world in case Claude can provide uplift for dangerous capabilities. Safeguards builds constitutional classifiers and probes that run over every Opus conversational turn to look for and suppress dangerous content and defend against jailbreaks.

This is a hard modeling task! The boundary between beneficial and risky content is thin and complicated. In some [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Many people who are concerned about existential risk from AI spend their time advocating for radical changes to how AI is handled. Most notably, they advocate for costly restrictions on how AI is developed now and in the future, e.g. the Pause AI people or the MIRI people. In contrast, I spend most of my time thinking about relatively cheap interventions that AI companies could implement to reduce risk assuming a low budget, and about how to cause AI companies to marginally increase that budget. I'll use the words "radicals" and "moderates" to refer to these two clusters of people/strategies. In this post, I’ll discuss the effect of being a radical or a moderate on your epistemics.

I don’t necessarily disagree with radicals, and most of the disagreement is unrelated to the topic of this post; see footnote for more on this.[1]

I often hear people claim that being [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

The conditions are: Lol, we’re Meta. Or lol we’re xAI.

This expands upon many previous discussions, including the AI Companion Piece.

Lol We’re Meta

I said that ‘Lol we’re Meta’ was their alignment plan.

It turns out their alignment plan was substantially better or worse (depending on your point of view) than that, in that they also wrote down 200 pages of details of exactly how much lol there would be over at Meta. Every part of this was a decision.

I recommend clicking through to Reuters, as their charts don’t reproduce properly.

Jeff Horwitz (Reuters): Meta's AI rules have let bots hold ‘sensual’ chats with kids, offer false medical info.

An internal Meta Platforms document detailing policies on chatbot behavior has permitted the company's artificial intelligence creations to “engage a child in conversations that are romantic or sensual,” generate false medical [...]

---

Outline:

(00:21) Lol We're Meta

(04:15) If 'Lol We're Meta' It Is Good To Write That Down

(06:06) We Can't Help But Notice What Meta Wrote Down

(11:55) xAI and Meta Companion Offerings Are Differently Awful

(16:40) You're The Only One That Groks Me

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As I discussed in a prior post, I felt like there were some reasonably compelling arguments for expecting very fast AI progress in 2025 (especially on easily verified programming tasks). Concretely, this might have looked like reaching 8 hour 50% reliability horizon lengths on METR's task suite[1] by now due to greatly scaling up RL and getting large training runs to work well. In practice, I think we've seen AI progress in 2025 which is probably somewhat faster than the historical rate (at least in terms of progress on agentic software engineering tasks), but not much faster. And, despite large scale-ups in RL and now seeing multiple serious training runs much bigger than GPT-4 (including GPT-5), this progress didn't involve any very large jumps.

The doubling time for horizon length on METR's task suite has been around 135 days this year (2025) while it was more like 185 [...]

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

I got suckered into paying attention to multiple non-AI political stories this month: The shooting of the messenger, in violation of the most sacred principles, via firing the head of the USA's Bureau of Labor Statistics, and the Online Safety Bill in the UK.

As a reminder, feel no obligation whatsoever to engage with either of these.

There are tons of other things worth paying attention to that are not that.

Highly Effective Altruism

I realize philanthropy has been handed quite a few sinking ships lately, but if you exclude AI there is one crisis I would prioritize above the rest, which is mRNA.

Which is: We have entered the War on Cancer, future pandemics and other diseases on the side of cancer and future pandemics and those other diseases, because we decided to hand this power over to RFK Jr.

As Rick [...]

---

Outline:

(00:36) Highly Effective Altruism

(07:06) Variously Effective Altruism

(10:39) Bad News

(15:51) Whisper Networks

(17:36) Opportunity Knocks

(19:06) Government Working

(20:45) Censorship In The UK

(45:50) Don't Shoot the Messenger

(59:57) Jones Act Watch

(01:00:12) The Incorrect Contrarian Cluster

(01:01:53) Talent Search

(01:04:19) While I Cannot Condone This

(01:11:41) Predictions Are Hard Especially About The Future

(01:14:01) Good News, Everyone

(01:15:49) For Your Entertainment

(01:19:42) Game Theory

(01:20:39) Gamers Gonna Game Game Game Game Game

(01:28:48) I Was Promised Flying Self-Driving Cars

(01:34:00) Sports Go Sports

(01:36:17) Antisocial Media

(01:41:50) Stop Deboosting Links On Twitter

(01:49:10) Technology Advances

(01:52:35) The Lighter Side

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I'd been working on a very long piece of writing to try to contextualize my views here but it seems like now's the time to try to talk about this stuff online directly.

(I was a resident at Maple for two years from 2019-2021, and then later for six months as a "villager" in 2022, and worked for them remotely for all of 2023.)

I've been frustrated by the criticism that's been made of Maple in public[1], which has seemed mostly unhelpful, and not a productive contribution to the discourse.[2] One person, who I mostly do respect, did a lot of angry and perhaps manipulative finagling behind the scenes about this stuff, but broadly didn't write about it etc., I'm not sure why.

One story I might tell for that person, and part of why I myself haven't written about Maple publicly yet, is that it's very difficult [...]

---

Outline:

(02:14) I think that criticism of Maple cant be made by sneering appeals to Western secular culture.

(04:57) However, I dont think what Maple is doing is working or will work.

(07:10) Maple is correct about the modern disease, in a very important way.

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Written with Humzah Khan. This post is crossposted from Substack. Original here.

The AI takeover story, so we are told, is the supreme pinnacle of rationalist theory-crafting. The idea of unstoppable intelligence is so emotionally appealing, so theoretically compelling, so damn good at raising money from billionaires, that the hundreds of Ivy League graduates dropping all other pursuits to work on it are just the most tragically roastable examples of a nerdish, youthful tendency to ignore the banalities, the slowness, the thinking-doesn't-get-you-anywhereness of the real-world. An entire generation of youths oneshotted by straight lines on graphs.1 AI takeover, in the 21st century, has taken over from scholastic God-contemplation as the canonical form of speculation.

But what if I were to tell you there was a precedent for the sudden, violent overthrow of an entrenched ruling class by alien servants of their own creation? And further, what if I [...]

---

Outline:

(02:17) Principals for agents

(05:33) Medieval Disempowerment

(08:35) Medieval ARA in practice

(11:44) Conclusion: Medieval Lessons for Modern Problems

(12:38) Epilogue

(15:12) Further Reading

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a response to https://www.lesswrong.com/posts/mXa66dPR8hmHgndP5/hyperbolic-trend-with-upcoming-singularity-fits-metr which claims that a hyperbolic model, complete with an actual singularity in the near future, is a better fit for the METR time-horizon data than a simple exponential model.

I think that post has a serious error in it and its conclusions are the reverse of correct. Hence this one.

(An important remark: although I think Valentin2026 made an important mistake that invalidates his conclusions, I think he did an excellent thing in (1) considering an alternative model, (2) testing it, (3) showing all his working, and (4) writing it up clearly enough that others could check his work. Please do not take any part of this post as saying that Valentin2026 is bad or stupid or any nonsense like that. Anyone can make a mistake; I have made plenty of equally bad ones myself.)

The models

Valentin2026's post compares the results of [...]

---

Outline:

(01:02) The models

(02:32) Valentin2026s fits

(03:29) The problem

(05:11) Fixing the problem

(06:15) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 46 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Authors: Andrew Qin*, Tim Hua*, Samuel Marks, Arthur Conmy, Neel Nanda

Andrew and Tim are co-first authors. This is a research progress report from Neel Nanda's MATS 8.0 stream. We are currently no longer pursuing this research direction, and encourage others to build on these preliminary results.

tl;dr. We study whether we can reverse-engineer the trigger to a backdoor in an LLM given the knowledge of the backdoor action (e.g. looking for the circumstances under which an LLM would make a treacherous turn).

---

Outline:

(01:52) Introduction

(01:55) Problem setting

(03:40) Methodology

(04:48) Our Takeaways

(05:12) Related Work

(06:45) Methods

(07:15) SAE-Attribution

(08:23) MELBO Extensions

(09:14) Unsupervised SAE-MELBO

(10:07) Supervised MELBO

(10:40) Supervised SAE-MELBO

(10:51) Implementation Details

(11:43) Model Settings

(11:46) Toy Backdoors

(13:57) Realistic Settings

(15:53) Results

(16:26) Toy Model Performance

(17:01) Results by Model

(20:29) Realistic Setting Performance

(20:33) Banana

(21:58) Vanilla Factual Recall

(23:27) Discussion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: I think labs keep some safety research internal-only, though the motivations for this are sometimes unclear. This post is an exploration into the possible incentives and dynamics at play, which I have tried to gain information about from reliable sources. With that said, I'm posting to start a discussion rather than to make definitive claims.

Executive Summary

1. Multiple sources point to frontier labs keeping at least some safety research internal-only.
2. Additionally, I think frontier labs may be incentivised to keep at least some important safety research proprietary, which I refer to as safety hoarding.
3. Theories for why labs may safety-hoard include: the labs want to hedge against stronger future AI legislation or an Overton window shift in public safety concerns, which would create a competitive advantage for compliant labs; reputational gains from being "the safe lab"; safety breakthroughs being too expensive to implement, meaning it's favourable [...]

---

Outline:

(00:32) Executive Summary

(02:46) Introduction

(04:02) Hoarding Incentives

(07:23) Neutral Explanations for Unpublished Work

(08:31) In Favour of Selective Withholding

(10:00) Consequences of Safety Hoarding

(10:04) Negative

(11:32) Possibly Positive

(12:19) The Current State

(13:05) OpenAI

(15:25) Google DeepMind

(16:27) Anthropic

(17:14) A Move Towards Openness

(19:30) Grey Areas

(20:14) Precedents and Parallels

(20:35) Case Study: AV Safety Hoarding

(23:45) Other Automotive Examples

(25:40) Conclusion

(27:02) Future Work

(27:36) Acknowledgements

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.