LessWrong (Curated & Popular)

Audio note: this article contains 31 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Lewis Smith*, Sen Rajamanoharan*, Arthur Conmy, Callum McDougall, Janos Kramar, Tom Lieberum, Rohin Shah, Neel Nanda

* = equal contribution

The following piece is a list of snippets about research from the GDM mechanistic interpretability team, which we didn’t consider a good fit for turning into a paper, but which we thought the community might benefit from seeing in this less formal form. These are largely things that we found in the process of a project investigating whether sparse autoencoders were useful for downstream tasks, notably out-of-distribution probing.

TL;DR

• To validate whether SAEs were a worthwhile technique, we explored whether they were useful on the downstream task of OOD generalisation when detecting harmful intent in user prompts
• [...]

---

Outline:

(01:08) TL;DR

(02:38) Introduction

(02:41) Motivation

(06:09) Our Task

(08:35) Conclusions and Strategic Updates

(13:59) Comparing different ways to train Chat SAEs

(18:30) Using SAEs for OOD Probing

(20:21) Technical Setup

(20:24) Datasets

(24:16) Probing

(26:48) Results

(30:36) Related Work and Discussion

(34:01) Is it surprising that SAEs didn't work?

(39:54) Dataset debugging with SAEs

(42:02) Autointerp and high frequency latents

(44:16) Removing High Frequency Latents from JumpReLU SAEs

(45:04) Method

(45:07) Motivation

(47:29) Modifying the sparsity penalty

(48:48) How we evaluated interpretability

(50:36) Results

(51:18) Reconstruction loss at fixed sparsity

(52:10) Frequency histograms

(52:52) Latent interpretability

(54:23) Conclusions

(56:43) Appendix

The original text contained 7 footnotes which were omitted from this narration.

---

First published:
March 26th, 2025

Source:
https://www.lesswrong.com/posts/4uXCAJNuPKtKBsi28/sae-progress-update-2-draft

---

Narrated by TYPE III AUDIO.

---

Images from the article:

This is a link post. When I was a really small kid, one of my favorite activities was to try and dam up the creek in my backyard. I would carefully move rocks into high walls, pile up leaves, or try patching the holes with sand. The goal was just to see how high I could get the lake, knowing that if I plugged every hole, eventually the water would always rise and defeat my efforts. Beaver behaviour.

One day, I had the realization that there was a simpler approach. I could just go get a big 5 foot long shovel, and instead of intricately locking together rocks and leaves and sticks, I could collapse the sides of the riverbank down and really build a proper big dam. I went to ask my dad for the shovel to try this out, and he told me, very heavily paraphrasing, 'Congratulations. You've [...]

---

First published:
April 10th, 2025

Source:
https://www.lesswrong.com/posts/rLucLvwKoLdHSBTAn/playing-in-the-creek

Linkpost URL:
https://hgreer.com/PlayingInTheCreek

---

Narrated by TYPE III AUDIO.

This is part of the MIRI Single Author Series. Pieces in this series represent the beliefs and opinions of their named authors, and do not claim to speak for all of MIRI.

Okay, I'm annoyed at people covering AI 2027 burying the lede, so I'm going to try not to do that. The authors predict a strong chance that all humans will be (effectively) dead in 6 years, and this agrees with my best guess about the future. (My modal timeline has loss of control of Earth mostly happening in 2028, rather than late 2027, but nitpicking at that scale hardly matters.) Their timeline to transformative AI also seems pretty close to the perspective of frontier lab CEO's (at least Dario Amodei, and probably Sam Altman) and the aggregate market opinion of both Metaculus and Manifold!

If you look on those market platforms you get graphs like this:

Both [...]

---

Outline:

(02:23) Mode ≠ Median

(04:50) Theres a Decent Chance of Having Decades

(06:44) More Thoughts

(08:55) Mid 2025

(09:01) Late 2025

(10:42) Early 2026

(11:18) Mid 2026

(12:58) Late 2026

(13:04) January 2027

(13:26) February 2027

(14:53) March 2027

(16:32) April 2027

(16:50) May 2027

(18:41) June 2027

(19:03) July 2027

(20:27) August 2027

(22:45) September 2027

(24:37) October 2027

(26:14) November 2027 (Race)

(29:08) December 2027 (Race)

(30:53) 2028 and Beyond (Race)

(34:42) Thoughts on Slowdown

(38:27) Final Thoughts

---

First published:
April 9th, 2025

Source:
https://www.lesswrong.com/posts/Yzcb5mQ7iq4DFfXHx/thoughts-on-ai-2027

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Short AI takeoff timelines seem to leave no time for some lines of alignment research to become impactful. But any research rebalances the mix of currently legible research directions that could be handed off to AI-assisted alignment researchers or early autonomous AI researchers whenever they show up. So even hopelessly incomplete research agendas could still be used to prompt future capable AI to focus on them, while in the absence of such incomplete research agendas we'd need to rely on AI's judgment more completely. This doesn't crucially depend on giving significant probability to long AI takeoff timelines, or on expected value in such scenarios driving the priorities.

Potential for AI to take up the torch makes it reasonable to still prioritize things that have no hope at all of becoming practical for decades (with human effort). How well AIs can be directed to advance a line of research [...]

---

First published:
April 9th, 2025

Source:
https://www.lesswrong.com/posts/3NdpbA6M5AM2gHvTW/short-timelines-don-t-devalue-long-horizon-research

---

Narrated by TYPE III AUDIO.

In this post, we present a replication and extension of an alignment faking model organism:

• Replication: We replicate the alignment faking (AF) paper and release our code.
• Classifier Improvements: We significantly improve the precision and recall of the AF classifier. We release a dataset of ~100 human-labelled examples of AF for which our classifier achieves an AUROC of 0.9 compared to 0.6 from the original classifier.
• Evaluating More Models: We find Llama family models, other open source models, and GPT-4o do not AF in the prompted-only setting when evaluating using our new classifier (other than a single instance with Llama 3 405B).
• Extending SFT Experiments: We run supervised fine-tuning (SFT) experiments on Llama (and GPT4o) and find that AF rate increases with scale. We release the fine-tuned models on Huggingface and scripts.
• Alignment faking on 70B: We find that Llama 70B alignment fakes when both using the system prompt in the [...]

---

Outline:

(02:43) Method

(02:46) Overview of the Alignment Faking Setup

(04:22) Our Setup

(06:02) Results

(06:05) Improving Alignment Faking Classification

(10:56) Replication of Prompted Experiments

(14:02) Prompted Experiments on More Models

(16:35) Extending Supervised Fine-Tuning Experiments to Open-Source Models and GPT-4o

(23:13) Next Steps

(25:02) Appendix

(25:05) Appendix A: Classifying alignment faking

(25:17) Criteria in more depth

(27:40) False positives example 1 from the old classifier

(30:11) False positives example 2 from the old classifier

(32:06) False negative example 1 from the old classifier

(35:00) False negative example 2 from the old classifier

(36:56) Appendix B: Classifier ROC on other models

(37:24) Appendix C: User prompt suffix ablation

(40:24) Appendix D: Longer training of baseline docs

---

First published:
April 8th, 2025

Source:
https://www.lesswrong.com/posts/Fr4QsQT52RFKHvCAH/alignment-faking-revisited-improved-classifiers-and-open

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Summary: We propose measuring AI performance in terms of the length of tasks AI agents can complete. We show that this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months. Extrapolating this trend predicts that, in under five years, we will see AI agents that can independently complete a large fraction of software tasks that currently take humans days or weeks.

The length of tasks (measured by how long they take human professionals) that generalist frontier model agents can complete autonomously with 50% reliability has been doubling approximately every 7 months for the last 6 years. The shaded region represents 95% CI calculated by hierarchical bootstrap over task families, tasks, and task attempts.

Full paper | Github repo



We think that forecasting the capabilities of future AI systems is important for understanding and preparing for the impact of [...]

---

Outline:

(08:58) Conclusion

(09:59) Want to contribute?

---

First published:
March 19th, 2025

Source:
https://www.lesswrong.com/posts/deesrjitvXM4xYGZd/metr-measuring-ai-ability-to-complete-long-tasks

---

Narrated by TYPE III AUDIO.

---

Images from the article:

“In the loveliest town of all, where the houses were white and high and the elms trees were green and higher than the houses, where the front yards were wide and pleasant and the back yards were bushy and worth finding out about, where the streets sloped down to the stream and the stream flowed quietly under the bridge, where the lawns ended in orchards and the orchards ended in fields and the fields ended in pastures and the pastures climbed the hill and disappeared over the top toward the wonderful wide sky, in this loveliest of all towns Stuart stopped to get a drink of sarsaparilla.”
— 107-word sentence from Stuart Little (1945)

Sentence lengths have declined. The average sentence length was 49 for Chaucer (died 1400), 50 for Spenser (died 1599), 42 for Austen (died 1817), 20 for Dickens (died 1870), 21 for Emerson (died 1882), 14 [...]



---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/xYn3CKir4bTMzY5eb/why-have-sentence-lengths-decreased

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In 2021 I wrote what became my most popular blog post: What 2026 Looks Like. I intended to keep writing predictions all the way to AGI and beyond, but chickened out and just published up till 2026.

Well, it's finally time. I'm back, and this time I have a team with me: the AI Futures Project. We've written a concrete scenario of what we think the future of AI will look like. We are highly uncertain, of course, but we hope this story will rhyme with reality enough to help us all prepare for what's ahead.

You really should go read it on the website instead of here, it's much better. There's a sliding dashboard that updates the stats as you scroll through the scenario!

But I've nevertheless copied the first half of the story below. I look forward to reading your comments.

Mid 2025: Stumbling Agents The [...]



---

Outline:

(01:35) Mid 2025: Stumbling Agents

(03:13) Late 2025: The World's Most Expensive AI

(08:34) Early 2026: Coding Automation

(10:49) Mid 2026: China Wakes Up

(13:48) Late 2026: AI Takes Some Jobs

(15:35) January 2027: Agent-2 Never Finishes Learning

(18:20) February 2027: China Steals Agent-2

(21:12) March 2027: Algorithmic Breakthroughs

(23:58) April 2027: Alignment for Agent-3

(27:26) May 2027: National Security

(29:50) June 2027: Self-improving AI

(31:36) July 2027: The Cheap Remote Worker

(34:35) August 2027: The Geopolitics of Superintelligence

(40:43) September 2027: Agent-4, the Superhuman AI Researcher

---

First published:
April 3rd, 2025

Source:
https://www.lesswrong.com/posts/TpSFoqoG2M5MAAesg/ai-2027-what-superintelligence-looks-like-1

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Back when the OpenAI board attempted and failed to fire Sam Altman, we faced a highly hostile information environment. The battle was fought largely through control of the public narrative, and the above was my attempt to put together what happened.My conclusion, which I still believe, was that Sam Altman had engaged in a variety of unacceptable conduct that merited his firing.In particular, he very much ‘not been consistently candid’ with the board on several important occasions. In particular, he lied to board members about what was said by other board members, with the goal of forcing out a board member he disliked. There were also other instances in which he misled and was otherwise toxic to employees, and he played fast and loose with the investment fund and other outside opportunities. I concluded that the story that this was about ‘AI safety’ or ‘EA (effective altruism)’ or [...] ---

Outline:

(01:32) The Big Picture Going Forward

(06:27) Hagey Verifies Out the Story

(08:50) Key Facts From the Story

(11:57) Dangers of False Narratives

(16:24) A Full Reference and Reading List

---

First published:
March 31st, 2025

Source:
https://www.lesswrong.com/posts/25EgRNWcY6PM3fWZh/openai-12-battle-of-the-board-redux

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Epistemic status: This post aims at an ambitious target: improving intuitive understanding directly. The model for why this is worth trying is that I believe we are more bottlenecked by people having good intuitions guiding their research than, for example, by the ability of people to code and run evals.

Quite a few ideas in AI safety implicitly use assumptions about individuality that ultimately derive from human experience.

When we talk about AIs scheming, alignment faking or goal preservation, we imply there is something scheming or alignment faking or wanting to preserve its goals or escape the datacentre.

If the system in question were human, it would be quite clear what that individual system is. When you read about Reinhold Messner reaching the summit of Everest, you would be curious about the climb, but you would not ask if it was his body there, or his [...]

---

Outline:

(01:38) Individuality in Biology

(03:53) Individuality in AI Systems

(10:19) Risks and Limitations of Anthropomorphic Individuality Assumptions

(11:25) Coordinating Selves

(16:19) Whats at Stake: Stories

(17:25) Exporting Myself

(21:43) The Alignment Whisperers

(23:27) Echoes in the Dataset

(25:18) Implications for Alignment Research and Policy

---

First published:
March 28th, 2025

Source:
https://www.lesswrong.com/posts/wQKskToGofs4osdJ3/the-pando-problem-rethinking-ai-individuality

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.