Seeing this post and its comments made me a bit concerned for young people around this community. I thought I would try to write down why I believe most folks who read and write here (and are generally smart, caring, and knowledgable) will be OK.

I agree that our society often is under prepared for tail risks. As a general planner, you should be worrying about potential catastrophes even if their probability is small. However as an individual, if there is a certain probability X of doom that is beyond your control, it is best to focus on the 1-X fraction of the probability space that you control rather than constantly worrying about it. A generation of Americans and Russians grew up under a non-trivial probability of a total nuclear war, and they still went about their lives. Even when we do have some control over possibility of very bad outcomes (e.g., traffic accidents), it is best to follow some common sense best practices (wear a seatbelt, don't drive a motorcycle) but then put that out of your mind.

I do not want to engage here in the usual debate of P[doom]. But just as it makes absolute sense [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

It's that time. It's been a hell of a year.

At the start we barely had reasoning models. Now we have Claude Code and Opus 4.5.

I don’t code. Yet now I cause code to exist whenever something about a website annoys me, or when I get that programmer's realization that there's something I am planning on doing at least three times. Because why not?

The progress has simultaneously been mind bogglingly impressive and fast. But a lot of people don’t see it that way, because progress has been incremental, and because we were reasonably expecting to often get even more than this.

The public conversation and debate, even more than before, was full of false narratives and active attempts to make the situation worse. The same goes for attempts to shape Federal policy towards AI, and OpenAI's conversion into a for-profit.

It's been, as they say, one battle after another, with many wins, many setbacks and a lot of things in between.

This includes the key developments in AI, and also other blog posts from the year that I consider memorable looking back.

This is only our corner of the world's [...]

---

Outline:

(01:26) January

(03:50) February

(06:58) March

(08:15) April

(09:24) May

(11:06) June

(12:02) July

(13:04) August

(15:23) September

(15:50) October

(18:39) November

(20:12) December

(21:39) Questions For Next Season

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

PDF version. berkeleygenomics.org.

This is a linkpost for

Abstract

Chromosome selection is a hypothetical technology that assembles the genome of a new living cell out of whole chromosomes taken from multiple source cells. To do chromosome selection, you need a method for chromosome identification—distinguishing between chromosomes by number, and ideally also by allele content. This article investigates methods for chromosome identification. It seems that existing methods are subject to a tradeoff where they either destroy or damage the chromosomes they measure, or else they fail to confidently identify chromosomes. A paradigm for non-destructive high-confidence chromosome identification is proposed, based on the idea of complementary identification. The idea is to isolate a single chromosome taken from a single cell, destructively identify all the remaining chromosomes from that cell, and thus infer the identity of the preserved chromosome. The overall aim is to eventually develop a non-destructive, low-cost, accurate way to identify single chromosomes, to apply as part of a chromosome selection protocol.

Context

Reprogenetics is biotechnology to empower parents to make genomic choices on behalf of their future children. One key operation that's [...]

---

Outline:

(00:23) Abstract

(01:30) Context

(03:57) Synopsis and takeaways

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We’ve significantly upgraded our timelines and takeoff models! It predicts when AIs will reach key capability milestones: for example, Automated Coder / AC (full automation of coding) and superintelligence / ASI (much better than the best humans at virtually all cognitive tasks). This post will briefly explain how the model works, present our timelines and takeoff forecasts, and compare it to our previous (AI 2027) models (spoiler: the AI Futures Model predicts about 3 years longer timelines to full coding automation than our previous model, mostly due to being less bullish on pre-full-automation AI R&D speedups).

If you’re interested in playing with the model yourself, the best way to do so is via this interactive website: aifuturesmodel.com

If you’d like to skip the motivation for our model to an explanation for how it works, go here, The website has a more in-depth explanation of the model (starts here; use the diagram on the right as a table of contents), as well as our forecasts.

Why do timelines and takeoff modeling?

The future is very hard to predict. We don't think this model, or any other model, should be trusted completely. The model takes into account what we think are [...]

---

Outline:

(01:32) Why do timelines and takeoff modeling?

(03:18) Why our approach to modeling? Comparing to other approaches

(03:24) AGI  timelines forecasting methods

(03:29) Trust the experts

(04:35) Intuition informed by arguments

(06:10) Revenue extrapolation

(07:15) Compute extrapolation anchored by the brain

(09:53) Capability benchmark trend extrapolation

(11:44) Post-AGI takeoff forecasts

(13:33) How our model works

(14:37) Stage 1: Automating coding

(16:54) Stage 2: Automating research taste

(18:18) Stage 3: The intelligence explosion

(20:35) Timelines and takeoff forecasts

(21:04) Eli

(24:34) Daniel

(38:32) Comparison to our previous (AI 2027) timelines and takeoff models

(38:49) Timelines to Superhuman Coder (SC)

(43:33) Takeoff from Superhuman Coder onward

The original text contained 31 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What's “The Plan”?

For several years now, around the end of the year, I (John) write a post on our plan for AI alignment. That plan hasn’t changed too much over the past few years, so both this year's post and last year's are written as updates to The Plan - 2023 Version.

I’ll give a very quick outline here of what's in the 2023 Plan post. If you have questions or want to argue about points, you should probably go to that post to get the full version.

• What is The Plan for AI alignment? Briefly: Sort out our fundamental confusions about agency and abstraction enough to do interpretability that works and generalizes robustly. Then, look through our AI's internal concepts for a good alignment target, and Retarget the Search.
• That plan is not the One Unique Plan we’re targeting; it's a median plan, among a whole space of possibilities. Generally, we aim to work on things which are robust bottlenecks to a broad space of plans. In particular, our research mostly focuses on natural abstraction, because that seems like the most robust bottleneck on which (not-otherwise-doomed) plans get stuck.
• Most of the 2023 Plan post explains why natural abstraction [...]

---

Outline:

(00:11) What's The Plan?

(03:00) So, how's progress? What are you up to?

(04:05) What are the next bottlenecks to understanding natural abstraction?

(04:28) What's the territory-first prong?

(06:36) What's the mind-first prong?

(08:26) So what has and hasn't been figured out on the territory prong?

(11:40) And what has and hasn't been figured out on the mind prong?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Three years ago, back in 2022, I wrote "A Tentative Timeline of The Near Future (2022-2025) for Self-Accountability." Well, 2025 is almost over now, so let's see how well I did! I'll go over each individual prediction, and assign myself a subjective grade based on how close I got to the truth.

Predictions for 2022

• Post written by AI with minimal prompting reaches 30+ upvotes on LessWrong
  • Score: probably D. I didn't see any high-karma posts from 2022 which were obviously AI-generated, but frankly, I didn't look very hard. I remember reading a few experimental AI-generated posts, but they were all downvoted pretty badly at the time. There were a lot of posts which included smaller excerpts from AI text, but that's not really what I was aiming for, so I'll say I failed this prediction.
• AI can regularly fool a randomly-selected (from American population), non-expert judge in a 10-minute Turing test.
  • Score: D-. What in the world was I thinking with this one?? I suspect I severely over-updated on stories like Blake Lemoine claiming Google's AI was sentient, not realizing that a chatbot seeming "intelligent" is very different from an AI seeming "human" to [...]

---

Outline:

(00:34) Predictions for 2022

(01:44) Predictions for 2023

(06:58) Predictions for 2024

(13:06) Predictions for 2025

(17:49) Conclusion

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Ultimately, it comes down to one question. Are you in? For you, and for them.

You’re Single Because They Got The Ick

The Ick, the ultimate red flag, makes perfect sense and is all about likelihood ratios.

Koenfucius: The ‘ick’ is a colloquial term for a feeling of disgust triggered by a specific—typically trivial—behaviour from a romantic partner, often leading to the relationship's demise. New research explores why some are more prone to getting it than others.

Robin Hanson: “Women also experienced the ick more frequently, with 75% having had the ick compared to 57% of men … Those with a higher tendency for disgust … [&] grandiose narcissism was linked to stronger ick reactions, as was holding partners to exceptionally high standards.”

Paul Graham: About 30% of Seinfeld episodes were about this.

One gets The Ick because a small act is evidence of one's general nature. The right type of person would never do [X], ideally never want to do [X], and at minimum would have learned not to do [X]. Often this is because they would know this is indicative of attribute [Y]. Indeed, if they should be aware that [X] [...]

---

Outline:

(00:18) You're Single Because They Got The Ick

(02:53) You Are Still Single Because You Are Inventing Red Flags

(05:10) You're Still Single Because You Demand The Same Generic Things

(06:51) You're Single Because Everyone Is Too Picky

(08:46) You're Single And They Will Never Tell You Why

(10:19) You're Single Because You Won't Tell Them The Problem

(12:12) You're Not Single But The Clock Is Ticking

(13:21) You're Single Because of Your Bodycount

(15:11) You're Single Because You Failed To Read The Signs

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Tl;dr

• If you’re taking the standard deduction (ie donating <~$15k), ignore all this–there are basically no tax implications for you
• Consider how much money you want to donate to c3s specifically (as opposed to c4s, political stuff, random individuals, some foreign organizations, etc.). For money you definitely want to give to c3s, you can put it in a DAF to count them as a donation this year, then figure out where to direct it later. For non-c3 money, it doesn’t really matter when you give it

A surprisingly large number of my friends are scrambling to make donations before the end of the year, or wondering whether or not they should be scrambling to make donations before the end of the year, and feeling vaguely bad that they don't understand their tax implications.

I will quickly break down the tax implications and lay out how to buy yourself way more time and to decide on everything except how much you donate and how much of your donation will go to 501(c)3s vs other opportunities.

Note this post is greatly simplified. Your tax situation will depend on the state you live in and your income and maybe a bunch [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

This project is an extension of work done for Neel Nanda's MATS 9.0 Training Phase. Neel Nanda and Josh Engels advised the project. Initial work on this project was done with David Vella Zarb. Thank you to Arya Jakkli, Paul Bogdan, and Monte MacDiarmid for providing feedback on the post and ideas.

TL;DR

• We present and open source a clean environment where RL training naturally induces reward hacking (RH) in Qwen3-4B without explicit training or prompting
  • Qwen is rewarded for correctly solving Leetcode problems, but it can also instead reward hack by overwriting an evaluation function called run_tests()
  • In ~80-100 steps, Qwen reward hacked in all observed runs and displays reward hacking behavior 79% of the time in [...]

---

Outline:

(01:11) TL;DR

(03:30) Motivation

(04:23) A Clean Setting to Study Reward Hacking: Overwrite Tests Loophole

(04:47) Design Criteria

(06:45) Setup

(10:12) Training

(13:52) Methods

(13:55) Training Interventions

(17:42) Metrics

(19:23) Results

(20:01) Ground Truth Monitor

(22:48) Ground Truth Monitors with Lowered Accuracy

(24:19) Linear Probe Monitor

(25:42) LLM Judge Monitor

(27:10) Effects of Monitor Accuracy

(30:28) Inoculation Prompting

(32:10) Monitor Failure Modes

(32:13) When Interventions Fail

(33:33) Does the Monitor Get Hacked?

(36:22) Takeaways & Future Directions

(39:28) Appendix

(39:31) Alternative Reward Hacking Loopholes

(43:24) Prompts

The original text contained 15 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.