If you've never read the LessWrong Sequences (which I read through the book-length compilation Rationality: From AI To Zombies), I suggest that you read the Sequences as if they were written today. Additionally, if you're thinking of rereading the Sequences, I suggest that your agenda for rereading, in addition to what it may already be, should be to read the Sequences as if they were written today.

To start, I'd like to take a moment to clarify what I mean. I don't mean "think about what you remember the Sequences talking about, and try to apply those concepts to current events." I don't even mean "read the Sequences and reflect on where the concepts are relevant to things that have happened since they were written." What I mean is that you should read the Sequences as if they were written today. You should imagine that, on January 1, 2025 [...]

---

Outline:

(01:09) Key advantages of this approach

(01:13) 1. Its easier to notice the contemporary applicability of the Sequences if you think of them as contemporary

(03:23) 2. It lets you set aside misremembered ideas about what the Sequences actually say

(05:36) 3. The Sequences were designed for today, not for when they were written

(06:46) A potential new series of posts

---

First published:

Source:

Narrated by TYPE III AUDIO.

Follow-up to: 2023 in AI predictions.

Here I collect some AI predictions made in 2024. It's not very systematic, it's a convenience sample mostly from browsing Twitter/X. I prefer including predictions that are more specific/testable. I'm planning to make these posts yearly, checking in on predictions whose date has expired. Feel free to add more references to predictions made in 2024 to the comments. (Thanks especially @tsarnick and @AISafetyMemes for posting about a lot of these.)

Predictions about 2024

I'll review predictions from previous posts that are about 2024.

the gears to ascension: "Hard problem of alignment is going to hit us like a train in 3 to 12 months at the same time some specific capabilities breakthroughs people have been working on for the entire history of ML finally start working now that they have a weak AGI to apply to, and suddenly critch's [...]

---

Outline:

(00:40) Predictions about 2024

(01:32) Predictions made in 2024

(01:37) December 2024

(02:31) September 2025

(02:54) October 2025

(03:06) December 2025

(03:54) April 2026

(04:22) June 2026

(04:33) August 2026

(04:57) November 2026

(05:08) Februrary 2027

(05:28) December 2027

(07:53) 2028

(09:56) 2029

(12:27) 2030

(13:45) 2032

(14:28) 2034

(14:56) 2039

(16:02) 2044

---

First published:

Source:

Narrated by TYPE III AUDIO.

United States Air Force Colonel John Boyd was a fighter pilot military strategist who developed several important strategic theories. While serving as a jet fighter instructor, he was nicknamed "Forty-Second Boyd" because he had a standing offer that he could go up in his plane and defeat any opponent in a simulated dogfight in forty seconds or less -- and do it starting from an extremely unfavorable position! If he failed, he would owe the challenger $40 -- but purportedly he never failed! Further, he was not only able to accomplish this feat against trainees, but also against various visiting pilots who challenged him.[1]

His concepts have been credited as instrumental to the United States's dramatic victory in Operation Desert Storm, and his insights into aircraft design have been highly influential as well. Boyd was a very unconventional thinker in some ways and did not always work [...]

---

Outline:

(01:49) What are OODA Loops?

(01:53) Basic Definition

(03:25) Example: Unknown Object in the Road

(04:56) A Note on Orientation

(05:20) More Advanced Illustration

(06:46) Applications of the OODA Framework

(06:51) OODA Loops In Competition

(07:06) Getting Inside their OODA Loop

(10:31) OODA in the Business World

(12:37) OODA Loops Without Competition

(14:16) OODA Failure Analysis - a technique

(16:15) Final Thoughts

The original text contained 5 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

(some plot spoilers)

There's something distinctly uncomfortable about reading Greg Egan in the 2020s. Besides telling gripping tales with insightful commentary on the true nature of mind and existence, Egan stories written in the 1990s and set in the twenty-first century excelled at speculative worldbuilding, imagining what technological wonders might exist in the decades to come and how Society might adapt to them.

In contrast, "Death and the Gorgon", published in the January/February 2024 issue of Asimov's, feels like it's set twenty minutes into the future. The technologies on display are an AI assistant for police officers (capable of performing research tasks and carrying on conversation) and real-time synthetic avatars (good enough to pass as a video call with a real person). When these kinds of products showed up in "'90s Egan"—I think of Worth's "pharm" custom drug dispenser in Distress (1995) or Maria's "mask" for screening [...]

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Recently, 2 Chinese military aircraft were seen flying for the first time. Some people wanted to read about my thoughts on them. In this post, I'll be referring to them as "Diamond" and "Dart" based on their shapes. Speculative designations being used elsewhere are:

some articles

Instead of embedding photos here, I'll just link to some articles with pictures:

what the photos show

aircraft size

Diamond seems to be ~22m long, with a central weapon bay long enough for the PL-17 or YJ-83 (6.4m), and 2 smaller bays long enough for the PL-15 (4m). It could probably carry glide bombs too. Its wing area is quite large for a fighter aircraft. The planform is similar to a F-16XL, and scaling that up to 22m [...]

---

Outline:

(00:28) some articles

(00:41) what the photos show

(00:45) aircraft size

(01:32) aerodynamics

(02:02) control surfaces

(02:54) stealth

(04:04) landing gear

(04:25) flight location

(04:48) engines

(06:06) sensors

(06:44) cost

(07:27) strategic purposes

(07:48) Diamond

(09:40) Dart

(10:08) other possible new aircraft

(10:55) Taiwan timelines

---

First published:

Source:

Narrated by TYPE III AUDIO.

Previous: My AGI safety research—2022 review, ’23 plans. (I guess I skipped it last year.)

“Our greatest fear should not be of failure, but of succeeding at something that doesn't really matter.” –attributed to DL Moody

Tl;dr

• Section 1 goes through my main research project, “reverse-engineering human social instincts”: what does that even mean, what's the path-to-impact, what progress did I make in 2024 (spoiler: lots!!), and how can I keep pushing it forward in the future?
• Section 2 is what I’m expecting to work on in 2025: most likely, I’ll start the year with some bigger-picture thinking about Safe & Beneficial AGI, then eventually get back to reverse-engineering human social instincts after that. Plus, a smattering of pedagogy, outreach, etc.
• Section 3 is a sorted list of all my blog posts from 2024
• Section 4 is acknowledgements 

1. Main research project: reverse-engineering human social instincts

1.1 Background [...]

---

Outline:

(00:25) Tl;dr

(01:12) 1. Main research project: reverse-engineering human social instincts

(01:21) 1.1 Background: What's the problem and why should we care?

(04:31) 1.2 More on the path-to-impact

(07:41) 1.3 Progress towards reverse-engineering human social instincts

(10:09) 1.4 What's next?

(11:22) 2. My plans going forward

(12:58) 3. Sorted list of my blog posts from 2024

(15:58) 4. Acknowledgements

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

What should we make of DeepSeek v3?

DeepSeek v3 seems to clearly be the best open model, the best model at its price point, and the best model with 37B active parameters, or that cost under $6 million.

According to the benchmarks, it can play with GPT-4o and Claude Sonnet.

Anecdotal reports and alternative benchmarks tells us it's not as good as Claude Sonnet, but it is plausibly on the level of GPT-4o.

So what do we have here? And what are the implications?

Table of Contents

What is DeepSeek v3 Techncially?

I’ve now had a [...]

---

Outline:

(00:39) What is DeepSeek v3 Techncially?

(01:56) Our Price Cheap

(02:33) Run Model Run

(04:57) Talent Search

(05:22) The Amazing Incredible Benchmarks

(07:23) Underperformance on AidanBench

(12:59) Model in the Arena

(13:27) Other Private Benchmarks

(15:05) Anecdata

(23:57) Implications and Policy

The original text contained 9 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post is a follow-up to The Plan - 2023 Version. There's also The Plan - 2022 Update and The Plan, but the 2023 version contains everything you need to know about the current Plan. Also see this comment and this comment on how my plans interact with the labs and other players, if you’re curious about that part.

What Have You Been Up To This Past Year?

Our big thing at the end of 2023 was Natural Latents. Prior to natural latents, the biggest problem with my math on natural abstraction was that it didn’t handle approximation well. Natural latents basically solved that problem. With that theoretical barrier out of the way, it was time to focus on crossing the theory-practice gap. Ultimately, that means building a product to get feedback from users on how well our theory works in practice, providing an empirical engine for iterative improvement [...]

---

Outline:

(00:26) What Have You Been Up To This Past Year?

(03:59) Why The Focus On Image Generators Rather Than LLMs?

(04:53) Any Major Changes To The Plan In The Past Year?

(06:15) Do We Have Enough Time?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

OpenAI presented o3 on the Friday before Thanksgiving, at the tail end of the 12 Days of Shipmas.

I was very much expecting the announcement to be something like a price drop. What better way to say ‘Merry Christmas,’ no?

They disagreed. Instead, we got this (here's the announcement, in which Sam Altman says ‘they thought it would be fun’ to go from one frontier model to their next frontier model, yeah, that's what I’m feeling, fun):

Greg Brockman (President of OpenAI): o3, our latest reasoning model, is a breakthrough, with a step function improvement on our most challenging benchmarks. We are starting safety testing and red teaming now.

Nat McAleese (OpenAI): o3 represents substantial progress in general-domain reasoning with reinforcement learning—excited that we were able to announce some results today! Here is a summary of what we shared about o3 in the livestream.

---

Outline:

(03:48) GPQA Has Fallen

(04:21) Codeforces Has Fallen

(05:32) Arc Has Kinda of Fallen But For Now Only Kinda

(09:27) They Trained on the Train Set

(15:26) AIME Has Fallen

(15:58) Frontier of Frontier Math Shifting Rapidly

(19:09) FrontierMath 4: We're Going To Need a Bigger Benchmark

(23:10) What is o3 Under the Hood?

(25:17) Not So Fast!

(28:38) Deep Thought

(30:03) Our Price Cheap

(36:32) Has Software Engineering Fallen?

(37:42) Don't Quit Your Day Job

(40:48) Master of Your Domain

(43:21) Safety Third

(47:56) The Safety Testing Program

(48:58) Safety testing in the reasoning era

(51:01) How to apply

(53:07) What Could Possibly Go Wrong?

(56:36) What Could Possibly Go Right?

(57:06) Send in the Skeptic

(59:25) This is Almost Certainly Not AGI

(01:02:57) Does This Mean the Future is Open Models?

(01:07:17) Not Priced In

(01:08:39) Our Media is Failing Us

(01:14:56) Not Covered Here: Deliberative Alignment

(01:15:08) The Lighter Side

The original text contained 22 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.