Or rather, we don’t actually have a proper o1 system card, aside from the outside red teaming reports. At all.

Because, as I realized after writing my first draft of this, the data here does not reflect the o1 model they released, or o1 pro?

I think what happened is pretty bad on multiple levels.

---

Outline:

(02:18) Where Art Thou o1 System Card?

(05:35) Introduction (Section 1)

(06:01) Model Data and Training (Section 2)

(06:13) Challenges and Evaluations (Section 3)

(09:38) Jailbreak Evaluations (Section 3.1.2)

(11:33) Regurgitation (3.1.3) and Hallucinations (3.1.4)

(12:30) Fairness and Bias (3.1.5)

(13:33) Jailbreaks Through Custom Developer Messages (3.2)

(14:41) Chain of Thought Safety (3.3)

(18:52) External Red Teaming Via Pairwise Safety Comparisons (3.4.1)

(19:57) Jailbreak Arena (3.4.2)

(20:25) Apollo Research (3.4.3) and the ‘Escape Attempts’

(21:38) METR (3.4.4) and Autonomous Capability

(25:22) Preparedness Framework Evaluations (Section 4)

(27:47) Mitigations

(30:27) Cybersecurity

(31:22) Chemical and Biological Threats (4.5)

(31:52) Radiological and Nuclear Threat Creation (4.6)

(32:21) Persuasion (4.7)

(32:49) Model Autonomy (4.8)

(34:45) Multilingual Performance

(34:55) Conclusion

The original text contained 17 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Six months ago, I was a high school English teacher.

I wasn’t looking to change careers, even after nineteen sometimes-difficult years. I was good at it. I enjoyed it. After long experimentation, I had found ways to cut through the nonsense and provide real value to my students. Daily, I met my nemesis, Apathy, in glorious battle, and bested her with growing frequency. I had found my voice.

At MIRI, I’m still struggling to find my voice, for reasons my colleagues have invited me to share later in this post. But my nemesis is the same.

Apathy will be the death of us. Indifference about whether this whole AI thing goes well or ends in disaster. Come-what-may acceptance of whatever awaits us at the other end of the glittering path. Telling ourselves that there's nothing we can do anyway. Imagining that some adults in the room will take care [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

At this point, we can confidently say that no, capabilities are not hitting a wall. Capacity density, how much you can pack into a given space, is way up and rising rapidly, and we are starting to figure out how to use it.

Not only did we get o1 and o1 pro and also Sora and other upgrades from OpenAI, we also got Gemini 1206 and then Gemini Flash 2.0 and the agent Jules (am I the only one who keeps reading this Jarvis?) and Deep Research, and Veo, and Imagen 3, and Genie 2 all from Google. Meta's Llama 3.3 dropped, claiming their 70B is now as good as the old 405B, and basically no one noticed.

This morning I saw Cursor now offers ‘agent mode.’ And hey there, Devin. And Palisade found that a little work made agents a lot more effective.

And OpenAI [...]

---

Outline:

(01:52) Language Models Offer Mundane Utility

(09:12) A Good Book

(12:24) Language Models Don’t Offer Mundane Utility

(14:25) o1 Pro Versus Claude

(15:25) AGI Claimed Internally

(16:52) Ask Claude

(23:19) Huh, Upgrades

(27:24) All Access Pass

(29:03) Fun With Image Generation

(35:28) Deepfaketown and Botpocalypse Soon

(37:49) They Took Our Jobs

(42:40) Get Involved

(43:50) Introducing

(44:11) In Other AI News

(48:14) OpenlyEvil AI

(55:39) Quiet Speculations

(01:00:14) Scale That Wall

(01:03:45) The Quest for Tripwire Capability Thresholds

(01:10:11) The Quest for Sane Regulations

(01:13:52) Republican Congressman Kean Brings the Fire

(01:18:35) CERN for AI

(01:23:34) The Week in Audio

(01:24:24) Rhetorical Innovation

(01:28:08) Model Evaluations Are Lower Bounds

(01:30:49) Aligning a Smarter Than Human Intelligence is Difficult

(01:35:38) I’ll Allow It

(01:38:10) Frontier AI Systems Have Surpassed the Self-Replicating Red Line

(01:42:50) People Are Worried About AI Killing Everyone

(01:43:56) Key Person Who Might Be Worried About AI Killing Everyone

(01:54:18) Other People Are Not As Worried About AI Killing Everyone

(01:56:22) Not Feeling the AGI

(01:59:19) Fight For Your Right

(02:01:44) The Lighter Side

The original text contained 23 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Just expose yourself to more social situations!” — Ah yes, you felt anxious the first 100 times, but the 101st will be the breakthrough!

“But exposure works!” people yell from across the street. “Like for fear of snakes - you know, those things you see once a year!”

Uh, it's pretty rational to fear things you have little experience with. But social anxiety… you interact with people everyday! Why would anything change after the first 100 attempts?

I don’t doubt that a couple of exposures can often reduce anxieties. However, if you still feel anxious even after hundreds of social situations and years of trying... then maybe your fear is actually doing something presently useful and you should reconnect with your intuitions.

At a 100% eye contact workshop I led earlier this year, most people became comfortable quickly with essentially a guided meditation.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The creation of a “mirrored” organism could “trigger severe ecological disruptions,” according to a 300-page technical report released today. Its authors claim such organisms could quickly spread across the world, fatally infect humans, and “directly drive vulnerable plant and animal species to extinction.” The report accompanies an article in Science, also released today, entitled “Confronting Risks of Mirror Life.”

But what, exactly, is a mirrored organism? To answer that, let's consider how extant life works.

Proteins, sugars, lipids, and nucleic acids — key molecules used by cellular life — are all “chiral,” a term derived from the Greek for “hand.” Just as our hands cannot be perfectly aligned on top of one another regardless of how they are rotated, despite being mirror images, the same is true of chiral molecules. The two mirror images of a chiral molecule are called its enantiomers; they possess the same atoms connected [...]

---

Outline:

(04:26) Molecular Hands

(09:52) Making Mirrors

(21:25) Infected World

(36:32) Creating Countermeasures

(43:16) The Arc of Progress

The original text contained 26 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

We're thinking about changing our narrator's voice.

There are three new voices on the shortlist. They're all similarly good in terms of comprehension, emphasis, error rate, etc. They just sound different—like people do.

We think they all sound similarly agreeable. But, thousands of listening hours are at stake, so we thought it'd be worth giving listeners an opportunity to vote—just in case there's a strong collective preference.

Listen and vote

Please listen here:

https://files.type3.audio/lesswrong-poll/

And vote here:

https://forms.gle/JwuaC2ttd5em1h6h8

It'll take 1-10 minutes, depending on how much of the sample you decide to listen to.

Don't overthink it—we'd just like to know if there's a voice that you'd particularly love (or hate).

We'll collect votes until Monday December 16th. Thanks!

---

Outline:

(00:58) Listen and vote

(01:30) Other feedback?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

A new article in Science Policy Forum voices concern about a particular line of biological research which, if successful in the long term, could eventually create a grave threat to humanity and to most life on Earth.

Fortunately, the threat is distant, and avoidable—but only if we have common knowledge of it.

What follows is an explanation of the threat, what we can do about it, and my comments.

Background: chirality

Glucose, a building block of sugars and starches, looks like this:

But there is also a molecule that is the exact mirror-image of glucose. It is called simply L-glucose (in contrast, the glucose in our food and bodies is sometimes called D-glucose):

This is not just the same molecule flipped around, or looked at from the other side: it's inverted, as your left hand is vs. your [...]

---

Outline:

(00:29) Background: chirality

(01:41) Mirror life

(02:47) The threat

(05:06) Defense would be difficult and severely limited

(06:09) Are we sure?

(07:47) Mirror life is a long-term goal of some scientific research

(08:57) What to do?

(10:22) We have time to react

(10:54) The far future

(12:25) Optimism, pessimism, and progress

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ultimately, I don’t want to solve complex problems via laborious, complex thinking, if we can help it. Ideally, I'd want to basically intuitively follow the right path to the answer quickly, with barely any effort at all.

For a few months I've been experimenting with the "How Could I have Thought That Thought Faster?" concept, originally described in a twitter thread by Eliezer:

Sarah Constantin: I really liked this example of an introspective process, in this case about the "life problem" of scheduling dates and later canceling them: malcolmocean.com/2021/08/int…

Eliezer Yudkowsky: See, if I'd noticed myself doing anything remotely like that, I'd go back, figure out which steps of thought were actually performing intrinsically necessary cognitive work, and then retrain myself to perform only those steps over the course of 30 seconds.

SC: if you have done anything REMOTELY like training yourself to do it in 30 seconds, then [...]

---

Outline:

(03:59) Example: 10x UI designers

(08:48) THE EXERCISE

(10:49) Part I: Thinking it Faster

(10:54) Steps you actually took

(11:02) Magical superintelligence steps

(11:22) Iterate on those lists

(12:25) Generalizing, and not Overgeneralizing

(14:49) Skills into Principles

(16:03) Part II: Thinking It Faster The First Time

(17:30) Generalizing from this exercise

(17:55) Anticipating Future Life Lessons

(18:45) Getting Detailed, and TAPS

(20:10) Part III: The Five Minute Version

---

First published:

Source:

Narrated by TYPE III AUDIO.

Given any particular concrete demonstration of an AI algorithm doing seemingly-bad-thing X, a knowledgeable AGI optimist can look closely at the code, training data, etc., and say:

“Well of course, it's obvious that the AI algorithm would do X under these circumstances. Duh. Why am I supposed to find that scary?”

And yes, it is true that, if you have enough of a knack for reasoning about algorithms, then you will never ever be surprised by any demonstration of any behavior from any algorithm. Algorithms ultimately just follow their source code.

(Indeed, even if you don’t have much of a knack for algorithms, such that you might not have correctly predicted what the algorithm did in advance, it will nevertheless feel obvious in hindsight!)

From the AGI optimist's perspective: If I'm not scared of AGI extinction right now, and nothing surprising has happened, then I won’t feel like I [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Adam Karvonen*, Can Rager*, Johnny Lin*, Curt Tigges*, Joseph Bloom*, David Chanin, Yeu-Tong Lau, Eoin Farrell, Arthur Conmy, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Samuel Marks, Neel Nanda *equal contribution

TL;DR

• We are releasing SAE Bench, a suite of 8 diverse sparse autoencoder (SAE) evaluations including unsupervised metrics and downstream tasks. Use our codebase to evaluate your own SAEs!
• You can compare 200+ SAEs of varying sparsity, dictionary size, architecture, and training time on Neuronpedia.
• Think we're missing an eval? We'd love for you to contribute it to our codebase! Email us.

🔍 Explore the Benchmark & Rankings

📊 Evaluate your SAEs with SAEBench

✉️ Contact Us

Introduction

Sparse Autoencoders (SAEs) have become one of the most popular tools for AI interpretability. A lot of recent interpretability work has been focused on studying SAEs, in particular on improving SAEs, e.g. the Gated [...]

---

Outline:

(00:31) TL;DR

(01:18) Introduction

---

First published:

Source:

Narrated by TYPE III AUDIO.