Linus Pauling was a chemist who won a Nobel Prize in Chemistry in 1954. He later became well-known for advocating large doses of Vitamin C.

I've heard that advocacy referred to as a cautionary tale, but I've long had a bit of sympathy for Linus, and I'm writing this post to explain how and why.

mainstream nutrition

One reason for my sympathy for Linus is that I've heard him used as an example of why you shouldn't go off on your own instead of trusting mainstream views in a field, yet his advice, while not particularly helpful, caused much less harm than contemporary "mainstream nutritional advice", such as:

I certainly wouldn't suggest trying [...]

---

Outline:

(00:25) mainstream nutrition

(01:18) my view of Vitamin C

(04:13) conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

(spoiler alert: may meta-spoil future iterations of the LOPH, if you

I knew early on that I wouldn’t be able to finish the LessOnline puzzle

I contributed to solving two of the first six puzzles, each of which

On discovering the existence of the meta-puzzle, I knew I would have to

---

Outline:

(01:04) Suspicion

(01:37) Sequence Breaker!

(03:22) Decision Theory

(04:32) Failure

(05:50) Afterward

---

First published:

Source:

Narrated by TYPE III AUDIO.

Without countermeasures, a scheming AI could escape.

A safety case (for deployment) is an argument that it is safe to deploy a particular AI system in a particular way.[1] For any existing LM-based system, the developer can make a great safety case by demonstrating that the system does not have dangerous capabilities.[2] No dangerous capabilities is a straightforward and solid kind of safety case. For future systems with dangerous capabilities, a safety case will require an argument that the system is safe to deploy despite those dangerous capabilities. In this post, I discuss the safety cases that the labs are currently planning to make, note that they ignore an important class of threats—namely threats from scheming AI escaping—and briefly discuss and recommend control-based safety cases.

1. Safety cases implicit in current safety plans: no dangerous capabilities and mitigations to prevent misuse

Four documents both (a) are endorsed by [...]

---

Outline:

(00:53) 1. Safety cases implicit in current safety plans: no dangerous capabilities and mitigations to prevent misuse

(03:32) 2. Scheming AI and escape during internal deployment

(07:41) 3. Control techniques and control-based safety cases

The original text contained 19 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

[Thanks to Aryan Bhatt, Ansh Radhakrishnan, Adam Kaufman, Vivek Hebbar, Hanna Gabor, Justis Mills, Aaron Scher, Max Nadeau, Ryan Greenblatt, Peter Barnett, Fabien Roger, and various people at a presentation of these arguments for comments. These ideas aren’t very original to me; many of the examples of threat models are from other people.]

In this post, I want to introduce the concept of a “rogue deployment” and argue that it's interesting to classify possible AI catastrophes based on whether or not they involve a rogue deployment. I’ll also talk about how this division interacts with the structure of a safety case, discuss two important subcategories of rogue deployment, and make a few points about how the different categories I describe here might be caused by different attackers (e.g. the AI itself, rogue lab insiders, external hackers, or [...]

---

Outline:

(01:01) Rogue deployments

(04:01) Rogue deployments and safety cases

(05:10) More on catastrophes with rogue deployment

(10:04) More on catastrophes without rogue deployment

(13:03) Different attacker profiles

(13:46) Hot takes that I can say using the above concepts

---

First published:

Source:

Narrated by TYPE III AUDIO.

These are some of my notes from reading Anthropic's latest research report, Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet.

TL;DR

In roughly descending order of importance:

1. Its great that Anthropic trained an SAE on a production-scale language model, and that the approach works to find interpretable features. Its great those features allow interventions like the recently-departed Golden Gate Claude. I especially like the code bug feature.
2. I worry that naming features after high-activating examples (e.g. "the Golden Gate Bridge feature") gives a false sense of security. Most of the time that feature activates, it is irrelevant to the golden gate bridge. That feature is only well-described as "related to the golden gate bridge" if you condition on a very high activation, and that's <10% of its activations (from an eyeballing of the graph).
3. This work does not address my major concern about dictionary learning: it is [...]

---

Outline:

(00:16) TL;DR

(02:23) A Feature Isnt Its Highest Activating Examples

(04:38) Finding Specific Features

(06:02) Architecture - The Classics, but Wider

(07:26) Correlations - Strangely Large?

(09:48) Future Tests

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

You can read the book on nanosyste.ms.

The book won the 1992 Award for Best Computer Science Book. The AI safety community often references it, as it describes a lower bound on what intelligence should be able to achieve.

Previously, you could only physically buy the book or read a PDF scan.

(Thanks to MIRI and Internet Archive for their scans.)

---

First published:

Source:

Narrated by TYPE III AUDIO.

Tomorrow I will fly out to San Francisco, to spend Friday through Monday at the LessOnline conference at Lighthaven in Berkeley. If you are there, by all means say hello. If you are in the Bay generally and want to otherwise meet, especially on Monday, let me know that too and I will see if I have time to make that happen.

Even without that hiccup, it continues to be a game of playing catch-up. Progress is being made, but we are definitely not there yet (and everything not AI is being completely ignored for now).

Last week I pointed out seven things I was unable to cover, along with a few miscellaneous papers and reports.

Out of those seven, I managed to ship on three of them: Ongoing issues at OpenAI, The Schumer Report and Anthropic's interpretability paper.

However, OpenAI developments continue. Thanks largely [...]

---

Outline:

(01:42) Language Models Offer Mundane Utility

(03:59) Not Okay, Google

(13:33) OK Google, Don’t Panic

(17:10) Not Okay, Meta

(23:11) Not Okay Taking Our Jobs

(28:36) They Took Our Jobs Anyway

(32:39) A New Leaderboard Appears

(35:24) Copyright Confrontation

(35:42) Deepfaketown and Botpocalypse Soon

(40:12) Get Involved

(40:29) Introducing

(44:33) In Other AI News

(47:48) GPT-5 Alive

(50:11) Quiet Speculations

(01:02:26) Open Versus Closed

(01:07:16) Your Kind of People

(01:09:42) The Quest for Sane Regulations

(01:16:01) Lawfare and Liability

(01:22:47) SB 1047 Unconstitutional, Claims Paper

(01:26:48) The Week in Audio

(01:28:58) Rhetorical Innovation

(01:32:34) Abridged Reports of Our Death

(01:33:38) Aligning a Smarter Than Human Intelligence is Difficult

(01:38:46) People Are Worried About AI Killing Everyone

(01:39:43) Other People Are Not As Worried About AI Killing Everyone

(01:40:51) The Lighter Side

---

First published:

Source:

Narrated by TYPE III AUDIO.

[This post is more opinion-heavy and aimlessly self-promoting than feels appropriate for Lesswrong. I wrote this post for my site, Eukaryote Writes Blog, to show off that I now have a substack. But it had all these other observations about the state of the internet and advice woven in, and THOSE seemed more at home on Lesswrong, and I'm a busy woman with a lot of pictures of fish to review, so I'm just going to copy it over as posted without laboriously extricating the self-advertisement. Sorry if it's weird that it's there!]

Eukaryote Writes Blog is now syndicating to Substack. I have no plans for paygating content at the time, and new and old posts will continue to be available at EukaryoteWritesBlog.com. Call this an experiment and a reaching-out. If you’re reading this on Substack, hi! Thanks for joining me.

I really don’t like [...]

---

Outline:

(02:01) Hey, the internet is getting weird these days

(02:24) Generative AI

(03:47) Search engines

(04:33) Censorship

(07:15) Survival tips

(08:02) USE MULTIPLE SEARCH ENGINES

(09:15) USE FIREFOX AS YOUR BROWSER

(11:11) READ BLOGS? GET AN RSS READER

(12:43) FIND SOME PEOPLE YOU TRUST.

(13:49) PAY FOR THINGS YOU LIKE TO INCENTIVIZE THEIR EXISTENCE.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Here is a little Q&A

Can you explain your position quickly?

I think autonomous replication and adaptation in the wild is under-discussed as an AI threat model. And this makes me sad because this is one of the main reasons I'm worried. I think one of AI Safety people's main proposals should first focus on creating a nonproliferation treaty. Without this treaty, I think we are screwed. The more I think about it, the more I think we are approaching a point of no return. It seems to me that open source is a severe threat and that nobody is really on the ball. Before those powerful AIs can self-replicate and adapt, AI development will be very positive overall and difficult to stop, but it's too late after AI is able to adapt and evolve autonomously because Natural selection favors AI over humans.

What is ARA?

Autonomous Replication and [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

I

When I was a child, I remember thinking: Where do houses come from? They are huge! Building one would take forever! Yet there are so many of them!

Having become a boring adult, I no longer have the same blue-eyed wonder about houses, but humanity does have an accomplishment or two I'm still impressed by.

When going to the airport, the metal boulders really stay up in the air without crashing. Usually they leave at the time they told me two weeks earlier, taking me to the right destination at close to the speed of sound.

There are these boxes with buttons that you can press to send information near-instantly anywhere. They are able to perform billions of operations a second. And you can just buy them at a store!

And okay, I admit that big houses - skyscrapers - still light up some of that child-like marvel [...]

---

Outline:

(00:04) I

(01:00) II

(02:20) III

(04:29) IV

(07:43) V

(09:03) VI

(10:44) VII

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.