Suppose we had to take moves in a high-stakes chess game, with thousands of lives at stake. We wouldn't just find a good chess player and ask them to play carefully. We would consult a computer. It would be deeply irresponsible to do otherwise. Computers are better than humans at chess, and more reliable.

We'd probably still keep some good chess players in the loop, to try to catch possible computer error. (Similarly we still have pilots for planes, even though the autopilot is often safer.) But by consulting the computer we'd remove the opportunity for humans to make a certain type of high stakes mistake.

A lot of the high stakes decisions people make today don't look like chess, or flying a plane. They happen in domains where computers are much worse than humans.

But that's a contingent fact about our technology level. [...]

---

Outline:

(03:08) What will this era look like?

(05:40) Central challenges to be borne by humans

(08:51) Trying to help at far remove

(12:14) What to make of this

The original text contained 3 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is second kind of obvious point to make, but if you are interested in AI, AI safety, or cognition in general, it is likely worth going to top ML conferences, such as NeurIPS, ICML or ICLR. In this post I cover some reasons why, and some anecdotal stories.

1. Parts of AI alignment and safety are now completely mainstream

Looking at the "Best paper awards" at ICML, you'll find these safety-relevant or alignment-relevant papers:

• Stealing part of a production language model by Carlini et al.
• Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo by Zhao et al.
• Debating with More Persuasive LLMs Leads to More Truthful Answers by Khan et al.
• Genie: Generative Interactive Environments Bruce et al.

which amounts to about one-third (!). "Because of safety concerns" is part of the motivation for hundreds of papers.

While the signal-to-noise ratio is even worse [...]

---

Outline:

(00:23) 1. Parts of AI alignment and safety are now completely mainstream

(01:22) 2. Conferences are an efficient way how to screen general ML research without spending a lot of time on X

(02:57) 3. ML research community as a control group

(04:04) 4. What experts think

(04:54) 5. Examples

(06:28) 6. Conclusion

---

First published:

Source:

Narrated by TYPE III AUDIO.

Book review: The Cancer Resolution?: Cancer reinterpreted through

In the grand tradition of outsiders overturning scientific paradigms,

Lintern spends too many pages railing against the medical establishment,

If you can push through this preamble, you'll find a treasure trove of

Lintern's central claim is that fungal infections, not genetic

---

Outline:

(01:14) Older Theories

(02:05) Evidence Discrediting the DNA Theory

(03:36) Evidence of Fungi

(04:07) Medical Establishment Inertia

(06:32) Many Cancers?

(07:23) Implications for Treatment

(10:10) Concluding Thoughts

---

First published:

Source:

Narrated by TYPE III AUDIO.

Epistemic status: possibly trivial, but I hadn't heard it before.

TL;DR: What I thought of as a "flaw" in PCA—its inability to isolate pure metrics—might actually be a feature that aligns with our cognitive processes. We often think in terms of composite concepts (e.g., "Age + correlated attributes") rather than pure metrics, and this composite thinking might be more natural and efficient

Introduction

I recently found myself describing Principal Component Analysis (PCA) and pondering its potential drawbacks. However, upon further reflection, I'm reconsidering whether what I initially viewed as a limitation might actually be a feature. This led me to think about how our minds — and, potentially, language models — might naturally encode information using correlated attributes.

An important aspect of this idea is the potential conflation between the metric we use to measure something and the actual concept we're thinking about. For instance, when we think about [...]

---

Outline:

(00:35) Introduction

(01:51) 1 - Age and Associated Attributes in Children

(03:52) 2 - Price and Quality of Goods

(04:38) 3 - Size and Weight of Objects

(05:30) Implications for Information Encoding

(06:27) Rethinking PCA and Dimensional Reduction

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It is monthly roundup time.

I invite readers who want to hang out and get lunch in NYC later this week to come on Thursday at Bhatti Indian Grill (27th and Lexington) at noon.

I plan to cover the UBI study in its own post soon.

I cover Nate Silver's evisceration of the 538 presidential election model, because we cover probabilistic modeling and prediction markets here, but excluding any AI discussions I will continue to do my best to stay out of the actual politics.

Bad News

Jeff Bezos’ rocket company Blue Origin files comment suggesting SpaceX Starship launches be capped due to ‘impact on local environment.’ This is a rather shameful thing for them to be doing, and not for the first time.

Alexey Guzey reverses course, realizes at 26 that he was a naive idiot at 20 and finds everything he [...]

---

Outline:

(00:37) Bad News

(02:37) Silver Bullet

(06:35) Shame on Kathy Hochul

(06:58) This is (One Reason) Why We Can’t Have Nice Things

(09:02) (Don’t) Hack the Planet

(10:16) The Laptop Trap

(12:24) Courage

(14:25) Friendship

(15:26) The Gravest Mistake

(21:01) You Need Functional Decision Theory

(23:40) Antisocial Media

(25:52) For Science!

(30:58) Truth Seeking

(34:37) Liar Liar

(36:51) Government Working

(47:55) For Your Entertainment

(52:18) Variously Effective Altruism

(55:09) News You Can Use

(55:31) Good News, Everyone

(58:03) Gamers Gonna Game Game Game Game Game

(01:01:22) Sports Go Sports

(01:05:10) I Was Promised Flying Self-Driving Cars

(01:05:50) While I Cannot Condone This

(01:12:55) The Lighter Side

The original text contained 15 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is an informal research note. It is the result of a few-day exploration into RMU through the lens of model internals. Code to reproduce the main result is available here.

This work was produced as part of Ethan Perez's stream in the ML Alignment & Theory Scholars Program - Summer 2024 Cohort. Thanks to Nina Panickssery, Mrinank Sharma, and Fabien Roger for helpful discussion.

Summary

We investigate RMU, a recent unlearning method proposed by Li et al. (2024), through the lens of model internals. Through this lens, we explain that RMU mostly works by flooding the residual stream with "junk" in hazardous contexts, resulting in incoherence. We then propose a simple intervention to "clear the junk" from the residual stream. This intervention mostly restores the model's coherence in hazardous contexts, and recovers a significant proportion (but not all) of its original hazardous knowledge. This suggests that the effectiveness [...]

---

Outline:

(00:33) Summary

(01:34) What is RMU?

(03:53) Examining an RMU model

(04:33) Prompting with hazardous instructions

(04:49) Looking at activations

(06:18) Trying to undo RMU via directional ablation

(07:31) Directional ablation mostly restores coherence

(08:05) Directional ablation mostly restores activations to baseline

(08:39) Does directional ablation recover unlearned knowledge?

(09:41) Evaluation on WMDP benchmark

(11:46) Author contributions statement

The original text contained 4 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There have been a lot of D&D.Sci scenarios, but there's a lot of variance between them in complexity and quality. Some are more difficult, and might not be a good place to start, while others are much simpler - some were very good, while others on reflection didn't flow quite right.

Unfortunately, LW karma doesn't track the quality of these scenarios very well: often mediocre scenarios are higher-karma than better scenarios (whether because they had good writing around a poor scenario, or because people upvoted before playing them, or just because more people happened to be online and see them).

If you're interested in playing D&D.Sci scenarios, but don't know where to start, this index (compiled by frequent authors abstractapplic and aphyer, we'll try to keep this updated going forwards) is a good reference point to make sure you can pick good scenarios at a difficulty level you're [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is an adaptation of an internal doc I wrote for Anthropic.

Recently I’ve been having a lot of conversations about how to structure and staff teams. One framework I’ve referenced repeatedly is to break down team leadership into a few different categories of responsibility.

This is useful for a couple reasons. One is that it helps you get more concrete about what leading a team involves; for new managers, having an exhaustive list of job responsibilities is helpful to make sure you’re tracking all of them.

More importantly, though, we often want to somehow split these responsibilities between people. Team leadership covers a huge array of things—as you can see from how long this post is—and trying to find someone who can be great at all of them is often a unicorn hunt. Even if you do find someone good-enough at all of them, they [...]

---

Outline:

(01:04) Categories

(01:07) Overall direction

(02:50) People management

(05:03) Project management

(07:13) Technical leadership

(09:00) Example divisions of responsibility

(09:11) The “tech lead manager”

(10:23) Engineering manager / tech lead

(11:20) Product manager / tech lead

(13:15) People manager / research lead

---

First published:

Source:

Narrated by TYPE III AUDIO.

A high production value 16-minute video that summarizes the popular safety concerns, featuring Hinton, Russell and Claude 3.5.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This article contains more than fifty uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.