Today we are publishing a significant update to our Responsible Scaling Policy (RSP), the risk governance framework we use to mitigate potential catastrophic risks from frontier AI systems. This update introduces a more flexible and nuanced approach to assessing and managing AI risks while maintaining our commitment not to train or deploy models unless we have implemented adequate safeguards. Key improvements include new capability thresholds to indicate when we will upgrade our safeguards, refined processes for evaluating model capabilities and the adequacy of our safeguards (inspired by safety case methodologies), and new measures for internal governance and external input. By learning from our implementation experiences and drawing on risk management practices used in other high-consequence industries, we aim to better prepare for the rapid pace of AI advancement.

The promise and challenge of advanced AI

As frontier AI models advance, they have the potential to bring about [...]

---

Outline:

(00:58) The promise and challenge of advanced AI

(02:30) A framework for proportional safeguards

(05:12) Implementation and oversight

(06:19) Learning from experience

(08:07) Looking ahead

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Audio note: this article contains 65 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Suppose two Bayesian agents are presented with the same spreadsheet - IID samples of data in each row, a feature in each column. Each agent develops a generative model of the data distribution. We'll assume the two converge to the same predictive distribution, but may have different generative models containing different latent variables. We'll also assume that the two agents develop their models independently, i.e. their models and latents don't have anything to do with each other informationally except via the data. Under what conditions can a latent variable in one agent's model be faithfully expressed in terms of the other agent's latents?

Let's put some math on that question.

The n “features” in the data are random [...]

---

Outline:

(02:37) The Main Argument

(05:27) Approximation

(06:47) Why Is This Interesting?

The original text contained 1 footnote which was omitted from this narration.

The original text contained 5 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A few months ago, I wrote a post about using slower computing substrates as a possibly new way to safely train and align ASI.

If you haven't read that post, basically the idea is that if we consider compute speed as a factor in Total Intelligence (alongside say, quality of intelligence), then it should be possible to keep quality the same and lower compute speed in order to lower Total Intelligence while keeping quality the same.

An intuition pump is to imagine a scenario where we are able to slow down Einstein's brain, by slowing actual biochemical and electrical processes, so that it produces the Theory of Relativity in 40 years instead of 10.

The obvious reason to do this would be to gain some degree of controllability so that a sharp left turn is less likely. However, a more compelling argument for this strategy is that it [...]

---

Outline:

(01:22) Running the first tests

(03:02) Challenges

(04:21) Opportunity

(05:00) Enter TextStarCraft II

(06:10) Results

(10:21) Observations and Discussion

(12:36) Where do we go from here?

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What if you could remove some information from the weights of an AI? Would that be helpful?

It is clearly useful against some misuse concerns: if you are concerned that LLMs will make it easier to build bioweapons because they have memorized such information, removing the memorized facts would remove this misuse concern.

In a paper Aghyad Deeb and I just released, we show it is tractable to evaluate the presence of certain undesirable facts in an LLM: take independent facts that should have all been removed, fine-tune on some of them, and see if accuracy increases on the other ones. The fine-tuning process should make the model “try” to answer, but if the information was removed from the weights (and if the facts are actually independent), then accuracy on the held-out facts should remain low.

Removing information from the weights is stronger than the usual notion of [...]

---

Outline:

(01:50) Do current unlearning techniques remove facts from model weights?

(04:24) Hopes for successful information removal

(06:51) Using information removal to reduce x-risk

(06:56) Information you should probably remove from the weights

(08:20) How removing information helps you

(09:20) Information you probably can’t remove - and why this won’t work for superintelligent AIs

The original text contained 5 footnotes which were omitted from this narration.

The original text contained 2 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 259 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Work done at Apollo Research. The bottom half of this post is just maths that you do not need to read to get the gist — the estimated reading time is misleading!

Tl;dr: We generalize the mathematical framework for computation in superposition from compressing many boolean logic gates into a neural network, to compressing many small neural networks into a larger neural network. The number of small networks we can fit into the large network depends on the small networks' total parameter count, not their neuron count.

Introduction

Background

Anthropic's toy model of superposition shows how to compress many sparsely activating variables into a low dimensional vector space and then read them out again. But it doesn't show [...]

---

Outline:

(00:53) Introduction

(00:57) Background

(01:25) What we do

(03:07) Generalising to circuits

(03:34) Some very tentative implications, maybe?

(06:17) Future work

(07:27) The Construction

(09:25) Read-in interference

(11:44) Maths

(13:01) Embedding Matrix

(13:37) Other layers

(13:55) Reading from the residual stream

(14:51) Writing to the neurons

(17:37) Writing back to the residual stream

(18:22) Error analysis

(19:16) Read-in interference

The original text contained 9 footnotes which were omitted from this narration.

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Stop AI just put out a press release, summarising our plan to barricade OpenAI's office entrances.

As an organiser, let me add some thoughts to nuance the text:

Plan

On October 21st, 2024 at 12:00 pm Stop AI will peacefully barricade, via sit-in, OpenAI's office entrance gate

The emphasis is on peacefully. We are a non-violent activism organisation, and always will be.

We could very easily stop the development of Artificial General Intelligence if a small group of people repeatedly barricaded entrances at AI company offices and data centers.

My take is that a small group barricading OpenAI is a doable way to be a thorn in OpenAI's side, while raising public attention to the recklessness of AI corporations. From there, stopping AI development requires many concerned communities acting together to restrict the data, work, uses, and hardware of AI.

We will [...]

---

Outline:

(00:16) Plan

(03:04) Risk of extinction

(10:23) Why restrict OpenAI

---

First published:

Source:

Narrated by TYPE III AUDIO.

The standard psychological questionnaire for depression doctors have given me is the PHQ-9. It names a symptom, and for each symptom it gives four possible responses in severity. The responses are worth points, the points are totaled, and the final score is supposed to be indicative of how severe a person's depression is.

This is what it looks like:

According to Wikipedia, the final score indicates the following:

To be perfectly blunt, while I suppose this functions for doctors to get an idea of where their patient is at, it's absolute garbage for giving the patient a sense of how depressed they are. Knowing how many times in the last two weeks you’ve felt ‘down, depressed, and hopeless’ isn’t super helpful when you’re trying to understand - or worse, explain to someone else - what it's like in your head.

To that end I’ve been slowly developing [...]

---

Outline:

(01:22) The Scale

(03:17) 0-10%

(04:17) 10-20%

(05:38) 20-30%

(06:33) 30-40%

(07:52) 40-50%

(09:04) 50-60%

(10:13) 60-70%

(11:22) 70-80%

(12:42) 80-90%

(14:03) 90-100%

(16:03) Conclusion

The original text contained 3 images which were described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

---

First published:

Source:

Narrated by TYPE III AUDIO.

As humanity gets closer to Artificial General Intelligence (AGI), a new geopolitical strategy is gaining traction in US and allied circles, in the NatSec, AI safety and tech communities. Anthropic CEO Dario Amodei and RAND Corporation call it the “entente”, while others privately refer to it as “hegemony" or “crush China”. I will argue that, irrespective of one's ethical or geopolitical preferences, it is fundamentally flawed and against US national security interests.

If the US fights China in an AGI race, the only winners will be machines

The entente strategy

Amodei articulates key elements of this strategy as follows:

"a coalition of democracies seeks to gain a clear advantage (even just a temporary one) on powerful AI by securing its supply chain, scaling quickly, and blocking or delaying adversaries’ access to key resources like chips and semiconductor equipment. This coalition would on one hand use AI to achieve robust [...]

---

Outline:

(00:51) The entente strategy

(02:22) Why it's a suicide race

(09:19) Loss-of-control

(11:32) A better strategy: tool AI

The original text contained 1 image which was described by AI.

---

First published:

Source:

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A man has just robbed a train. A Rock Island Rail car was held up in the desert of North Texas on route from Chicago carrying dozens of passengers and tens of thousands of dollars in a Wells Fargo express compartment. The men were slain and the women and children are kidnapped.

You’re the sheriff in charge of the investigation. In a case like this, tried and true procedure is to put a bounty on the man's head. That gets everyone interested in looking and bringing this man to justice. An everyday 2-bit robber might get a bounty of a thousand or ten, but this is an emergency. You set the bounty at $200,000, dead or alive.

This bandit ain’t getting away easy.

With two major hurricanes in the last couple of weeks, “price gouging” is in the news. Whether it's $10 for a gallon [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.