It's interesting to me how chill people sometimes are about the non-extinction future AI scenarios. Like, there seem to be opinions around along the lines of “pshaw, it might ruin your little sources of ‘meaning’, Luddite, but we have always had change and as long as the machines are pretty near the mark on rewiring your brain it will make everything amazing”. Yet I would bet that even that person, if faced instead with a policy that was going to forcibly relocate them to New York City, would be quite indignant, and want a lot of guarantees about the preservation of various very specific things they care about in life, and not be just like “oh sure, NYC has higher GDP/capita than my current city, sounds good”.

I read this as a lack of engaging with the situation as real. But possibly my sense that a non-negligible number [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Modern AI is trained on a huge fraction of the internet, especially at the cutting edge, with the best models trained on close to all the high quality data we’ve got.[1] And data is really important! You can scale up compute, you can make algorithms more efficient, or you can add infrastructure around a model to make it more useful, but on the margin, great datasets are king. And, naively, we’re about to run out of fresh data to use.

It's rumored that the top firms are looking for ways to get around the data wall. One possible approach is having LLMs create their own data to train on, for which there is kinda-sorta a precedent from, e.g. modern chess AIs learning by playing games against themselves.[2] Or just finding ways to make AI dramatically more sample efficient with the data we’ve already got: the [...]

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Have you heard this before? In clinical trials, medicines have to be compared to a placebo to separate the effect of the medicine from the psychological effect of taking the drug. The patient's belief in the power of the medicine has a strong effect on its own. In fact, for some drugs such as antidepressants, the psychological effect of taking a pill is larger than the effect of the drug. It may even be worth it to give a patient an ineffective medicine just to benefit from the placebo effect. This is the conventional wisdom that I took for granted until recently.

I no longer believe any of it, and the short answer as to why is that big meta-analysis on the placebo effect. That meta-analysis collected all the studies they could find that did "direct" measurements of the placebo effect. In addition to a placebo group that could [...]

---

Outline:

(01:58) Examples: Depression

(08:08) Example: The common cold

(11:40) The status of the placebo effect in science

---

First published:

Source:

Narrated by TYPE III AUDIO.

In past few years I've been blogging in Slovak, that is, downscaling from writing in English, a language with 1457 million speakers to a language with 7 million speakers.

From the point of view of the writer, this has been a very different experience. It's not only that for a topic that interests one million English speakers, the equivalent is five thousand in Slovakia, scaling down by factor of 200. It's also that topic that interests 100 English speakers, interests one half of a hypothetical Slovak speaker. In fact, not everybody reads blogs, so the population in question is likely smaller by an order of magnitude or even two, resulting in even more fractional readers... In other words, the reading population is not as big as to fill in all the possible niches and the writing thus has to become much more generic.

It must also be [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Understanding the alignment problem can be challenging, especially with its complex theoretical underpinnings and technical jargon. To make this crucial topic more accessible, I've created a comic that breaks down the key concepts in a simple, intuitive, and engaging way. Whether you're new to the subject or looking for a fresh perspective, this visual guide aims to clarify why alignment is so important and what it means for the future of AI. Dive in and explore the alignment problem through a story that's both informative and easy to grasp.

(Not AI generated art obviously)

I hope you enjoyed this brief overview. For the full comic visit:
https://milanrosko.substack.com/p/button

---

First published:

Source:

Narrated by TYPE III AUDIO.

(Part 2 of the CAST sequence)

As a reminder, here's how I’ve been defining “corrigible” when introducing the concept: an agent is corrigible when it robustly acts opposite of the trope of "be careful what you wish for" by cautiously reflecting on itself as a flawed tool and focusing on empowering the principal to fix its flaws and mistakes.

This definition is vague, imprecise, and hides a lot of nuance. What do we mean by “flaws,” for example? Even the parts that may seem most solid, such as the notion of there being a principal and an agent, may seem philosophically confused to a sufficiently advanced mind. We’ll get into trying to precisely formalize corrigibility later on, but part of the point of corrigibility is to work even when it's only loosely understood. I’m more interested [...]

---

Outline:

(04:00) Emergent Desiderata

(04:04) Communication

(05:01) Low-Impact

(05:32) Reversibility

(06:31) Efficiency

(07:17) Relevance

(07:56) Transparency

(08:21) Obedience

(09:19) Mild-Optimization

(10:07) Protectiveness

(10:57) Local Scope

(11:32) Simple Self-Protectiveness

(11:50) Stop Button

(12:53) Graceful Shutdown

(13:25) Configurable Verbosity

(14:27) Disambiguation/Concreteness

(15:17) Honesty

(16:02) Handling Antagonists

(17:20) Straightforwardness

(18:03) Proactive Reflection

(18:44) Cognitive Legibility

(19:53) Infohazard Caution

(20:42) Resource Accumulation

(21:46) Non-Manipulation

(23:25) Sub-Agent Stability

(24:22) Principal-Looping

(25:01) Graceful Obsolescence

(26:00) Handling Trolley-Tradeoffs

(26:46) Handling Time-Pressure

(28:32) Expandable Concerns

(29:21) Navigating Conflict

(29:32) Simple Conflict

(30:15) Violent Conflict

(30:45) Authority Conflict

(31:18) Shutdown Conflict

(31:55) Emergent Downsides

(31:59) Intrusiveness

(32:45) Indifference

(33:31) Rigidity

(34:17) Immorality

(34:52) Irresponsibility

(35:28) Myopia

(36:02) Incorrigible Counter-Examples

(36:24) Honesty

(36:41) Protectiveness

(37:05) Proactive Benevolence

(37:22) Kindness

(37:40) Human-In-Loop

(38:02) Moral Learning

(38:14) Balancing Needs

(38:37) Broad Perspective

(38:55) Top-Level-Goal Focus

(39:13) Nearby Concepts that Aren’t Synonyms for Corrigible

(39:43) Correctability

(41:58) “The Thing Frontier Labs Are Currently Aiming For”

(44:32) Preference Satisfaction

(48:07) Empowerment (in general)

(50:20) Caution

(51:47) Servility

(54:30) Tool/Task-ishness

(56:21) Discussion

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

People talk about model weight security being really hard and crucial around the advent of AGI. (E.g. RAND report, Leopold; see here for some distinctions in these thread models that I think are important.) But I think that the thinking on this has not been sufficiently attentive to the fact that during that crucial time period, by assumption we’ll have access to powerful AIs. I think that such access might make security wildly easier, by a bunch of different mechanisms, some of which I’ll describe in this post.

The story I’m telling here is pretty different from the main story I've heard people talk about in the past about AIs helping with computer security, which is that AIs can help with hardening software and other infrastructure. Though I agree that that seems really useful and important, here [...]

---

Outline:

(02:41) Four strategies for using powerful AI to improve security

(02:46) Monitoring

(03:49) Trust displacement

(04:53) Fine-grained permission management

(06:13) AI investigation of automatically detected suspicious activity

(06:37) How vulnerable to jailbreaks or trickery is this?

(08:51) These techniques seem really powerful

---

First published:

Source:

Narrated by TYPE III AUDIO.

(Part 1 of the CAST sequence)

AI Risk Introduction

(TLDR for this section, since it's 101 stuff that many readers will have already grokked: Misuse vs Mistake; Principal-Agent problem; Omohundro Drives; we need deep safety measures in addition to mundane methods. Jump to “Sleepy-Bot” if all that seems familiar.)

Earth is in peril. Humanity is on the verge of building machines capable of intelligent action that outstrips our collective wisdom. These superintelligent artificial general intelligences (“AGIs”) are almost certain to radically transform the world, perhaps very quickly, and likely in ways that we consider catastrophic, such as driving humanity to extinction. During this pivotal period, our peril manifests in two forms.

The most obvious peril is that of misuse. An AGI which is built to serve the interests of one person or party, such as jihadists or [...]

---

Outline:

(00:10) AI Risk Introduction

(05:53) Aside: Sleepy-Bot

(07:46) The Corrigibility-As-Singular-Target Strategy

(12:58) How Can We Get Corrigibility?

(17:32) What Makes Corrigibility Special

(28:08) Contra Impure or Emergent Corrigibility

(31:01) How to do a Pivotal Act

(34:47) Cruxes and Counterpoints

(36:47) “Anti-Naturality” and Hardness

(40:37) Prosaic Methods Make Anti-Naturality Worse

(41:30) Solving Anti-Naturality at the Architectural Layer

(42:52) Aside: Natural Concepts vs Antinatural Properties

(43:46) The Effect Size of Anti-Naturality is Unclear

(45:32) “Corrigibility Isn’t Actually a Coherent Concept”

(47:07) “CAST is More Complex than Diamond, and We Can’t Even Do That”

(50:04) “General Intelligence Demands Consequentialism”

(53:11) Desiderata Lists vs Single Unifying Principle

(56:58) “Human-In-The-Loop Can’t Scale”

(59:17) Identifying the Principal is Brittle

(01:02:56) “Reinforcement Learning Only Creates Thespians”

(01:05:47) “Largely-Corrigible AGI is Still Lethal in Practice”

The original text contained 16 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

What the heck is up with “corrigibility”? For most of my career, I had a sense that it was a grab-bag of properties that seemed nice in theory but hard to get in practice, perhaps due to being incompatible with agency.

Then, last year, I spent some time revisiting my perspective, and I concluded that I had been deeply confused by what corrigibility even was. I now think that corrigibility is a single, intuitive property, which people can learn to emulate without too much work and which is deeply compatible with agency. Furthermore, I expect that even with prosaic training methods, there's some chance of winding up with an AI agent that's inclined to become more corrigible over time, rather than less (as long as the people who built it understand corrigibility and want that agent [...]

---

Outline:

(07:30) Overview

(07:33) 1. The CAST Strategy

(08:15) 2. Corrigibility Intuition (Coming Saturday)

(08:49) 3a. Towards Formal Corrigibility (Coming Sunday)

(09:27) 3. Formal (Faux) Corrigibility ← the mathy one (Also Sunday)

(10:12) 4. Existing Writing on Corrigibility (Coming Monday)

(10:33) 5. Open Corrigibility Questions (Also Monday)

(10:58) Bibliography and Miscellany

---

First published:

Source:

Narrated by TYPE III AUDIO.

This article contains more than fifty uses of logical or mathematical notation, so an audio narration would be too hard to follow. You'll find a link to the original text in the episode description.

---

First published:

Source:

Narrated by TYPE III AUDIO.