Disclaimer: This is highly incomplete. I am not an expert in the field. There might be some unfamiliar terms. While I will try to explain things, explaining every single term would be beyond this post. You will usually be able to get a sufficient understanding by clicking the links or googling it.

I think everyone, if they read about the chip industry long enough, has a moment where they have to put down a book or pause a podcast and simply remain stunned at the fact that it is possible to design and build something that is so incredibly impressive.

The Apple A17 chip contains 183 million transistors per square millimeter. All placed in a coherent manner and produced with extremely high reliability.

This is exactly why it is so fascinating to learn more about how it is actually done. On [...]

---

Outline:

(00:25) Introduction

(01:16) Background Knowledge

(03:43) The Design Process

(03:52) Definition and Planning

(04:18) Design and Verification

(05:40) Logic Synthesis

(06:54) Physical Design

(07:57) Signoff and Tapeout

(08:28) Takeaways

(09:16) Appendix

(09:19) 1 Verilog example

(09:30) 2 Gate-Level Netlist example

The original text contained 7 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Goal of This Post

I have never seen anyone give a satisfying intuitive explanation of what corrigibility (in roughly Eliezer's sense of the word) is. There's lists of desiderata, but they sound like scattered wishlists which don’t obviously point to a unified underlying concept at all. There's also Eliezer's extremely meta pointer:

We can imagine, e.g., the AI imagining itself building a sub-AI while being prone to various sorts of errors, asking how it (the AI) would want the sub-AI to behave in those cases, and learning heuristics that would generalize well to how we would want the AI to behave if it suddenly gained a lot of capability or was considering deceiving its programmers and so on.

… and that's basically it.[1]

In this post, we’re going to explain a reasonably-unified concept which seems like a decent match to “corrigibility” in Eliezer's sense.

Tools

Starting point: we think [...]

---

Outline:

(00:05) Goal of This Post

(01:03) Tools

(02:40) Respecting Modularity

(04:49) Visibility and Correctability

(06:00) Let's Go Through A List Of Desiderata

(11:53) What Would It Look Like To Use A Powerful Corrigible AGI?

(14:11) From Cognition to Real Patterns?

The original text contained 6 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

It is known that many birds are able to sense the direction of Earth's magnetic field. Here's a wikipedia page on that general phenomenon. There have been 2 main theories of how that works.

One theory is that birds have magnets in their beak that act like a compass. We know this is the correct theory because:

The other theory is that birds have some sensing mechanism in their eyes that uses magneto-optical effects. We know this is the correct theory because:

We also know those theories probably aren't [...]

---

Outline:

(00:08) introduction

(01:15) organic magneto-optics

(05:55) the problem

(06:34) magnetite

(07:31) compass

(08:30) two systems

---

First published:

Source:

Narrated by TYPE III AUDIO.

Childhood roundup #5 excluded all developments around college. So this time around is all about issues related to college or graduate school, including admissions.

Tuition and Costs

What went wrong with federal student loans? Exactly what you would expect when you don’t check who is a good credit risk. From a performance perspective, the federal government offered loans to often-unqualified students to attend poor-performing, low-value institutions. Those students then did not earn much and were often unable to repay the loans. The students are victims here too, as we told them to do it.

Alas, none of the proposed student loan solutions involve fixing the underlying issue. If you said ‘we are sorry we pushed these loans on students and rewarded programs and institutions that do not deserve it, and we are going to stop giving loans for those programs and institutions and offer help to [...]

---

Outline:

(00:18) Tuition and Costs

(03:07) What Your Tuition Buys You

(05:45) Decline of Academia

(07:24) Grading and Stress

(14:29) Lower Standards

(15:43) Degree Value

(19:39) Shifting Consumer Preferences

(21:01) Standardized Tests in College Admissions

(21:44) Discrimination in College Admissions

(25:18) Required Classes and Choosing Your Major

(28:31) Everything I Need To Know That Waited Until Graduate School

(31:19) When You See Fraud Say Fraud

(33:03) Free Speech

(35:28) Harvard Goes Mission First

(37:03) The Waterloo Model

(38:33) DEI

(45:20) In Other News

---

First published:

Source:

Narrated by TYPE III AUDIO.

A general thing I hadn’t noticed about debts until lately:

These might be tiny incentives, and not overwhelm for instance Bob's many reasons for not wanting Alice to disappear.

But the bigger the owing, the more relevant the incentives. When big enough, the former comes up as entities being “too big to fail”, and potentially rescued from destruction by those who would like them to repay or provide something expected of them in future. But the opposite must exist also: too big to succeed—where the abundance owed to you is so off-putting to provide that those responsible for it would [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This represents work from several people at the NAO. Thanks especially to Dan Rice for implementing the duplicate junction detection, and to @Will Bradshaw and @mike_mclaren for editorial feedback.

If someone were to intentionally cause a stealth pandemic today, one of the ways they might do it is by modifying an existing virus. Over the past few months we’ve been working on building a computational pipeline that could flag evidence of this kind of genetic engineering, and we now have an initial pipeline working end to end. When given 35B read pairs of wastewater sequencing data it raises 14 alerts for manual review, 13 of which are quickly dismissible false positives and one is a known genetically engineered sequence derived from HIV. While it's hard to get a good estimate before actually going and doing it, our best guess is that [...]

---

Outline:

(00:22) Summary

(01:14) System Design

(02:34) Evaluation

(02:49) Simulation

(05:27) Real World Evaluation

(08:28) System Sensitivity

(11:32) Future Work

---

First published:

Source:

Narrated by TYPE III AUDIO.

I've been thinking about memetically fit Schelling points in the AGI policy space. I'll describe four such "Schelling policies", and use them as pedagogical examples.

Shut it all down

MIRI's new stated objective is the clearest example of a Schelling policy: "Shut it all down". MIRI states that they want governments to coordinate to pause all AI research that involves smarter-than-human systems. Laypeople will find this policy easy to understand, since they can rely on the shared cultural knowledge of CFC bans and international nuclear disarmament as case studies.

If you want to coordinate a large number of people coherently towards furthering a particular policy, "you get about five words" that you can make 'common knowledge' such that people can coordinate in a specific direction. The ease of communicating the policy makes a big difference in such conditions.

When you attempt to communicate an idea [...]

---

Outline:

(00:17) Shut it all down

(02:28) Effective Accelerationism (e/acc)

(04:58) We must beat China

(06:58) Only One Org

(09:36) Parting words

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Acknowledgements

The vision here was midwifed originally in the wild and gentle radiance that is Abram's company (though essentially none of the content is explicitly his).

The PIBBSS-spirit has been infused in this work from before it began (may it infuse us all), as have meetings with the Agent Foundations team at MIRI over the past ~2 years.

More recently, everyone who has been loving the High Actuation project into form (very often spontaneously and without being encumbered by self-consciousness of this fact):[1] individuals include Steve Petersen, Mateusz Baginski, Aditya Prasad, Harmony, TJ, Chris Lakin; the AISC 2024 team, Murray Buchanan, Matt Farr, Arpan Agrawal, Adam, Ryan, Quinn; various people from Topos, ALIFE, MAPLE, EA Bangalore. Published while at CEEALAR.

Disclaimers

• Very occasionally there are small remarks/questions from a remarkable human named Steve, since this and the next two posts are an edited transcript of me [...]

---

Outline:

(00:05) Acknowledgements

(01:01) Disclaimers

(01:40) Taking Intelligence Seriously

(03:04) Failing to Take Intelligence Seriously

(05:02) Opportunities from Moderate Intelligence at Scale

(09:14) Live Interfaces

(12:57) Live Theory (preface)

(14:33) Navigation

(17:02) Live Theory (teaser)

(17:06) Possibility

The original text contained 14 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

The progress movement has grown a lot in the last few years. We now have progress journals, think tanks, and fellowships. The progress idea has spread and evolved into the “abundance agenda”, “techno-optimism”, “supply-side progressivism”, “American dynamism”. All of us want to see more scientific, technological, and economic progress for the good of humanity, and envision a bold, ambitious, flourishing future.

What we haven’t had so far is a regular gathering of the community.

Announcing Progress Conference 2024, a two-day event to connect people in the progress movement. Meet great people, share ideas in deep conversations, catalyze new projects, get energized and inspired.

Hosted by: the Roots of Progress Institute, together with the Foresight Institute, HumanProgress.org, the Institute for Humane Studies, the Institute for Progress, and Works in Progress magazine

When: October 18–19, 2024

Where: Berkeley, CA—at the Lighthaven campus, an inviting space perfect [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

Throughout this post, we’re going to follow the Cognition -> Convergence -> Consequences methodology[1]. That means we’ll tackle tool-ness in three main stages, each building on the previous:

• Cognition: What does it mean, cognitively, to view or model something as a tool?
• Convergence: Insofar as different minds (e.g. different humans) tend to convergently model the same things as tools, what are the “real patterns” in the environment which give rise to that convergence?
• Consequences: Having characterized the real patterns convergently recognized as tool-ness, what other properties or consequences of tool-ness can we derive? What further predictions does our characterization make?

We’re not going to do any math in this post, though we will gesture at the spots where proofs or quantitative checks would ideally slot in.

Cognition: What does it mean, cognitively, to view or model something as a tool?

Let's start with a mental model of (the cognition [...]

---

Outline:

(01:02) Cognition: What does it mean, cognitively, to view or model something as a tool?

(03:27) Convergence: Insofar as different minds (e.g. different humans) tend to convergently model the same things as tools, what are the “real patterns” in the environment which give rise to that convergence?

(06:12) Consequences: From that characterization, what other properties or consequences of tool-ness can we derive?

(06:26) Modularity

(08:34) Loopiness

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.