In a new preprint, Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models, my coauthors and I introduce a technique, Sparse Human-Interpretable Feature Trimming (SHIFT), which I think is the strongest proof-of-concept yet for applying AI interpretability to existential risk reduction.[1] In this post, I will explain how SHIFT fits into a broader agenda for what I call cognition-based oversight. In brief, cognition-based oversight aims to evaluate models according to whether they’re performing intended cognition, instead of whether they have intended input/output behavior.

In the rest of this post I will:

1. Articulate a class of approaches to scalable oversight I call cognition-based oversight.
2. Narrow in on a model problem in cognition-based oversight called Discriminating Behaviorally Identical Classifiers (DBIC). DBIC is formulated to be a concrete problem which I think captures most of the technical difficulty [...]

---

Outline:

(01:41) Cognition-based oversight

(02:10) Discriminating models: a simplification of scalable oversight

(05:20) Cognition-based oversight for discriminating behaviorally identical models

(07:02) Discriminating Behaviorally Identical Classifiers

(11:51) Hard vs. Relaxed DBIC

(13:48) SHIFT as a technique for (hard) DBIC

(19:09) Limitations and next steps

(21:13) Direction 1: better ways to understand interpretable units in deep networks

(23:02) Direction 2: identifying especially leveraged settings for cognition-based oversight

(24:47) Conclusion

The original text contained 8 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I just finished reading The Mom Test for the second time. I took raw notes here. In this post I'll first write up a bullet-point summary and then ramble off some thoughts that I have.

Introduction:

• Trying to learn from customer conversations is like trying to excavate a delicate archeological site. The truth is down there somewhere, but it's fragile. When you dig you get closer to the truth, but you also risk damaging or smashing it.
• Bad customer conversations are worse than useless because they mislead you, convincing you that you're on the right path when instead you're on the wrong path.
• People talk to customers all the time, but they still end up building the wrong things. How is this possible? Almost no one talks to customers correctly.
• Why another book about this? Why this author?
  • Rob is a techie, not a sales [...]

---

Outline:

(10:42) Thoughts

(10:45) Hierarchy of evidence

(11:49) Genuinely helpful takeaways

(12:06) Ask for intros

(14:07) Ask what theyve tried

(17:36) Things on my radar

(17:40) Organize an event

(18:35) Half reminders, half learnings

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

For this iteration I will exclude discussions involving college or college admissions.

There has been a lot of that since the last time I did one of these, along with much that I need to be careful with lest I go out of my intended scope. It makes sense to do that as its own treatment another day.

Bullying

Why do those who defend themselves against bullies so often get in more trouble than bullies? This is also true in other contexts but especially true in school. Thread is extensive, these are the highlights translated into my perspective. A lot of it is that a bully has experience and practice, they know how to work the system, they know what will cause a response, and they are picking the time and place to do something. The victim has to respond in the moment, and by responding [...]

---

Outline:

(00:22) Bullying

(02:34) Truancy

(03:56) Against Active Shooter Drills

(05:14) Censorship

(06:32) Woke Kindergarden

(09:01) Tracking

(10:43) The Case Against Education

(13:56) Home Schooling

(14:54) Despair

(17:02) Goals

(19:02) Taking the Developing World to School

(26:05) Primary School

(27:46) Guessing the Teacher's Password

(28:26) Correcting the Teacher's Incentives

(29:58) Mathematics

(31:08) Let Kids Be Kids

(32:27) Mandatory Work We Encourage

(34:03) Mandatory Work We Discourage

(35:32) Air Quality Matters

(36:18) School Choice

(39:01) Full Access to Smartphones Is Not Good For Children

(47:17) Lifetime Learning

---

First published:

Source:

Narrated by TYPE III AUDIO.

Posted on Twitter:

• Opus can operate as a Turing machine. 
• given only existing tapes, it learns the rules and computes new sequences correctly. 
• 100% accurate over 500+ 24-step solutions (more tests running). 
• for 100% at 24 steps, the input tapes weigh 30k tokens*. 
• GPT-4 cannot do this.

Here is the prompt code for the Turing machine: https://github.com/SpellcraftAI/turing

This is the fully general counterpoint to the @VictorTaelin's A::B challenge (he put money where his mouth is and got praise for that from Yudkowsky).

Attention is Turing Complete was a claim already in 2021:

Theorem 6 The class of Transformer networks with positional encodings is Turing complete. Moreover, Turing completeness holds even in the restricted setting in which the only non-constant values in positional embedding pos(n) of n, for n ∈ N, are n, 1/n, and 1/n2 , and Transformer networks have a single [...]

---

First published:

Source:

Linkpost URL:
https://twitter.com/ctjlewis/status/1779740038852690393

Narrated by TYPE III AUDIO.

TLDR: I am investigating whether to found a spiritual successor to FHI, housed under Lightcone Infrastructure, providing a rich cultural environment and financial support to researchers and entrepreneurs in the intellectual tradition of the Future of Humanity Institute. Fill out this form or comment below to express interest in being involved either as a researcher, entrepreneurial founder-type, or funder.

The Future of Humanity Institute is dead:

I knew that this was going to happen in some form or another for a year or two, having heard through the grapevine and private conversations of FHI's university-imposed hiring freeze and fundraising block, and so I have been thinking about how to best fill the hole in the world that FHI left behind.

I think FHI was one of the best intellectual institutions in history. Many of the most important concepts[1] in my intellectual vocabulary were developed and popularized under its [...]

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Yesterday Adam Shai put up a cool post which… well, take a look at the visual:

Yup, it sure looks like that fractal is very noisily embedded in the residual activations of a neural net trained on a toy problem. Linearly embedded, no less.

I (John) initially misunderstood what was going on in that post, but some back-and-forth with Adam convinced me that it really is as cool as that visual makes it look, and arguably even cooler. So David and I wrote up this post / some code, partly as an explainer for why on earth that fractal would show up, and partly as an explainer for the possibilities this work potentially opens up for interpretability.

One sentence summary: when tracking the hidden state of a hidden Markov model, a Bayesian's beliefs follow a chaos game (with the observations randomly selecting the update at each time), so [...]

---

Outline:

(01:11) Background: Fractals and Symmetry

(04:02) Background: Chaos Games

(05:58) Bayesian Belief States For A Hidden Markov Model

(09:35) Why Would That Show Up In A Neural Net?

(10:13) Why This Sort Of Thing Might Be A Pretty Big Deal

---

First published:

Source:

Narrated by TYPE III AUDIO.

On May 25th, 2023, someone posted a review of How Minds Change on LessWrong. It talked about Street Epistemology, Deep Canvassing, and Smart Politics, ways of handling disagreements that open the possibility of rational belief progression through amicable discussions. Summarized quickly, they rely on active listening, sharing personal stories and socratic questioning.

You can now learn all of those three techniques online, for free, in 4 hours, and in a Deliberate Practice setting. If interested, you can also learn them in an in-person workshop spanning anytime between 2 hours and a full weekend -just shoot me an email with the object EHD (at the time of writing, I’m based in Paris, France).

You can enroll on the website (see bottom for subsrcibing to the mailing list), and join the discord server.

About the workshop:

What would you learn?

When you find yourself in disagreement with someone on a significant [...]

---

Outline:

(00:59) About the workshop:

(04:14) Very long FAQ:

(06:45) Meta

---

First published:

Source:

Narrated by TYPE III AUDIO.

After 7 years at Deep End (and 4 more years in other group houses before that), Janos and I have moved out to live near a school we like and some lovely parks. The life change is bittersweet - we will miss living with our friends, but also look forward to a logistically simpler life with our kids. Looking back, here are some thoughts on what worked and didn't work well about living in a group house with kids.

Pros. There were many things that we enjoyed about living at Deep End, and for a long time I couldn't imagine ever wanting to leave. We had a low-effort social life - it was great to have spontaneous conversations with friends without arranging to meet up. This was especially convenient for us as new parents, when it was harder to make plans and get out [...]

---

First published:

Source:

Linkpost URL:
https://vkrakovna.wordpress.com/2024/04/17/moving-on-from-community-living/

Narrated by TYPE III AUDIO.

---

First published:

Source:

Linkpost URL:
https://www.futureofhumanityinstitute.org/

Narrated by TYPE III AUDIO.

xlr8harder writes:

In general I don’t think an uploaded mind is you, but rather a copy. But one thought experiment makes me question this. A Ship of Theseus concept where individual neurons are replaced one at a time with a nanotechnological functional equivalent.

Are you still you?

Presumably the question xlr8harder cares about here isn't semantic question of how linguistic communities use the word "you", or predictions about how whole-brain emulation tech might change the way we use pronouns.

Rather, I assume xlr8harder cares about more substantive questions like:

1. If I expect to be uploaded tomorrow, should I care about the upload in the same ways (and to the same degree) that I care about my future biological self?
2. Should I anticipate experiencing what my upload experiences?
3. If the scanning and uploading process requires destroying my biological brain, should I say yes to the procedure?

My answers:

1. ---

   Outline:

   (05:02) Why Humans Feel Like They Persist

   (10:12) Sleep and Film Reels

   (14:17) Weird-Futuristic-Technology Anxiety

   (16:20) To Change Experience, You Have to Change Physics, Not Just Metaphysics

   (18:35) ... And You Cant Change Experience With Just Any Old Change to Physics

   (22:59) Having Multiple Futures

   The original text contained 1 footnote which was omitted from this narration.

   ---

First published:

Source:

Narrated by TYPE III AUDIO.