He was 90 years old.

His death was confirmed by his stepdaughter Deborah Treisman, the fiction editor for the New Yorker. She did not say where or how he died.

---

First published:

Source:

Linkpost URL:
https://www.washingtonpost.com/obituaries/2024/03/27/daniel-kahneman-dead/

Narrated by TYPE III AUDIO.

Description:

A greyhound catching the mechanical lure—what would he actually do with it? Has he given this any thought?

Bostrom's previous book, Superintelligence: Paths, Dangers, Strategies changed the global conversation on AI and became a New York Times bestseller. It focused on what might happen if AI development goes wrong. But what if things go right?

Suppose that we develop superintelligence safely, govern it well, and make good use of the cornucopian wealth and near magical technological powers that this technology can unlock. If this transition to the machine intelligence era goes well, human labor becomes obsolete. We would thus enter a condition of "post-instrumentality", in which our efforts are not needed for any practical purpose. Furthermore, at technological maturity, human nature becomes entirely malleable.

Here we confront a challenge that is not technological but philosophical and spiritual. In such a solved world, what [...]

---

First published:

Source:

Linkpost URL:
https://nickbostrom.com/deep-utopia/

Narrated by TYPE III AUDIO.

Quick update from AE Studio: last week, Judd (AE's CEO) hosted a panel at SXSW with Anil Seth, Allison Duettmann, and Michael Graziano, entitled “The Path to Conscious AI” (discussion summary here[1]).

We’re also making available an unedited Otter transcript/recording for those who might want to read along or increase the speed of the playback.

Why AI consciousness research seems critical to us

With the release of each new frontier model seems to follow a cascade of questions probing whether or not the model is conscious in training and/or deployment. We suspect that these questions will only grow in number and volume as these models exhibit increasingly sophisticated cognition.

If consciousness is indeed sufficient for moral patienthood, then the stakes seem remarkably high from a utilitarian perspective that we do not commit the Type II error of behaving as if these and future systems are [...]

---

Outline:

(00:37) Why AI consciousness research seems critical to us

(03:45) Resources related to AI consciousness

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

There's a particular kind of widespread human behavior that is kind on the surface, but upon closer inspection reveals quite the opposite. This post is about four such patterns.

Computational Kindness

One of the most useful ideas I got out of Algorithms to Live By is that of computational kindness. I was quite surprised to only find a single mention of the term on lesswrong. So now there's two.

Computational kindness is the antidote to a common situation: imagine a friend from a different country is visiting and will stay with you for a while. You're exchanging some text messages beforehand in order to figure out how to spend your time together. You want to show your friend the city, and you want to be very accommodating and make sure all their preferences will be met. So you simply ask them: "What do you want to do"? [...]

---

Outline:

(00:18) Computational Kindness

(02:36) Responsibility Offloading

(06:49) Opt-In vs Opt-Out

(09:07) The Fake Exit Option

(13:26) Isnt This Just Overcomplicating Things?

The original text contained 5 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

This is my personal opinion, and in particular, does not represent anything like a MIRI consensus; I've gotten push-back from almost everyone I've spoken with about this, although in most cases I believe I eventually convinced them of the narrow terminological point I'm making.

In the AI x-risk community, I think there is a tendency to ask people to estimate "time to AGI" when what is meant is really something more like "time to doom" (or, better, point-of-no-return). For about a year, I've been answering this question "zero" when asked.

This strikes some people as absurd or at best misleading. I disagree.

The term "Artificial General Intelligence" (AGI) was coined in the early 00s, to contrast with the prevalent paradigm of Narrow AI. I was getting my undergraduate computer science education in the 00s; I experienced a deeply-held conviction in my professors that the correct response to any talk [...]

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

On 16 March 2024, I sat down to chat with New York Times technology reporter Cade Metz! In part of our conversation, transcribed below, we discussed his February 2021 article "Silicon Valley's Safe Space", covering Scott Alexander's Slate Star Codex blog and the surrounding community.

The transcript has been significantly edited for clarity. (It turns out that real-time conversation transcribed completely verbatim is full of filler words, false starts, crosstalk, "uh huh"s, "yeah"s, pauses while one party picks up their coffee order, &c. that do not seem particularly substantive.)

ZMD: I actually have some questions for you.

CM: Great, let's start with that.

ZMD: They're critical questions, but one of the secret-lore-of-rationality things is that a lot of people think criticism is bad, because if someone criticizes you, it hurts your reputation. But I think criticism is good, because if I write a bad blog [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

This posts assumes basic familiarity with Sparse Autoencoders. For those unfamiliar with this technique, we highly recommend the introductory sections of these papers.

TL;DR

Neuronpedia is a platform for mechanistic interpretability research. It was previously focused on crowdsourcing explanations of neurons, but we’ve pivoted to accelerating researchers for Sparse Autoencoders (SAEs) by hosting models, feature dashboards, data visualizations, tooling, and more.

Important Links

• Explore: The SAE research focused Neuronpedia. Current SAEs for GPT2-Small:
  • RES-JB: Residuals - Joseph Bloom (294k feats)
  • ATT-KK: Attention Out - Connor Kissane + Robert Kryzanowski (344k feats)
• Upload: Get your SAEs hosted by Neuronpedia: fill out this <10 minute application 
• Participate: Join #neuronpedia on the Open Source Mech Interp Slack

Neuronpedia has received 1 year of funding. Johnny Lin is full-time on engineering, design, and product, while Joseph Bloom is supporting with high-level direction and product management. We’d love to talk to you [...]

---

Outline:

(00:21) TL;DR

(00:43) Important Links

(01:48) Why Neuronpedia?

(02:37) What Its Already Useful For

(03:29) Making It More Useful

(05:08) Strategy: Iterate Very Quickly

(06:17) Current Neuronpedia Functionality

(06:21) Hosting SAE Feature Dashboards

(07:05) Feature Testing

(07:45) Automatic Explanations and UMAP for Exploration

(08:23) Live Feature Inference

(09:13) Enabling Collaboration

(11:05) Future Work

(11:19) Circuit Analysis

(11:42) Understanding + Red-Teaming Features

(12:15) Quality Control and Benchmarking

(12:40) FAQ

(12:44) Who's involved with the project?

(13:35) I’d like to upload my SAE weights to Neuronpedia. How do I do this?

(13:48) This sounds cool, how can I help?

(14:44) You seem to be super into SAEs, what if SAEs suck?

(15:27) Is Neuronpedia an AI safety org?

(15:52) Are there dual-use risks associated with SAEs?

---

First published:

Source:

Narrated by TYPE III AUDIO.

I just started thinking about what I would write to someone who disagreed with me on the claim "Rationalists would be better off if they were more spiritual/religious", and for this I'd need to define what I mean by "spiritual".

Here are some things that I would classify under "spirituality":

• Rationalist Solstices (based on what I've read about them, not actually having been in one)
• Meditation, especially the kind that shows you new things about the way your mind works
• Some forms of therapy, especially ones that help you notice blindspots or significantly reframe your experience or relationship to yourself or the world (e.g. parts work where you first shift to perceiving yourself as being made of parts, and then to seeing those parts with love)
• Devoting yourself to the practice of some virtue, especially if it is done from a stance of something like "devotion", "surrender" [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.

(Cross-posted from my website. Podcast version here, or search for "Joe Carlsmith Audio" on your podcast app.

This essay is part of a series that I'm calling "Otherness and control

"You, moon, You, Aleksander, fire of cedar logs.

~ Czeslaw Milosz, "Winter"

"Poplars (Autumn)," by Claude Monet (image source

My last

The original text contained 19 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

Previously: General Thoughts on Secular Solstice.

This blog post is my scattered notes and ramblings about the individual components (talks and songs) of Secular Solstice in Berkeley. Talks have their title in bold, and I split the post into two columns, with the notes I took about the content of the talk on the left and my comments on the talk on the right. Songs have normal formatting.

Bonfire

The Circle

This feels like a sort of whig history: a history that neglects most of the complexities and culture-dependence of the past in order to advance a teleological narrative. I do not think that whig histories are inherently wrong (although the term has negative connotations). Whig histories should be held to a very strict standard because they make claims about how most or all of human history functions.

The song describes morality in terms of an expanding circle of [...]

---

Outline:

(00:35) Bonfire

(00:38) The Circle

(03:40) Life Is Too Short to Fold Underwear

(05:27) Time Wrote the Rocks

(06:05) Underrated Reasons to Be Thankful

(06:12) Bold Orion

(06:27) Embers

(06:30) The Fallen Star

(07:39) On Wanting Things

(07:55) Night

(07:58) Bitter Wind Lullaby

(08:44) Hymn to Breaking Strain

(09:03) For Every Ailment Under the Sun

(09:09) No One Survives

(09:15) Songs Stay Sung

(10:34) Call and Response: Defiance

(11:17) Dawn

(11:20) Brighter Than Today

(11:30) Half Monkey, Half God

(11:37) Song of the Artesian Water

(12:55) I Have Seen the Tops of Clouds

(13:06) Morning

(13:09) We Will Not Die of This

(13:18) Ballad of Smallpox Gone

(13:33) Tomorrows

(13:48) Level Up

(13:59) The Orange

(14:05) The Great Transhumanist Future

(15:48) Five Thousand Years

(16:42) Summary by MC

(16:52) What a Wonderful World

The original text contained 12 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.