This essay contains an examination of handling information which is unpleasant to learn. Also, more references to spiders than most people want.

CW: Pictures of spiders.

1. Litanies and Aspirations

If the box contains a diamond,
I desire to believe that the box contains a diamond;
If the box does not contain a diamond,
I desire to believe that the box does not contain a diamond;
Let me not become attached to beliefs I may not want.

-Litany of Tarski

I read these words when I was around eighteen. They left a strong impression on me. While it's not frequent that I do so aloud, sometimes I respond to learning things have gone wrong by muttering them quietly. I've used them as words of comfort when people are anxiously awaiting important news.

The litany of Tarski is aspirational for me. It isn't always literally true.

If my mother's biopsy contains breast cancer, I really want the biopsy to contain no breast cancer. I am attached to that belief. Of course, the exact text of Tarski's litany is "I desire to believe that the box contains a diamond" not "I desire that the box contained a diamond."

---

Outline:

(00:23) 1. Litanies and Aspirations

(01:44) 2. Youve Got Mail Spiders

(05:34) 3. Merrin and the Spider

(13:49) 4. Do you desire to believe there is a spider in this essay?

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's been eight months since I released my last story, so you could be forgiven for thinking that I’d given up on writing fiction. In fact, it's the opposite. I’m excited to announce that I’m releasing my first fiction collection—The Gentle Romance: Stories of AI and Humanity—with Encour Press in mid-December!

It contains 22 stories, most of which are revised versions of the best stories I’ve posted online. The thread that connects them is the struggle to hold onto our identities in the face of radical technological change—the same thread that winds through many of our own lives.

⁠⁠I’ve also written three new stories for the collection, which are some of my favorites:

• Lentando is set in a future inspired by Charles Stross’ masterpiece Accelerando. Through it we follow Liza, a zero-knowledge consultant whose soon-to-be-deleted copies struggle to hold the world together.
• The Biggest Short is the story of two traders who make a fortune buying and selling reputations while struggling to preserve their own.
• Kuhn's Ladder is about a simulated utopia that starts experiencing inexplicable glitches which seem designed to remain hidden.

You can preorder the book here.

Looking forward, it's hard to [...]

---

First published:

Source:

Linkpost URL:
https://www.narrativeark.xyz/p/book-announcement

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Manifest X DC was this weekend, hopefully the first of many local spin-offs of Manifest. Despite a late prediction market surge, there were no fires. An attendee wrote a nice post of observations, though he left the afterparty before the lights dimmed and the duels began.

I gave an opening talk, which ended up being more personal and emotional than I'd set out to write. Several attendees asked for the text, so I am posting a lightly edited version of my draft. This is not the version as delivered, I improvised a fair amount, and I think the tone in delivery did a lot of the work. Still, I'm sharing here for those curious.

Good morning. I wanted to say a few words to set the tone for today. I decided to tell a story about a book club that got out of hand.

In college or group houses, community is easy. Years ago, after grad school, I got my own apartment and lived alone for the first time in my life. As my college friends grew up, I noticed that I’d have to be more deliberate about finding community.

I tried out several groups, for board games [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Looking back, it appears that much of my intellectual output could be described as legibilizing work, or trying to make certain problems in AI risk more legible to myself and others. I've organized the relevant posts and comments into the following list, which can also serve as a partial guide to problems that may need to be further legibilized, especially beyond LW/rationalists, to AI researchers, funders, company leaders, government policymakers, their advisors (including future AI advisors), and the general public.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Condensation: a theory of concepts is a model of concept-formation by Sam Eisenstat. Its goals and methods resemble John Wentworth's natural abstractions/natural latents research.[1] Both theories seek to provide a clear picture of how to posit latent variables, such that once someone has understood the theory, they'll say "yep, I see now, that's how latent variables work!".

The goal of this post is to popularize Sam's theory and to give my own perspective on it; however, it will not be a full explanation of the math. For technical details, I suggest reading Sam's paper.

Brief Summary

Shannon's information theory focuses on the question of how to encode information when you have to encode everything. You get to design the coding scheme, but the information you'll have to encode is unknown (and you have some subjective probability distribution over what it will be). Your objective is to minimize the total expected code-length.

Algorithmic information theory similarly focuses on minimizing the total code-length, but it uses a "more objective" distribution (a universal algorithmic distribution), and a fixed coding scheme (some programming language). This allows it to talk about the minimum code-length of specific data (talking about particulars rather than average [...]

---

Outline:

(00:45) Brief Summary

(02:35) Shannons Information Theory

(07:21) Universal Codes

(11:13) Condensation

(12:52) Universal Data-Structure?

(15:30) Well-Organized Notebooks

(18:18) Random Variables

(18:54) Givens

(19:50) Underlying Space

(20:33) Latents

(21:21) Contributions

(21:39) Top

(22:24) Bottoms

(22:55) Score

(24:29) Perfect Condensation

(25:52) Interpretability Solved?

(26:38) Condensation isnt as tight an abstraction as information theory.

(27:40) Condensation isnt a very good model of cognition.

(29:46) Much work to be done!

The original text contained 15 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

Rationalist Winter Solstice is (in most cities) a singalong event.

This worked extremely straightforwardly well in a living room in 2012, and in my family's Christmas Eve singalong that Rationalist Solstice was inspired by.

It works way less well in an auditorium.

When I seek out advice about making people more singalongable, there's a cluster of advice I get from folksinger people that... seems totally "correct", but, feels... insufficiently ambitious or something.

The advice includes things like "try to have people come in singing on the chorus, and not worry so much about the verses". Or "teach people the song beforehand" and "hold practice singalongs before the event or send out music so people can learn it", or "teach people music."

This... totally makes sense as advice, and I do do it. The way people bring it up feels a bit alien to me, like it's clearly intervening on the wrong level. It's optimizing a thing that has a lower overall effect size than another variable that I've seen accomplish way more singalongability way harder.

The other variable is "actually make people feel in their bones that singing along is okay/fine even if they're bad at singing [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

When I put on my LLM skeptic hat, sometimes I think things like “LLMs don’t really understand what they’re saying”. What do I even mean by that? What's my mental model for what is and isn’t going on inside LLMs minds?

First and foremost: the phenomenon precedes the model. That is, when interacting with LLMs, it sure feels like there's something systematically missing which one could reasonably call “understanding”. I’m going to articulate some mental models below, but even if I imagine all those mental models are wrong, there's still this feeling that LLMs are missing something and I’m not quite sure what it is.

That said, I do have some intuitions and mental models for what the missing thing looks like. So I’ll run the question by my intuitions a few times, and try to articulate those models.

First Pass: A Bag Of Map-Pieces

Imagine taking a map of the world, then taking a bunch of pictures of little pieces of the map - e.g. one picture might be around the state of Rhode Island, another might be a patch of Pacific Ocean, etc. Then we put all the pictures in a bag, and forget about the original [...]

---

Outline:

(01:02) First Pass: A Bag Of Map-Pieces

(02:05) Second Pass: Consistent Domains

(03:30) Third Pass: Aphantasia

(05:00) Fourth Pass: Noticing And Improving

The original text contained 1 footnote which was omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

In this post, I describe a mindset that is flawed, and yet helpful for choosing impactful technical AI safety research projects.

The mindset is this: future AI might look very different than AI today, but good ideas are universal. If you want to develop a method that will scale up to powerful future AI systems, your method should also scale down to MNIST. In other words, good ideas omniscale: they work well across all model sizes, domains, and training regimes.

Putting the omniscaling mindset into practice is straightforward. Any time you come across a clever-sounding machine learning idea, ask: "can I apply this to MNIST?" If not, then it's not a good idea. If so, run an experiment to see if it works. If it doesn't, then it's not a good idea. If it does, then it might be a good idea, and you can continue as usual to more realistic experiments or theory.

In this post, I will:

1. Share how MNIST experiments have informed my [...]

---

Outline:

(01:58) Applications to MNIST

(02:42) Gradient routing

(04:43) Distillation robustifies unlearning

(08:39) Subliminal learning

(10:37) Why you should do it on MNIST

(11:30) MNIST is not sufficient (and other tips)

(14:25) The omniscaling assumption is false

(17:09) Code and more ideas

(18:40) Closing thoughts

The original text contained 7 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Audio note: this article contains 49 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.

Löb's Theorem:

• If _vdash Box x to x_, then _vdash x_.
• Or, as one formula: _Box (Box x to x) to Box x_

Payor's Lemma:

• If _vdash Box (Box x to x) to x_, then _vdash x_.
• Or, as one formula: _Box big(Box (Box x to x) to xbig) to Box x_.

In the following discussion, I'll say "reality" to mean _x_, "belief" to mean _Box x_, "reliability" to mean _Box x to x_ (ie, belief is reliable when belief implies reality), and "trust" to mean _Box (Box x to x)_ (belief-in-reliability).

Löb says that if you have trust, you have belief.

Payor says that if you can prove that trust implies reality, then you have belief.

So, both results give conditions for belief. Indeed, both results give conditions equivalent to belief, since in both cases the inference can also be reversed:

• Bidirectional Löb: _vdash Box x to x iff vdash x_
• Bidirectional Payor: _vdash Box (Box x to x) to x iff vdash x_ 

Furthermore, both results relate reliability with [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

The barriers between us and what we want are often entirely imagined. It is true: you can learn how to paint, change careers, write a paper or run a marathon. These things are hard, but we shouldn’t pretend that they are impossible. You can just do them.

But this mindset has a dangerous edge case: it can make you skip asking for permission precisely when you should ask. The anxiety you're overriding might be trying to tell you something important: that sometimes the fence is there for a reason.

Here are two illustrative examples from my life (with details changed to protect the privacy of those involved) that motivated my thinking:

1. A close friend in high school (we call him friend #1) was faced with the pressure of finding a date to the prom lest he look like a total loser. Under the pressure of desperately trying to find someone to go to prom with, he realized that there was nothing stopping him from asking the girl that he had liked for years. He realized he could “just do things.” The worst she could say is no!

   Friend #1 asked out his love interest and she said [...]

---

First published:

Source:

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.