Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Sapient Algorithms, published by Valentine on July 17, 2023 on LessWrong.

I notice my mind runs lots of cached programs. Like "walk", "put away the dishes", "drive home", "go to the bathroom", "check phone", etc.

Most of these can run "on autopilot". I don't know how to define that formally. But I'm talking about how, e.g., I can start driving and get lost in thought and suddenly discover I'm back home - sometimes even if that wasn't where I was trying to go!

But some programs cannot run on autopilot. The algorithm has something like a "summon sapience" step in it. Even if the algorithm got activated due to autopilot, some step turns it off.

When I look at the examples of sapient algorithms that I run, I notice they have a neat kind of auto-generalization nature to them. I have some reason to think that property is general. It's the opposite of how, e.g., setting up webpage blockers can cause my fingers to autopilot learn how to bypass them.

I'll try to illustrate what I mean via examples.

Example: Look at my car keys

I got tired of risking locking my keys in my car. So I started making a habit of looking at my keys before closing the door.

Once, right after I'd closed the locked car door, I realized I'd looked at the phone in my hand and shut the door anyway. Luckily the key was in my pocket. But I noticed that this autopilot program just wasn't helping.

So I modified it (as a TAP): If I was about to close the car door, I would look at my hand, turn on consciousness, and check if I was actually looking at my keys.

First, that TAP just worked. To this day I still do this when stepping out of a car.

Second, it generalized without my trying to:

After a while it would fire whenever I was about to close any locked door.

It then generalized to anyone I was with. If they were about to close a locked door, I would sort of "pop awake" with a mental question about whether someone had the key.

It then generalized even more. It now fires when I'm, say, preparing for international travel. Crossing a border feels a bit like going through a door that locks behind me. So now I "wake up" and check that I and my travel companions all have passports. (I would usually check before, but now it's specifically this mental algorithm that summons sapience. It's reliable instead of being an extra thing to remember.)

This generalization wasn't intentional. But it's been really good. I haven't noticed any problems at all from this program sort of spreading on its own.

Example: Taste my food

When I'm in autopilot mode while eating, it can feel at the end like my food kind of vanished. Like I wasn't there for the meal.

So I installed a TAP: If I'm about to put food in my mouth, pause & remember emptiness.

"Remember emptiness" has a "summon sapience" type move embedded in it. It's something like "Turn on consciousness, pause, and really look at my sensory input." It's quite a bit deeper than that, but if this kind of emptiness just sounds like gobbledegook to you then you can pretend I said the simplified version.

In this case, the TAP itself didn't install as cleanly as with the car keys example. Sometimes I just forget. Sometimes the TAP fires only after I've taken my first bite.

But all the same, the algorithm still sort of auto-generalized. When I'm viewing a beautiful vista, or am part of a touching conversation, or hear some lovely music, the TAP sometimes fires (about as regularly as with food). One moment there are standard programs running, and then all of a sudden "I'm there" and am actually being touched by whatever it is (the same way I'm actually tasting my food when I'm "there").

Yesterday I noticed this sapient algorithm booting up in a conversation. Someone asked me "Can I speak plainly?" and I knew she was about to say something I'd find challenging to receive. My autopilot start...