Link to original article

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Fundamental question: What determines a mind's effects?, published by Tsvi Benson-Tilsen on September 3, 2023 on The AI Alignment Forum.

[Metadata: crossposted from. First completed April 9, 2023.]

A mind has some effects on the world. What determines which effects a mind has? To eventually create minds that have large effects that we specify, this question has to first be answered.

Slippery questions

A slippery question is a question that, when you try to approach it looking for an answer, it slips out of your grasp. It runs away or withdraws, it cloaks itself in false expressions. It leads you astray, replacing itself with a similar-sounding question, maybe one that's easier to answer. It's hard to hug a slippery question. The question withdraws.

Or worse, it pretends to be made out of ungrounded abstract concepts, so that you're not sure anymore that there's even a real question there. This slipperiness is especially likely for questions that involve "big" things, such as minds.

The question might seem obvious and even tautological. Or it might seem simplistic, failing to carve reality at the joints. These appearances usually somewhat accurately reflect the reality - - but they do not imply necessarily that there is not a real question behind the appearances. A question that can be pointed at, but only imperfectly, inchoately, ostensively, or preliminarily, is at risk of being question-substituted or being ignored.

The discomfort of not having an answer or even a clear question pushes people to run ahead of the question. When they make an error that would have been prevented by meditating on the question, even if they notice the error, they don't notice that they are in general not focusing on the question. If the question asks: There is something, it has to do with concrete things A, B, and C, and it is important - - what is it? Then someone will analyze A and B in detail, gain some understanding, and declare the whole question - - including C, and the inexplicit core question - - solved.

This essay will ask a slippery question over and over in different words, hoping to arrive at a firmer grasp on the real questions. The starting point:

What determines the effects of a mind?

The word "effects"

Suppose there is a mind M. As a result of M existing, the cosmos (the entirety of the world, in any aspect or abstraction) is different from how it would have been if M hadn't existed. Those differences are the effects of M.

What determines the difference in the cosmos that results from the presence of the mind?

The effects of a mind depend on how strong the mind is. But the question wants to ask about the sort of effects, not their size:

What determines the direction of a mind's effects?

Minds pursue instrumental subgoals, which affect the world. These effects might matter, but they are subordinate to their supergoals. Any reversible effect might be reversed. The question asks about the final effects of the mind:

What determines the direction of a mind's ultimate effects on the cosmos?

"Direction" is only a metaphor. It suggests that there's something compactly understandable about the effects of the mind. The metaphor may not hold well. To nod in that direction, the question can at least be rephrased to suggest multi-dimensionality:

What determines the directions of a mind's ultimate effects on the cosmos?

The word "mind"

What does "mind" mean here? The choice of the word "mind" reflects the guess that things that have large effects will be integrated - - will speak in internal languages, pass information, correct errors, and in general become coherent.

Rather than "mind", the question could refer to [things that have large effects], reading:

What determines the effects of a large-effect-haver?

But this is too broad. The cosmos as a whole is a large-effect-haver, in...