tl;dr: Mixing goal-directedness into cognitive processes that are working to truth-seek about possible futures tends to undermine both truth-seeking and effective pursuit of your goals. Cleanly separating them has nice properties.

[Epistemic Status: Mostly empirically observed rather than rigorously justified, but have seen it many times across different people and within myself so fairly confident there's something here, and have some sketch of the process]

Minds are composed of circuits/programs/parts which model goal-states. Or, to put another way, have sub-patterns with preferences over how the world should be.

We imagine possible futures that could come out of different actions using our world models, and use this prediction of outcome as an input to action selection[1] (including internal actions like choice of thoughts).

I'd like to draw attention to one axis of this process, which I'll bind to the following words[2]:

• Intention is doing something close to unbiased simulation[3], then applying agentic choice only at the current timestep, trusting future yous at future timesteps to run their own agency with their greater information. It accepts everything at the truth-seeking stage of predicting the future, and only decides things that are immediately actionable.
• Trying is when you're applying pressure into [...]

The original text contained 9 footnotes which were omitted from this narration.

---

First published:

Source:

---

Narrated by TYPE III AUDIO.