Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Goodhart's Law inside the human mind, published by Kaj Sotala on April 17, 2023 on LessWrong.

Some time back, I saw a tweet from somebody that read:

Much of social psychology seems to be premised on the bizarre assumption that what people really care about is not real-world outcomes but the state of their own mind: self-esteem, a positive self-image, dissonance reduction, feelings of control, reducing uncertainty, etc.

I've certainly seen versions of the same myself. Maybe the most poignant example comes from this book review, which suggested that gambling addicts get hooked on a sense of control - even though someone who's hooked on gambling to the point of ruining their life clearly isn't in much control of anything:

The primary objective that machine gambling addicts have is not to win, but to stay in the zone. The zone is a state that suspends real life, and reduces the world to the screen and the buttons of the machine. Entering the zone is easiest when gamblers can get into a rhythm. Anything that disrupts the rhythm becomes an annoyance. This is true even when the disruption is winning the game. Many gamblers talk about how winning the game brings them out of the zone, and they actually dislike winning for that reason. For some gamblers, the very act of pressing buttons to play the game disrupts the rhythm. These gamblers use autoplay modes on games that offer them, and jerryrig an autoplay mode on machines that don’t by jamming something into buttons to keep them pressed. They don’t want to chase a win or pick their lucky numbers, they want to disappear into the zone. [...]

The book is full of heartbreaking stories about what gamblers endure on their path to extinction. They sacrifice their bodies, their time, and their relationships. Sharon, for example, spent four days at a casino, trying to lose all her money to reach extinction. At the end of this ordeal, she came home to sleep, but she found three nickels in her bedroom. The thought of not having spent all her money bothered her so much that she drove back to a casino immediately to lose those last three nickels. [...]

I used to think that gambling addicts “lost control” when they gambled excessively. But the addicts in the book use machines as a way to gain control in their lives. In front of a machine, the world is simple: they place bets and lose a little bit of money on each turn. The gamblers are in control of this machine world. It is the world away from machines where the prospect of losing control in frightening ways looms. Away from the machines, life is long and full of terrors.

This certainly sounds odd and destructive—seeking control in a way that destroys one's life.

But it's also one that resonates with me - it feels like it describes the relationship I've often had with social media. Picking up my phone and checking Facebook can give me a sense of autonomy, like I'm choosing to leave the current situation and momentarily visit another world. This feels like it's the case even when the phone-checking becomes compulsive; a part of me craves that feeling of control so much that it gets out of control.

While I've never smoked, I understand that many smokers describe their relationship with cigarettes similarly. Smoking offers a socially acceptable reason to step away from a dull conversation or meeting for a moment. Briefly, you can tune out and regain a sense of control.

There's something peculiar about this. The initial tweet mentioned the "bizarre" assumption in social psychology that people prioritize internal mental states over actual control (or any other attribute the feelings seem to track). It's strange that a gambling addict would chase the feeling of control even when it leads their life to spiral out of control.

However, I believe it's relatively simple to explain. First, there's a...