Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Heritability, Behaviorism, and Within-Lifetime RL, published by Steven Byrnes on February 2, 2023 on LessWrong.

I’m a firm subscriber to both:

(A) The theory that people’s personalities are significantly predictable from their genes, and mostly independent of how their parents raised them (at least within the typical distribution, i.e. leaving aside cases of flagrant abuse and neglect etc.). See e.g. popular expositions of this theory by Judith Harris or by Bryan Caplan for the fine print.

(B) The theory that we should think of people’s beliefs and goals and preferences developing via within-lifetime learning, and more specifically via within-lifetime Model-based Reinforcement Learning (details), with randomly-initialized (“learning-from-scratch”) world-model and value function.

I feel like there’s an idea in the air that these two beliefs are contradictory. For example, one time someone politely informed me that (A) is true and therefore obviously (B) must be false.

Needless to say, I don’t think they’re contradictory. Indeed, I think that (B) naturally implies (A).

But I admit that they sorta feel contradictory. Why do they feel that way? I think because:

(A) is sorta vaguely affiliated with cognitive science, evolutionary psychology, etc.

(B) is sorta vaguely affiliated with B.F. Skinner-style behaviorism,

.and those two schools-of-thought are generally considered to be bitter enemies.

In this short post I want to explain why we should put aside that baggage and see (A) & (B) as natural allies.

Two dubious steps to get from (B) to Behaviorism

Here’s the fleshed-out argument as I see it:

I’ll go through the two dubious steps in the opposite order.

Dubious step #1: “No more learning / unlearning after the kid grows up”

Here are two stories:

“RL with continuous learning” story: The person has an internal reward function in their head, and over time they’ll settle into the patterns of thought & behavior that best tickle their internal reward function.

If they spend a lot of time in the presence of their parents, they’ll gradually learn patterns of thought & behavior that best tickle their innate internal reward function in the presence of their parents.

If they spend a lot of time hanging out with friends, they’ll gradually learn patterns of thought & behavior that best tickle their innate internal reward function when they’re hanging out with friends.

As adults in society, they’ll gradually learn patterns of thought & behavior that best tickle their innate internal reward function as adults in society.

“RL learn-then-get-stuck” story: The kid learns patterns of thoughts & behavior in childhood, and then sticks with those patterns for the rest of their lives no matter what.

Claim: I think the “RL with continuous learning” story, not the “RL learn-then-get-stuck” story, is how we should generally be thinking about things. At least in humans. (Probably also in non-human animals, but that’s off-topic.)

I am not making a strong statement that the “RL learn-then-get-stuck” story is obviously and universally wrong and stupid nonsense. Indeed, I think there are edge cases where the “learn-then-get-stuck” story is true. For example, childhood phobias can sometimes persist into adulthood, and certainly childhood regional accents do. Some related discussion is at Scott Alexander’s blog post “Trapped priors”.

Instead, I think we should mainly believe the “RL with continuous learning” story for empirical reasons:

Heritability studies: See top. More specifically, note that (IIRC) parenting style can have some effect on what a kid believes and how they behave while a child, but these effects fade out when the kid grows up.

Culture shifts: Culture shifts are in fact possible, contrary to the “RL learn-then-get-stuck” story. For example, almost everybody in the USA opposed gay marria...