Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why You Should Never Update Your Beliefs, published by Arjun Panickssery on July 29, 2023 on LessWrong.

Epistemic status: Invincible

Since Cavalry scouts are often in direct contact with the enemy, their job can be considered one of the most dangerous jobs the Army has to offer.

something called "Operation Military Kids"

There's some irony that Julia Galef's rationalist self-help book The Scout Mindset compares favorably the scout, who hunts for new and reliable evidence, to the soldier, who fights off threats. But scouts have one of the most dangerous military occupations. To quote a random website, "cavalry scouts and recon units tread uncharted ground when it comes to conflict zones. They are usually at the tip of any advance and, therefore, meet the brunt of whatever resistance is lying in wait for them."

Uncharted epistemic territory is dangerous because it's awash with incorrect arguments which might convince you of their false conclusions. Many of these arguments are designed to be persuasive regardless of their accuracy. Scott Alexander describes succumbing to an "epistemic learned helplessness" after his failure to refute crackpots whose arguments are too carefully crafted to refute in any reasonable length of time:

What finally broke me out wasn't so much the lucidity of the consensus view so much as starting to sample different crackpots. Some were almost as bright and rhetorically gifted as Velikovsky, all presented insurmountable evidence for their theories, and all had mutually exclusive ideas. After all, Noah's Flood couldn't have been a cultural memory both of the fall of Atlantis and of a change in the Earth's orbit, let alone of a lost Ice Age civilization or of megatsunamis from a meteor strike. So given that at least some of those arguments are wrong and all seemed practically proven, I am obviously just gullible in the field of ancient history.

Given a total lack of independent intellectual steering power and no desire to spend thirty years building an independent knowledge base of Near Eastern history, I choose to just accept the ideas of the prestigious people with professorships in Archaeology, rather than those of the universally reviled crackpots who write books about Venus being a comet.

You could consider this a form of epistemic learned helplessness, where I know any attempt to evaluate the arguments is just going to be a bad idea so I don't even try. If you have a good argument that the Early Bronze Age worked completely differently from the way mainstream historians believe, I just don't want to hear about it. If you insist on telling me anyway, I will nod, say that your argument makes complete sense, and then totally refuse to change my mind or admit even the slightest possibility that you might be right.

(This is the correct Bayesian action: if I know that a false argument sounds just as convincing as a true argument, argument convincingness provides no evidence either way. I should ignore it and stick with my prior.)

The solution is to ignore most evidence that would change your views. This strategy is well-supported by epistemology and psychology:

Critical thinking is altogether on dubious footing. See Michael Huemer's "Is Critical Thinking Epistemically Responsible?" (the link goes to his blog post summary; the full text is available at his website in Papers Epistemology). He discusses the rationality of three strategies for forming a view on a "publicly-discussed issue":"Credulity: You canvass the opinions of a number of experts, and adopt the belief held by most of them. In the best case, you find a poll of the experts; failing that, you may look through several books and articles and identify their overall conclusions.Skepticism: You give upon finding the answer, i.e., immediately suspend judgement.Critical Thinking: You gather ...