Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Evidence under Adversarial Conditions, published by PeterMcCluskey on January 9, 2023 on LessWrong.

I recently noticed similarities between how I decide what stock market evidence to look at, and how the legal system decides what lawyers are allowed to tell juries.

This post will elaborate on Eliezer's Scientific Evidence, Legal

Evidence, Rational

Evidence. In particular, I'll try to generalize about why there's a large class of information that I actively avoid treating as Bayesian evidence.

Here's an example of a briefly popular meme that helped teach me to ignore certain types of information:

Companies need to be forgetting organizations. Enron Corp., which has repeatedly been tagged as the nation's most innovative corporation, is exhibit A as a world-class forgetting organization. It's not wedded to what it did yesterday. Enron chiefs Kenneth Lay and Jeff Skilling have figured out how to operate like a band of pirates. Got an idea? Don't dally. Go for it while it's an original!

Everything here seems at least technically true (The Media Very Rarely

Lies). I suspect it was mostly intended to be honest and helpful. But as a reader, I found it discouraged me from focusing on some key questions, such as: who was tagging Enron as innovative? What were the goals of those taggers? What was their track record at helping readers identify good investments? How much of a connection is there between innovation and good investments? What are some examples of Enron's innovation, and how do they resemble the kind of innovations that have caused other companies to succeed?

Enron seems to have done a few things which qualify as fraud. But most of the Enron stock price bubble was generated through misdirecting investor's attention (and maybe the attention of Lay and Skilling) away from the risks that Enron was taking.

I was in fact tempted to invest in Enron when I listened to that kind of meme. Whenever I felt that temptation, I looked at its financial statements and charts of its stock price, and decided I wouldn't miss much if I waited for better evidence.

The standard Bayesian worldview asks me to update on all relevant information. Why do both I as an investor and the legal system exclude large classes of information from that rule?

Both contexts involve some zero sum or negative sum conflicts. That makes it likely that someone is packaging information in order to mislead.

If I, or a jury, only notice a datum because someone thought it would mislead me or the jury, odds are that updating on that information would reduce the accuracy of my (or the jury's) beliefs. We reject that information in order to ensure that its expected value will be zero, because the alternative tends to be that its expected value is negative.

In both contexts, rules have been developed to admit evidence only if it's hard to manipulate or if it comes from a source that's unlikely to manipulate it.

Accounting rules strongly limit how many ways key information can be presented. They also ensure that someone can in principle be punished if the information misleads.

Similar rules ensure that company press releases are fairly safe for investors to read. The company will often be sued if the press release contains questionable claims that subsequently become linked to investor losses. That deters a wide variety of harmful propaganda (and presumably a fair amount of helpful evidence). It still leaves some gray areas for vaguely misleading memes, such as talk of Enron's innovation. Companies have mostly shifted to getting other respected sources to promote those ideas for them, rather than putting them in their own press releases.

Still, I consider it an important skill to read those press releases in somewhat the same way that courts admit evidence: carefully distinguishing between "facts" and everything else....