https://astralcodexten.substack.com/p/mantic-monday-mantic-matt-y

 

The current interest in forecasting grew out of Iraq-War-era exasperation with the pundit class. Pundits were constantly saying stuff, like "Saddam definitely has WMDs, trust me, I'm an expert", then getting proven wrong, then continuing to get treated as authorities and thought leaders. Occasionally they would apologize, but they'd be back to telling us what we Had To Believe the next week.

You don't want a rule that if a pundit ever gets anything wrong, we stop trusting them forever. Warren Buffett gets some things wrong, Zeynep Tufecki gets some things wrong, even Nostradamus would have gotten some things wrong if he'd said anything clearly enough to pin down what he meant. The best we can hope for is people with a good win-loss record. But how do you measure win-loss record? Lots of people worked on this (especially Philip Tetlock) and we ended up with the kind of probabilistic predictions a lot of people use now.

But not pundits. We never did get the world where pundits, bloggers, and other commentators post predictions clearly in a way where they can check up on them later.