Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Assessment of intelligence agency functionality is difficult yet important, published by trevor on August 24, 2023 on LessWrong.

Summary: When it comes to observing intelligence agencies, it's hard to see the hardened parts and easy to observe the soft corrupt parts. This leads to a bias where very large numbers of people overestimate how prevalent the easily-observed soft and harmless parts are. This can sometimes even result in a dangerous and prevalent estimation, among people whose careers are much further ahead than yours, that the entire intelligence agency is harmless and irrelevant, when it actually isn't. Intelligence agencies are probably a mix of both less-functional, less-relevant parts, and also more-functional, more-relevant parts that have a disproportionately large influence over governments and policies; and it is a mistake to assume that intelligence agencies are homogenously composed of non-functional non-relevant parts that aren't worth paying any attention to, even if such a belief is a popular norm.

Why intelligence agencies are dangerous

There are a wide variety of situations where intelligence agencies suddenly becomes relevant, without warning. For example, most or all of the US Natsec establishment might suddenly and unanimously change its stance on Gain of Function research, such as if US-China relations or US-Russian relations once again hit a new 25-year low (which has actually been happening very frequently over the last few years).

Either the leadership of an agency, or a powerful individual in an agency with authority to execute operations, or a corrupt clique, might personally make a judgement that the best way to expedite or restart GOF research is to target various people who are the most efficient or effective at opposing GOF research.

This need not be anywhere near the most effective way to expedite or protect GOF research, it just needs to look like that, sufficiently for someone to sign off on that, or even for them to merely thing that it would look good to their boss.

Competent or technologically advanced capabilities can obviously be mixed with incompetent administration/decisionmaking in the mixed competence model of intelligence agencies. An intelligence agency that is truly harmless, irrelevant, and not worth paying attention to (as opposed to having an incentive to falsely give off the appearance of harmlessness, irrelevance, or not being worth paying attention to) would have to be an intelligence agency that is both technologically unsophisticated and too corrupt for basic functioning, such as running operations.

This would be an extremely naive belief to have about the intelligence agencies in the US, Russia, and China; particularly the US and China, which have broad prestige, sophisticated technology, and also thriving private sector skill pools to recruit talent from.

When calculating the expected value from policy advocacy tasks that someone somewhere absolutely must carry out, like pushing sensible policymaking on GOF research that could cause human extinction, many people are currently aware that the risk of that important community disappearing or dissolving substantially reduces the expected value calculations of everything produced by that important community; e.g. a 10% chance of the community ceasing to exist or dissolving reduces the expected value produced by that entire community by something like ~10%.

Most people I've encountered have in mind a massive totalitarian upheaval, like the ones in the early-mid 20th century, and such an upheaval is a hard boundary between being secure and not being secure. However, in the 21st century, especially after COVID and the 2008 recession, experts and military planners are now more focused on the international balance of power (e.g. the strength of the US, Russia, and ...