Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Yglesias on EA and politics, published by Richard Y Chappell on May 23, 2022 on The Effective Altruism Forum.

[Updated to a full cross-post, with thanks to Matt for permitting this.]

Last Tuesday, the most expensive open House primary in history wrapped up with Andrea Salinas, a well-qualified state legislator, winning handily against a large field in which the second-place contender was a political neophyte named Carrick Flynn.

Millions of dollars were spent on behalf of Salinas, largely by Bold PAC, the super PAC for Hispanic House Democrats. But a much larger sum of money was spent on Flynn’s behalf by a super PAC called Protect Our Future, which is largely funded by Sam Bankman-Fried, CEO of the cryptocurrency exchange FTX.

While Flynn and SBF had never met before the campaign, Flynn is a longtime effective altruist and member of the EA community (see his interview with Vox’s EA vertical Future Perfect), and SBF has recently emerged as the biggest donor in the EA space.

Daniel Eth of the Oxford Future of Humanity Institute wrote a good piece on lessons for the EA movement from the campaign, but speaking from a media and politics perspective, I want to underline with the boldest possible stroke that I think the most important lesson from the race is the extent to which Flynn was understood to be the “crypto candidate” rather than the “EA candidate.”

Dave Wasserman @Redistrict

I've seen enough. State Rep. Andrea Salinas (D) wins the Dem primary for the new #OR06, defeating crypto-backed Carrick Flynn (D). Salinas would be the first Latina elected to Congress from Oregon. @CookPolitical November rating: Likely D.

May 18th 2022

1,116 Retweets7,990 Likes

That’s in part because of hostile messaging from his opponents (which is fair enough, it’s politics) but also because Flynn didn’t really talk about effective altruism until quite late. His campaign did discuss pandemic prevention — the top current priority for SBF and a topic Flynn has a background in — but not much about the movement or the ideas until the final weeks of the race. The judgment that EA is unfamiliar to most voters and sounds a bit weird was perfectly reasonable. But you can’t dump $12 million into a random House race without people wondering what’s up.

I think SBF’s ideas, as I understand them, are pretty good, and I hope that his efforts to spend money to influence American politics succeed. But to succeed, I think ultimately he’s going to have to persuade people that his agenda really is what he says it is.

Part of that will require fostering a broader familiarity with EA, and to that end, I wanted to write about the space as I understand it and its significance and insights.

Consequentialism briefly explained

I can’t speak to contemporary academia, but when I was in college the family of moral theories known as consequentialism were pretty unfashionable in philosophy departments. My understanding is that this has been true for most of the past two or three generations.

Probably the easiest way to gloss consequentialism is with reference to the famous trolley problem thought experiment: a consequentialist is someone who wants to throw the switch and cause one person to be run over in order to avoid the deaths of five people.

Perhaps more scandalously, opponents of consequentialism often argue that it follows from this position that it is morally appropriate (perhaps even obligatory) to murder innocent people and harvest their organs in order to save multiple lives.

Barbara Fried, a Stanford legal theorist and philosopher who also happens to be Bankman-Fried’s mother, has a great paper arguing that we should talk less about trolley problem scenarios. Her point is that the many variations of trolley problems share an artificial certainty about outcomes that is very unusual for a policy choice. A real...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some unfun lessons I learned as a junior grantmaker, published by Linch on May 23, 2022 on The Effective Altruism Forum.

I’ve been a part-time grantmaker since January 15th or so, first (and currently) as a guest manager on the Long-Term Future Fund, and more recently did non-trivial other grantmaking as well.

In doing so, I learned a number of useful but not fun lessons, some of which may be useful to others on this forum. I mostly learned these lessons through consulting more senior grantmakers and other advisors, and secondarily from thinking through these problems abstractly, rather than independently learning them the hard way from contact with reality with my own grants.

Standard disclaimers apply: all views are my own. I do not speak for any of the people who advised me on grantmaking. I do not speak for the Long-Term Future Fund, other grantmakers, or my full-time employer (Rethink Priorities).

I believe my points are more true for the longtermist and meta spaces (I have zero experience with animal or global health grantmaking), and more true for longtermism than meta (in particular, I think the points about feedback especially are directionally true for meta but less so than in longtermist grantmaking). Most of my actual grants have been in forecasting, but I actually do not believe my points below are more true for forecasting grants than for other subfields of longtermist grantmaking.

In this article, I tried my best to convey knowledge that is both true and at least somewhat novel to me/this Forum. In doing so, I may say things that are systematically not-yet-said because others believe that these messages are difficult to hear and convey publicly with the appropriate level of care and nuance. I tried my best to be both honest and kind, but I likely still screwed up the balance at various parts. I ask for your patience.

Summary

Important lessons

(Refresher) The top grants are much much better than the marginal grants, even ex ante.

All of the grantmakers are extremely busy

Most (within-scope) projects are usually rejected for personal factors or person-project fit reasons

Some of your most impactful work in grantmaking won’t look like evaluating grants

Most grantmakers don’t (and shouldn’t) spend a lot of time on evaluating any given grant.

It’s rarely worth your time to give detailed feedback

Less Important Lessons

The value of time for people in EA is incredibly high

The direct counterfactual impact of grant evaluation is lower than I initially would have guessed

People are often grateful to you for granting them money. This is a mistake.

Conflicts of Interests are unavoidable if you want to maximize impact as a grantmaker in EA.

Acceptance/Rejection rates can easily be extremely misleading

(Refresher) The top grants are much much better than the marginal grants, even ex ante.

I think it’s well understood in EA that for most important values of X in the EA space (charitable interventions, organizations, careers, etc,), the best X are much more impactful than the marginal X.

I think I have internalized this more after I started grantmaking than before. In particular, I now understood that (from the perspective of many grantmakers), their top grants are ex ante much more impactful than their marginal grants. This lesson has publicly been discussed by other grantmakers, see e.g.

I think the top decile is at least an order of magnitude more impactful (per dollar) than the bottom decile grantmaking that we’re doing

Claire

I think that more than half of the impact I have via my EAIF grantmaking is through the top 25% of the grants I make. And I am able to spend more time on making those best grants go better, by working on active grantmaking or by advising grantees in various ways.

Buck

In practice, this often means that your highest impact actions as a grant...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Are human imitators superhuman models with explicit constraints on capabilities?, published by Chris van Merwijk on May 22, 2022 on LessWrong.

epistemic status: A speculative hypothesis, don't know if this already exists. The only real evidence I have for this is a vague analogy based on some other speculative (though less speculative) hypothesis. I don’t think this is particularly likely to be true, having thought about it for about a minute. Edit: having thought about it for two minutes, it seems somewhat more likely.

There is a hypothesis that has been floating around in the context of explaining double descent, that what so called over-parameterized models do is store in parallel two algorithms: (1) the simplest model of the data without (label) noise, and (2) a “lookup table” for deviations from that model, to represent the (label) noise, because this is the most simple representation of the data and big neural nets are biased towards simplicity.

Maybe something vaguely similar would happen if we throw sufficient compute at generative models of collections of humans, e.g. language models:

Hypothesis: The simplest way to represent the data generated by various humans on the internet, in the limit of infinite model size and data, is to have (1) a single idealized super-human model for reasoning and writing and knowledge-retrieval and so forth, and (2) memorized constraints on this model for various specific groups of humans and contexts to represent their deviations from rationality, specific biases, cognitive limitations, lack of knowledge of certain areas, etc.

This maybe implies, using some eye squinting and handwaving and additional implicit assumptions, some of the following (vague, speculative) implications about GPT-N:

In the limit of N, GPT-N will produce text that sufficiently looks like human-written text within contexts (prompts) that humans typically produce. GPT-N will use human-level reasoning, world-modeling, and planning abilities to produce this text. However, if you give it sufficiently out-of-distribution prompts, its lookup table for specific irrationalities will not apply, and it will apply superhuman planning and world-modeling and reasoning abilities that are more competent and more free of biases than the most rational human.

In the limit of N, If you take GPT-N and fine tune it on an RL task that requires good reasoning, it might be possible to get a system that seems to behave far more intelligently than it seemed to be on the imitation task, as the fine tuning essentially turns off constraints on its reasoning abilities.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Hiring: How to do it better, published by Joseph Lemien on May 23, 2022 on The Effective Altruism Forum.

The waste of human resources caused by poor selection procedures should pain the professional conscience of I–O psychologists.

Short Summary: I think that EA organizations can do hiring a lot better. If you are involved in hiring, you should do these three things: avoid unstructured interviews, train your interviewers to do structured interviews, and have a clear idea of what your criteria are before you start to review and filter applicants. Feel free to jump to the “Easy steps you should implement” section if you don’t want to go through all the details.

Intro

If we view a hiring process as an attempt to predict which applicant will be successful in a role, then we want to be able to predict this as accurately as we can. That is what this post will be about: how can we increase the accuracy of our predictions for something that is as complex as human beings. Whenever I write about predicting success in this post, it is about trying to predict whether (and to what extent) a job applicant would be successful if hired.

This post will be full of suggestions about hiring. Like all suggestions anyone gives you about how to do things, feel free to ignore aspects that you think aren’t applicable to your situation.

What this article is not, and epistemic status

This isn’t original research. This is mostly just me summarizing things I’ve read, with a dash of my own perspectives sprinkled in here and there. I think it should be quite clear from context which ideas are my own, but if you have any questions about that please do make a comment and I will clarify.

This also isn’t legal advice for employment law, for avoiding adverse impact, etc.

This isn’t an operational plan for how to implement a hiring system. While I have designed hiring systems in the past and most good hiring systems should make use of the techniques/tools described here, describing how to put these ideas into practice would be a separate post.

And of course, the standard caveat that these general recommendations might not fit the particular situation you face.

Although I asked a few strangers to review this, this is not a professionally edited piece of writing: all errors are my own.

My background

I ran a good deal of operations and a good deal of hiring (as well as some other HR functions) for a couple of small organizations (headcount 20-60), and I had some insight into the hiring practices of a larger organization (headcount of about 2500) when I worked as a director in the HR department there. I've read some books that are relevant to hiring; this includes books on mistakes/biases in decision making, books specifically about human resources, as well general business books which have given me relevant tidbits. I am wrapping up a graduate level course in Organizational Behavior as part of my effort to earn a master's degree in management. I've recently become aware of the Scientist–Practitioner Gap (sometimes called the The Research-Practice Gap), and I'm interested in it.

I've discovered a few academic papers recently by following and exploring the sources cited in my textbook and in my professors lectures, and Stubborn Reliance on Intuition and Subjectivity in Employee Selection has inspired me to write this post.

I want to emphasize that I am not an expert on personnel selection. I am a man who has a few years of work experience in operations and HR at small organizations, and who designed hiring processes for a couple of startup-y teams. Over the past few months I've read some relevant journal articles, over the past several years I've read a lot of relevant popular press articles and books, and I am about 1/12 of the way through a relevant master's degree. I'd encourage you to view this not as an...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Funds donation platform is moving to Giving What We Can, published by Luke Freeman on May 23, 2022 on The Effective Altruism Forum.

Since late 2021 Effective Altruism Funds (EA Funds) and Giving What We Can (GWWC) have been going through a restructure. EA Funds has moved towards focussing primarily on grantmaking and GWWC has taken over the management of the donation platform.

In April 2022 we soft-launched a rebranded GWWC donation portal. Over the coming months the donation specific functionality of funds.effectivealtruism.org will be retired and redirected to GWWC's version of the donation platform (the pages related to the four EA Funds and grantmaking will continue on funds.effectivealtruism.org and the homepage will become more grantee focused). A new GWWC website is currently under development which will include a fully integrated donation experience (as well as improving our pledge dashboard and signup process).

As well as the donation platform, GWWC will continue to support the donor lottery, EffectiveCrypto.org and any of the other formerly EA Funds activities relating to fundraising while maintaining our existing work promoting effective giving (e.g. pledge, community, guides, talks, marketing campaigns and research).

EA Funds will continue to manage the grantmaking activities of their four Funds and will at some point post an update about their plans moving forward and this includes some of the reasoning for this restructure decision. GWWC has also recently posted an update about our strategy which is very relevant to this decision.

We will be consulting with donors, stakeholders and the broader community about the future of the donation platform and how we can best support effective giving within the community.

Please don't hesitate to get in touch with any feedback, suggestions, requests, or concerns.

We look forward to this next chapter and are excited to continue our mission to create a world where giving effectively and significantly is a cultural norm!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AXRP Episode 15 - Natural Abstractions with John Wentworth, published by DanielFilan on May 23, 2022 on The AI Alignment Forum.

Google Podcasts link

Why does anybody care about natural abstractions? Do they somehow relate to math, or value learning? How do E. coli bacteria find sources of sugar? All these questions and more will be answered in this interview with John Wentworth, where we talk about his research plan of understanding agency via natural abstractions.

Topics we discuss:

Agency in E. coli

Agency in financial markets

Inferring agency in real-world systems

Selection theorems

(Natural) abstractions

Information at a distance

Why the natural abstraction hypothesis matters

Unnatural abstractions in humans

Probability, determinism, and abstraction

Whence probabilities in deterministic universes?

Abstraction and maximum entropy distributions

Natural abstractions and impact

Learning human values

The shape of the research landscape

Following John’s work

Daniel Filan: Hello, everybody. Today I’ll be speaking with John Wentworth, an independent AI alignment researcher who focuses on formalizing abstraction. For links to what we’re discussing, you can check the description of this episode and you can read the transcripts at axrp.net. Well, welcome to AXRP, John.

John Wentworth: Thank you. Thank you to our live studio audience for showing up today.

Agency in E. coli

Daniel Filan: So I guess the first thing I’d like to ask is I see you as being interested in resolving confusions around agency, or things that we don’t understand about agency. So what don’t we understand about agency?

John Wentworth: Whew. All right. Well, let’s start chronologically from where I started. I started out, first, in two directions in parallel. One of them was in biology, like systems biology and, to some extent, synthetic biology, looking at E. coli. Biologists always describe E. coli as collecting information from their environment and using that to keep an internal model of what’s going on with the world, and then making decisions based on that in order to achieve good things.

John Wentworth: So they’re using these very agency-loaded intuitions to understand what’s going on with the E. coli, but the actual models they have are just these simple dynamical systems, occasionally with some feedback loops in there. They don’t have a way to take the low-level dynamics of the E. coli and back out these agency primitives that they’re talking about, like goals and world models and stuff.

Daniel Filan: Sorry, E. coli, it’s a single-celled organism, right?

John Wentworth: Yes.

Daniel Filan: Is the claim that the single cells are taking in information about their environment and like . Right?

John Wentworth: Yes. One simple example of this is chemotaxis. So you’ve got this E. coli. It’s swimming around in a little pool of water and you drop in a sugar cube. There’ll be a chemical gradient of sugar that drops off as you move away from the grain of sugar. The E. coli will attempt to swim up that gradient, which is actually an interesting problem because when you’re at a length scale that small, the E. coli’s measurements of the sugar gradient are extremely noisy. So it actually has to do pretty good tracking of that sugar gradient over time to keep track of whether it’s swimming up the gradient or down the gradient.

Daniel Filan: Is it using some momentum algorithm, or is it just accepting the high variance and hoping it’ll wash out over time?

John Wentworth: It is essentially a momentum algorithm.

Daniel Filan: Okay, which is basically, roughly, continuing to move in the direction it used to move, something like that.

John Wentworth: Yes. Basically it tracks the sugar concentration over time. If it’s trending upwards, it keeps swimming and if it’s trending downwards, it just stops and tumbles in place...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: PSA: The Sequences don't need to be read in sequence, published by kave on May 23, 2022 on LessWrong.

This week, I hung out with the LessWrong team while they talked to relatively new users. New users often had a vague intention to read Eliezer's original Sequences, but were blocked on the size of the project. They thought the Sequences would only work in, well, sequence.

I just polled eight people (including me) who have read the Sequences whether they only work in sequence.

7 people said they work out of sequence (though 2 noted that it might be better to read a given sequence in order)

1 person said they thought it was necessary to read any given sequence in order, but it didn't matter if you read one sequence (e.g. A Human's Guide to Words) before or after another (e.g. Mysterious Answers to Mysterious Questions)

A typical sequence post has many links to other sequence posts. But these are mostly context and elaboration. The posts tend to work well standalone.

Here are three posts you might get started with:

Fake Explanations. The first post in the Mysterious Answers to Mysterious Questions sequence, described by Eliezer as "probably the most important core sequence in Less Wrong".

Leave a Line of Retreat. Letting go of a belief that's important to you is hard. Particularly if you think stuff you care about depends on it (e.g. if you think being good depends on moral realism, it could be hard to reexamine your belief in moral realism). This post describes the phenomenon, and gives advice for dealing with it.

The Hidden Complexity of Wishes. Imagine you had a device that could cause any concrete statement to become true. This post explores the difficulties you would have getting what you want with the device.

(Thanks to various people at the Lightcone offices for beta reading this post, particularly Ben Pace).

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Gradations of Agency, published by Daniel Kokotajlo on May 23, 2022 on The AI Alignment Forum.

[Epistemic status: This post written hastily, for Blog Post Day.]

I’ve de-prioritised this sequence due to other recent posts which cover a lot of the same ground. Moreover I begin to suspect that the theory of agency I’m building is merely the glowing brain and that Yudkowsky-Veedrac's is the cosmic galaxy brain. Perhaps mine can be a useful stepping-stone.

Consider the messy and large real world, with creatures reproducing, competing, and evolving. The behavior of each entity is controlled by some sort of cognition/computation—some sort of algorithm.

Inspired by something Dan Dennett wrote (h/t Ramana Kumar), I propose the following loose hierarchy of algorithm families. As we move up the hierarchy, things get more complicated and computationally expensive, but also more powerful and general.

Level 1: Do what worked in the past for your ancestors. You have some “input channel” or “senses” and then you respond to it somehow, doing different things in response to different inputs.

Example: You swim towards warmth and away from cold and hot. You run from red things and towards green things.

This is more complex and computationally expensive, but (if done right) more powerful and general than algorithms which aren’t environment-responsive: There are many situations, many niches in the environment, where being able to sense and adapt quickly helps you survive and thrive.

Level 2: Do what worked in the past for you. How you respond to inputs is itself responsive to inputs; you learn over the course of your life. This typically involves some sort of memory and a special input channel for positive and negative reward.

Example: You start off reacting randomly to inputs, but you learn to run from red things and towards green things because when you ran towards red things you got negative reward and when you ran towards green things you got positive reward.

This is more complex and computationally expensive, but (if done right) more powerful and general:

For a particular ecological niche, you & your relatives will reproduce and fill the niche more quickly than rivals from Level 1. This is because your rivals in some sense keep doing the same thing until they die, and will thus only “learn” to exploit the niche by gradual natural selection. Whereas you & your kind can be dropped straight into the niche and learn to

Also, there are some niches that Level 1 creatures simply cannot survive in, that you can. Example: In Pond X, the color of food and the color of predators constantly changes on timescales of about a lifetime, too fast for a population of Level 1 creatures to adapt.

Level 3: Do what worked in the past for things similar to you in similar situations. You have some ability to judge similarity, and in particular to notice and remember when your current situation is similar to the situation of something else you saw, something you classify as similar to you. Armed with this ability you can learn not just from your own experience, but from the experience of others—you can identify successful others and imitate them.

This is more complex and computationally expensive, but (if done right) more powerful and general:

For a particular ecological niche, you & your relatives will reproduce and fill the niche more quickly than rivals from level 2. This is because you can learn from the experiences of others while level 2 creatures have to experience things themselves.

Also, there are some niches that level 2 creatures simply cannot survive in, that you can. For example, niches in which certain behaviors in certain situations are deadly.

Level 4: Do what worked in your head when you imagined various plans. Whereas with Level 3 you processed your memories into the sort of world-model tha...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Adversarial attacks and optimal control, published by Jan Hendrik Kirchner on May 22, 2022 on The AI Alignment Forum.

Meta: After a fun little motivating section, this post goes deep into the mathematical weeds. This thing aims to explore the mathematical properties of adversarial attacks from first principles. Perhaps other people are not as confused about this point as I was, but hopefully, the arguments are still useful and/or interesting to some. I'd be curious to hear if I'm reinventing the wheel.

TL;DR:

When we invest the appropriate effort, the probability of random catastrophic events tends to decrease exponentially with a rate given by the rate function.

One way of estimating the rate function is to solve an optimal control problem, where we have to determine the "least surprising" way that the catastrophic event comes about. The rate function equals the catastrophic event's surprise (in nats).

Thus, intuitively, as we invest the effort to decrease the probability of random catastrophic events, the "difficulty" of performing an adversarial attack only increases linearly.

Real talk and real estate

Zillow is an American tech real-estate marketplace company that recently had (what the experts call) a small snafu. They decided they were done just being a marketplace and started buying up homes, completing light renovations, and then selling them with a profit. The whole thing went poorly; they bought houses too expensive and had to sell at a loss, costing the company $420 million and leading to large lay-offs.

This story is not very interesting for anyone not directly involved. The reason I remember the whole affair is a Twitter thread that also caught the attention of Eliezer:

The thread proceeds to lay out how Zillow relied too much on their automatic estimates, which (ex-ante) looked good on average, but (ex-post) manifested in the worst possible way. Or, in the words of the OP:

"[They mistook] an adversarial environment for a random one."

I have approximately zero knowledge about Zillow or the real estate business, so I can't comment on whether this actually is what went wrong (edit: see this comment for a more informed take!). But the distinction between a random and an adversarial environment applies beyond the Zillow example and is relevant for research in AI Safety: from adversarial examples, via adversarial training, all the way to conceivable existential risk scenarios. It is the distinction between a world where it's fine to go outside because getting struck by lightning is sufficiently unlikely and a world where somebody is trying to smite you.

This post works towards achieving a mathematical understanding of what distinguishes these two worlds. Perhaps other people are not as confused about this point as I was, but hopefully, the arguments are still useful and/or interesting to some. I'll introduce a bit of extreme value theory. Then I'll demonstrate a neat connection to control theory via large deviation theory, which shows that an exponentially decreasing risk of failure translates into a linearly increasing difficulty of adversarial attacks.

A random environment

I'll go over the basics quickly: When you (i.e. Zillow) think of your environment as random, each house you buy is essentially a coin flip with a biased coin. If you make sure that

the expected value of a deal is positive and

that all the deals are (mostly) independent of each other,

then you're guaranteed to accrue profits eventually.

This guarantee is the law of large numbers from probability theory 101 and probably not too revolutionary for you, dear reader. But what is the probability of things going very badly? What is the probability of an extreme event or a large deviation? It should decrease (given the law of large numbers), but how fast does it decrease?

Let's continue the exampl...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Future of Nuclear War, published by avturchin on May 21, 2022 on LessWrong.

Here I present a view on the future of nuclear war which takes into account the expected technological progress as well as global political changes.

There are three main directions in which technological progress in nuclear weapons may happen:

1) Many gigaton weapons.

2) Cheaper nuclear bombs which are based on the use of the reactor-grade plutonium, laser isotope separation or are hypothetical pure fusion weapons. Also, advanced nanotechnology will provide the ability to quickly build large nuclear arsenals and AI could be used in designing, manufacturing and nuclear strategy planning.

3) Specialized nuclear weapons like nuclear-powered space lasers, hafnium bombs and nuclear-powered space ships as kinetic weapons.

Meanwhile, the nuclear war strategy also has changed as the bipolar world has ended and as new types of weapons are becoming available.

The first strategy change is that Doomsday weapons for blackmail will become attractive for weaker countries which can’t use ICBM to penetrate the anti-missile defense of the enemies.

Secondly, the cheaper nukes will become available to smaller actors, who may be involved in “worldwide guerilla”. Cheaper nukes and a larger number of actors also encourage regional nuclear wars, nuclear terrorism and anonymous or false-flag warfare. This will result in disruption of social complexity and global food production shortages because of effects on climate.

The third change of the strategy is the use of nukes not against primary military targets but on other objects, which could increase their effects: nuclear power plants, supervolcanos, EMP, tsunamis, taiga fires for global cooling, and even more hypothetical things like asteroid deflection to Earth.

All these unconventional means and strategies of nuclear war could become available somewhere in the 21st century and may cause or contribute to the civilizational collapse and even human extinction.

Permalink:

Introduction

When we speak about global catastrophic risks connected with new technologies, we assume that in the decades from now these technologies will advance by many orders of magnitude. AI will become much stronger, nanobots will appear and biohackers will get access to better manufacturing capabilities. But the discussions about nuclear war assumed that this war will be the same as it was envisioned in the 20th century: a nuclear exchange between two superpowers, which, in the worst case, will cause a “nuclear winter” for several years, which could cause mass starvation of survivors, but not a human extinction. The highest number of deployed nukes was achieved in the 1980s and declined several times after that. So, it looks like the scale of possible nuclear war and thus its danger has declined.

However, if we assume that technological progress will continue also in the nuclear field, the same way as it happens in other fields, such a decline in nuclear numbers maybe only a temporary fluke.

It is reasonable to suggest that assumed exponential growth in AI and nanotech will be reflected in exponential growth in nuclear technologies, but we currently observe such growth only in civilian nuclear applications, like the number of approaches to fusion.

Objections to the idea of the significant future of nuclear weapons

There are several objections which immediately appear in mind when we speak about the future of nuclear war, and which generally prevent us from digging deeper into the topic. I will shortly address these objections first:

Drones and precise weapons. One may object that there is no need in nuclear development as drones could inflict similar damage via precision strikes (Turchin & Denkenberger, 2018). But owning nukes was always seen as a symbol of ultimate military power, as they c...