Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How much is reducing catastrophic and extinction risk worth, assuming XPT forecasts?, published by rosehadshar on July 25, 2023 on The Effective Altruism Forum.

This is a post I drafted some months ago, in the course of analysing some XPT data and reading Shulman and Thornley. It's not very sophisticated, I haven't checked the workings, and I haven't polished the language; but I'm posting anyway because that seems better than not posting. Note that it's a personal take and doesn't represent FRI's views.

Thanks to Josh Rosenberg at FRI and Elliot Thornley for help and comments.

BLUF: if you make a bunch of assumptions, then even quite low absolute risk forecasts like the XPT ones imply quite high spending on reducing GCRs, conditional on there being sufficiently cost-effective ways to do so.

In 2022, what has become the Forecasting Research Institute ran the Existential Risk Persuasion Tournament (XPT). Over 200 forecasters, including superforecasters and domain experts, spent 4 months making forecasts on various questions related to existential and catastrophic risk.

You can see the results from the tournament overall here, and a discussion of the XPT AI risk forecasts in particular here.

These are the main XPT forecasts on catastrophic and extinction risk:

Biological--Engineered pathogens--Natural pathogens--AI (superforecasters)AI (domain experts)NuclearNon-anthropogenicTotal catastrophic riskBiological--Engineered pathogens--Natural pathogens--AI (superforecasters)AI (domain experts)NuclearNon-anthropogenicTotal extinction risk

2030

2050

2100

Catastrophic risk (>10% of humans die in 5 years)

1.8%

0.8%

1%

0.01%

0.73%

2.13%

0.35%

5%

12%

0.50%

1.83%

4%

0.0026%

0.015%

0.05%

0.85%

3.85%

9.05%

Extinction risk (human population <5000)

0.012%

0.01%

0.0018%

0.0001%

0.03%

0.38%

0.02%

1.1%

3%

0.001%

0.01%

0.074%

0.0004%

0.0014%

0.0043%

0.01%

0.3%

1%

If we take these numbers at face value, how much is catastrophic and extinction risk reduction worth?

One approach is to take the XPT forecasts, convert them into deaths in expectation, then assume a value of a statistical life and a discount rate, and estimate how much averting those deaths is 'worth'. (I'm stealing this method directly from Shulman and Thornley.)

Using the XPT superforecasts and OWID population projections gives us the following deaths in expectation, in millions:

Bio----AINuclearNon-anthropogenicTotalBio----AINuclearNon-anthropogenicTotal

Deaths in expectation (millions)

2030

2050

2100

Catastrophic risk (>10% of humans die in 5 years)

18.6

0.09

7.1

22.0

4.3

17.8

41.4

0.02

0.15

0.5

7.3

37.4

93.7

Extinction risk (human population <5000)

1.2

0.01

2.9

39.3

0.09

1.0

7.7

0.03

0.1

0.4

0.9

29.1

103.5

Some notes:

Workings here.

For catastrophic risks, the deaths in expectation should be read as a lower bound, because they assume 10% deaths and the question includes scenarios with >10% deaths.

That's deaths in expectation worldwide. But the value of a statistical life varies by country: governments have different resources and the cost of interventions in different places varies.

So the most straightforward way to think about the worth of catastrophic and extinction risk reduction is to ask how much this would be worth in a given country. Let's take the US as an example.

First we need US deaths in expectation:

Bio----AINuclearNon-anthropogenicTotalBio----AINuclearNon-anthropogenicTotal

US deaths in expectation (millions)

2030

2050

2100

Catastrophic risk (>10% of humans die in 5 years)

0.7

0.004

0.3

0.8

0.2

0.7

1.6

0.001

0.006

0.02

0.3

1.4

3.6

Extinction risk (human population <5000)

0.05

0.0004

0.1

1.5

0.004

0.04

0.3

0.001

0.01

0.02

0.04

1.1

3.9

Workings here.

We can then assume a US value for a statistical life, and a discount rate, and use these to estimate how much averting the deaths in expectation is 'worth' to the...