Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: We Need Holistic AI Macrostrategy, published by NickGabs on January 15, 2023 on LessWrong.

Summary

AI Macrostrategy is the study of high level questions having to do with prioritizing the use of resources on the current margin in order to achieve good AI outcomes. AI macrostrategy seems important if it is tractable. However, while few people are working on estimating particular parameters relevant to macrostrategy, even fewer are working on developing holistic macrostrategic models that combine estimates for different parameters to guide our actions. Moreover, while macrostrategy was less tractable in the past, recent developments (especially increased evidence for <10 year timelines) have made macrostrategy substantially more tractable. Thus, using the importance/tractability/neglectedness heuristics from EA, I conclude that on current margins macrostrategy should be a top priority.

Acknowledgements:

Thanks to Chinmay Deshpande, Carson Ezell, Nikola Jurkovic, and others for helping me to develop many of the ideas in this post, and to Thomas Larsen for both doing this and helping to directly edit the post.

Epistemic status:

Speculative, but I think the arguments are pretty straightforward and so I have >70% confidence in the main conclusion that more macrostrategy work should be done on current margins relative to other kinds of alignment work.

What is AI Macrostrategy?

AI Macrostrategy (henceforth just macrostrategy) is the study of high level questions having to do with prioritizing the use of resources to achieve good AGI outcomes on the current margin.

Macrostrategic work can be divided broadly into two categories:

Parameter estimates: attempts to forecast key variables such as timelines, takeoff speeds, and the difficulty of aligning AGI

Holistic macrostrategy: attempts to combine these estimates and other pieces of data into a coherent, action-guiding model of AI alignment.

For examples of macrostrategic questions, Holden mentions several central macrostrategic questions in this post.

Importance of Macrostrategy

I think that attempting to answer macrostrategic questions is extremely important for four primary reasons.

Importance of Prioritization in Heavy Tailed Domains

It is widely accepted among Effective Altruists that the distribution of impactfulness among different cause areas is heavy tailed. However, while I expect that the distribution of impactfulness among different AI interventions is not as heavy tailed as the distribution of impactfulness among cause areas in general, I do expect it to be at least somewhat heavy tailed, with the best interventions being >2 orders of magnitude more effective in expectation than the median intervention. Thus, it is critical to identify the best interventions rather than settling for interventions that seem vaguely pointed in the direction of solving alignment/making AGI go well. However, identifying these interventions requires some kind of macrostrategic model. Thus, applying the basic heuristic that prioritization is important in heavy tailed domains already suggests that macrostrategy is quite important.

Achieving The Best Long Term Outcomes Requires Macrostrategy

In addition to the distribution of the impactfulness of different AI interventions, the distribution of value across possible long run futures is also likely to be heavy tailed. This is because if what happens in the long run future will be controlled by powerful optimizers such as superintelligent AI, then due to the fact that tails come apart, most of the expected value relative to a particular utility function lies in futures where the powerful optimizers controlling the future in question are optimizing that specific utility function (or something extremely close to it). As a result, if you have consequentialist values, you should be focused on tryi...