Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Cyborg Periods: There will be multiple AI transitions, published by Jan Kulveit on February 22, 2023 on LessWrong.

It can be useful to zoom out and talk about very compressed concepts like ‘AI progress’ or ‘AI transition’ or ‘AGI timelines’. But from the perspective of most AI strategy questions, it’s useful to be more specific.

Looking at all of human history, it might make sense to think of ourselves as at the cusp of an AI transition, when AI systems overtake humans as the most powerful actors. But for practical and forward-looking purposes, it seems quite likely there will actually be multiple different AI transitions:

There will be AI transitions at different times in different domains

In each of these domains, transitions may move through multiple stages:

DescriptionPresent day examplesHumans clearly outperform AIs. At some point, AIs start to be a bit helpful.Alignment research, high-level organisational decisions. Humans and AIs are at least comparably powerful, but have different strengths and weaknesses. This means that human+AI teams outperform either unaided humans, or pure AIs.Visual art, programming, trading.AIs overtake humans. Humans become obsolete and their contribution is negligible to negative.Chess, go, shogi.

Stage

[ = more powerful than]

Human period:

Humans AIs

Cyborg period: Human+AI teams humans

Human+AI teams AIs

AI period:

AIs humans

(AIs ~ human+AI teams)

Some domains might never enter an AI period. It’s also possible that in some domains the cyborg period will be very brief, or that there will be a jump straight to the AI period. But:

We’ve seen cyborg periods before

Global supply chains have been in a cyborg period for decades

Chess and go both went through cyborg periods before AIs became dominant

Arguably visual art, coding and trading are currently in cyborg periods

Even if cyborg periods are brief, they may be pivotal

More on this below

This means that for each domain, there are potentially two transitions: one from the human period into the cyborg period, and one from the cyborg period into the AI period.

Transitions in some domains will be particularly important

The cyborg period in any domain will correspond to:

An increase in capabilities (definitionally, as during that period human+AI teams will be more powerful than humans were in the human period)

An increase in the % of that domain which is automated, and therefore probably an increase in the rate of progress

Some domains where increased capabilities/automation/speed seem particularly strategically important are:

Research, especially

AI research

AI alignment research

Human coordination

Persuasion

Cultural evolution

AI systems already affect cultural evolution by speeding it up and influencing which memes spread. However, AI doesn’t yet play a significant role in creating new memes (although we are at the very start of this happening). This is similar to the way that humans harnessed the power of natural evolution to create higher yield crops without being able to directly engineer at the genetic level

Meme generation may also become increasingly automated, until most cultural change happens on silica rather than in brains, leading to different selection pressures

Strategic goal seeking

Currently, broad roles involving long-term planning and open domains like "leading a company" are in the human period

If this changes, it would give cyborgs additional capabilities on top of the ones listed above

Some other domains which seem less centrally important but could end up mattering a lot are:

Cybersecurity

Military strategy

Nuclear command and control

Some kinds of physical engineering/manufacture/nanotech/design

Chip design

Coding

There are probably other strategically important domains we haven’t listed.

A common feature of the domains listed is that increased capabilities in...