Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The AGI Race Between the US and China Doesn’t Exist., published by Eva B on June 3, 2023 on LessWrong.

When I write “China”, I refer to the political and economic entity, the People’s Republic of China, founded in 1949.

Leading US AI companies are currently rushing towards developing artificial general intelligence (AGI) by building and training increasingly powerful large language models (LLMs), as well as building architecture on top of them. This is frequently framed in terms of a great power competition for technological supremacy between the US and China.

However, China has neither the resources nor any interest in competing with the US in developing artificial general intelligence (AGI) primarily via scaling Large Language Models (LLMs).

In brief, China does not compete with the US in developing AGI via LLMs because of:

Resources and technology: China does not have access to the computational resources (compute, here specifically data centre-grade GPUs) needed for large-scale training runs of large language models. This gap will only widen over time; China is failing to develop a domestic semiconductor industry, despite massive efforts to do so, and is increasingly cut off from international semiconductor supply chains.

Political goals: The Chinese Communist Party (CCP) is above all concerned with ensuring social stability and staying in power. It will not tolerate large language models that cannot reliably be controlled and prevented from leaking sensitive information, raising politically sensitive topics, or challenging the regime. Reliably prohibiting LLMs from doing any of these things is an unsolved technical problem. This has and will continue to stifle the development and deployment of Chinese LLMs for the Chinese market, as a single political faux pas will spell doom for the companies involved.

Therefore, as it currently stands, there is no AGI race between China and the US.

China Cannot Compete with the US to Reach AGI First Via Scaling LLMs

Today, training powerful LLMs like GPT-4 requires massive GPU clusters. And training LLMs on a larger number of GPUs is the most common and reliable way of increasing their capabilities. This is known as the bitter lesson in machine learning; increasing the compute a model is trained on leads to higher progress in the model's capabilities than any changes in the algorithm's architecture. So if China wants to “win the AGI race” against the US, it would need to maximise its domestic GPU capacities. However, China lacks domestic semiconductor production facilities for producing data centre-grade GPUs. So to increase domestic GPU capacities, it has to import GPUs or semiconductor manufacturing equipment (SME).

However, since the US introduced its export controls on advanced chips and SMEs in October 2022, China can no longer purchase GPUs or SMEs abroad. And with the semiconductor industry advancing at lightning speed, China is falling further and further behind every day. These policies are just the most recent ones in a row of ongoing measures by the US restricting the transfer of semiconductor technology to Chinese companies, beginning with Huawei in 2019. In early 2023, the Netherlands and Japan, which play a key role in the global semiconductor manufacturing chain, joined the US and introduced SME export controls targeting China. Crucially, these combined export controls apply to ASML’s most advanced lithography machines, the only machines in the world which can produce cutting-edge chips.

The problem for China lies not so much in designing cutting-edge GPUs, but in designing and building SMEs to produce the advanced GPUs needed for training LLMs, a far more challenging task. But China has been trying - and failing - to build domestic semiconductor manufacturing chains for cutting-edge chips since at least the 199...