Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Dial of Progress, published by Zvi on June 13, 2023 on LessWrong.

“There is a single light of science. To brighten it anywhere is to brighten it everywhere.” – Isaac Asimov

You cannot stand what I’ve becomeYou much prefer the gentleman I was beforeI was so easy to defeat, I was so easy to controlI didn’t even know there was a war

– Leonard Cohen, There is a War

“Pick a side, we’re at war.”

– Steven Colbert, The Colbert Report

Recently, both Tyler Cowen in response to the letter establishing consensus on the presence of AI extinction risk, and Marc Andreessen on the topic of the wide range of AI dangers and upsides, have come out with posts whose arguments seem bizarrely poor.

These are both excellent, highly intelligent thinkers. Both clearly want good things to happen to humanity and the world. I am confident they both mean well. And yet.

So what is happening?

A Theory

My theory is they and similar others believe discourse in 2023 cannot handle nuance.

Instead, effectively there is a single Dial of Destiny Progress, based on the extent our civilization places restrictions, requires permissions and places strangleholds on human activity, from AI to energy to housing and beyond.

If we turn this dial down, and slow or end such progress across the board, our civilization will perish. If you turn the dial up, we will prosper.

In this view, any talk of either extinction risks or other AI downsides is functionally an argument for turning the dial down, saying Boo Progress, when we instead desperately need it turn the dial up, and say Yay Progress.

It would, again in this view, be first best to say Yay Progress in most places, while making a careful narrow exception that lets us guard against extinction risks. Progress is no use if you are dead.

Alas, this is too nuanced, and thus impossible. Trying will not result in the narrow thing that would protect us. Instead, trying turns the dial down which does harm everywhere, and also does harm within AI because the new rules will favor insiders and target mundane utility without guarding against danger, and the harms you do elsewhere inhibit sane behavior.

Thus, the correct thing to do is shout Yay Progress from the rooftops, by whatever means are effective. One must think in terms of the effects of the rhetoric on the dial and the vibe, not on whether the individual points track underlying physical reality. Caring about individual points and how they relate to physical reality, in this model, is completely missing the point.

This doesn’t imply there is nothing to be done to reduce extinction risks. Tyler Cowen in particular has supported at least technical private efforts to do this. Perhaps people in National Security or high in government, or various others who could help, could have their minds changed in good directions that would let us do nuanced useful things. But such efforts must be done quietly, among the cognoscenti and behind the scenes, a la ‘secret congress.’

While I find this model incomplete, and wish for higher epistemic standards throughout, I agree that in practice this single Dial of Progress somewhat exists.

Also Yay Progress.

Robin Hanson explicitly endorses the maximalist Yay Progress position, expecting human extinction but considering the alternative even worse.

This post fleshes out the model, its implications, and my view of both.

Consider a Dial

What if, like the metaphorical single light of science, there was also a single knob of (technological, scientific, economic) progress?

Collectively, the sum of little decisions, the lever is moved.

If we turn the dial up, towards Yay Progress, we get more progress.

If we turn the dial down, towards Boo Progress, we get less progress.

As the dial is turned up, people are increasingly empowered to do a wide variety of useful and productive things, without ...