Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The algorithm isn't doing X, it's just doing Y., published by Cleo Nardo on March 16, 2023 on LessWrong.

Introduction

Mutual reduction implies equivalence

Here's my most load-bearing intuition

If two tasks reduce to one another, then it is meaningless to ask if a machine is 'really doing' one task versus the other.

Moreover

This intuition grounds my perspective on intelligence, AI, alignment, philosophy, etc.

This intuition is load-bearing for other people who share my views.

This intuition is a crux for much of the disagreement we have with other people.

In this article, I'll formalise this intuition in two ways, computational and physical.

Motivation

People often say "the algorithm isn't doing X, it's just doing Y".

X is normally some impressive high-level human-y thing, such as

writing poetry

causal reasoning

recognising emotions

interpreting art

writing music

making ethical decisions

planning actions

telling jokes

understanding concepts

simulating agents, etc.

Y is normally some unimpressive low-level computery thing, such as

predicting tokens

sampling from a distribution

querying a lookup table

multiplying matrices

sorting numbers

clustering data points

compressing text

searching a tree

manipulating bitstrings

polarising magnetic strips, etc.

Rather than address each example individually, I think it'll be more efficient to construct a general criterion by which we can assess each example.

Click here for the specific example of LLMs.

This criterion doesn't actually matter

I should stress that this criterion doesn't actually matter for AI x-risk, because you can always reframe the risks in terms of Y, and not mention X at all. However, that might cost you more ink.

ME, a visionary: GPT-4 is misaligned because it's simulating deceptive agents.YOU, a fool: GPT-4 isn't simulating any agents, it's just predicting which tokens continue a prompt.ME, a correct-opinion-haver: Fine, whatever... GPT-4 is misaligned because it predicts the tokens continuing a prompt by applying a function parameterised in a high-dimensional space to minimise cross-entropy loss across the internet corpus and the internet corpus contains a lot of conversations where one character deceives another and therefore GPT-4 will respond in the same way that a deceptive character would do so.

The X-Y Criterion

Informal statement

Okay, here's the X-Y Criterion:

If two tasks reduce to one another, then it is meaningless to ask if a machine is 'really doing' one task versus the other.

Don't worry, later in the article we'll formalise what "task", "reduce", and "doing" means.

First draft — computational reduction

Our first draft will be "computational reduction".

A task X is about processing classical information, i.e. X:{0,1}∗{0,1}∗.

An algorithm A achieves a particular task X if it processes classical information in that way.

In order to achieve a task X, the algorithm A expends certain quantities of computational resources, e.g. time, memory, samples, bandwidth, etc. These resources are abstract and non-physical.

A task X reduces to task Y if and only if...For every algorithm A that solves task Y, there exists another algorithm B such that...(1) B solves task X by interacting with A.(2) The combined algorithm (A⊗B) doesn't expend much more computational resources to solve X as A expends to solve Y.

X-Y Criterion: If two tasks X and Y reduce to one another, then it is meaningless to ask if an algorithm A is 'really doing' one task versus the other.

This is what computer scientists mean when they say that one problem "reduces" to another task, e.g. when they say that all NP problems reduce to 3SAT.

Second draft — physical reduction

The second-draft formalisation will be "physical reduction".

A task X is about changing the state of the world, i.e. X:ΩΩ.

A machine A achieves a particular task X if it change...