Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A note on 'semiotic physics', published by metasemi on February 11, 2023 on The AI Alignment Forum.

Introduction

This is an attempt to explain to myself the concept of semiotic physics that appears in the original Simulators post by janus and in a later post by Jan Hendrik Kirchner. Everything here comes from janus and Jan's work, but any inaccuracies or misinterpretations are all mine.

TL;DR

The prototypical simulator, GPT, is sometimes said to "predict the next token" in a text sequence. This is accurate, but incomplete.

It's more illuminating to consider what happens when GPT, or any simulator, is run repeatedly to produce a multi-token forward trajectory, as in the familiar scenario of generating a text completion in response to a prompt.

The token-by-token production of output is stochastic, with a branch point at every step, making the simulator a multiverse generator analogous to the time evolution operator of quantum mechanics.

In this analogical sense, a simulator such as GPT implements a "physics" whose "elementary particles" are linguistic tokens. When we experience the generated output text as meaningful, the tokens it's composed of are serving as semiotic signs. Thus we can refer to the simulator's physics-analogue as semiotic physics.

We can explore the simulator's semiotic physics through experimentation and careful observation of the outputs it actually produces. This naturalistic approach is complementary to analysis of the model's architecture and training.

Though GPT's outputs often contain remarkable renditions of the real world, the relationship between semiotic physics and quantum mechanics remains analogical. It's a misconception to think of semiotic physics as a claim that the simulator's semantic world approximates or converges on the real world.

Trajectories

GPT, the prototypical simulator, is often said to "predict the next token" in a sequence of text. This is true as far as it goes, but it only partially describes typical usage, and it misses a dynamic that's essential to GPT's most impressive performances. Usually, we don't simply have GPT predict a single token to follow a given prompt; we have it roll out a continuous passage of text by predicting a token, appending that token to the prompt, predicting another token, appending that, and so on.

Thinking about the operation of the simulator within this autoregressive loop better matches typical scenarios than thinking about single token prediction, and is thus a better fit to what we typically mean when we talk about GPT. But there's more to this distinction than descriptive point of view. Crucially, the growing sequence of prompt+output text, repeatedly fed back into the loop, preserves information and therefore constitutes state, like the tape of a Turing machine.

In the Simulators post, janus writes:

I think that implicit type-confusion is common in discourse about GPT. “GPT”, the neural network, the policy that was optimized, is the easier object to point to and say definite things about. But when we talk about “GPT’s” capabilities, impacts, or alignment, we’re usually actually concerned about the behaviors of an algorithm which calls GPT in an autoregressive loop repeatedly writing to some prompt-state...

The Semiotic physics post defines the term trajectory to mean the sequence of tokens—prompt plus generated-output-so-far—after each iteration of the autoregressive loop. In semiotic physics, as is common in both popular and technical discourse, by default we talk about GPT as a generator of (linguistic) trajectories, not context-free individual tokens.

Simulators are multiverse generators

GPT's token-by-token production of a trajectory is stochastic: at each autoregressive step, the trained model generates an output probability distribution over the token vocabulary, samples from t...