How should we represent robot actions for autoregressive transformers? Most robot policies use diffusion or flow to generate continuous action sequences, but this isn’t how large language models work; they predict output tokens, which has many advantages. But coming up with a set of useful action tokens, so we can skip the slow and expensive diffusion steps, is difficult.

Chaoqi Liu says action tokens need three qualities: reasonable compression, universal decodability, and a left-to-right causally ordered token space, and he proposes Ordered Action Tokenization as a solution to all three.

Watch Episode 66 of RoboPapers now, with Michael Cho and Chris Paxton, to learn more!

Abstract:

Autoregressive policies offer a compelling foundation for scalable robot learning by enabling discrete abstraction, token-level reasoning, and flexible inference. However, applying autoregressive modeling to continuous robot actions requires an effective action tokenization scheme. Existing approaches either rely on analytical discretization methods that produce prohibitively long token sequences, or learned latent tokenizers that lack structure, limiting their compatibility with next-token prediction. In this work, we identify three desiderata for action tokenization — reasonable compression, universal decodability, and a left-to-right causally ordered token space — and introduce Ordered Action Tokenization (OAT), a learned action tokenizer that satisfies all three. OAT discretizes action chunks into an ordered sequence of tokens using transformer with register tokens, finite scalar quantization, and ordering-inducing training mechanisms. The resulting token space aligns naturally with autoregressive generation and enables prefix-based detokenization, yielding an anytime trade-off between inference cost and action fidelity. Across more than 20 tasks spanning four simulation benchmarks and real-world settings, autoregressive policies equipped with OAT consistently outperform prior tokenization schemes and diffusion-based baselines, while offering significantly greater flexibility at inference time.

Project Site: https://ordered-action-tokenization.github.io/

ArXiV: https://arxiv.org/abs/2602.04215

Blog Post on X

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com