A new paper by Finlayson et al. describes how to exploit the softmax bottleneck in large language models to infer the model dimension of closed-source LLMs served to the public via an API. I'll briefly explain the method they use to achieve this and provide a toy model of the phenomenon, though the full paper has many practical details I will elide in the interest of simplicity. I recommend reading the whole paper if this post sounds interesting to you.

Background

First, some background: large language models have a model dimension that corresponds to the size of the vector that each token in the input is represented by. Knowing this dimension _ d_{text{model}} _ and the number of layers _ n_{text{layers}} _ of a dense model allows one to make a fairly rough estimate _ approx 10 n_{text{layers}} d_{text{model}}^2 _ of the [...]

---

Outline:

(00:33) Background

(01:44) The method of attack

(03:39) Results

(06:43) What will labs do about this?

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Linkpost URL:
https://arxiv.org/abs/2403.09539

Narrated by TYPE III AUDIO.