
Sign up to save your podcasts
Or


I.
In The Argument, Kelsey Piper gives a good description of the ways that AIs are more than just "next-token predictors" or "stochastic parrots" - for example, they also use fine-tuning and RLHF. But commenters, while appreciating the subtleties she introduces, object that they're still just extra layers on top of a machine that basically runs on next-token prediction.
I want to approach this from a different direction. I think overemphasizing next-token prediction is a confusion of levels. On the levels where AI is a next-token predictor, you are also a next-token (technically: next-sense-datum) predictor. On the levels where you're not a next-token predictor, AI isn't one either.
By Jeremiah4.8
129129 ratings
I.
In The Argument, Kelsey Piper gives a good description of the ways that AIs are more than just "next-token predictors" or "stochastic parrots" - for example, they also use fine-tuning and RLHF. But commenters, while appreciating the subtleties she introduces, object that they're still just extra layers on top of a machine that basically runs on next-token prediction.
I want to approach this from a different direction. I think overemphasizing next-token prediction is a confusion of levels. On the levels where AI is a next-token predictor, you are also a next-token (technically: next-sense-datum) predictor. On the levels where you're not a next-token predictor, AI isn't one either.

32,110 Listeners

2,116 Listeners

2,672 Listeners

26,254 Listeners

4,270 Listeners

2,448 Listeners

2,271 Listeners

890 Listeners

292 Listeners

4,168 Listeners

1,625 Listeners

316 Listeners

3,819 Listeners

577 Listeners

691 Listeners