
Sign up to save your podcasts
Or
I want to highlight this paper (from Sept 29, 2025) of an alternative to RL (for fine-tuning pre-trained LLMs) which:
They claim the magic sauce behind all this is the evolutionary strategy optimizing over distributions of model parameters. Surprisingly, they've scaled this to optimize over billion-parameter models.
Let's get into their method.
Evolutionary Strategy (ES) Algorithm
They start w/ a "Basic ES Algorithm" which is:
In other words, we're gonna sample noise around the original model's weights N times (ie we're going to explore around the model weights where the variance I is the identity covariance).
[Below is an example explaining more in depth, feel free to skip [...]
---
Outline:
(00:54) Evolutionary Strategy (ES) Algorithm
(02:41) New ES Implementation
(03:28) Task 1: Countdown task
(05:05) Task 2: Conciseness
(06:00) Future Work
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
I want to highlight this paper (from Sept 29, 2025) of an alternative to RL (for fine-tuning pre-trained LLMs) which:
They claim the magic sauce behind all this is the evolutionary strategy optimizing over distributions of model parameters. Surprisingly, they've scaled this to optimize over billion-parameter models.
Let's get into their method.
Evolutionary Strategy (ES) Algorithm
They start w/ a "Basic ES Algorithm" which is:
In other words, we're gonna sample noise around the original model's weights N times (ie we're going to explore around the model weights where the variance I is the identity covariance).
[Below is an example explaining more in depth, feel free to skip [...]
---
Outline:
(00:54) Evolutionary Strategy (ES) Algorithm
(02:41) New ES Implementation
(03:28) Task 1: Countdown task
(05:05) Task 2: Conciseness
(06:00) Future Work
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
26,376 Listeners
2,429 Listeners
8,187 Listeners
4,155 Listeners
92 Listeners
1,553 Listeners
9,799 Listeners
89 Listeners
488 Listeners
5,472 Listeners
16,144 Listeners
531 Listeners
131 Listeners
96 Listeners
510 Listeners