
Sign up to save your podcasts
Or
I want to highlight this paper (from Sept 29, 2025) of an alternative to RL (for fine-tuning pre-trained LLMs) which:
They claim the magic sauce behind all this is the evolutionary strategy optimizing over distributions of model parameters. Surprisingly, they've scaled this to optimize over billion-parameter models.
Let's get into their method.
Evolutionary Strategy (ES) Algorithm
They start w/ a "Basic ES Algorithm" which is:
In other words, we're gonna sample noise around the original model's weights N times (ie we're going to explore around the model weights where the variance I is the identity covariance).
[Below is an example explaining more in depth, feel free to skip [...]
---
Outline:
(00:54) Evolutionary Strategy (ES) Algorithm
(02:41) New ES Implementation
(03:28) Task 1: Countdown task
(05:05) Task 2: Conciseness
(06:00) Future Work
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
I want to highlight this paper (from Sept 29, 2025) of an alternative to RL (for fine-tuning pre-trained LLMs) which:
They claim the magic sauce behind all this is the evolutionary strategy optimizing over distributions of model parameters. Surprisingly, they've scaled this to optimize over billion-parameter models.
Let's get into their method.
Evolutionary Strategy (ES) Algorithm
They start w/ a "Basic ES Algorithm" which is:
In other words, we're gonna sample noise around the original model's weights N times (ie we're going to explore around the model weights where the variance I is the identity covariance).
[Below is an example explaining more in depth, feel free to skip [...]
---
Outline:
(00:54) Evolutionary Strategy (ES) Algorithm
(02:41) New ES Implementation
(03:28) Task 1: Countdown task
(05:05) Task 2: Conciseness
(06:00) Future Work
---
First published:
Source:
---
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
26,373 Listeners
2,429 Listeners
8,189 Listeners
4,159 Listeners
92 Listeners
1,554 Listeners
9,812 Listeners
88 Listeners
484 Listeners
5,476 Listeners
16,145 Listeners
532 Listeners
133 Listeners
96 Listeners
510 Listeners