Sign up to save your podcasts
Or
Share Module 2: The MLP Layer - Where Transformers Store Knowledge
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Module 2: The MLP Layer - Where Transformers Store Knowledge
Shay explains where a transformer actually stores knowledge: not in attention, but in the MLP (feed-forward) layer. The episode frames the transformer block as a two-step loop: attention moves information between tokens, then the MLP transforms each token’s representation independently to inject learned knowledge.
View all episodes
By Sheetal ’Shay’ DharShay explains where a transformer actually stores knowledge: not in attention, but in the MLP (feed-forward) layer. The episode frames the transformer block as a two-step loop: attention moves information between tokens, then the MLP transforms each token’s representation independently to inject learned knowledge.