<p>Kimi K2 and Moonshot AI's history, avoiding loss spikes during training, the muon optimizer, and data parallelism</p>

Kimi K2 and Moonshot AI's history, avoiding loss spikes during training, the muon optimizer, and data parallelism

Training a 1 trillion parameter model

Two Stanford friends who flooded their kitchen using dish soap instead of dishwasher detergent have somehow become AI experts. One builds startups, the other studies at Cambridge - together they break down LLMs and machine learning with zero BS and maximum banter.

Share Training a 1 trillion parameter model

Sign up to save your podcasts

Training a 1 trillion parameter model

Training a 1 trillion parameter model