
Sign up to save your podcasts
Or
Kimi k1.5 is a multimodal LLM trained with reinforcement learning (RL). Key aspects include: long context scaling to 128k, improving performance with increased context length; improved policy optimization using a variant of online mirror descent; and a simplistic framework that enables planning and reflection without complex methods. It uses a reference policy in its off-policy RL approach, and long2short methods such as model merging and DPO to transfer knowledge from long-CoT to short-CoT models, achieving state-of-the-art reasoning performance. The model is jointly trained on text and vision data.
5
22 ratings
Kimi k1.5 is a multimodal LLM trained with reinforcement learning (RL). Key aspects include: long context scaling to 128k, improving performance with increased context length; improved policy optimization using a variant of online mirror descent; and a simplistic framework that enables planning and reflection without complex methods. It uses a reference policy in its off-policy RL approach, and long2short methods such as model merging and DPO to transfer knowledge from long-CoT to short-CoT models, achieving state-of-the-art reasoning performance. The model is jointly trained on text and vision data.
272 Listeners
441 Listeners
298 Listeners
331 Listeners
217 Listeners
156 Listeners
192 Listeners
9,170 Listeners
409 Listeners
121 Listeners
75 Listeners
479 Listeners
94 Listeners
31 Listeners
43 Listeners