This October 10, 2025 joint collaboration between Meta Superintelligence Labs, FAIR at Meta, and The Ohio State University academic paper proposes and evaluates a training paradigm called **"early experience"** for language agents to bridge the gap between **Imitation Learning (IL)** and **Reinforcement Learning (RL)**, especially in environments lacking reliable rewards. The core idea is to generate **scalable supervision** from the agent's own exploratory actions through two methods: **Implicit World Modeling (IWM)**, which trains the agent to predict the next state after an action, and **Self-Reflection (SR)**, where the agent generates reasoning to explain why an expert action is better than its alternatives. Experiments across eight environments—including web navigation and multi-turn tool-use—show that early experience consistently **outperforms pure imitation learning** and provides a **stronger initialization** for subsequent RL training, even using less expert data and across different model scales. This method improves both in-domain performance and **out-of-domain generalization**, offering a practical path toward developing agents that learn effectively from their own interactions without external reward signals.

Source:

https://arxiv.org/pdf/2510.08558