This paper introduces **GameTalk**, a novel framework designed to train large language models (LLMs) for **strategic, multi-turn conversations**. While standard LLM training typically focuses on static, single-turn tasks, this research optimizes models to achieve **long-term goals** through complex interactions like negotiation and coordination. The authors adapt advanced fine-tuning methods—specifically **DPO, GRPO, and STaR**—to incorporate rewards based on the outcome of entire dialogues across various game environments. To diagnose and improve performance, the study utilizes three behavioral signals: **Internal State Evaluation**, **State-Relative Performance**, and **Leverage Opportunity**. Experimental results across games like Rock-Paper-Scissors and bargaining scenarios demonstrate that **DPO** is particularly effective at teaching models to use language as a persuasive tool. Ultimately, the framework shifts the focus of AI development toward **dynamic, goal-oriented reasoning** in interactive settings.

This paper introduces Self-Distillation Policy Optimization (SDPO), a novel reinforcement learning framework designed to improve how large language models learn from complex environments. While traditional methods often rely on simple scalar rewards that create information bottlenecks, SDPO utilizes rich textual feedback, such as runtime errors or descriptive evaluations, to provide denser learning signals. By treating the current model as a self-teacher that re-evaluates its own attempts in light of this feedback, the algorithm distills corrected predictions back into the policy without needing external human or AI mentors. Research shows that this approach significantly enhances sample efficiency and reasoning accuracy across tasks like scientific problem-solving and competitive programming. Furthermore, SDPO qualitatively produces concise reasoning and avoids the repetitive verbosity common in other reinforcement learning techniques. At test-time, the method also accelerates the discovery of solutions for exceptionally difficult problems by iteratively refining the model’s internal logic.

This research explores the theoretical alignment between self-supervised contrastive learning (CL) and supervised learning, specifically investigating why label-agnostic training produces organized semantic clusters. The authors prove that standard CL objectives implicitly approximate a negatives-only supervised contrastive loss (NSCL), with the gap between the two vanishing as the number of dataset classes increases. Their analysis identifies that global minimizers of this loss exhibit augmentation collapse, within-class collapse, and a simplex equiangular tight frame structure, mirroring the "neural collapse" found in supervised models. The paper introduces a new few-shot error bound based on directional feature variability, which explains how these models support high-accuracy label recovery with minimal supervision. Empirical tests across diverse vision datasets confirm that minimizing the unsupervised CL loss effectively drives down the supervised NSCL loss. Ultimately, the study provides a robust mathematical framework to justify the success of contrastive pre-training in downstream classification tasks.

This research explores the mathematical and empirical relationship between Contrastive Learning (CL) and Non-Contrastive Supervised Contrastive Learning (NSCL). The authors demonstrate that CL and NSCL converge toward highly similar structural representations, a phenomenon they validate using metrics like Centered Kernel Alignment (CKA) and Representational Similarity Analysis (RSA). Their theoretical framework identifies key variables—such as temperature, batch size, and learning rate—that determine the proximity of these two methods in similarity space. Experimental results on datasets like CIFAR and ImageNet confirm that these training dynamics lead to nearly identical attention maps and feature distributions. Ultimately, the paper provides a formal proof that unsupervised contrastive models inherently approximate their supervised counterparts under specific optimization constraints.

The provided text explores whether multi-agent systems (MAS) can be effectively replaced by a single agent simulating complex workflows through multi-turn conversations. Research indicates that homogeneous workflows, where multiple agents use the same base model, can be replicated by one agent with significant computational efficiency gains via KV cache reuse. The authors introduce OneFlow, an automated algorithm that utilizes dual meta-LLMs and Monte Carlo Tree Search to design streamlined, high-performance workflows specifically for single-agent execution. Experimental results across various benchmarks demonstrate that this single-agent approach matches the accuracy of multi-agent setups while reducing inference costs. However, the study acknowledges that heterogeneous workflows involving different base models still offer unique benefits that a single model cannot yet fully capture. Consequently, these findings establish the single-LLM implementation as a powerful new baseline for future multi-agent research.

This research explores Reinforcement Learning from Human Feedback (RLHF) under the KL-regularized contextual bandits framework. While traditional methods rely on complex optimistic or pessimistic estimates to manage uncertainty, the authors prove that greedy sampling—directly using empirical estimates—is surprisingly efficient. By leveraging the structural property that optimal policies remain within a bounded likelihood ratio of the reference policy, the study establishes logarithmic regret in online settings and optimal sample complexity for offline learning. These findings apply to both the Bradley-Terry reward-based model and general preference models, offering a more computationally efficient approach to aligning large language models. The theoretical results are further validated through simulations that show greedy sampling performs comparably to more sophisticated, resource-intensive algorithms.

This research paper establishes a formal learning theoretic framework to analyze the performance of zero-shot prediction (ZSP) in multimodal models like CLIP. The authors decompose prediction error into three distinct components: prompt bias, which measures the suitability of a prompting strategy; residual dependence, which quantifies the information lost when using text as a proxy for image features; and estimation error from finite data. By avoiding common but unrealistic assumptions of conditional independence, the study provides theoretical guarantees for how pre-training distributions and prompting methods influence downstream task accuracy. The framework introduces two primary mathematical approaches—conditional mean and information density—to evaluate how indirect predictors compare to direct supervised learners. Finally, the authors validate their theory through empirical simulations and image data experiments, demonstrating that minimizing residual dependence and prompt bias is essential for optimizing zero-shot performance.

This paper introduces TTT-Discover, an innovative system designed to solve complex science and engineering problems through test-time training. Unlike traditional static models, this approach enables an open-source AI to continuously learn and refine its policy while actively seeking solutions for a specific task. By utilizing an entropic objective and adaptive reinforcement learning, the system successfully established new state-of-the-art results in mathematics, GPU kernel engineering, and biology. The researchers demonstrate that this method can outperform elite human experts and powerful closed-frontier models at a fraction of the typical computational cost. This framework effectively transforms the problem-solving process into an iterative search and learning environment where the model improves itself until a breakthrough is reached. Notably, the paper details successful applications in Erdős’ minimum overlap problem and high-performance algorithm design.

This paper explores how the statistical properties of pretraining data determine the success of in-context learning (ICL) in transformer models. By developing a theoretical framework that unifies task selection and generalization, the authors demonstrate that heavy-tailed pretraining distributions significantly enhance a model's robustness to distribution shifts. Conversely, while light-tailed distributions excel at familiar tasks, they require fewer examples to generalize effectively. The study also highlights that stronger temporal dependencies within data sequences increase the volume of training tasks necessary for reliable performance. Through experiments on numerical tasks like stochastic differential equations, the findings suggest that careful distribution design is essential for building reliable and adaptable AI systems.

This research paper addresses the challenge of anytime reasoning, where large language models (LLMs) must provide high-quality solutions under strict computational or token budgets. The authors introduce a novel evaluation metric called the Anytime Index, which measures how effectively a model’s solution quality improves as more reasoning tokens are generated. To enhance this efficiency, they propose Preference Data Prompting (PDP), an inference-time method where models learn from self-generated contrastive examples of successful and unsuccessful reasoning. Testing across diverse benchmarks like NaturalPlan, AIME, and GPQA shows that this technique consistently boosts both intermediate and final performance across various model families. Ultimately, the framework helps distinguish "fast-thinking" models that reach accuracy quickly from those that require exhaustive computation. This work proves that LLMs can become more resource-efficient by following guided, high-quality reasoning patterns without requiring human supervision or fine-tuning.