Best AI papers explained

This paper introduces Learning-to-Measure (L2M) to address the challenges of meta-Active Feature Acquisition (meta-AFA), a sequential decision-making problem. Traditional AFA methods often struggle with scalability because they are designed for a single task and fail when trained on retrospective data containing systematic missingness in features. L2M overcomes these limitations by formalizing the meta-AFA problem to allow learning acquisition policies across diverse tasks and leveraging a pre-trained sequence-modeling or autoregressive approach to provide reliable uncertainty quantification. By coupling this uncertainty quantification with a greedy policy that maximizes conditional mutual information, L2M can select the next feature to acquire in-context without requiring retraining for every new task, demonstrating superior performance, especially when labeled data is scarce or missingness is high.

Today, instead of introducing new research, we go deeper into Andrej Karpathy's insights. In his recent interview, he presents his perspectives on the current state and future of Artificial General Intelligence (AGI) and Large Language Models (LLMs). Karpathy argues that AGI is still about a decade away, asserting that the challenges, while tractable, are difficult and require incremental progress across many domains, including better datasets, hardware, and algorithms. He frequently contrasts current machine learning paradigms, particularly reinforcement learning (RL), with human and animal learning, suggesting that RL is "terrible" and that LLMs currently suffer from cognitive deficits like "model collapse" and an over-reliance on memorized knowledge rather than a "cognitive core" of pure intelligence. The discussion also touches on the long timeline for developing self-driving technology, the continuous nature of technological progress blending into the established 2% GDP growth rate, and Karpathy's new focus on education to empower humanity in an increasingly automated future.

This research paper, by authors affiliated with NVIDIA, Carnegie Mellon University, Boston University, and Stanford University, focuses on the optimal strategy for incorporating reasoning data into Large Language Model (LLM) training. The central finding challenges the conventional approach of relying solely on post-training, demonstrating that "front-loading" reasoning data during the pretraining phase is critical, yielding a durable 19% average performance gain on expert-level tasks. The research establishes an asymmetric principle for data allocation: pretraining benefits most from broad diversity and scale in reasoning patterns, while supervised fine-tuning (SFT) is most sensitive to high data quality. The study concludes that early investment in reasoning creates a foundational capacity that cannot be fully replicated by later-stage fine-tuning, advising against naively scaling mixed-quality SFT data.

This paper investigates the effectiveness of deliberate exploration in enhancing the reasoning capabilities of large language models (LLMs) trained with reinforcement learning (RL). The authors propose and evaluate a novel representation-based exploration (RepExp) strategy, which uses a bonus derived from the LLM's hidden states to encourage the discovery of diverse and novel behaviors. The study employs a two-pronged evaluation methodology, first testing RepExp in an inference-time setting for selecting diverse responses and then integrating it into the RL post-training pipeline. Key findings indicate that this exploration method significantly improves verifier efficiency and mitigates the "diversity collapse" phenomenon observed in standard RL methods, suggesting that the approach moves beyond merely sharpening existing model capabilities. The results show RepExp provides substantial improvements in pass@k rates and is especially beneficial for stronger models and harder reasoning problems across various tasks like MATH and GSM8K.

The academic paper discusses the critical flaws in current methods used to evaluate the robustness of large language model (LLM) defenses against jailbreaks and prompt injections. The authors argue that testing defenses with static or computationally weak attacks yields a false sense of security, as demonstrated by the fact that they successfully bypassed twelve different recent defenses with an attack success rate exceeding 90% in most cases. Instead, they propose that robustness must be measured against adaptive attackers who systematically tune and scale optimization techniques, including gradient descent, reinforcement learning, search-based methods, and human red-teaming. The paper emphasizes that human creativity remains the most effective adversarial strategy, and future defense work must adopt stronger, adaptive evaluation protocols to make reliable claims of security.

The research empirically investigates the role of pretraining distribution and a new concept of task diversity in the emergence of ICL, particularly using models trained on linear functions. Findings indicate that increasing task diversity causes transformers to shift from a specialized solution to one that can generalize across the entire task space, a transition also observed in nonlinear regression problems. The authors constructed a phase diagram to characterize how task diversity and the number of pretraining tasks interact, while also examining the influence of factors like model depth and problem dimensionality.

This paper studies scaling reinforcement learning (RL) compute for large language models (LLMs), introducing a principled framework to predict performance. The authors develop ScaleRL, a best-practice recipe derived from ablating various algorithmic choices, and demonstrate its predictable scaling trajectory using a sigmoidal function to fit compute-performance curves. Accompanying figures illustrate validation performance over increasing GPU hours (log scale) for different RL configurations, showing that ScaleRL achieves higher asymptotic performance and efficiency than prevalent methods while maintaining stability across various scaling axes, including model size and batch size. The work establishes that predictable scaling laws, similar to those in LLM pre-training, can be applied to the RL fine-tuning stage.

This white paper by Anthropic, UK AI Security Institute, and The Alan Turing Institute demonstrates that a small, fixed number of malicious documents—as few as 250—can successfully create a "backdoor" vulnerability in LLMs, regardless of the model's size or the total volume of clean training data. This finding challenges the previous assumption that attackers need to control a percentage of the training data, suggesting that these poisoning attacks are more practical and accessible than previously believed. The study specifically tested a denial-of-service attack that causes the model to output gibberish upon encountering a specific trigger phrase like , and the authors share these results to encourage further research into defenses against such vulnerabilities.

This paper discusses dual goal representations for goal-conditioned reinforcement learning (GCRL), a novel method for encoding a state based on its temporal distance relation to all other states within an environment. The authors theoretically establish that this representation is sufficient for recovering an optimal goal-reaching policy and is invariant to extraneous noise within the state observations. Building on this theory, they propose a practical implementation using an inner product parameterization and offline value learning, demonstrating that this approach consistently improves goal-reaching performance across a suite of robotic navigation and manipulation tasks, outperforming existing representation learning methods. The overall aim is to enhance the efficiency and generalization capability of GCRL agents by providing a robust and structured goal representation.

This "bitter lesson"-style position paper by David Silver and Richard S. Sutton introduces a shift in artificial intelligence from the "Era of Human Data" to the "Era of Experience." The authors argue that AI progress is slowing because the available pool of human-generated data is being exhausted, necessitating a new approach where agents learn predominantly from their own self-generated experience and interaction with the environment. This transition is characterized by agents that inhabit streams of experience rather than short interactions, utilize richly grounded actions and observations, optimize for grounded rewards from the environment, and engage in non-human planning and reasoning to achieve truly superhuman capabilities. The paper positions this new era as a critical next step that will reconcile the task-generality achieved by large language models with the self-discovery of knowledge exemplified by earlier reinforcement learning systems.