AI Post Transformers

This September 2025 paper explores the critical role of statistical methods in enhancing the reliability and functionality of Generative AI (GenAI), which inherently lacks guarantees regarding correctness or safety. It discusses various statistical applications, including improving and altering model behavior through techniques like output trimming and abstention based on risk scores, often utilizing conformal prediction for provable guarantees. The text also covers diagnostics and uncertainty quantification (UQ), differentiating between epistemic and aleatoric uncertainty and addressing challenges like semantic multiplicity and the need for calibration in GenAI outputs. Furthermore, it highlights the importance of statistical inference in evaluating GenAI models, particularly with limited data, and examines interventions and experiment design, such as using "steering vectors" and "causal mediation analysis" to understand and mitigate biases within these complex systems. Source: https://arxiv.org/pdf/2509.07054

This September 2025 paper provides a comprehensive overview of Reinforcement Learning (RL) as applied to Large Reasoning Models (LRMs). It breaks down the field into foundational components such as reward design and policy optimization, explaining various algorithms like PPO and GRPO. The document also discusses training resources, distinguishing between static corpora and dynamic environments, and highlights diverse applications of RL in LRMs, including coding, agentic tasks, and multimodal understanding, with a focus on models from 2025. Ultimately, the paper aims to identify future directions for scaling RL in LRMs towards achieving Artificial Superintelligence (ASI). Source: https://arxiv.org/pdf/2509.08827

This April 2025 paper introduces HyperController, a novel and computationally efficient algorithm designed to optimize hyperparameters during the training of reinforcement learning neural networks. Hyperparameter optimization is crucial for improving machine learning models, but traditional methods can be slow and computationally intensive. HyperController addresses these challenges by modeling the hyperparameter optimization problem as an unknown Linear Gaussian Dynamical System and leveraging the Kalman filter for efficient prediction. The algorithm is validated through experiments on various OpenAI Gymnasium environments, where it demonstrates faster training times and superior or comparable performance compared to existing methods, achieving the highest median reward in four out of five environments. The research highlights HyperController's potential for stable and efficient training of reinforcement learning neural networks, offering advantages like quicker deployment and easier fine-tuning. Source: https://arxiv.org/pdf/2504.19382

This August 2025 paper introduces NOVELTYBENCH, a new benchmark designed to evaluate how well large language models (LLMs) generate diverse and high-quality outputs, addressing the problem of "mode collapse" where models produce repetitive responses. The research found that current state-of-the-art LLMs consistently generate less diversity than human writers, with larger models often exhibiting even lower diversity than their smaller counterparts. The benchmark uses a unique approach to measure functional equivalence between generations, ensuring that diversity is meaningful to users. While certain prompting strategies, like in-context regeneration, can enhance diversity, the study suggests that this capability is not inherent in the models themselves, highlighting the need for new training and evaluation paradigms that prioritize both diversity and quality in LLM development. Source: https://arxiv.org/pdf/2504.05228

This September 10, 2025 technical report from Tencent AI Lab introduces Parallel-R1, a novel reinforcement learning (RL) framework designed to enhance large language models (LLMs) with parallel thinking capabilities for complex mathematical reasoning tasks. Unlike previous methods relying on supervised fine-tuning (SFT) over synthetic data, Parallel-R1 utilizes a progressive curriculum to address the cold-start problem in RL, initially using SFT on simpler tasks to instill the basic format of parallel thinking before transitioning to RL for exploration and generalization on more challenging problems. The research highlights that parallel thinking evolves from an exploratory strategy to a multi-perspective verification tool during training, and can also serve as a mid-training exploration scaffold to unlock higher performance ceilings. The framework demonstrated significant accuracy improvements across various math benchmarks, and the authors plan to open-source their model, data, and code. Source: https://arxiv.org/pdf/2509.07980

This July 2025 paper introduces AdLlama, a new large language model (LLM) for generating Facebook ad text, trained using Reinforcement Learning with Performance Feedback (RLPF). Unlike previous models that relied on supervised fine-tuning to imitate curated ads, AdLlama utilizes historical ad performance data, specifically click-through rates (CTR), as a reward signal to optimize its text generation. A large-scale A/B test on Facebook, involving nearly 35,000 advertisers, demonstrated that AdLlama significantly improved advertiser-level CTR by 6.7% and increased the number of ad variations advertisers created by 18.5%. The findings highlight RLPF as a promising, generalizable approach for metric-driven LLM post-training, moving beyond subjective human preferences to achieve tangible business outcomes in real-world settings like online advertising. Source: https://arxiv.org/pdf/2507.21983

This July 2024 paper introduces ByteCheckpoint, a novel PyTorch-native system designed for Large Language Model (LLM) development. This system addresses critical challenges in LLM training, particularly the high I/O costs associated with saving and loading checkpoints, and the complexities of checkpoint resharding across different parallel configurations and training frameworks. ByteCheckpoint achieves this through a data/metadata disaggregated storage architecture and asynchronous tensor merging, enabling automatic online resharding and multi-framework support. The paper highlights ByteCheckpoint's significant performance improvements in reducing checkpoint saving and loading times compared to existing methods, making LLM development more efficient and robust. The authors detail various I/O performance optimization techniques. Source: https://arxiv.org/html/2407.20143v1

This April 2025 paper introduces SODA, a novel framework designed to enhance digital advertising strategies by making opaque AI systems more understandable for marketers. The authors highlight the current challenges faced by advertisers due to the lack of transparency in major ad platforms like Meta, which often results in wasted ad spend and reliance on intuition. To address this, SODA integrates Large Language Models (LLMs) with explainable AI techniques to provide clear, actionable insights into ad performance. The framework initially employs an improved Click-Through Rate (CTR) prediction model, SoWide-v2, which also offers visual explanations through attention maps. Furthermore, SODA leverages LLMs to automate comprehensive ad analysis, including identifying target audiences, brand personas, and key messaging, providing marketers with summarized and comparative data that was previously difficult to obtain without extensive manual effort. A case study with marketing professionals validated the framework's practical value and potential to streamline decision-making in fast-paced advertising environments. Source: https://arxiv.org/pdf/2504.20064

The May - June 2025 sources introduce AlphaEvolve, a novel AI coding agent developed by Google DeepMind in collaboration with mathematicians like Javier Gómez Serrano and Terence Tao. This Gemini-powered tool utilizes an evolutionary process, similar to natural selection, to generate and iteratively refine code solutions for complex problems. AlphaEvolve has demonstrated its capability in scientific and algorithmic discovery, successfully tackling open mathematical challenges such as improving bounds for matrix multiplication and the kissing number problem in 11 dimensions. Beyond theoretical advancements, it has also been applied to optimize critical components within Google's computing infrastructure, including data center scheduling, Gemini kernel engineering, and hardware circuit design. The sources highlight AlphaEvolve's potential to accelerate research by combining human intuition with machine exploration, marking a significant shift in how complex problems are approached.Sources:https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/AlphaEvolve.pdfhttps://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/https://www.crm.cat/javier-gomez-serrano-collaborates-with-terence-tao-and-deepmind-on-an-ai-project-to-solve-open-mathematical-problems/

This July 2002 paper introduced BLEU (Bilingual Evaluation Understudy), an automatic and inexpensive method for evaluating machine translation (MT) quality. It highlights the limitations of human evaluation, such as its high cost and time consumption, and proposes BLEU as a quick, language-independent alternative that correlates strongly with human judgment. The core concept of BLEU involves measuring the "closeness" of a machine translation to one or more human reference translations through a modified n-gram precision metric and a brevity penalty. The paper details the mathematical formulation of the BLEU score, its evaluation against human and machine translations, and its proven correlation with human assessment across various languages. Source: https://dl.acm.org/doi/10.3115/1073083.1073135