AI Post Transformers

This September 2025 paper introduces Diversity-Aware Reinforcement Learning (Darling), a novel framework designed to enhance both the quality and semantic diversity of large language model (LLM) generations. Recognizing that traditional post-training methods often sacrifice diversity for accuracy, Darling integrates a learned partition function to measure semantic diversity beyond simple lexical variations. This diversity signal is then multiplied with a quality reward during online reinforcement learning, which encourages LLMs to produce responses that are not only high-quality but also distinct and novel. Experiments on both non-verifiable tasks, such as creative writing, and verifiable tasks, like competition math, demonstrate that Darling consistently outperforms quality-only baselines, achieving improved scores in both quality metrics and diversity measures (e.g., pass@1 and pass@k). A key finding is that explicitly optimizing for diversity can catalyze exploration in online RL, leading to a simultaneous improvement in the overall quality of the generated responses. Source: https://arxiv.org/pdf/2509.02534

This February 2025 paper introduces INF2, a novel framework designed to enhance the generative inference throughput of large language models (LLMs) by utilizing computational storage devices (CSDs). The core innovation, attention-near storage (ANS), offloads memory-intensive self-attention operations directly to accelerators within these storage devices, significantly reducing data transfer bottlenecks over the system interconnect. To further boost performance, INF2 incorporates delayed KV cache writeback which minimizes storage write latency by batching updates to the KV cache, and cooperative X-cache, which optimizes host memory usage by storing input activations instead of key-value caches for cooperative processing between the GPU and CSDs. Through these methods, INF2 demonstrates substantial throughput improvements, achieving up to 3.46 times faster performance compared to existing state-of-the-art baselines in real-world evaluations. Source: https://arxiv.org/html/2502.09921v1

The September 9 2025 press release and paper announce and detail K2 Think, an advanced open-source AI reasoning system developed by the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) and G42 in the UAE. K2 Think stands out for its parameter efficiency, achieving performance comparable to much larger models, particularly in mathematical reasoning, with only 32 billion parameters. This breakthrough is attributed to a six-pillar approach, including supervised fine-tuning, reinforcement learning with verifiable rewards, agentic planning, test-time scaling, and optimization for Cerebras Wafer-Scale Engine hardware for rapid inference. The system also undergoes red-teaming for safety evaluation, demonstrating a solid baseline in content refusal and conversational robustness, while identifying areas for improvement in cybersecurity and jailbreak resistance.Sources:https://k2think-about.pages.dev/assets/tech-report/K2-Think_Tech-Report.pdfhttps://k2think-about.pages.dev/assets/announcement/K2_Think_-_Sep_9_English_Release.pdfhttps://www.k2think.ai/k2think

This September 2025 paper analyzes the theoretical benefits and limitations of Masked Diffusion Models (MDMs) for text generation, contrasting them with auto-regressive models. While MDMs can sample multiple tokens in parallel, offering a potential for efficiency, the research demonstrates that their actual performance depends heavily on the evaluation metric. Specifically, MDMs can achieve near-optimal fluency (low Token Error Rate) with a constant number of sampling steps, regardless of sequence length. However, when assessed for correctness (low Sequence Error Rate), particularly for tasks requiring logical reasoning, MDMs necessitate a number of sampling steps that scales linearly with sequence length, effectively negating their efficiency advantage. Empirical results using formal languages and large open-sourced MDMs support these theoretical findings, indicating that MDMs are better suited for fluent text generation but less so for accuracy-critical reasoning tasks. Source: https://arxiv.org/pdf/2502.09622

This September 2025 paper introduces Mini-o3, a Vision-Language Model (VLM) designed to overcome the limitations of existing VLMs in handling complex visual search tasks that require multi-turn reasoning and trial-and-error exploration. The researchers developed a three-component training recipe, including the creation of the Visual Probe Dataset with challenging, high-resolution images, a pipeline for synthesizing diverse multi-turn trajectories for supervised finetuning, and an over-turn masking technique in reinforcement learning. This masking prevents penalization of long, incomplete reasoning paths, encouraging deeper exploration without increasing training time. Mini-o3 demonstrates state-of-the-art performance on various visual search benchmarks, showcasing its enhanced ability for complex, adaptive visual understanding through iterative observation, thought, and action. Source: https://arxiv.org/pdf/2509.07969

This September 2025 paper introduces "Behavioral Fingerprinting," a novel framework designed to evaluate Large Language Models (LLMs) beyond traditional performance scores like MMLU. It aims to understand how models "think," creating a multi-faceted profile of their intrinsic cognitive and interactive styles. The methodology employs a diagnostic prompt suite and an automated evaluation pipeline where a powerful LLM acts as a judge, analyzing eighteen different models across four key dimensions: internal world model, reasoning abilities, biases and personality (including sycophancy), and semantic robustness. Findings indicate a convergence in core reasoning abilities among top models but a significant divergence in alignment-related behaviors such as sycophancy and robustness, which are influenced by specific developer strategies. The framework also identifies default personality clustering (ISTJ/ESTJ types), reflecting common training paradigms that reward logical, structured, and decisive responses. Source: https://arxiv.org/pdf/2509.04504

This September 2025 paper investigates the reliability and robustness of Large Language Models (LLMs) when evaluated using traditional benchmarks. The authors systematically paraphrased questions across six common benchmarks and observed how 34 different LLMs performed. Their findings indicate that while LLM rankings remain relatively consistent, their absolute effectiveness scores significantly decline when faced with reworded questions, suggesting a lack of robustness to linguistic variability. The study highlights that current benchmark evaluations may overstate LLM generalization abilities and advocates for more robustness-aware evaluation methodologies that better reflect real-world language use. Source: https://arxiv.org/pdf/2509.04013

This September 2025 paper introduces TraceRL, a novel reinforcement learning framework designed to enhance diffusion language models (DLMs) across various architectural types. The core idea behind TraceRL is to align the training process with the preferred inference trajectories of the model, which demonstrably improves performance on complex reasoning tasks like mathematics and coding. The authors also propose a diffusion-based value model to boost training stability. Through experiments, the paper showcases the effectiveness of TraceRL, yielding state-of-the-art DLMs called TraDo that outperform larger autoregressive models. Furthermore, the source provides an open-source framework to facilitate the development, training, and deployment of these advanced DLMs, including accelerated inference techniques and diverse post-training methods. Source: https://arxiv.org/pdf/2509.06949

This March 2025 paper examines the input/output (I/O) characteristics of offloading large language model (LLM) components to NVMe SSDs during inference, a critical solution for overcoming GPU memory limitations with ever-growing LLMs. Researchers analyzed block-layer I/O traces from two prominent LLM frameworks, DeepSpeed and FlexGen, to understand how model weights and key-value (KV) caches are handled. The findings indicate that asynchronous I/O using libaio significantly outperforms POSIX for tensor transfers, although neither method fully saturates the NVMe SSD's theoretical bandwidth. For model offloading, I/O is predominantly characterized by 128KiB reads, primarily occurring at the beginning of the inference process, while KV cache offloading involves both reads and writes of similar size, with read bandwidth being substantially higher. Ultimately, the research suggests that modern NVMe SSDs are capable of supporting current LLM inference workloads but highlights opportunities for further optimization in SSD design and KV cache management. Source: https://dl.acm.org/doi/10.1145/3719330.3721230

This blog post series from Chris Lattner extensively examines CUDA's pervasive dominance in AI compute, detailing its evolution from a graphics processor to a layered software platform integral to NVIDIA's success, while also highlighting the challenges and complexities it presents to developers and alternative hardware vendors. The articles critically assess various attempts to democratize AI compute, including OpenCL, TVM, XLA, and MLIR, explaining why these alternatives largely failed to dislodge CUDA due to fragmentation, misaligned incentives, and a lack of unified vision. Ultimately, the texts introduce Modular's approach to addressing these issues through its Mojo language, MAX framework, and Mammoth cluster management system, aiming to provide a portable, performant, and programmable solution for the rapidly evolving Generative AI landscape. Source: https://www.modular.com/blog/democratizing-compute-part-1-deepseeks-impact-on-ai