AI Post Transformers

This June 2025 paper introduces a novel methodology called Test-Time Reinforcement Learning (TTRL), which enables Large Language Models (LLMs) to improve their performance on reasoning tasks using unlabeled test data. The core innovation addresses the challenge of reward estimation without ground-truth labels by employing Test-Time Scaling (TTS) practices, specifically majority voting, to generate effective pseudo-labels and rule-based rewards. TTRL facilitates the self-evolution of LLMs during inference, demonstrating substantial performance gains—up to a 211% boost on challenging mathematical benchmarks like AIME 2024—and even surpassing the performance ceiling of the initial majority voting signal. This unsupervised online learning approach is shown to be compatible with different reinforcement learning algorithms and effective across various models, suggesting a path toward continually learning AI systems less reliant on extensive human annotation. Source: https://arxiv.org/pdf/2504.16084

The text introduces the Retrieval Embedding Benchmark (RTEB), a new standard designed to accurately evaluate the retrieval accuracy of embedding models for real-world applications. The authors argue that existing benchmarks fail due to a generalization gap and misalignment with modern enterprise AI applications, often leading to inflated scores from models that are "teaching to the test." RTEB addresses this with a hybrid strategy using both transparent open datasets and impartial private datasets to measure true generalization. Emphasizing multilingual and domain-specific enterprise use cases like law and finance, RTEB aims to be a reliable, community-trusted standard, using NDCG@10 as its primary evaluation metric. Source: https://huggingface.co/blog/rteb

This September 30 2025 academic paper, introduces Regression Language Models (RLMs) as a unified method for code-to-metric regression, which is the task of predicting numerical outcomes from source code or computation graphs. This approach simplifies traditional methods by directly using text input—such as high-level programming languages like Haskell and Python or low-level ONNX graph representations—to predict metrics like accuracy, memory consumption, and execution latency. The RLM, initialized from a pretrained T5Gemma encoder, is shown to perform competitively against specialized models like Graph Neural Networks (GNNs) across various tasks, including predicting performance in Neural Architecture Search (NAS) and estimating memory usage in competitive programming. The findings highlight the RLM's versatility and ability to model multiple objectives concurrently, suggesting a shift toward generic, text-based regression in computational graph analysis. Source: https://arxiv.org/pdf/2509.26476

The provided sources offer a comprehensive look at memory management for GPU-accelerated computing, focusing heavily on Heterogeneous Memory Management (HMM) and NVIDIA Unified Virtual Memory (UVM). One source details the release of the CUDA Toolkit 12.2, highlighting new features like HMM support, NVIDIA Hopper (H100) GPU compatibility, and Confidential Computing for secure data environments. Another source focuses exclusively on HMM, explaining how this feature simplifies GPU programming by allowing direct access to system-allocated memory, thereby eliminating the need for explicit memory calls like `cudaMallocManaged`. The third source, a technical paper, performs an in-depth performance analysis of UVM, examining the overhead associated with transparent paging and migration, identifying key performance bottlenecks like Host OS interactions (e.g., page unmapping) and the efficiency of fault batching and prefetching mechanisms. Source: https://tallendev.github.io/assets/papers/sc21.pdf https://developer.nvidia.com/blog/nvidia-cuda-toolkit-12-2-unleashes-powerful-features-for-boosting-applications/ https://developer.nvidia.com/blog/simplifying-gpu-application-development-with-heterogeneous-memory-management/

The May 2025 academic paper introduces BurstGPT, a novel, real-world workload dataset consisting of over ten million traces from regional Azure OpenAI GPT services collected over 213 days, which aims to optimize Large Language Model (LLM) serving systems. The authors argue that existing LLM serving optimizations are often evaluated using unrealistic synthetic or non-LLM workloads, leading to performance degradation in real-world deployments. BurstGPT provides empirical data on user concurrency patterns, conversation structures, model response lengths, and system failures to facilitate more accurate system evaluation and refinement of scheduling, caching, and resource provisioning strategies. The source presents BurstGPT-Perf, a benchmark suite using the dataset to demonstrate how realistic, bursty workloads reveal declines in efficiency, stability, and reliability in serving systems like vLLM. Ultimately, the work advocates for data-driven methodologies in optimizing LLM serving for better efficiency and quality of service. Source: https://arxiv.org/pdf/2401.17644

The June 2025 paper characterizes and optimizes the Key-Value Cache (KV$) workload patterns associated with serving large language models (LLMs) at a major cloud provider. Using real-world production traces from customer-facing (to-C) and business-facing (to-B) workloads, the authors analyze KV$ reuse behaviors, noting that reuses are significantly skewed, with single-turn requests being as important as multi-turn requests, especially in API-dominated workloads. Crucially, the analysis reveals that KV$ lifespan is ephemeral and that reuse probability follows predictable exponential distributions within specific request categories. Based on these findings, the researchers propose a workload-aware cache eviction policy that significantly improves the cache hit ratio and reduces the query time to first token compared to standard policies like LRU and LFU. Source: https://arxiv.org/pdf/2506.02634v1

These two 2025 research papers collaboratively examine Moravec's Paradox, which posits that skills effortless for humans (like perception and mobility) are computationally difficult for machines, while complex reasoning tasks (like math or chess) are comparatively easy for AI. The Wikipedia entry introduces the paradox, explaining its evolutionary basis: skills acquired over millions of years are deeply encoded and hard to reverse-engineer, while abstract thought is evolutionarily recent and less efficient. A research paper further demonstrates this gap with an "auditory Turing test," where state-of-the-art AI models catastrophically fail (achieving less than 7% accuracy) on simple human listening tasks involving overlapping speech and noise, confirming the paradox in the auditory domain. Finally, an economics preprint incorporates the paradox into a model of economic growth, arguing that the high or infinite computational cost of automating physical, sensorimotor tasks means human labor in these bottleneck areas will persist, preventing the labor share of income from collapsing to zero as some AGI models predict.Sources:https://arxiv.org/pdf/2507.23091https://arxiv.org/pdf/2509.24466https://en.wikipedia.org/wiki/Moravec%27s_paradox

These May and September 2025 technical reports introduce and evaluate two distinct but related large language models: the Qwen3 family and the Qwen3-Omni multimodal system. The first source focuses on the text-based Qwen3 models, highlighting their development process, which includes a sophisticated multilingual data annotation system and a multi-stage training pipeline incorporating Strong-to-Weak Distillation and "Thinking Mode" for complex tasks like coding and mathematics. The second, more comprehensive source describes Qwen3-Omni, a single model designed to excel across text, image, audio, and video modalities without performance degradation, utilizing a novel Thinker–Talker Mixture-of-Experts (MoE) architecture for real-time speech and reasoning. Crucially, Qwen3-Omni achieves state-of-the-art performance in audio tasks and boasts extremely low first-packet latency for interactive applications, supporting a wide range of multilingual capabilities in both speech and text.Sources:https://arxiv.org/pdf/2505.09388https://arxiv.org/pdf/2509.17765

This September 26 2025 paper is an excerpt from a research paper introducing a variational reasoning framework designed to enhance the reasoning capabilities of language models (LLMs). This framework conceptualizes thinking traces as latent variables and uses variational inference to optimize them, building upon the Evidence Lower Bound (ELBO) and extending it with a tighter, multi-trace IWAE-style bound. Crucially, the paper proposes a forward-KL objective for stabilizing the training of the variational posterior, which samples high-quality thinking paths. The research also interprets existing methods like Rejection Sampling Finetuning (RFT) and binary-reward Reinforcement Learning (RL) as local forward-KL objectives, highlighting a previously unrecognized bias toward easier questions in these traditional approaches. Empirical validation on the Qwen 2.5 and Qwen 3 models across diverse benchmarks confirms that this principled probabilistic perspective leads to consistent performance improvements and greater training stability compared to strong baselines. Source: https://arxiv.org/pdf/2509.22637

This Meta September 24 2025 paper provides an extensive overview of Code World Model (CWM), a 32-billion-parameter dense decoder-only Transformer designed for coding and reasoning tasks, highlighting its architecture and multi-stage training process. Training involves pre-training, mid-training, and post-training stages which include supervised fine-tuning (SFT) and joint reinforcement learning (RL) across environments like software engineering (SWE) tasks, coding problems, and mathematics. A core feature is CWM's ability to process and predict Python execution traces and perform agentic interactions using a minimal set of tools within containerized environments. The document details the model's competitive performance against other large language models on benchmarks like SWE-bench Verified and discusses infrastructure choices, such as asynchronous RL and fp8 matrix multiplication, used to achieve training efficiency.Sources:https://ai.meta.com/temp/research/publications/cwm-an-open-weights-llm-for-research-on-code-generation-with-world-models/https://scontent-lax3-2.xx.fbcdn.net/v/t39.2365-6/553592426_661450129912484_4072750821656455102_n.pdf?_nc_cat=103&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=0-g1m0kIX7cQ7kNvwFJRCs6&_nc_oc=AdnQYyoahmWWXKZWyQuh4F09IuhBGd08Uwox14N8BdY_tMilZ5_Tl5u7P82HLIJ9RSc8nDy188xiuxmmByXhkJ1S&_nc_zt=14&_nc_ht=scontent-lax3-2.xx&_nc_gid=RcYYIy-y9eIernCj-naXRQ&oh=00_AfYC3ol0MNn_PokD4H3hoOcpxg6OoOsgWfD4pDyThszIpA&oe=68DD28B5