AI Post Transformers

This January 2019 academic paper addresses the common issue of poor generalization in adaptive gradient optimization methods like Adam, compared to traditional Stochastic Gradient Descent (SGD) with momentum. The authors demonstrate that L2 regularization and weight decay are not equivalent for adaptive optimizers, unlike for standard SGD, leading to suboptimal performance in Adam. They propose a simple modification called "decoupled weight decay" (AdamW), which separates the weight decay step from the gradient-based updates. Empirical evidence shows that AdamW significantly improves Adam's generalization performance on image classification tasks and simplifies hyperparameter tuning by decoupling the learning rate and weight decay factors. Furthermore, the paper introduces AdamWR, incorporating warm restarts to further enhance AdamW's anytime performance, ultimately making Adam competitive with SGD with momentum. Source: https://arxiv.org/pdf/1711.05101

This August 2025 paper introduces Oaken, a novel acceleration solution for serving Large Language Models (LLMs) that addresses the significant challenges of memory bandwidth and capacity bottlenecks inherent in batched LLM inference. Oaken achieves this through a co-designed algorithm and hardware architecture, featuring an online-offline hybrid KV cache quantization technique. This technique efficiently reduces the memory footprint and access requirements of the Key-Value (KV) cache by categorizing data into "inliers" and "outliers" using offline threshold profiling and applying group-shift quantization. Furthermore, Oaken integrates custom quantization/dequantization engines and memory management units into LLM accelerators to translate algorithmic gains into tangible performance improvements, demonstrating increased throughput and minimal accuracy loss compared to existing methods. Source: https://arxiv.org/html/2503.18599v2

This August 2025 paper introduces Pimba, a novel Processing-in-Memory (PIM) accelerator designed to enhance the efficiency of Large Language Model (LLM) serving for both traditional transformer-based models and emerging post-transformer architectures. The authors highlight that memory bandwidth is a critical bottleneck for both types of LLMs, specifically during attention operations in transformers and state updates in post-transformers. Pimba addresses this by integrating PIM technology with LLM quantization, using a State-update Processing Unit (SPU) shared between memory banks to maximize hardware resource sharing and area efficiency. The system employs MX-based quantized arithmetic within its State-update Processing Engine (SPE), which is identified as a Pareto-optimal choice for balancing accuracy and area overhead. Evaluations show Pimba significantly boosts token generation throughput and reduces latency and energy consumption compared to existing GPU and GPU+PIM systems, providing a unified and scalable solution for diverse LLM serving demands. Source: https://arxiv.org/pdf/2507.10178

This February 2025 research addresses the critical issue of training instability in Large Language Models (LLMs), which often stems from sudden, massive "gradient spikes" that can be thousands of times larger than typical gradients. The authors introduce Spike-Aware Adam with Momentum Reset (SPAM), a novel optimizer designed to counteract these spikes through periodic momentum resets and spike-aware gradient clipping, which scales down rather than zeroes out large gradients. Experiments demonstrate that SPAM consistently outperforms existing optimizers like Adam and Adafactor across various LLM sizes during both pre-training and fine-tuning. Furthermore, SPAM offers a memory-efficient version leveraging sparse momentum, enabling better performance under memory constraints compared to other state-of-the-art memory-efficient optimizers. The study highlights the detrimental impact of gradient spikes and presents an effective optimization strategy to enhance LLM training stability and resource efficiency. Source: https://arxiv.org/pdf/2501.06842

This academic paper addresses the inherent challenges in training Recurrent Neural Networks (RNNs), specifically the vanishing and exploding gradient problems. The authors explore these issues from analytical, geometrical, and dynamical systems perspectives, building upon previous work. They propose and empirically validate a gradient norm clipping strategy to combat exploding gradients and a soft regularization constraint to mitigate vanishing gradients. The research demonstrates that these solutions significantly improve RNN performance on both synthetic pathological tasks requiring long-term memory and natural language processing and music prediction problems. Source: https://arxiv.org/pdf/1211.5063

This August 2025 paper introduces ComoRAG, a novel framework designed to enhance long-context narrative comprehension in Large Language Models (LLMs) by simulating human metacognitive regulation. It addresses the limitations of existing Retrieval-Augmented Generation (RAG) methods, which struggle with stateful reasoning and integrating contradictory evidence over extended narratives. ComoRAG employs a dynamic cognitive loop that includes a hierarchical knowledge source (veridical, semantic, and episodic layers) and a dynamic memory workspace to continuously acquire new evidence and consolidate knowledge. Experimental results demonstrate ComoRAG's superior performance, particularly in solving complex narrative and inferential queries across various datasets, showcasing its robustness and flexibility as a model-agnostic, plug-and-play solution. Source: https://arxiv.org/pdf/2508.10419

This August 2025 paper presents Google's comprehensive methodology for measuring the environmental impact of AI inference workloads in a large-scale production environment. It addresses a critical gap in existing research by accounting for the full stack of AI serving infrastructure, including active AI accelerator power, host system energy, idle machine capacity, and data center overhead. The paper reveals that a median Gemini Apps text prompt consumes significantly less energy, carbon emissions, and water than many prior public estimates. Furthermore, it highlights Google's efforts in software efficiency and clean energy procurement, which have led to substantial reductions in the environmental footprint of AI serving over the past year. The authors advocate for a standardized, comprehensive measurement framework to accurately compare AI models and incentivize further efficiency gains across the industry. Source: https://arxiv.org/pdf/2508.15734

This August 2025 paper introduces ODYSSEY, a comprehensive framework for open-world mobile manipulation that integrates robotic mobility, manipulation, and real-time perception. It highlights a novel approach that uses large language models for high-level task planning and vision-language models for fine-grained action guidance, enabling robots to adaptively interact in complex environments. A significant contribution is the first comprehensive benchmark for long-horizon mobile manipulation, featuring diverse daily tasks in both indoor and outdoor settings to thoroughly evaluate embodied reasoning, planning, navigation, and manipulation capabilities. The system demonstrates strong sim-to-real transfer performance, although challenges with precise control and robust perception in real-world scenarios are identified. The paper details the coarse-to-fine task planner and a reinforcement learning-based whole-body policy, both crucial for generalizing across varied terrains and overcoming the gap between simulation and reality. Source: https://arxiv.org/pdf/2508.08240

This August 2025 academic paper explores the application of post-training quantization (PTQ) to diffusion large language models (dLLMs), a promising alternative to traditional autoregressive LLMs for natural language generation. The authors conduct a systematic study to understand how existing PTQ techniques, commonly used for compressing AR LLMs, perform with dLLMs. A key finding is the prevalence of activation outliers in dLLMs, which pose a significant challenge for low-bit quantization. The research also evaluates the effectiveness of various quantization methods, bit-widths, task types, and model variants, concluding that 4-bit quantization is optimal for weight-only methods like GPTQ, while 8-bit is tolerable for weight-activation quantization, with rotation-based methods like DuQuant showing superior performance. The study ultimately aims to facilitate the efficient deployment of dLLMs on resource-constrained devices by providing practical insights into their quantization behavior. Source: https://arxiv.org/pdf/2508.14896

This August 2025 academic paper, titled "Has GPT-5 Achieved Spatial Intelligence? An Empirical Study," examines the spatial understanding and reasoning capabilities of advanced multi-modal AI models, including the recently released GPT-5. The authors propose a new taxonomy for spatial tasks and evaluate both proprietary and open-source models against eight key benchmarks, utilizing over a billion tokens for their study. Their findings indicate that while GPT-5 shows unprecedented strength in spatial intelligence, it still falls short of human performance across a broad range of tasks. The research also identifies specific challenging problems for multi-modal models and notes that proprietary models don't consistently outperform open-source options on the most difficult problems. The study further includes a qualitative evaluation of scenarios that are intuitive for humans but prove difficult for even the most advanced AI models. Source: https://arxiv.org/abs/2508.13142