AI Post Transformers

This September 2025 paper introduces LoFT, a novel framework designed to improve Long-Tailed Semi-Supervised Learning (LTSSL) by leveraging parameter-efficient fine-tuning of pre-trained foundation models. The core idea is to enhance confidence calibration and generate more reliable pseudo-labels, which are crucial for addressing the imbalance inherent in long-tailed datasets. Furthermore, the paper extends this approach to open-world scenarios with LoFT-OW, specifically incorporating mechanisms to detect and filter out-of-distribution (OOD) samples from unlabeled data. The authors demonstrate that these fine-tuned models achieve superior performance on various benchmarks, even when utilizing significantly less unlabeled data compared to previous methods. Source: https://arxiv.org/pdf/2509.09926

This July 2025 paper discusses advanced memory optimization techniques for Large Language Models (LLMs), particularly focusing on KV cache management in multi-tenant serving environments. The primary subject, MIRAGE, introduces parameter remapping, a novel method that dynamically repurposes GPU memory allocated for model parameters to expand KV cache capacity, outperforming traditional CPU-offloading and KV cache swapping by reducing latency and increasing throughput. Complementary research highlights challenges in on-device LLM deployment and proposes solutions like quantization (AWQ) for model compression and two-level scheduling (FineServe, Nexus) for efficient GPU sharing to mitigate memory fragmentation and improve performance. Overall, the papers underscore the critical need for innovative memory management to address the growing memory demands of LLMs and enhance their inference serving efficiency across diverse hardware configurations. Source: https://www.researchgate.net/publication/393724496_MIRAGE_KV_Cache_Optimization_through_Parameter_Remapping_for_Multi-tenant_LLM_Serving

This September 2025 paper describes QuantAgent, a novel multi-agent large language model (LLM) framework designed for high-frequency quantitative trading based solely on price-derived market signals. The system decomposes trading decisions into four specialized agents—IndicatorAgent, PatternAgent, TrendAgent, and DecisionAgent—which analyze market dynamics from complementary perspectives and communicate through structured prompts. QuantAgent consistently outperforms baseline models across diverse assets, including commodities, equities, and cryptocurrencies, demonstrating robust generalization and achieving high directional accuracy in predicting price movements. A key feature is its ability to produce traceable, language-native explanations for its trading decisions, enhancing transparency and interpretability in a field traditionally marked by opaque algorithms. The framework also includes a local, browser-based interface that allows users to visualize market data and interact with LLM-generated analyses, emphasizing user control and understanding. Source: https://arxiv.org/pdf/2509.09995

This April 2025 paper introduces ShadowKV, an innovative inference system for long-context Large Language Models (LLMs) designed to significantly enhance throughput and support larger batch sizes without compromising accuracy. It achieves this by strategically managing the Key-Value (KV) cache: specifically, it compresses the low-rank pre-Rotary Position Embedding (RoPE) key cache on the GPU and offloads the value cache to the CPU. ShadowKV further optimizes performance through an accurate KV selection strategy that reconstructs minimal sparse KV pairs on-the-fly, thus minimizing decoding latency. Empirical evaluations demonstrate that ShadowKV can support up to 6x larger batch sizes and boost throughput by up to 3.04x on an A100 GPU across various LLMs and benchmarks, even outperforming theoretical infinite memory scenarios. Source: https://arxiv.org/pdf/2410.21465

This May 2025 paper introduces TailorKV, a novel hybrid framework designed to optimize Key-Value (KV) cache management in large language models (LLMs) for long-context inference. It addresses challenges like high GPU memory consumption and inference latency that arise from the linear growth of KV cache size with sequence length. TailorKV categorizes Transformer layers into quantization-friendly and sparsity-friendly based on their attention patterns, applying 1-bit quantization to the former and dynamic retrieval of Top-K tokens from CPU memory for the latter. This tailored approach significantly reduces memory usage and decoding latency while maintaining model accuracy, enabling LLMs to operate efficiently on resource-limited hardware. Source: https://arxiv.org/pdf/2505.19586

This September 2025 paper introduces WebSailor-V2, an open-source deep research agent developed by Alibaba Group's Tongyi Lab. The paper details a post-training pipeline that uses a novel synthetic data construction scheme, SailorFog-QA-V2, and a dual-environment reinforcement learning framework. WebSailor-V2, built on the Qwen3-30B-A3B model, demonstrates state-of-the-art performance among open-source agents and is competitive with leading proprietary systems on various web-agent benchmarks, including BrowseComp and Humanity's Last Exam. The authors emphasize that high-quality data and a stable training environment are more crucial than the specific RL algorithm for developing robust AI agents. Source: https://arxiv.org/pdf/2509.13305

This May 2025 paper introduces a resource-aware algorithm designed to optimize the performance of Large Language Models (LLMs) for low-latency inference on edge computing devices. The core innovation lies in its fine-grained partitioning of the Transformer architecture, specifically at the attention head-level, rather than coarser layer-level divisions. This approach allows for dynamic reassignment and migration of these individual attention heads and their associated Key/Value (K/V) caches across heterogeneous edge devices. By managing the expanding memory footprint of K/V caches and exploiting parallel execution of attention heads, the proposed method significantly reduces inference latency and memory usage compared to existing static or layer-based partitioning strategies. Source: https://arxiv.org/pdf/2505.02533

This February 2025 paper introduce CodeI/O, a novel training method for Large Language Models (LLMs) that enhances general reasoning abilities by transforming code into an input-output prediction task. Instead of focusing on generating code, CodeI/O trains models to predict inputs or outputs of a given code in natural language Chain-of-Thought (CoT) rationales. This approach allows LLMs to learn universal reasoning primitives embedded in code, such as logic flow and decision-making, while decoupling them from specific programming syntax. An improved version, CodeI/O++, further refines training data through multi-turn revision based on execution feedback. Experimental results demonstrate that both CodeI/O and CodeI/O++ lead to consistent and balanced performance improvements across a wide range of symbolic, scientific, logical, and mathematical reasoning tasks in LLMs. Source: https://arxiv.org/html/2502.07316v1

This December 2024 paper introduces a collaborative inference framework designed for large-scale models in 5G smart city edge computing environments, addressing the challenge of limited memory and computing capacity on individual edge nodes. The framework partitions large models into sub-models deployed across multiple edge nodes and incorporates an early exit mechanism to accelerate inference. To manage the complexities of heterogeneous systems and dynamic environments, the authors propose a distributed algorithm called DTO-EE, which jointly optimizes task offloading strategies and confidence thresholds for early exits. Experimental results demonstrate that DTO-EE significantly reduces response delay and improves inference accuracy compared to existing methods. Source: https://arxiv.org/pdf/2412.08284

This August 2025 paper examines the evolving landscape of Federated Large Language Models (FedLLM), focusing on how large language models are post-trained while preserving user data privacy. The authors introduce a novel taxonomy that categorizes FedLLM approaches based on model accessibility (white-box, gray-box, and black-box) and parameter efficiency. It highlights various techniques within these categories, such as adapter-based tuning and prompt tuning, which reduce computational and communication overhead. The paper also discusses the growing importance of inference-only black-box settings for future FedLLM development and identifies open challenges like federated value alignment and enhanced security in constrained environments. Source: https://arxiv.org/html/2508.16261v1