AI Post Transformers

This September 20 2025 paper introduce a novel, efficient architecture for training retrieval models used in retrieval-augmented generation (RAG) systems. This architecture addresses the inefficiency of fine-tuning large models by combining adapters for soft embeddings with a Classifier-as-Retriever (CaR) approach. The soft embeddings, created by lightweight layers in a frozen small language model (SLM), efficiently adapt the model to new corpora, while the CaR replaces static maximum inner product search (MIPS) with a trainable classifier for significantly higher accuracy (up to 99%). Furthermore, the methods integrate naturally with federated learning (FL) to achieve distributed training speedups (up to 2.6x faster) and utilize differential privacy (DP) techniques to safeguard client data during edge device training. This combined approach results in a lighter, faster, and privacy-preserving solution for domain-specific RAG.Sources:https://www.webai.com/blog/federated-learning-with-soft-embeddings-a-new-efficient-way-to-train-retrieval-modelshttps://arxiv.org/pdf/2509.16508

This September 18 2025 paper introduces a research project that applies Schoenfeld’s Episode Theory, a classic cognitive framework for analyzing human mathematical problem-solving, to understand the reasoning processes of Large Reasoning Models (LRMs). The authors created a novel, publicly available benchmark by annotating thousands of sentences and paragraphs from model-generated solutions to math problems, using seven cognitive labels such as Plan, Implement, and Verify. This approach offers a theoretically grounded methodology for interpreting LRM cognition, demonstrating that machine reasoning exhibits structured, episodic patterns similar to human behavior. The resulting annotated corpus and analytical protocol aim to enable the development of more transparent and controllable reasoning systems. Source: https://arxiv.org/pdf/2509.14662

The September 25 2035 paper introduces a novel reinforcement learning (RL) algorithm, Controlling Entropy via Gradient-Preserving Policy Optimization (CE-GPPO), designed to fine-tune large language models (LLMs) for complex reasoning tasks. The authors analyze how policy entropy, which represents the balance between exploration and exploitation, becomes unstable in existing methods like Proximal Policy Optimization (PPO) due to the clipping of low-probability tokens. CE-GPPO addresses this by reintroducing gradients from these clipped tokens—specifically Positive-advantage Low-Probability (PA&LP) and Negative-advantage Low-Probability (NA&LP) tokens—in a bounded and controlled manner. The goal is to regulate entropy dynamics and prevent both entropy collapse and entropy explosion. Empirical results on mathematical reasoning benchmarks show that CE-GPPO consistently outperforms strong baselines by maintaining more stable and optimal entropy throughout training. Source: https://arxiv.org/pdf/2509.20712

The September 24 2025 paper introduces EmbeddingGemma, a novel, lightweight text embedding model developed by Google DeepMind, built upon the Gemma 3 language model family. The paper details the innovative training methodology, which involves encoder-decoder initialization and geometric embedding distillation from larger models like Gemini Embedding, alongside a "spread-out" regularizer and model souping for improved expressiveness and generalizability. Through extensive evaluation on the Massive Text Embedding Benchmark (MTEB), the 308M-parameter model is shown to achieve state-of-the-art performance among models under 500M parameters across multilingual, English, and code tasks, often rivaling models double its size, thus offering an exceptional performance-to-cost ratio suitable for low-latency, on-device applications. Ablation studies support the design choices, concluding that the encoder-decoder initialization and mean pooling provide the strongest foundation for high-quality embeddings. Source: https://arxiv.org/pdf/2509.20354

The September 25 2025 dated sources introduce GDPval, a novel benchmark created by OpenAI to evaluate the performance of AI models on economically valuable, real-world tasks. This evaluation spans 44 knowledge work occupations across the top nine sectors contributing to the U.S. GDP, using tasks meticulously crafted by experienced industry professionals. Results indicate that the best frontier models are approaching human expert quality on these tasks, with models like Claude Opus 4.1 and GPT-5 demonstrating strengths in different areas, such as aesthetics and accuracy, respectively. Furthermore, the analysis suggests that integrating AI can potentially lead to significant speed and cost improvements in expert workflows, while noting that model performance is still limited by the real-world complexity of multi-draft and ambiguous tasks. Finally, OpenAI is open-sourcing a subset of tasks and an automated grader to facilitate further research in tracking AI capabilities.Sources:https://openai.com/index/gdpval/https://cdn.openai.com/pdf/d5eb7428-c4e9-4a33-bd86-86dd4bcf12ce/GDPval.pdf

The September 24 2025 paper is a technical report from ByteDance Seed detailing the Seedream 4.0 system, an advanced multimodal image generation model. This single framework efficiently unifies text-to-image synthesis, image editing, and multi-image composition. The core innovation is an efficient diffusion transformer and powerful VAE that facilitates fast generation of high-resolution images (up to 4K), outperforming competitors like GPT-4o and Gemini 2.5 in human and automatic evaluations. The system uses sophisticated training techniques, including multi-modal post-training and adversarial acceleration methods, to achieve state-of-the-art results and ultra-fast inference speed. Seedream 4.0 is designed for both creative and professional applications, featuring capabilities like in-context reasoning and advanced text rendering. Source: https://arxiv.org/pdf/2509.20427

The September 25 2025 paper introduces Tree-based Group Relative Policy Optimization (Tree-GRPO), a new reinforcement learning (RL) method designed to enhance the agentic capabilities of large language models (LLMs) in multi-turn tasks where supervision is typically sparse. Tree-GRPO addresses the challenges of sparse rewards and heavy rollout costs associated with existing chain-based RL by employing a tree-search sampling strategy where each node represents a complete agent interaction step, allowing for prefix sharing and reduced budget use. This tree structure inherently creates finer-grained process supervision signals from outcome rewards, a mechanism shown to be structurally equivalent to step-level direct preference learning. Empirical results across multiple datasets demonstrate that the tree-based approach consistently achieves higher performance with less rollout budget compared to chain-based methods. Source: https://arxiv.org/pdf/2509.21240

These 14 research papers provide an overview of various compression techniques for Large Language Models (LLMs), primarily focusing on reducing the size and computational overhead of the Key-Value (KV) cache to handle long contexts more efficiently. Several novel methods are detailed, including GVote, an adaptive compression algorithm using query sampling and voting to find an optimal cache budget, and SnapKV, which selects clustered, important KV positions based on an "observation" window to maintain performance while increasing speed and memory efficiency. Other approaches include POD (Proximal tokens over Distant tokens), which reduces redundancy by sharing key states across layers for distant tokens while preserving proximal ones, and DecoQuant, a quantization method utilizing matrix decomposition to reduce errors. The sources also examine prompt compression methods like LLMLingua and LongLLMLingua, and describe CASC (Context-Adaptive Synthesis and Compression), a Retrieval-Augmented Generation (RAG) framework that intelligently synthesizes and compresses multi-document contexts to improve answer accuracy in complex domains.Sources:https://arxiv.org/pdf/2509.08315https://arxiv.org/html/2509.09199v1https://arxiv.org/html/2509.03136v1https://aclanthology.org/2025.acl-long.1394.pdfhttps://proceedings.neurips.cc/paper_files/paper/2024/file/fd0705710bf01b88a60a3d479ea341d9-Paper-Conference.pdfhttps://arxiv.org/html/2412.14838v1https://arxiv.org/pdf/2412.02252https://aclanthology.org/2024.acl-long.133.pdfhttps://arxiv.org/html/2508.19357v1https://aclanthology.org/2024.acl-long.91.pdfhttps://arxiv.org/html/2310.05736v2https://aclanthology.org/2025.naacl-long.368.pdfhttps://arxiv.org/pdf/2404.14469https://aclanthology.org/2024.findings-emnlp.266.pdf

The September 2025 academic paper introduces LLM-Interleaved (LLM-I), a novel, flexible framework for interleaved image-text generation that reframes the task as a tool-use problem to overcome the "one-tool" limitation of unified models. Authored by researchers from Zhejiang University and ByteDance, BandAI, the system uses a central Large Language Model (LLM) or Multimodal LLM (MLLM) agent to orchestrate a diverse toolkit of specialized visual tools, including online image search, diffusion generation, code execution, and image editing. The agent is trained using a Reinforcement Learning (RL) framework featuring a hybrid reward system that combines rule-based logic with LLM and MLLM evaluators. The research demonstrates that LLM-I achieves state-of-the-art performance across four benchmarks by moving from an "omniscient solver" to a "proficient tool-user" paradigm, allowing for factually grounded and programmatically precise visual outputs. Source: https://arxiv.org/pdf/2509.13642

This September 2025 paper source is a research paper from Tencent AI Lab and academic collaborators that introduces EVOL-RL, an Evolution-Oriented and Label-free Reinforcement Learning framework for Large Language Models (LLMs). The paper addresses a critical flaw, termed entropy collapse, in existing label-free self-improvement methods like Test-Time Reinforcement Learning (TTRL), where reliance solely on a majority vote leads to a shrink in solution diversity and poor generalization. EVOL-RL overcomes this by incorporating a novel reward system that explicitly balances majority-based selection (for stability) with a novelty-aware reward (for variation), preventing the model from converging to repetitive, low-entropy solutions. Experimental results on mathematical reasoning benchmarks demonstrate that EVOL-RL significantly improves accuracy and generalization by sustaining diverse, longer chains of thought compared to the TTRL baseline. Source: https://arxiv.org/pdf/2509.15194