AI Post Transformers

This January 2025 paper introduces HybridServe, an LLM inference system designed to enhance throughput and cost-effectiveness for large language models by optimizing memory usage and host-GPU communication. It tackles the challenges of host memory offloading, where model parameters and KV cache are stored on slower host memory to reduce costs but can lead to GPU underutilization due to limited transfer bandwidth. HybridServe proposes a novel activation checkpointing technique with a KV-Activation hybrid caching scheme that stores intermediate activations, allowing for faster recomputation of the KV cache while model parameters are transferred. This system dynamically balances communication overhead and recomputation time to maximize throughput, demonstrating significant improvements over existing state-of-the-art methods like FlexGen. Source: https://arxiv.org/pdf/2501.01792

This August 2025 paper explores the critical area of fact-checking and factuality evaluation in Large Language Models (LLMs). It systematically analyzes the challenges of misinformation generation, particularly hallucinations, which are factually incorrect but fluent outputs from LLMs. The paper investigates various mitigation strategies, including fine-tuning, instruction tuning, and Retrieval-Augmented Generation (RAG), which grounds LLM outputs in external knowledge. It further examines evaluation metrics, datasets, and prompting strategies used to assess and enhance the factual accuracy of these models, highlighting the need for more robust, explainable, and domain-specific fact-checking frameworks. The review concludes by identifying open issues and future research agendas to foster more trustworthy and context-aware LLMs. Source: https://arxiv.org/pdf/2508.03860

This September 2025 paper presents MetaGraph, a novel methodology for constructing knowledge graphs from scientific literature, specifically applied to Financial Natural Language Processing (NLP) research between 2022 and 2025. The authors utilized Large Language Models (LLMs) to extract key information from 681 papers, including tasks, datasets, models, motivations, and limitations, and organized it into a structured, queryable format. The analysis highlights three phases in Financial NLP's evolution: initial LLM adoption and task/dataset innovation, subsequent critical reflection on LLM limitations, and a current trend toward integrating peripheral techniques into modular systems. The research reveals a shift toward Financial Question Answering (QA), increased use of synthetic data, and a growing emphasis on open-source models and system-level solutions like Retrieval-Augmented Generation (RAG). Ultimately, MetaGraph offers a reusable framework for quantitatively mapping scientific progress and understanding changing priorities in a rapidly evolving field. Source: https://www.arxiv.org/pdf/2509.09544

This September 2023 paper introduces PyTorch Fully Sharded Data Parallel (FSDP), an advanced solution designed to scale the training of exceptionally large machine learning models. It addresses limitations of previous methods like Distributed Data Parallel (DDP) by sharding model parameters, gradients, and optimizer states across multiple GPUs, thereby drastically reducing individual GPU memory consumption. FSDP employs various techniques, including deferred initialization, flexible sharding strategies, and optimizations for communication overlap and prefetching, to ensure high efficiency and a user-friendly experience. The research demonstrates FSDP's effectiveness in training models with billions of parameters, achieving near-linear scalability and enabling broader access to large model development. The authors also discuss interoperability with other parallelization paradigms and known limitations. Source: https://arxiv.org/pdf/2304.11277

This September 2025 paper explores the concept of long-horizon execution in Large Language Models (LLMs), arguing that marginal gains in single-step accuracy can lead to exponential improvements in the length of tasks LLMs can complete. The authors introduce a novel framework to isolate execution capabilities by providing models with necessary knowledge and plans, revealing that larger models can execute significantly more steps, even when smaller models achieve perfect single-turn accuracy. A key finding is the "self-conditioning effect," where LLMs become more prone to errors when their past mistakes are present in the context, a challenge not fully mitigated by increasing model size. However, the paper concludes that "thinking" models, which employ sequential test-time compute, effectively address this self-conditioning and can execute substantially longer tasks in a single turn. Source: https://arxiv.org/pdf/2509.09677

This May 2025 paper explores the structural patterns of knowledge within Large Language Models (LLMs) by adopting a graph-based perspective. The authors quantify LLM knowledge at both the triplet and entity levels, analyzing its relationship with graph properties like node degree. Key findings include the discovery of knowledge homophily, where closely connected entities exhibit similar knowledgeability, and a positive correlation between an entity's degree and its knowledge. These insights further motivate the development of graph machine learning models to predict entity knowledge, which can then be used to strategically select less-known triplets for fine-tuning LLMs, leading to improved performance. The study evaluates several prominent LLMs across diverse knowledge graphs, highlighting domain-specific variations in knowledge distribution and the consistent presence of structural patterns. Source: https://arxiv.org/html/2505.19286v2

On this November 2025 paper the Meta Llama Team's paper introduces Llama 3, a new family of large language models featuring 8B, 70B, and 405B parameters, designed with native multilingual support, coding, reasoning, and tool usage capabilities. The development emphasizes data quality and diversity, employing extensive filtering, de-duplication, and heuristic cleaning processes for both English and multilingual data, alongside scaling laws to optimize model size and training budgets. The models utilize a standard dense Transformer architecture with minor adaptations like grouped query attention and an attention mask for multi-document sequences, demonstrating comparable performance to leading models such as GPT-4 across various benchmarks. Furthermore, the research explores integrating multimodal capabilities—image, video, and speech—through compositional approaches involving specialized encoders and adapters, which are trained through multi-stage pre-training and fine-tuning. A significant focus is also placed on safety and responsible development, incorporating comprehensive data cleaning, iterative safety finetuning with reward models and DPO, and robust red teaming efforts to address risks like insecure coding and prompt injection, while publicly releasing Llama Guard 3 as a system-level safety classifier. Source: https://arxiv.org/pdf/2407.21783

This September 2025 published research investigates how large language models (LLMs) perform mental math, particularly focusing on the flow of information and computational processes within their transformer architecture. The authors introduce two novel techniques, Context-Aware Mean Ablation (CAMA) and Attention-Based Peeking (ABP), to identify a minimal computational subgraph called All-for-One (AF1). This subgraph reveals that for mental math tasks, input-specific computation is largely deferred to later layers and primarily handled by the final token, which receives necessary information from other tokens during a few specific intermediate layers. The study demonstrates that this sparse AF1 subgraph is sufficient and necessary for high performance across various arithmetic expressions and models, offering significant insights into the mechanistic interpretability of LLM arithmetic reasoning. Source: https://www.arxiv.org/pdf/2509.09650

This October 2024 paper introduces synthetic continued pretraining (synthetic CPT), a novel method designed to enhance language model knowledge acquisition from small, specialized text collections. Current large language models often struggle with data efficiency and learning niche facts from limited sources. The core of this approach is EntiGraph, a synthetic data augmentation algorithm that extracts entities and their relationships from a small corpus to generate a much larger, more diverse synthetic dataset. Experiments using the QuALITY dataset demonstrate that EntiGraph CPT significantly improves a model's ability to answer questions and summarize content about these specialized domains, outperforming direct training on raw data or simple rephrasing techniques. The authors also explore the scaling properties of EntiGraph, revealing distinct phases of knowledge acquisition. Source: https://arxiv.org/pdf/2409.07431

These September 2025 papers present a technical report on SpikingBrain, a novel family of large language models (LLMs) that draw inspiration from brain mechanisms to address the efficiency challenges of traditional Transformer architectures. The research focuses on efficient long-context training and inference by developing hybrid linear attention architectures and an adaptive threshold spiking neuron scheme. A significant aspect of this work is the successful training and deployment of these models on non-NVIDIA GPU clusters, specifically the MetaX platform, demonstrating the feasibility of large-scale LLM development on alternative hardware. The authors highlight substantial speedups in inference for long sequences and significant reductions in energy consumption through the sparse, event-driven spiking design, with performance comparable to established Transformer baselines while using considerably less training data. This work ultimately aims to advance the design of energy-efficient and scalable brain-inspired LLMs for next-generation computing systems, including neuromorphic hardware.Sources:https://arxiv.org/pdf/2509.05276https://arxiv.org/html/2509.05276v1