AI Post Transformers

This is a classic review of a now old but yet still important paper, the original Flash Attention paper. We review this in light of advances in compiler technology. The June 23, 2022 Stanford paper describes the original FlashAttention, an innovative, IO-aware algorithm designed to significantly enhance the efficiency of the attention mechanism in Transformer models by optimizing memory usage and access. Standard attention suffers from complexity that scales quadratically ($O(N^2)$) with sequence length ($N$) for both memory footprint and access to slow High Bandwidth Memory (HBM), which creates a performance bottleneck. FlashAttention overcomes this by employing tiling and recomputation within a single customized CUDA kernel, dramatically reducing the memory footprint to scale linearly ($O(N)$) and eliminating the quadratic term in HBM access complexity. While the algorithm does not reduce the total Floating Point Operations (FLOPs) and even slightly increases them due to recomputation, the massive reduction in slow memory transfers results in substantial wall-clock runtime speedups during both training and inference. Source: https://arxiv.org/pdf/2205.14135

The October 22, 2025 GigaAI paper introduces GigaBrain-0, a novel Vision-Language-Action (VLA) model designed for general-purpose robotic systems, which is primarily trained using a combination of real-world robot data and synthetic data generated by a world model called GigaWorld. This approach aims to enhance generalization across various real-world conditions by leveraging diverse synthetic data streams like Real2Real Transfer, Sim2Real Transfer, and View Transfer. Architecturally, GigaBrain-0 incorporates RGB-D input modeling for better spatial reasoning and uses an embodied Chain-of-Thought (CoT) framework that generates intermediate reasoning steps such as manipulation trajectories and subgoal language. Experimental results across dexterous manipulation, long-horizon, and mobile manipulation tasks demonstrate that the model, particularly when augmented with world model-generated data, achieves superior performance and robustness compared to baseline models like $\pi0$. The paper also presents GigaBrain-0-Small, an optimized variant for efficient hardware deployment. Source: https://arxiv.org/pdf/2510.19430

This March 27, 2025 Anthropic paper provides an overview and detailed excerpts from two related Anthropic papers concerning the interpretability of large language models, specifically focusing on Claude 3.5 Haiku. The core objective is to reverse engineer the internal computational mechanisms, or "circuits," that drive the model's behavior, analogous to studying biology or neuroscience. The research introduces a circuit tracing methodology that uses attribution graphs and feature analysis to examine how the model handles various tasks, including multi-step reasoning, planning in poems, multilingual translation, and arithmetic. Findings reveal sophisticated strategies like internal planning and the existence of "default" refusal circuits that must be inhibited by "known answer" features for the model to respond to questions, illuminating the mechanisms behind hallucinations and jailbreaks.Sources:https://transformer-circuits.pub/2025/attribution-graphs/biology.htmlhttps://www.anthropic.com/research/tracing-thoughts-language-model

On October 20, 2025 Hugging Face released MTEB v2, a significant refactoring of the Massive Text Embedding Benchmark, which was originally designed for evaluating text embedding models across various tasks like classification and retrieval. The update addresses package bloating and the need for broader support by introducing a more consistent interface, better typing, and improved documentation. Key new features include support for multimodal evaluation (text, images, and audio), unified retrieval and reranking tasks, and an easier evaluation process using the new `mteb.evaluate` function and `ResultCache` for managing results. The article also provides detailed instructions for upgrading from MTEB v1, including how to convert old models and datasets to the new v2 format. Source: https://huggingface.co/blog/isaacchung/mteb-v2

The provided text is an academic paper titled "Active Use of Latent Constituency Representation in both Humans and Large Language Models," which explores how sentences are internally represented in both the human brain and large language models (LLMs) like ChatGPT. The authors introduce a novel one-shot learning word deletion task where participants infer a deletion rule from a single example; they found that both humans and LLMs tend to delete a complete linguistic constituent rather than a nonconstituent word string, suggesting that latent, hierarchical linguistic structures emerge in both. Furthermore, the study demonstrates that the deletion behavior can be used to reconstruct a constituency tree representation that is structurally consistent with linguistically defined trees. The research also investigates how language-dependent rules are inferred and finds that native speakers primarily rely on syntactic structure over semantic plausibility in this task. Source: https://arxiv.org/pdf/2405.18241

The October 7, 2025 technical release by Liquid AI introducing their new model, LFM2-8B-A1B, an on-device Mixture-of-Experts (MoE) designed for efficiency on consumer hardware. This model boasts 8.3 billion total parameters but only uses 1.5 billion active parameters per token, allowing it to achieve larger model quality with significantly reduced compute requirements. The document highlights the model's superior quality and speed compared to similar-sized dense models, detailing its architecture which is optimized for low-latency and energy consumption on devices like phones and laptops. Furthermore, the text presents extensive evaluation benchmarks across knowledge, instruction following, math, and coding tasks, demonstrating strong performance and outlining the customized inference stacks developed for both CPU and GPU to maximize the model’s efficiency. Source: https://www.liquid.ai/blog/lfm2-8b-a1b-an-efficient-on-device-mixture-of-experts

This October 23, 2025 Xidian University academic survey systematically reviews the transformative impact of Large Language Models (LLMs) on the three core stages of Knowledge Graph (KG) construction: ontology engineering, knowledge extraction, and knowledge fusion. The text explains that LLMs are shifting the paradigm from rigid, rule-based systems to unified, adaptive, and generative frameworks. The paper is structured to first revisit traditional KG methodologies before examining emerging LLM-driven approaches, which are categorized into schema-based (emphasizing structure) and schema-free (emphasizing flexibility) paradigms across all stages. The authors outline how LLMs function as either ontology assistants (top-down) or as consumers of KGs for grounding and memory (bottom-up), culminating in a discussion of future directions such as KG-based reasoning and dynamic knowledge memory for agentic systems. Ultimately, the work aims to clarify the evolving relationship between symbolic knowledge engineering and neural semantic understanding. Source: https://arxiv.org/pdf/2510.20345

This 2025 CMU paper introduces LithOS, a novel operating system designed to improve the efficiency and utilization of Graphics Processing Units (GPUs) for machine learning (ML) workloads in data centers. The authors argue that current GPU management solutions, such as NVIDIA's MPS and MIG, are too coarse-grained, leading to low utilization and high latency in multi-tenant environments. LithOS proposes a transparent, OS-level approach featuring a TPC Scheduler for fine-grained resource control, a Kernel Atomizer that breaks up monolithic kernels to reduce head-of-line blocking, and mechanisms for hardware right-sizing and transparent power management (DVFS). Evaluation results demonstrate that LithOS significantly reduces tail latencies (up to 13× compared to MPS) and improves aggregate throughput in both inference-only and hybrid inference/training scenarios while achieving substantial capacity and energy savings. Overall, the work establishes a foundation for developing true operating systems for GPUs to address the growing efficiency crisis in ML infrastructure. Source: https://www.cs.cmu.edu/~dskarlat/publications/lithos_sosp25.pdf

The September 25, 2025 collaboration between Sea AI Lab, SUTD, NUS, NTU and University of Waterloo paper proposes an alternative to traditional Reinforcement Learning (RL) for Large Language Models (LLMs) by introducing the Feedback-Conditional Policy (FCP), which learns directly from rich verbal feedback instead of compressing it into scalar rewards. The authors argue that scalarization leads to information loss, ambiguity, and imbalanced reward scales, hindering effective learning from natural language critiques. FCP reframes learning as a conditional generation problem, approximating the feedback-conditional posterior through maximum likelihood training on offline data and then using an online bootstrapping stage conditioned on positive feedback to refine the policy. This approach, which draws inspiration from text-to-image generation's ability to combine mixed captions (as shown in the accompanying image), allows LLMs to leverage their inherent linguistic priors for better control and performance, matching or surpassing scalar-based RL methods on reasoning tasks. Source: https://arxiv.org/pdf/2509.22638

The October 3, 2025 paper by Tencent introduces a reinforcement learning technique called Low-probability Regularization (Lp-Reg) designed to overcome the exploration collapse bottleneck in Reinforcement Learning with Verifiable Rewards (RLVR) for large language models. The authors identify that performance plateaus because training systematically eliminates crucial, low-probability tokens, termed reasoning sparks, which are necessary for diverse reasoning paths. Previous methods relying on overall policy entropy fail because they indiscriminately amplify both these valuable sparks and irrelevant noise tokens. Lp-Reg addresses this by constructing a less-noisy proxy distribution that filters out irrelevant tokens and regularizes the policy to preserve the valuable low-probability sparks, leading to stable on-policy training and achieving state-of-the-art accuracy on mathematical reasoning benchmarks. Source: https://arxiv.org/pdf/2510.03222