AI Post Transformers

This April 2025 paper introduces Self-Principled Critique Tuning (SPCT), a novel method designed to enhance the inference-time scalability of Generative Reward Models (GRMs) for various domains. It details how SPCT, through a combination of rejective fine-tuning and rule-based online reinforcement learning, facilitates the adaptive generation of principles and critiques, thereby improving the quality and inference-time scalability of GRMs. The paper compares different reward generation paradigms (scalar, semi-scalar, generative) and scoring patterns (pointwise, pairwise), demonstrating that the proposed DeepSeek-GRM models, particularly when guided by a meta Reward Model, consistently outperform existing methods across multiple benchmarks without significant domain biases. The research highlights the potential for GRMs to serve as a versatile interface for generalist reward systems, advancing Large Language Model (LLM) post-training and inference, while also acknowledging ethical considerations regarding bias and the importance of human-in-the-loop frameworks. Source: https://arxiv.org/pdf/2504.02495

This August 2025 paper introduces the Hierarchical Reasoning Model (HRM), a novel AI architecture inspired by the human brain's hierarchical and multi-timescale processing. This model aims to overcome the limitations of current large language models (LLMs) and Chain-of-Thought (CoT) techniques in complex reasoning tasks, which often suffer from computational inefficiencies and extensive data requirements. HRM utilizes two interdependent recurrent modules: a high-level module for abstract planning and a low-level module for detailed computations, enabling it to achieve significant computational depth. Notably, HRM demonstrates exceptional performance on challenging reasoning benchmarks like Sudoku and maze navigation with minimal training data (around 1000 samples) and without pre-training or CoT data. The research further explores the brain-like hierarchical dimensionality organization within HRM, where the high-level module operates in a higher-dimensional space, mirroring principles observed in the mammalian cortex. Source: https://arxiv.org/pdf/2506.21734

This January 2025 paper introduces Janus-Pro, an enhanced artificial intelligence model for multimodal understanding and generation. It builds upon its predecessor, Janus, through optimized training strategies, expanded data, and increased model size. The authors demonstrate that Janus-Pro achieves significant improvements in both multimodal understanding benchmarks and text-to-image generation capabilities, producing more stable and aesthetically pleasing outputs. This work highlights the benefits of decoupling visual encoding for understanding and generation tasks within a unified autoregressive transformer architecture. Source: https://arxiv.org/pdf/2501.17811

This February 2025 paper introduces Native Sparse Attention (NSA), a novel approach to address the computational demands of long-context modeling in large language models. NSA combines algorithmic innovations like a dynamic hierarchical sparse strategy with hardware-aligned optimizations to significantly improve efficiency. The paper highlights NSA's ability to maintain or even surpass the performance of traditional "Full Attention" models across various benchmarks, including general language, long-context tasks, and instruction-based reasoning, while achieving substantial speedups in decoding, forward, and backward propagation. It critically analyzes the shortcomings of existing sparse attention methods, particularly their failure to achieve practical speedups and support end-to-end training, thus motivating NSA's natively trainable and hardware-efficient design. NSA's architecture incorporates token compression, blockwise token selection, and a sliding window mechanism, underpinned by a specialized kernel designed for optimal GPU utilization. Source: https://arxiv.org/pdf/2502.11089

This June 2025 paper introduces Non-Penetrative Tensor Partitioning (NPTP), a novel method designed to improve the speed of collaborative inference for Deep Neural Networks (DNNs) on Internet of Things (IoT) devices. It addresses the common challenge of limited resources and strict latency requirements by minimizing the communication overhead that typically arises when large images are divided and processed across multiple devices. Unlike existing methods that utilize penetrative partitioning, which leads to substantial data sharing between devices, NPTP employs a non-penetrative approach and a Multilevel Partitioning Algorithm (MPA) to reduce this inter-device communication. Experimental results demonstrate that NPTP significantly outperforms state-of-the-art collaborative inference algorithms like CoEdge, achieving notable inference speedups, particularly for larger DNN models and image sizes, while maintaining device memory efficiency. The paper details the computational and communication overhead formulations, along with the algorithm design for optimal tensor partitioning. Source: https://arxiv.org/pdf/2501.04489

This March 2023 paper introduces PETALS, a novel system designed to facilitate the collaborative inference and fine-tuning of large language models (LLMs) by pooling resources from multiple participants. It addresses the significant computational and memory demands of LLMs, which typically restrict access for many researchers. PETALS proposes an alternative to traditional methods like slow RAM offloading or inflexible inference APIs by allowing distributed processing across a network of consumer GPUs, enhancing speed and flexibility. The system incorporates optimizations like 8-bit quantization and dynamic load balancing to improve performance and reliability. Ultimately, PETALS aims to democratize access to powerful LLMs, enabling broader research and application development that was previously cost-prohibitive. Source: https://arxiv.org/pdf/2209.01188

This August 2025 paper introduces UQ, a novel evaluation framework designed to challenge large language models (LLMs) with complex, unsolved questions sourced from platforms like Stack Exchange, where no definitive ground truth answers currently exist. The framework consists of three main components: UQ-Dataset, a collection of 500 hand-filtered, difficult, and unsolved questions; UQ-Validators, a set of LLM-based validation strategies that assess candidate solutions by leveraging the observation that models are often better at verifying answers than generating them; and UQ-Platform, which facilitates community engagement and human verification. The paper highlights the generator-validator gap, demonstrating that LLMs show improved performance in validating answers as their capabilities increase, and emphasizes that UQ aims to accelerate research in domains lacking clear ground-truth verification. Ultimately, UQ offers a new paradigm for evaluating advanced AI by focusing on realistic, open-ended problems that push the boundaries of current model capabilities. Source: https://arxiv.org/pdf/2508.17580

This July 2024 paper introduces Activation-aware Weight Quantization (AWQ), a novel method for compressing Large Language Models (LLMs) by quantizing weights to low-bit integers for efficient deployment on edge devices. It highlights that AWQ identifies and protects crucial "salient" weights by observing activation distributions, which significantly reduces quantization error without requiring computationally intensive training or overfitting to specific datasets. Complementing AWQ, the paper also presents TinyChat, an inference framework specifically designed to optimize and accelerate these 4-bit quantized LLMs on various hardware, including mobile GPUs and even resource-constrained devices like the Raspberry Pi, achieving substantial speedups compared to traditional implementations. The combination of AWQ and TinyChat aims to make powerful LLMs accessible for on-device applications, addressing challenges like memory limitations and power consumption. Source: https://arxiv.org/pdf/2306.00978

This June 2023 paper introduces FlexGen, a novel high-throughput generation engine designed to overcome the substantial computational and memory demands of large language model (LLM) inference on limited hardware, specifically a single commodity GPU. It details FlexGen's ability to aggregate memory and computation across the GPU, CPU, and disk, employing an optimized scheduling approach and a linear programming-based policy search to store and access tensors efficiently. Furthermore, FlexGen incorporates 4-bit compression for model weights and attention caches, which significantly reduces memory footprint with minimal accuracy loss. The research demonstrates FlexGen's superior performance, achieving substantially higher throughput compared to existing offloading systems, even enabling the operation of models as large as OPT-175B on a single 16GB GPU. Source: https://arxiv.org/pdf/2303.06865

This September 2018 paper introduces GraphSAGE, a novel inductive framework designed to generate node embeddings for large, evolving graphs, addressing limitations of prior transductive methods that struggle with unseen data. Instead of learning a specific embedding for each node, GraphSAGE learns a function that generates these embeddings by sampling and aggregating features from a node's local neighborhood. The authors evaluate various aggregator architectures, including mean, LSTM, and pooling functions, demonstrating that GraphSAGE significantly outperforms strong baselines on node classification tasks across diverse datasets, such as citation networks, Reddit posts, and protein-protein interaction graphs. The research also highlights GraphSAGE's computational efficiency and provides a theoretical analysis of its capability to learn local graph structural information, like clustering coefficients. Source: https://arxiv.org/pdf/1706.02216