This May 2025 paper introduces a resource-aware algorithm designed to optimize the performance of Large Language Models (LLMs) for low-latency inference on edge computing devices. The core innovation lies in its fine-grained partitioning of the Transformer architecture, specifically at the attention head-level, rather than coarser layer-level divisions. This approach allows for dynamic reassignment and migration of these individual attention heads and their associated Key/Value (K/V) caches across heterogeneous edge devices. By managing the expanding memory footprint of K/V caches and exploiting parallel execution of attention heads, the proposed method significantly reduces inference latency and memory usage compared to existing static or layer-based partitioning strategies.

Source:

https://arxiv.org/pdf/2505.02533

This August 2025 paper introduces UQ, a novel evaluation framework designed to challenge large language models (LLMs) with complex, unsolved questions sourced from platforms like Stack Exchange, where no definitive ground truth answers currently exist. The framework consists of three main components: UQ-Dataset, a collection of 500 hand-filtered, difficult, and unsolved questions; UQ-Validators, a set of LLM-based validation strategies that assess candidate solutions by leveraging the observation that models are often better at verifying answers than generating them; and UQ-Platform, which facilitates community engagement and human verification. The paper highlights the generator-validator gap, demonstrating that LLMs show improved performance in validating answers as their capabilities increase, and emphasizes that UQ aims to accelerate research in domains lacking clear ground-truth verification. Ultimately, UQ offers a new paradigm for evaluating advanced AI by focusing on realistic, open-ended problems that push the boundaries of current model capabilities.

Source:

https://arxiv.org/pdf/2508.17580

This March 2023 paper introduces PETALS, a novel system designed to facilitate the collaborative inference and fine-tuning of large language models (LLMs) by pooling resources from multiple participants. It addresses the significant computational and memory demands of LLMs, which typically restrict access for many researchers. PETALS proposes an alternative to traditional methods like slow RAM offloading or inflexible inference APIs by allowing distributed processing across a network of consumer GPUs, enhancing speed and flexibility. The system incorporates optimizations like 8-bit quantization and dynamic load balancing to improve performance and reliability. Ultimately, PETALS aims to democratize access to powerful LLMs, enabling broader research and application development that was previously cost-prohibitive.

Source:

https://arxiv.org/pdf/2209.01188

This July 2024 paper introduces Activation-aware Weight Quantization (AWQ), a novel method for compressing Large Language Models (LLMs) by quantizing weights to low-bit integers for efficient deployment on edge devices. It highlights that AWQ identifies and protects crucial "salient" weights by observing activation distributions, which significantly reduces quantization error without requiring computationally intensive training or overfitting to specific datasets. Complementing AWQ, the paper also presents TinyChat, an inference framework specifically designed to optimize and accelerate these 4-bit quantized LLMs on various hardware, including mobile GPUs and even resource-constrained devices like the Raspberry Pi, achieving substantial speedups compared to traditional implementations. The combination of AWQ and TinyChat aims to make powerful LLMs accessible for on-device applications, addressing challenges like memory limitations and power consumption.

Source:

https://arxiv.org/pdf/2306.00978

This January 2025 paper introduces HybridServe, an LLM inference system designed to enhance throughput and cost-effectiveness for large language models by optimizing memory usage and host-GPU communication. It tackles the challenges of host memory offloading, where model parameters and KV cache are stored on slower host memory to reduce costs but can lead to GPU underutilization due to limited transfer bandwidth. HybridServe proposes a novel activation checkpointing technique with a KV-Activation hybrid caching scheme that stores intermediate activations, allowing for faster recomputation of the KV cache while model parameters are transferred. This system dynamically balances communication overhead and recomputation time to maximize throughput, demonstrating significant improvements over existing state-of-the-art methods like FlexGen.

Source:

https://arxiv.org/pdf/2501.01792

This June 2023 paper introduces FlexGen, a novel high-throughput generation engine designed to overcome the substantial computational and memory demands of large language model (LLM) inference on limited hardware, specifically a single commodity GPU. It details FlexGen's ability to aggregate memory and computation across the GPU, CPU, and disk, employing an optimized scheduling approach and a linear programming-based policy search to store and access tensors efficiently. Furthermore, FlexGen incorporates 4-bit compression for model weights and attention caches, which significantly reduces memory footprint with minimal accuracy loss. The research demonstrates FlexGen's superior performance, achieving substantially higher throughput compared to existing offloading systems, even enabling the operation of models as large as OPT-175B on a single 16GB GPU.

Source:

https://arxiv.org/pdf/2303.06865

This September 2018 paper introduces GraphSAGE, a novel inductive framework designed to generate node embeddings for large, evolving graphs, addressing limitations of prior transductive methods that struggle with unseen data. Instead of learning a specific embedding for each node, GraphSAGE learns a function that generates these embeddings by sampling and aggregating features from a node's local neighborhood. The authors evaluate various aggregator architectures, including mean, LSTM, and pooling functions, demonstrating that GraphSAGE significantly outperforms strong baselines on node classification tasks across diverse datasets, such as citation networks, Reddit posts, and protein-protein interaction graphs. The research also highlights GraphSAGE's computational efficiency and provides a theoretical analysis of its capability to learn local graph structural information, like clustering coefficients.

Source:

https://arxiv.org/pdf/1706.02216

This September 2025 paper presents MetaGraph, a novel methodology for constructing knowledge graphs from scientific literature, specifically applied to Financial Natural Language Processing (NLP) research between 2022 and 2025. The authors utilized Large Language Models (LLMs) to extract key information from 681 papers, including tasks, datasets, models, motivations, and limitations, and organized it into a structured, queryable format. The analysis highlights three phases in Financial NLP's evolution: initial LLM adoption and task/dataset innovation, subsequent critical reflection on LLM limitations, and a current trend toward integrating peripheral techniques into modular systems. The research reveals a shift toward Financial Question Answering (QA), increased use of synthetic data, and a growing emphasis on open-source models and system-level solutions like Retrieval-Augmented Generation (RAG). Ultimately, MetaGraph offers a reusable framework for quantitatively mapping scientific progress and understanding changing priorities in a rapidly evolving field.

Source:

https://www.arxiv.org/pdf/2509.09544

This August 2025 paper explores the critical area of fact-checking and factuality evaluation in Large Language Models (LLMs). It systematically analyzes the challenges of misinformation generation, particularly hallucinations, which are factually incorrect but fluent outputs from LLMs. The paper investigates various mitigation strategies, including fine-tuning, instruction tuning, and Retrieval-Augmented Generation (RAG), which grounds LLM outputs in external knowledge. It further examines evaluation metrics, datasets, and prompting strategies used to assess and enhance the factual accuracy of these models, highlighting the need for more robust, explainable, and domain-specific fact-checking frameworks. The review concludes by identifying open issues and future research agendas to foster more trustworthy and context-aware LLMs.

Source:

https://arxiv.org/pdf/2508.03860

This September 2025 paper explores the concept of long-horizon execution in Large Language Models (LLMs), arguing that marginal gains in single-step accuracy can lead to exponential improvements in the length of tasks LLMs can complete. The authors introduce a novel framework to isolate execution capabilities by providing models with necessary knowledge and plans, revealing that larger models can execute significantly more steps, even when smaller models achieve perfect single-turn accuracy. A key finding is the "self-conditioning effect," where LLMs become more prone to errors when their past mistakes are present in the context, a challenge not fully mitigated by increasing model size. However, the paper concludes that "thinking" models, which employ sequential test-time compute, effectively address this self-conditioning and can execute substantially longer tasks in a single turn.

Source:

https://arxiv.org/pdf/2509.09677