AI Post Transformers

FAST26: CacheSlide: Unlocking Cross Position-Aware KV Cache Reuse for Accelerating LLM Serving


Listen Later

This FAST26 February 24, 2026 paper introduces CacheSlide, an innovative system designed to accelerate Large Language Model (LLM) serving by improving KV cache reuse. The researchers identify that complex agent-based workflows often cause positional misalignment, where cached data becomes unusable when text segments shift in absolute position. To resolve this, they propose Relative-Position-Dependent Caching (RPDC), which focuses on preserving the order of segments rather than their exact locations. The architecture utilizes Chunked Contextual Position Encoding (CCPE) to minimize drift and Weighted Correction Attention to efficiently restore cross-attention between static and dynamic segments. Additionally, the SLIDE manager optimizes system performance by decoupling data loading from computation and implementing dirty-aware eviction to reduce storage bottlenecks. Experimental results demonstrate that this approach significantly reduces latency and increases throughput across various agentic benchmarks with minimal impact on accuracy. February 24, 2026 CacheSlide: Unlocking Cross Position-Aware KV Cache Reuse for Accelerating LLM Serving Yang Liu and Yunfei Gu, Shanghai Jiao Tong University; Liqiang Zhang, Jinan Inspur Data Technology Co., Ltd; Chentao Wu, Guangtao Xue, Jie Li, and Minyi Guo, Shanghai Jiao Tong University; Junhao Hu, Peking University; Jie Meng, Huawei Cloud https://www.usenix.org/system/files/fast26-liu-yang.pdf
...more
View all episodesView all episodes
Download on the App Store

AI Post TransformersBy mcgrof