The sources detail a novel method called MHA2MLA (Multi-Head Attention to Multi-Head Latent Attention), which efficiently adapts pre-trained large language models (LLMs) to the memory-saving Multi-head Latent Attention (MLA) architecture without requiring full retraining. This framework achieves significant Key-Value (KV) cache compression (up to 96.87% reduction in Llama2-7B) through two main components: partial-Rotary Positional Embedding (RoPE) removal based on attention score contribution and low-rank approximation using Singular Value Decomposition (SVD). Crucially, MHA2MLA requires only a minimal amount of fine-tuning data (0.6% to 1%) and demonstrates strong compatibility with other compression techniques like KV cache quantization, maintaining performance across various commonsense reasoning and long-context tasks.Sources:https://arxiv.org/pdf/2405.04434https://arxiv.org/pdf/2502.07864https://arxiv.org/pdf/2502.14837