This August 2025 paper examines the evolving landscape of Federated Large Language Models (FedLLM), focusing on how large language models are post-trained while preserving user data privacy. The authors introduce a novel taxonomy that categorizes FedLLM approaches based on model accessibility (white-box, gray-box, and black-box) and parameter efficiency. It highlights various techniques within these categories, such as adapter-based tuning and prompt tuning, which reduce computational and communication overhead. The paper also discusses the growing importance of inference-only black-box settings for future FedLLM development and identifies open challenges like federated value alignment and enhanced security in constrained environments.

Source:

https://arxiv.org/html/2508.16261v1