DeepSpeed-FastGen is a system that uses Dynamic SplitFuse to improve the deployment and scaling of large language models, achieving higher throughput and lower latency compared to existing systems. The code is available for community engagement.

https://arxiv.org/abs//2401.08671

YouTube: https://www.youtube.com/@ArxivPapers

TikTok: https://www.tiktok.com/@arxiv_papers

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers