SiRA is a sparse mixture of low rank adaption approach that leverages sparse computation to improve the performance of large language models on downstream tasks. It outperforms other approaches in single task and multitask settings.

https://arxiv.org/abs//2311.09179

YouTube: https://www.youtube.com/@ArxivPapers

TikTok: https://www.tiktok.com/@arxiv_papers

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers