Researchers propose a new optimizer, LOMO, that reduces memory usage to enable full parameter fine-tuning of large language models on a single machine.

https://arxiv.org/abs//2306.09782