Summary of two papers on Transformers and Titans by researchers at Google.

Sources :

Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin

https://arxiv.org/pdf/1706.03762

Titans: Learning to Memorize at Test Time Ali Behrouz†, Peilin Zhong†, and Vahab Mirrokni†

https://arxiv.org/pdf/2501.00663

Thes papers reference several other important works in this field. Please refer to the full paper for acomprehensive list.

Disclaimer:

Please note that parts or all this episode was generatedby AI. While the content is intended to be accurate and informative, it is recommended that you consult the original research papers for a comprehensive understanding.