Sign up to save your podcasts
Or
Share MAGMA - Multimodal Augmentation of Generative Models through Adapter-based Finetuning
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
MAGMA - Multimodal Augmentation of Generative Models through Adapter-based Finetuning
Large-scale pretraining is fast becoming the norm in Vision-Language (VL) modeling. However, prevailing VL approaches are limited by the requirement for labeled data and the use of complex multi-step pretraining objectives. We present MAGMA a simple method for augmenting generative language models with additional modalities using adapter-based finetuning. Building on Frozen [52], we train a series of VL models that autoregressively generate text from arbitrary combinations of visual and textual input.
2021: Constantin Eichenberg, Sid Black, Samuel Weinbach, Letitia Parcalabescu, A. Frank
https://arxiv.org/pdf/2112.05253v1.pdf
View all episodes
By Rob
3.7
33 ratings
Large-scale pretraining is fast becoming the norm in Vision-Language (VL) modeling. However, prevailing VL approaches are limited by the requirement for labeled data and the use of complex multi-step pretraining objectives. We present MAGMA a simple method for augmenting generative language models with additional modalities using adapter-based finetuning. Building on Frozen [52], we train a series of VL models that autoregressively generate text from arbitrary combinations of visual and textual input.
2021: Constantin Eichenberg, Sid Black, Samuel Weinbach, Letitia Parcalabescu, A. Frank
https://arxiv.org/pdf/2112.05253v1.pdf