Retrieval Augmentation

Desmond Elliott
Associate Professor
Rita Ramos
Rita Ramos
(co-advised with Bruno Martins)
Wenyan Li
(co-advised with Anders Søgaard)

Retrieval augmentation has proven a useful approach to improving the quality and relevance of generated texts using in-context examples, as opposed to relying on what can be learned and store in model weights. We have worked on applying retrieval augmentation in a multimodal setting, with multimodal encoders, independent pretrained models, or in a multilingual setting without any supervised training.

Related Publications

2024
Understanding Retrieval Robustness for Retrieval-augmented Image Captioning.
Wenyan Li, Jiaang Li, Rita Ramos, Raphael Tang, and Desmond Elliott.
Proceedings of ACL.
2024
PAELLA: Parameter-Efficient Lightweight Language-Agnostic Captioning Model.
Rita Ramos, Emanuele Bugliarello, Bruno Martins, and Desmond Elliott.
Findings of NAACL.
2023
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting.
Rita Ramos, Bruno Martins, and Desmond Elliott.
Findings of ACL.
2023
SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation.
Rita Ramos, Bruno Martins, Desmond Elliott, and Yova Kementchedjhieva.
Proceedings of CVPR.
2023
Retrieval-augmented Image Captioning.
Rita Ramos, Desmond Elliott, and Bruno Martins.
Proceedings of EACL.