Retrieval augmentation has proven a useful approach to improving the quality and relevance of generated texts using in-context examples, as opposed to relying on what can be learned and store in model weights. We have worked on applying retrieval augmentation in a multimodal setting, with multimodal encoders, independent pretrained models, or in a multilingual setting without any supervised training.
2024 |
Understanding Retrieval Robustness for Retrieval-augmented Image Captioning.
Proceedings of ACL. |
2024 |
PAELLA: Parameter-Efficient Lightweight Language-Agnostic Captioning Model.
Findings of NAACL. |
2023 |
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting.
Findings of ACL. |
2023 |
SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation.
Proceedings of CVPR. |
2023 |
Retrieval-augmented Image Captioning.
Proceedings of EACL. |