We are the Language and Multimodal Processing Group at the Department of Computer Science at the University of Copenhagen. Our recent work includes representation learning for multilingual and multimodal data, resource creation, and language modelling.

News

Older posts…

Research Areas

Pixel Language Models

We are developing language models that can process any written language.

Retrieval Augmentation

Retrieval augmentation of multimodal models for image captioning.

Multilingual Multimodality

Design, understand, and evaluate multimodality in multilingual models

Recent Publications

2023
Text Rendering Strategies for Pixel Language Models.
Jonas F. Lotz, Elizabeth Salesky, Phillip Rust, and Desmond Elliott.
Proceedings of EMNLP.
2023
Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models.
Laura Cabello, Emanuele Bugliarello, Stephanie Brandl, and Desmond Elliott.
Proceedings of EMNLP.
2023
PHD: Pixel-Based Language Modeling of Historical Documents.
Nadav Borenstein, Phillip Rust, Desmond Elliott, and Isabelle Augenstein.
Proceedings of EMNLP.
2023
Visual Prediction Improves Zero-Shot Cross-Modal Machine Translation.
Tosho Hirasawa, Emanuele Bugliarello, Desmond Elliott, and Mamoru Komachi.
Proceedings of WMT.
2023
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting.
Rita Ramos, Bruno Martins, and Desmond Elliott.
Findings of ACL.
2023
SmallCap: Lightweight Image Captioning Prompted With Retrieval Augmentation.
Rita Ramos, Bruno Martins, Desmond Elliott, and Yova Kementchedjhieva.
Proceedings of CVPR.
2023
Language Modelling with Pixels.
Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, Elizabeth Salesky, Miryam Lhoneux, and Desmond Elliott.
Proceedings of ICLR. Notable Top 5% Paper.
2023
Cleaner Categories Improve Object Detection and Visual-Textual Grounding.
Davide Rigoni, Desmond Elliott, and Stella Frank.
Image Analysis.
2023
Retrieval-augmented Image Captioning.
Rita Ramos, Desmond Elliott, and Bruno Martins.
Proceedings of EACL.
2023
MultiFin: A Dataset for Multilingual Financial NLP.
Rasmus Jørgensen, Oliver Brandt, Mareike Hartmann, Xiang Dai, Christian Igel, and Desmond Elliott.
Findings of EACL.

Faculty

Desmond Elliott
Assistant Professor

Graduate Students

Rita Ramos
Rita Ramos
Ph.D Fellow
(co-advised with Bruno Martins)
Wenyan Li
Ph.D Fellow
(co-advised with Anders Søgaard)
Jonas F. Lotz
Jonas F. Lotz
Ph.D Fellow
(co-advised with Christian Igel)

Collaborators

Stella Frank University of Copenhagen
Bruno Martins Instituto Superior Técnico
CoAStaL University of Copenhagen

Alumni

Emanuele Bugliarello Ph.D 2023. Now Research Scientist at Google
Emiel van Miltenburg PhD 2018. Now at Tilburg University.
Xiang Dai Postdoc. Now Research Scientist at CSIRO Data61
Elham Pezhhan Digitaliseringskonsulent at Thisted Kommune
Mitja Nikolaus Now Ph.D Fellow at Aix-Marseille University