Library

Image captioning in different languages

This short position paper provides a manually curated list of non-English image captioning datasets (as of May 2024). Through this list, we can observe the dearth of datasets in different languages: only 23 different languages are represented. With the addition of the Crossmodal-3600 dataset (Thapliyal et al., 2022, 36 languages) this number increases somewhat, but still this number is small compared to the +/-500 institutional languages that are out there. This paper closes with some open questions for the field of Vision&Language.

cs.CLcs.CVLinguisticsComputer Science
View graphSource

Paper

References (63)

01Tiny TR-CAP: A novel small-scale benchmark dataset for general-purpose image captioning tasksAbbas Memiş, S. Yıldız2025 · Engineering Science and Technology, an International Journal · 4 citations
02JEEM: Vision-Language Understanding in Four Arabic DialectsKarima Kadaoui, Hanin Atwany, Hamdan Hamid Al-Ali et al.2025 · Conference of the European Chapter of the Association for Computational Linguistics · 12 citations In Library
03Framed Multi30K: A Frame-Based Multimodal-Multilingual DatasetMarcelo Viridiano, Arthur Lorenzi, Tiago Timponi Torrent et al.2024 · International Conference on Language Resources and Evaluation · 5 citations
04Indonesian Language Image Captioning Using Encoder-Decoder With Attention ApproachJason Jeremy Wijadi, Bryn Ghiffar, N. N. Qomariyah2024 · 2024 2nd International Conference on Software Engineering and Information Technology (ICoSEIT) · 3 citations
05Violet: A Vision-Language Model for Arabic Image Captioning with Gemini DecoderAbdelrahman Mohamed, Fakhraddin Alwajih, El Moatez Billah Nagoudi et al.2023 · ARABICNLP · 23 citations In Library
06Automatic Turkish Image Captioning: The Impact of Deep Machine TranslationS. Yıldız, Abbas Memiş, Songül Çarlı2023 · 2023 8th International Conference on Computer Science and Engineering (UBMK) · 5 citations
07Thangka Image Captioning Based on Semantic Concept Prompt and Multimodal Feature OptimizationWenjin Hu, Lang Qiao, Wendong Kang et al.2023 · Journal of Imaging · 10 citations
08UIT-OpenViIC: A Novel Benchmark for Evaluating Image Captioning in VietnameseD. C. Bui, Nghia Hieu Nguyen, Khang Nguyen2023 · arXiv.org · 4 citations In Library
09Transformation of Visual Information into Bangla Textual RepresentationNafisa Nawer, Md. Shakiful Islam Khan, Md. Mustakin Alam et al.2023 · Computing and Communication Workshop and Conference · 3 citations
10A complete human verified Turkish caption dataset for MS COCO and performance evaluation with well-known image caption models trained against itSina Berk Golech, Saltuk Buğra Karacan, Elena Battini Sönmez et al.2022 · 2022 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME) · 5 citations
11ThaiTC:Thai Transformer-based Image CaptioningTeetouch Jaknamon, S. Marukatat2022 · 2022 17th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP) · 13 citations
12English to Tamil Multi-Modal Image Captioning TranslationVishnu Kumar, Lalithamani V H2022 · International Workshop on Artificial Intelligence and Cognition · 6 citations

Scroll for more · 38 remaining

Similar papers

Semantic and Expressive Variation in Image Captions Across LanguagesAndre Ye, Sebastin Santy, Jena D. Hwang et al.2025 In Library
Semantic and Expressive Variation in Image Captions Across LanguagesAndre Ye, Sebastin Santy, Jena D. Hwang et al.2023 · Computer Vision and Pattern Recognition · 6 citations In Library
Image Caption Datasets: A Review and a Lithuanian Translation-Based CorpusTautvydas Kvietkauskas, Dmitrij Kamyšanskis, Ūla Šveiterytė2026 In Library
Crossmodal-3600: A Massively Multilingual Multimodal Evaluation DatasetAshish V. Thapliyal, Jordi Pont Tuset, Xi Chen et al.2022 In Library
Crossmodal-3600: A Massively Multilingual Multimodal Evaluation DatasetAshish V. Thapliyal, Jordi Pont-Tuset, Xi Chen et al.2022 · arXiv (Cornell University) · 124 citations In Library
A Survey on Image Caption Generation in Various LanguagesHaneen serag Ibrahim, Narjis Mezaal Shati2024 · Iraqi Journal of Science In Library
No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 LanguagesYoussef Mohamed, Runjia Li, I. S. Ahmad et al.2024 · Conference on Empirical Methods in Natural Language Processing · 21 citations In Library
Language-based machine perception: linguistic perspectives on the compilation of captioning datasetsLaura Hekanaho, Maija Hirvonen, Tuomas Virtanen2024 · Digital Scholarship in the Humanities In Library
MIRCaps: A Large-Scale Mixed-Domain Dataset with Image-Level and Region-Level Captions for Fine-Grained Vision-Language LearningArlindo Luciano Tulumba Roberto, Hyungjoon Kim2026 · 0 citations In Library
Multilingual Diversity Improves Vision-Language RepresentationsThao Nguyen, Matthew Wallingford, Sebastin Santy et al.2024 · Neural Information Processing Systems · 18 citations In Library
© 2026 NYSGPT2525 LLC