Any2AnyMultilingualRetrieval¶

Number of tasks: 3

WITT2IRetrieval¶

Retrieve images based on multilingual descriptions.

Dataset: mteb/wit • License: cc-by-sa-4.0 • Learn more →

Task category	Score	Languages	Domains	Annotations Creators	Sample Creation
text to image (t2i)	ndcg_at_10	ara, bul, dan, ell, eng, ... (11)	Encyclopaedic, Written	derived	found

Citation

@inproceedings{bugliarello2022iglue,
  author = {Bugliarello, Emanuele and Liu, Fangyu and Pfeiffer, Jonas and Reddy, Siva and Elliott, Desmond and Ponti, Edoardo Maria and Vuli{\'c}, Ivan},
  booktitle = {International Conference on Machine Learning},
  organization = {PMLR},
  pages = {2370--2392},
  title = {IGLUE: A benchmark for transfer learning across modalities, tasks, and languages},
  year = {2022},
}

XFlickr30kCoT2IRetrieval¶

Retrieve images based on multilingual descriptions.

Dataset: floschne/xflickrco • License: cc-by-sa-4.0 • Learn more →

Task category	Score	Languages	Domains	Annotations Creators	Sample Creation
text to image (t2i)	ndcg_at_10	deu, eng, ind, jpn, rus, ... (8)	Encyclopaedic, Written	derived	found

Citation

@inproceedings{bugliarello2022iglue,
  author = {Bugliarello, Emanuele and Liu, Fangyu and Pfeiffer, Jonas and Reddy, Siva and Elliott, Desmond and Ponti, Edoardo Maria and Vuli{\'c}, Ivan},
  booktitle = {International Conference on Machine Learning},
  organization = {PMLR},
  pages = {2370--2392},
  title = {IGLUE: A benchmark for transfer learning across modalities, tasks, and languages},
  year = {2022},
}

XM3600T2IRetrieval¶

Retrieve images based on multilingual descriptions.

Dataset: floschne/xm3600 • License: cc-by-sa-4.0 • Learn more →

Task category	Score	Languages	Domains	Annotations Creators	Sample Creation
text to image (t2i)	ndcg_at_10	ara, ben, ces, dan, deu, ... (36)	Encyclopaedic, Written	derived	found

Citation

@inproceedings{thapliyal2022crossmodal,
  author = {Thapliyal, Ashish V and Tuset, Jordi Pont and Chen, Xi and Soricut, Radu},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing},
  pages = {715--729},
  title = {Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset},
  year = {2022},
}