Cross-lingual Adaptation for Recipe Retrieval with Mixup

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

2 Scopus Citations
View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationICMR '22 - Proceedings of the 2022 International Conference on Multimedia Retrieval
PublisherAssociation for Computing Machinery, Inc
Pages258-267
Number of pages10
ISBN (print)9781450392389
Publication statusPublished - 2022

Publication series

NameICMR - Proceedings of the International Conference on Multimedia Retrieval

Conference

Title12th ACM International Conference on Multimedia Retrieval (ICMR 2022)
PlaceUnited States
CityNewark
Period27 - 30 June 2022

Abstract

Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different languages. Moreover, only recipes are available for training in the target domain. A novel recipe mixup method is proposed to learn transferable embedding features between the two domains. Specifically, recipe mixup produces mixed recipes to form an intermediate domain by discretely exchanging the section(s) between source and target recipes. To bridge the domain gap, recipe mixup loss is proposed to enforce the intermediate domain to locate in the shortest geodesic path between source and target domains in the recipe embedding space. By using Recipe 1M dataset as source domain (English) and Vireo-FoodTransfer dataset as target domain (Chinese), empirical experiments verify the effectiveness of recipe mixup for cross-lingual adaptation in the context of image-to-recipe retrieval.

Research Area(s)

  • cross-lingual, domain adaptation, mixup, recipe retrieval

Citation Format(s)

Cross-lingual Adaptation for Recipe Retrieval with Mixup. / Zhu, Bin; Ngo, Chong-Wah; Chen, Jingjing et al.
ICMR '22 - Proceedings of the 2022 International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2022. p. 258-267 (ICMR - Proceedings of the International Conference on Multimedia Retrieval).

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review