Cross-lingual Adaptation for Recipe Retrieval with Mixup
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Title of host publication | ICMR '22 - Proceedings of the 2022 International Conference on Multimedia Retrieval |
Publisher | Association for Computing Machinery, Inc |
Pages | 258-267 |
Number of pages | 10 |
ISBN (print) | 9781450392389 |
Publication status | Published - 2022 |
Publication series
Name | ICMR - Proceedings of the International Conference on Multimedia Retrieval |
---|
Conference
Title | 12th ACM International Conference on Multimedia Retrieval (ICMR 2022) |
---|---|
Place | United States |
City | Newark |
Period | 27 - 30 June 2022 |
Link(s)
Abstract
Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies unsupervised domain adaptation for image-to-recipe retrieval, where recipes in source and target domains are in different languages. Moreover, only recipes are available for training in the target domain. A novel recipe mixup method is proposed to learn transferable embedding features between the two domains. Specifically, recipe mixup produces mixed recipes to form an intermediate domain by discretely exchanging the section(s) between source and target recipes. To bridge the domain gap, recipe mixup loss is proposed to enforce the intermediate domain to locate in the shortest geodesic path between source and target domains in the recipe embedding space. By using Recipe 1M dataset as source domain (English) and Vireo-FoodTransfer dataset as target domain (Chinese), empirical experiments verify the effectiveness of recipe mixup for cross-lingual adaptation in the context of image-to-recipe retrieval.
Research Area(s)
- cross-lingual, domain adaptation, mixup, recipe retrieval
Citation Format(s)
ICMR '22 - Proceedings of the 2022 International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 2022. p. 258-267 (ICMR - Proceedings of the International Conference on Multimedia Retrieval).
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review