Learning to estimate smooth and accurate semantic correspondence

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)1-11
Journal / PublicationNeurocomputing
Volume436
Online published12 Jan 2021
Publication statusPublished - 14 May 2021

Abstract

We tackle the problem of estimating dense semantic correspondence between two images depicting different instances of the same category. In this paper, we consider semantic context and correspondence information from the neighborhood in order to overcome the drawback of previous works that estimate the correspondence of each pixel or patch independently. To this end, a novel network, called SANet, with a trainable spatial aggregation module is proposed, which is trained in an end-to-end manner and outputs semantic flow. We train this SANet by adopting two complementary loss terms: landmark loss, focusing on keypoints with ground truth, and consistency loss, applicable to all pixels without ground truth. Qualitative and quantitative experimental results demonstrate the improved network achieves a better balance between accuracy and smoothness comparing with the baseline and warps images with better visual quality.

Research Area(s)

  • Neighborhood context, Semantic correspondence, Shift-sum, Spatially adaptive aggregation

Citation Format(s)

Learning to estimate smooth and accurate semantic correspondence. / Xu, Huaiyuan; Chen, Xiaodong; Xi, Jiaqi; Liao, Jing.

In: Neurocomputing, Vol. 436, 14.05.2021, p. 1-11.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review