Going From RGB to RGBD Saliency : A Depth-Guided Transformation Model

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal

9 Scopus Citations
View graph of relations

Author(s)

  • Runmin Cong
  • Jianjun Lei
  • Huazhu Fu
  • Qingming Huang

Detail(s)

Original languageEnglish
Article number8807367
Pages (from-to)3627-3639
Number of pages13
Journal / PublicationIEEE Transactions on Cybernetics
Volume50
Issue number8
Online published20 Aug 2019
Publication statusPublished - Aug 2020

Abstract

Depth information has been demonstrated to be useful for saliency detection. However, the existing methods for RGBD saliency detection mainly focus on designing straightforward and comprehensive models, while ignoring the transferable ability of the existing RGB saliency detection models. In this article, we propose a novel depth-guided transformation model (DTM) going from RGB saliency to RGBD saliency. The proposed model includes three components, that is: 1) multilevel RGBD saliency initialization; 2) depth-guided saliency refinement; and 3) saliency optimization with depth constraints. The explicit depth feature is first utilized in the multilevel RGBD saliency model to initialize the RGBD saliency by combining the global compactness saliency cue and local geodesic saliency cue. The depth-guided saliency refinement is used to further highlight the salient objects and suppress the background regions by introducing the prior depth domain knowledge and prior refined depth shape. Benefiting from the consistency of the entire object in the depth map, we formulate an optimization model to attain more consistent and accurate saliency results via an energy function, which integrates the unary data term, color smooth term, and depth consistency term. Experiments on three public RGBD saliency detection benchmarks demonstrate the effectiveness and performance improvement of the proposed DTM from RGB to RGBD saliency.

Research Area(s)

  • Depth cue, energy function optimization, refined depth shape prior (RDSP), RGBD images, saliency detection, transformation model

Citation Format(s)

Going From RGB to RGBD Saliency : A Depth-Guided Transformation Model. / Cong, Runmin; Lei, Jianjun; Fu, Huazhu; Hou, Junhui; Huang, Qingming; Kwong, Sam.

In: IEEE Transactions on Cybernetics, Vol. 50, No. 8, 8807367, 08.2020, p. 3627-3639.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal