ASIF-Net : Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

30 Scopus Citations
View graph of relations

Author(s)

  • Chongyi Li
  • Runmin Cong
  • Huazhu Fu
  • Guopu Zhu
  • Dingwen Zhang
  • Qingming Huang

Detail(s)

Original languageEnglish
Pages (from-to)88-100
Journal / PublicationIEEE Transactions on Cybernetics
Volume51
Issue number1
Online published13 Feb 2020
Publication statusPublished - Jan 2021

Abstract

Salient object detection from RGB-D images is an important yet challenging vision task, which aims at detecting the most distinctive objects in a scene by combining color information and depth constraints. Unlike prior fusion manners, we propose an attention steered interweave fusion network (ASIF-Net) to detect salient objects, which progressively integrates cross-modal and cross-level complementarity from the RGB image and corresponding depth map via steering of an attention mechanism. Specifically, the complementary features from RGB-D images are jointly extracted and hierarchically fused in a dense and interweaved manner. Such a manner breaks down the barriers of inconsistency existing in the cross-modal data and also sufficiently captures the complementarity. Meanwhile, an attention mechanism is introduced to locate the potential salient regions in an attention-weighted fashion, which advances in highlighting the salient objects and suppressing the cluttered background regions. Instead of focusing only on pixelwise saliency, we also ensure that the detected salient objects have the objectness characteristics (e.g., complete structure and sharp boundary) by incorporating the adversarial learning that provides a global semantic constraint for RGB-D salient object detection. Quantitative and qualitative experiments demonstrate that the proposed method performs favorably against 17 state-of-the-art saliency detectors on four publicly available RGB-D salient object detection datasets. The code and results of our method are available at https://github.com/Li-Chongyi/ASIF-Net.

Research Area(s)

  • Adversarial learning, depth cue, interweave fusion, residual attention, RGB-D images, saliency detection

Citation Format(s)

ASIF-Net : Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection. / Li, Chongyi; Cong, Runmin; Kwong, Sam; Hou, Junhui; Fu, Huazhu; Zhu, Guopu; Zhang, Dingwen; Huang, Qingming.

In: IEEE Transactions on Cybernetics, Vol. 51, No. 1, 01.2021, p. 88-100.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review