NRGlassNet : Glass surface detection from visible and near-infrared image pairs

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

1 Scopus Citations
View graph of relations

Author(s)

  • Tao Yan
  • Shufan Xu
  • Hao Huang
  • Helong Li
  • Lu Tan
  • Xiaojun Chang

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number111722
Journal / PublicationKnowledge-Based Systems
Volume294
Online published3 Apr 2024
Publication statusPublished - 21 Jun 2024

Abstract

Glass surfaces are ubiquitous in human life environment, such as glass windows, glass doors, glass guardrails and glass walls. Most glass surfaces are transparent without intrinsic texture and color characteristics. Such characteristics pose significant challenges for artificial intelligence systems to identify glass surfaces. We observed that reflections on glass surfaces of near-infrared (NIR) images are always significantly suppressed compared with that of regular RGB images captured from the same scene. Thus, we propose an effective glass surface detection network, called NRGlassNet, which takes NIR-RGB image pair captured from the same scene as input. Our NRGlassNet employ a dual-branch structure consisting of powerful Swin-Transformer blocks to extract features from the NIR image and the RGB image separately. We also propose a novel Multi-modal Context Contrast (MCC) module to modulate the differences of reflection intensities in the NIR image and the RGB image for identifying glass surfaces. In addition, for learning our proposed network, we propose a new dataset, called RNGD, which consists of 1378 NIR-RGB image pairs captured from real-world scenes as well as their ground-truth glass surface annotations. Quantitative and qualitative evaluations demonstrate the effectiveness and superiority of our proposed method. Our code and dataset will be available at: https://github.com/YT3DVision/NRGlassNet. © 2024 Elsevier B.V.

Research Area(s)

  • Deep learning, Glass surface detection, Near-infrared image, RGB-NIR image pair, Visible image

Bibliographic Note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Citation Format(s)

NRGlassNet: Glass surface detection from visible and near-infrared image pairs. / Yan, Tao; Xu, Shufan; Huang, Hao et al.
In: Knowledge-Based Systems, Vol. 294, 111722, 21.06.2024.

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review