NRGlassNet : Glass surface detection from visible and near-infrared image pairs

Glass surfaces are ubiquitous in human life environment, such as glass windows, glass doors, glass guardrails and glass walls. Most glass surfaces are transparent without intrinsic texture and color characteristics. Such characteristics pose significant challenges for artificial intelligence systems to identify glass surfaces. We observed that reflections on glass surfaces of near-infrared (NIR) images are always significantly suppressed compared with that of regular RGB images captured from the same scene. Thus, we propose an effective glass surface detection network, called NRGlassNet, which takes NIR-RGB image pair captured from the same scene as input. Our NRGlassNet employ a dual-branch structure consisting of powerful Swin-Transformer blocks to extract features from the NIR image and the RGB image separately. We also propose a novel Multi-modal Context Contrast (MCC) module to modulate the differences of reflection intensities in the NIR image and the RGB image for identifying glass surfaces. In addition, for learning our proposed network, we propose a new dataset, called RNGD, which consists of 1378 NIR-RGB image pairs captured from real-world scenes as well as their ground-truth glass surface annotations. Quantitative and qualitative evaluations demonstrate the effectiveness and superiority of our proposed method. Our code and dataset will be available at: https://github.com/YT3DVision/NRGlassNet. © 2024 Elsevier B.V.