Structure-Informed Shadow Removal Networks

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Qing Guo
  • Lan Fu
  • Wei Feng
  • Ivor W. Tsang

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)5823-5836
Journal / PublicationIEEE Transactions on Image Processing
Volume32
Online published17 Oct 2023
Publication statusPublished - 2023

Abstract

Existing deep learning-based shadow removal methods still produce images with shadow remnants. These shadow remnants typically exist in homogeneous regions with low-intensity values, making them untraceable in the existing image-to-image mapping paradigm. We observe that shadows mainly degrade images at the image-structure level (in which humans perceive object shapes and continuous colors). Hence, in this paper, we propose to remove shadows at the image structure level. Based on this idea, we propose a novel structure-informed shadow removal network (StructNet) to leverage the image-structure information to address the shadow remnant problem. Specifically, StructNet first reconstructs the structure information of the input image without shadows and then uses the restored shadow-free structure prior to guiding the image-level shadow removal. StructNet contains two main novel modules: (1) a mask-guided shadow-free extraction (MSFE) module to extract image structural features in a non-shadow-to-shadow directional manner, and (2) a multi-scale feature & residual aggregation (MFRA) module to leverage the shadow-free structure information to regularize feature consistency. In addition, we also propose to extend StructNet to exploit multi-level structure information (MStructNet), to further boost the shadow removal performance with minimum computational overheads. Extensive experiments on three shadow removal benchmarks demonstrate that our method outperforms existing shadow removal methods, and our StructNet can be integrated with existing methods to improve them further. © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission

Research Area(s)

  • Data mining, Feature extraction, Image color analysis, Image restoration, Lighting, Periodic structures, Single-image shadow removal Image structure Structure-level shadow removal, Task analysis

Citation Format(s)

Structure-Informed Shadow Removal Networks. / Liu, Yuhao; Guo, Qing; Fu, Lan et al.
In: IEEE Transactions on Image Processing, Vol. 32, 2023, p. 5823-5836.

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review