Semantic-Aware Gated Fusion Network For Interactive Colorization

Deep neural networks boost many successful colorization methods, including automatic, interactive, and exemplar-based methods. Among them, interactive methods with global and/or local inputs are probably the most flexible to accurately add colors to a gray image. However, due to the sparseness of input-semantic correspondences, existing methods encounter difficulties in distributing inputs into correct regions. Moreover, they simply add or concatenate the features of different inputs to the network before color reconstruction, which cannot balance the influences of different inputs. To this end, we propose a novel interactive colorization network, which explicitly builds input-semantic correspondences with an attention mechanism and proposes a gated feature fusion module to balance the influences of global and local inputs. We further apply a differentiable histogram loss to impose a smooth impact of the global inputs. Extensive experiments demonstrate that our method can flexibly control the results and outperforms other state-of-the-art interactive methods. © 2023 IEEE.