OASG-Net : Occlusion Aware and Structure-Guided Network for Face De-Occlusion

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

View graph of relations

Author(s)

  • Yuewei Fu
  • Buyun Liang
  • Zhongyuan Wang
  • Baojin Huang
  • Tao Lu
  • Chao Liang

Related Research Unit(s)

Detail(s)

Original languageEnglish
Journal / PublicationIEEE Transactions on Biometrics, Behavior, and Identity Science
Publication statusOnline published - 9 Oct 2024

Abstract

During the COVID-19 coronavirus epidemic, almost everyone wears a facial mask, which poses a huge challenge for face recognition. Therefore, it is urgent to improve the performance of face de-occlusion of masked faces. However, previous inpainting approaches are limited as they require the knowledge of a given mask, and in the past there was also a lack of real-world masked face datasets suitable for the face de-occlusion task. To tackle above issues, we pioneer a real-world masked face de-occlusion dataset (RMFDD) with accurate mask labels. Further, we propose an occlusion aware and structure-guided network (OASG-Net) for face de-occlusion, consisting of mask prediction subnet, structure prediction subnet, and face de-occlusion subnet. In particular, due to the mask prediction subnet, OASG-Net achieves face de-occlusion without a given external mask. We also use face structure to guide OASG-Net, which makes the recovered face topology more natural and realistic. Besides, we design the mask aware layer to avoid the hard 0-1 mask updating of partial convolution in the face de-occlusion subnet. Extensive results on both face de-occlusion and face recognition tasks demonstrate the superiority of our OASG-Net over the state-of-the-art competitors. Code is available at https://github.com/WHUfreeway/OASG-Net-Occlusion-Aware-and-Structure-Guided-Network-for-Face-De-Occlusion. © 2019 IEEE.

Research Area(s)

  • Face de-occlusion, face recognition, generative adversarial network, masked face dataset, structure-guided

Citation Format(s)