Semi-Supervised Landmark-Guided Restoration of Atmospheric Turbulent Images

Image degradation due to atmospheric turbulence (AT), which is common while capturing images at long ranges, adversely affects the performance of tasks such as face alignment and face recognition. To the best of our knowledge, there does not exist any dataset consisting of turbulence-degraded face images along with their annotated landmarks and ground-truth clean images, making supervised training challenging. In this paper, we present a semisupervised method for jointly extracting facial landmarks and restoring the degraded images by exploiting the semantic information from the landmarks. The proposed approach learns to generate AT images by combining the content from a clean image and turbulence information from AT images in an unpaired manner. Next, we use heatmaps from the landmark localization network as a prior to the image restoration module. Subsequently, we impose heatmap consistency loss and heatmap confidence loss to regularize the restored images. Extensive experiments demonstrate the effectiveness of the proposed network, which achieves an NME of 2.797 on the task of landmark localization for strong turbulent images and yields improved restoration results compared to state-of-the-art methods.

© 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.