High-resolution facial expression image restoration via adaptive total variation regularization for classroom learning environment

Infrared videos play an important role in recording the learning process. Facial expression images are an important part of infrared videos. High-resolution facial images can reflect the emotion of students or teachers in the classroom. However, recorded infrared videos inevitably have random noise and image blur, which influence facial expression recognition and head pose estimation. In this study, we introduce a blind image restoration method with wavelet transform and total variation regularization. The difference between the low-resolution facial expression image and high-resolution one is revealed by the wavelet transform and total variation regularization. The distribution of the wavelet transform coefficient of high-resolution images is sparser than the coefficient distribution of original low-resolution images. The major novelty of this work is that the sparsity of coefficient distribution is described by the wavelet transform and total variation regularization. Furthermore, the proposed method is conducted on real facial images to verify the effectiveness of priori knowledge. Numerical experiments demonstrate that the proposed method can recover high-resolution facial image and facilitate the application on facial expression recognition. © 2022 Elsevier B.V. All rights reserved.