Deep Restoration of All-in-focus Images Using Light Field-based Dataset


Student thesis: Doctoral Thesis

View graph of relations


Related Research Unit(s)


Awarding Institution
Award date11 Nov 2021


Image blur is a long-standing problem since the advent of imaging devices. Generally, image blur can be roughly divided into two categories: 1) motion blur, which is easily caused by actions like camera shake, object movement during the capturing process; 2) defocus blur, which is due to the usage of lens and large aperture in the imaging system. Restoring a clear image with sharp content from its corrupted version with such degradations is highly important and beneficial to computer vision and image processing tasks, such as object detection, face detection, image segmentation, image stitching, misfocus correction, despite being a severely ill-posed problem. This dissertation offers a novel perspective to address the restoration problem drawing on the recent advanced deep learning and light field techniques. Specifically, two novel convolutional neural network architectures are proposed to restore the degraded images.

The first one is called All-in-Focus Network (AIFNet) that removes spatially-varying defocus blur from a single defocused image. To remedy the lack of realistic defocused image datasets, we leverage the synthetic aperture and refocusing techniques in light field to generate a large set of realistic defocused and all-in-focus image pairs depicting a variety of natural scenes for network training. AIFNet consists of three modules: defocus map estimation, deblurring and domain adaptation. The effects and performance of various network components are extensively evaluated. We also compare our method with existing solutions using several publicly available datasets. Quantitative and qualitative evaluations demonstrate that AIFNet shows the state-of-the-art performance on spatially-varying defocus blur removal tasks.

Although showing outstanding performance on spatially-varying defocus blur removal, AIFNet cannot handle cases when motion blur and spatially-varying defocus blur coexist in the image. Thus another deep convolutional architecture is built to tackle this complex blur. Most of the existing deep learning approaches only concentrate on either motion or defocus blur but not both, which commonly occurs due to the combined effects of camera motion, aperture and lens setting during capturing, and is extremely difficult to resolve. We address this problem by adopting three modules - a motion removal, defocus map prediction, and image deblurring modules. For the training data, we simulate the complex blur by convolving our proposed defocus blur dataset with motion kernels. Experimental results show that our proposed method outperforms relevant state-of-the-art methods.