Advancing High Dynamic Range Imaging: Reconstruction and Quality Assessment

Abstract

High Dynamic Range (HDR) imaging has garnered significant attention in recent years owing to its profound efficiency in preserving intricate details within over- and under-exposed regions, which is a challenge prevalent in Low Dynamic Range (LDR) images. Nevertheless, the direct generation of high-quality HDR content remains a time-consuming and costly process. Additionally, there exists a demand to enhance the quality of classical LDR content by converting it into HDR content. Furthermore, the high contrast and expansive data range inherent in HDR images make the evaluation of HDR content unsuitable using existing LDR Image Quality Assessment (IQA) metrics. In this thesis, we aim to reconstruct a high-quality HDR image from a single LDR image, coupled with a comprehensive subjective and objective evaluation of the HDR content. Our study aspires to present innovative perspectives on both the reconstruction and assessment of HDR content, and is expected to contribute to the provision of stable and exceptional content requisite for subsequent large-scale applications in artificial intelligence. The three sub-tasks of this thesis are succinctly summarized as follows.

In the first part, we aim to reconstruct HDR images from LDR ones. A predominant challenge of this issue lies in the absence of texture and structural details in the under/over-exposed regions. Motivated by the above observation, we propose an efficient and stable HDR reconstruction method, namely Exposure-Induced Network (EIN), for a single LDR image with arbitrary exposure and content. Specifically, two exposure-gated detail recovering branches are designed to infer the texture and structural details in under/over-exposed regions, where the learned confidence maps are progressively generated to provide the confidence level of whether the corresponding regions need further restoration. Simultaneously, the dynamic range expansion branch that interacts with the exposure-gated detail recovering branches is designed to expand the global dynamic range of the image. The features from these three interactional branches are adaptively merged in the feature fusion stage to reconstruct the final HDR image. Extensive experimental results demonstrate that the proposed model consistently improves the visual quality for input LDR images with different exposures.

The second part is dedicated to exploring the challenges associated with the objective evaluation of HDR images. We propose an effective IQA algorithm based on frequency disparity for HDR images, termed the Local-Global Frequency feature-based Model (LGFM). Motivated by the assumption that the Human Visual System (HVS) excels at extracting structural information and partial frequencies during visual perception, the Gabor and the Butterworth filters are applied to the luminance component of the HDR image to extract the local and global frequency features, respectively. Sequentially, the similarity measurement and feature pooling strategy are performed on the frequency features to calculate the predicted quality score. The experiments conducted on four widely used benchmarks demonstrate that the proposed LGFM can provide a higher consistency with subjective perception compared with the state-of-the-art HDR IQA methods.

The third part studies the effectiveness of existing IQA metrics for HDR images, where a novel HDR compression (HDRC) database is introduced to serve as a benchmark for developing full-reference HDR IQA algorithms, particularly in addressing new HDR compression distortions. Specifically, the proposed HDRC database is the first HDR-IQA database to incorporate Versatile Video Coding (VVC) compression distortions, which are closely associated with real-world application scenarios. It is worth noting that the proposed HDRC database is currently the largest HDR-IQA database, including 80 reference images and 400 distorted images. Comprehensive experiments are undertaken to assess the performance concerning existing HDR-IQA databases, evaluating three HDR-specific IQA models and nine IQA models prevalent for LDR content, revealing the challenges the proposed HDRC database brings. The results indicate that the existing IQA models demonstrate noticeable decreases in accuracy when assessing new compression distortions, emphasizing the necessity for the development of innovative HDR-IQA models. Furthermore, the suggested HDRC database holds potential as a valuable resource for HDR-IQA research, fostering a comprehensive exploration of the associated fields.

To sum up, this thesis aims to innovate both the reconstruction and evaluation of HDR content, contributing to the development of stable and high-quality content for large-scale Artificial Intelligence (AI) applications. In this thesis, we conducted an in-depth and systematic study of the advantages of deep neural networks and exposure priors in reconstructing high-quality HDR images. Furthermore, we validated the effectiveness of the contrast-sensitive function in evaluating image quality in the frequency domain. Additionally, we developed an HDR compression (HDRC) database. The introduction of this database addresses a critical gap in current research, providing a comprehensive benchmark for evaluating HDR-IQA algorithms under new compression distortions. By ensuring that HDR images can be accurately assessed and efficiently generated, our research will have a significant impact on both academic research and industry practices, facilitating the broader adoption and utilization of HDR technology.

Date of Award	23 Aug 2024
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Shiqi WANG (Supervisor) & Tak Wu Sam KWONG (External Co-Supervisor)