The light field describes the radiance of the light rays permeating the 3D free space as a function of their positions and directions. The light field image can be interpreted as a series of 2D images observed from different viewpoints, which implicitly encodes the depth information of the 3D scene. A high-quality 4D light field image with high resolution on spatial and angular dimensions records rich information of the scene in both appearance and geometry, and thus, enables worldwide applications in the fields of computer graphics and computer vision, such as novel view rendering, post-capture refocusing, scene reconstruction, and virtual/augmented reality. However, the high-quality light field data pose significant challenges to acquisition. Traditional approaches, including camera array and time-sequential capture, are either bulky and expensive or limited to static scenes. The recently commercialized hand-held plenoptic camera can conveniently capture the light field images but suffers from a trade-off between the spatial and angular resolution.

This thesis explores computational approaches to reconstruct the high-quality 4D light field image from low-cost inputs that are sparsely sampled in spatial or angular domains. We focus on three kinds of inputs, i.e., an angularly sparse light field, a spatially low-resolution light field, and a hybrid data containing a high-resolution 2D image and a spatially low-resolution light field image. Moreover, due to the high-dimensional characteristics of the light field images, modeling the intrinsic structure of such data is quite different from handling traditional 2D single images. To address this challenge, we take advantage of the advanced deep learning techniques and carefully design effective modules to implicitly or explicitly model the angular relations for light field reconstruction. The main works are summarized as follows:

(1) We propose a novel learning-based method, which accepts light fields sparsely sampled in angular domain with irregular structure, and produces densely sampled light fields with arbitrary angular resolution accurately and efficiently. We also propose a simple yet effective method for optimizing the angular sampling pattern. Comprehensive experimental evaluations demonstrate the superiority of our method on both real-world and synthetic light field images when compared with state-of-the-art methods. In addition, we illustrate the benefits and advantages of the proposed approach when applied in various light field-based applications, including image-based rendering and depth estimation enhancement.

(2) We propose a novel learning-based light field spatial super-resolution framework, in which each view of a light field image is first individually super-resolved by exploring the complementary information among views with combinatorial geometry embedding. For accurate preservation of the parallax structure among the reconstructed views, a regularization network trained over a structure-aware loss function is subsequently appended to enforce correct parallax relationships over the intermediate estimation. Our proposed approach is evaluated over datasets with a large number of testing images, including both synthetic and real-world scenes. Experimental results demonstrate the advantage of our approach over state-of-the-art methods.

(3) We propose a novel end-to-end learning-based approach to reconstruct high-quality light field images from hybrid lenses, including a high-resolution camera surrounded by multiple low-resolution cameras. Our approach can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Besides, to promote the effectiveness of our method trained with simulated hybrid data on real hybrid data captured by a hybrid light field imaging system, we carefully design the network architecture and the training strategy. Extensive experiments on both real and simulated hybrid data demonstrate the significant superiority of our approach over state-of-the-art ones. To the best of our knowledge, this is the first end-to-end deep learning method for light field reconstruction from a real hybrid input.