Deep Spatial-Angular Regularization for Compressive Light Field Reconstruction over Coded Apertures

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review

8 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2020, 16th European Conference, Proceedings
EditorsAndrea Vedaldi, Horst Bischof, Thomas Brox
PublisherSpringer
Pages278-294
VolumePart II
ISBN (Electronic)9783030585365
ISBN (Print)9783030585358
Publication statusPublished - Aug 2020

Publication series

NameLecture Notes in Computer Science
Volume12347
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Title16th European Conference on Computer Vision (ECCV 2020)
LocationOnline
PlaceUnited Kingdom
CityGlasgow
Period23 - 28 August 2020

Abstract

Coded aperture is a promising approach for capturing the 4-D light field (LF), in which the 4-D data are compressively modulated into 2-D coded measurements that are further decoded by reconstruction algorithms. The bottleneck lies in the reconstruction algorithms, resulting in rather limited reconstruction quality. To tackle this challenge, we propose a novel learning-based framework for the reconstruction of high-quality LFs from acquisitions via learned coded apertures. The proposed method incorporates the measurement observation into the deep learning framework elegantly to avoid relying entirely on data-driven priors for LF reconstruction. Specifically, we first formulate the compressive LF reconstruction as an inverse problem with an implicit regularization term. Then, we construct the regularization term with an efficient deep spatial-angular convolutional sub-network to comprehensively explore the signal distribution free from the limited representation ability and inefficiency of deterministic mathematical modeling. Experimental results show that the reconstructed LFs not only achieve much higher PSNR/SSIM but also preserve the LF parallax structure better, compared with state-of-the-art methods on both real and synthetic LF benchmarks. In addition, experiments show that our method is efficient and robust to noise, which is an essential advantage for a real camera system. The code is publicly available at https://github.com/angmt2008/LFCA.

Research Area(s)

  • Light field, Coded aperture, Deep learning, Regularization, Observation model

Citation Format(s)

Deep Spatial-Angular Regularization for Compressive Light Field Reconstruction over Coded Apertures. / Guo, Mantang; Hou, Junhui; Jin, Jing; Chen, Jie; Chau, Lap-Pui.

Computer Vision – ECCV 2020, 16th European Conference, Proceedings. ed. / Andrea Vedaldi; Horst Bischof; Thomas Brox. Vol. Part II Springer, 2020. p. 278-294 (Lecture Notes in Computer Science; Vol. 12347).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review