Fast Light Field Reconstruction with Deep Coarse-to-Fine Modeling of Spatial-Angular Clues

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)

10 Scopus Citations
View graph of relations

Author(s)

  • Henry Wing Fung Yeung
  • Junhui Hou
  • Jie Chen
  • Yuk Ying Chung
  • Xiaoming Chen

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2018
Subtitle of host publication15th European Conference, 2018, Proceedings
EditorsVittorio Ferrari, Martial Hebert, Cristian Sminchisescu, Yair Weiss
PublisherSpringer, Cham
Pages138-154
ISBN (Electronic)9783030012311
ISBN (Print)9783030012304
Publication statusPublished - Sep 2018

Publication series

NameLecture Notes in Computer Science
Volume11210
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Title15th European Conference on Computer Vision (ECCV 2018)
PlaceGermany
CityMunich
Period8 - 14 September 2018

Abstract

Densely-sampled light fields (LFs) are beneficial to many applications such as depth inference and post-capture refocusing. However, it is costly and challenging to capture them. In this paper, we propose a learning based algorithm to reconstruct a densely-sampled LF fast and accurately from a sparsely-sampled LF in one forward pass. Our method uses computationally efficient convolutions to deeply characterize the high dimensional spatial-angular clues in a coarse-to-fine manner. Specifically, our end-to-end model first synthesizes a set of intermediate novel sub-aperture images (SAIs) by exploring the coarse characteristics of the sparsely-sampled LF input with spatial-angular alternating convolutions. Then, the synthesized intermediate novel SAIs are efficiently refined by further recovering the fine relations from all SAIs via guided residual learning and stride-2 4-D convolutions. Experimental results on extensive real-world and synthetic LF images show that our model can provide more than 3 dB advantage in reconstruction quality in average than the state-of-the-art methods while being computationally faster by a factor of 30. Besides, more accurate depth can be inferred from the reconstructed densely-sampled LFs by our method.

Research Area(s)

  • Convolutional neural network, Deep learning, Light field, Super resolution, View synthesis

Citation Format(s)

Fast Light Field Reconstruction with Deep Coarse-to-Fine Modeling of Spatial-Angular Clues. / Yeung, Henry Wing Fung; Hou, Junhui; Chen, Jie ; Chung, Yuk Ying; Chen, Xiaoming.

Computer Vision – ECCV 2018: 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Martial Hebert; Cristian Sminchisescu; Yair Weiss. Springer, Cham, 2018. p. 138-154 (Lecture Notes in Computer Science; Vol. 11210).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)