Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review

4 Scopus Citations
View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Title of host publicationMM'20
Subtitle of host publicationProceedings of the 28th ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery
Pages193-201
ISBN (Print)9781450379885
Publication statusPublished - Oct 2020

Publication series

NameMM - Proceedings of the ACM International Conference on Multimedia

Conference

Title28th ACM International Conference on Multimedia (MM 2020)
LocationVirtual
PlaceUnited States
CitySeattle
Period12 - 16 October 2020

Abstract

This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple low-resolution cameras. To tackle this challenge, we propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation; the other one constructs another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations via the learned attention maps, leading to the final high-resolution LF image. Extensive experiments demonstrate the significant superiority of our approach over state-of-the-art ones. That is, our method not only improves the PSNR by more than 2 dB, but also preserves the LF structure much better. To the best of our knowledge, this is the first end-to-end deep learning method for reconstructing a high-resolution LF image with a hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and also be beneficial to LF data storage and transmission. The code is available at https://github.com/jingjin25/LFhybridSR-Fusion.

Research Area(s)

  • Light field, hybrid imaging system, deep learning, attention

Bibliographic Note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Citation Format(s)

Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses. / Jin, Jing; Hou, Junhui; Chen, Jie; Kwong, Sam; Yu, Jingyi.

MM'20: Proceedings of the 28th ACM International Conference on Multimedia. Association for Computing Machinery, 2020. p. 193-201 (MM - Proceedings of the ACM International Conference on Multimedia).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review