Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Jing Jin; Junhui Hou; Jie Chen; Henry Yeung; Sam Kwong

doi:10.1109/ICDSP.2018.8631859

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Jing Jin, Junhui Hou, Jie Chen, Henry Yeung, Sam Kwong

Department of Computer Science

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

4 Citations (Scopus)

Abstract

Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

Original language	English
Title of host publication	2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)
Publisher	IEEE
ISBN (Electronic)	9781538668115
ISBN (Print)	9781538668122
DOIs	https://doi.org/10.1109/ICDSP.2018.8631859
Publication status	Published - Nov 2018
Event	23rd IEEE International Conference on Digital Signal Processing (DSP 2018) - ShanghaiTech University, Shanghai, China Duration: 19 Nov 2018 → 21 Nov 2018 http://dsp2018.org/home.asp

Publication series

Name	International Conference on Digital Signal Processing, DSP
Volume	2018-November
ISSN (Print)	1546-1874
ISSN (Electronic)	2165-3577

Conference

Conference	23rd IEEE International Conference on Digital Signal Processing (DSP 2018)
Abbreviated title	DSP 2018
Place	China
City	Shanghai
Period	19/11/18 → 21/11/18
Internet address	http://dsp2018.org/home.asp

Research Keywords

deep learning
Light field
super-resolution

Access Output

10.1109/ICDSP.2018.8631859

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Jin, J., Hou, J., Chen, J., Yeung, H., & Kwong, S. (2018). Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. In 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP) Article 8631859 (International Conference on Digital Signal Processing, DSP; Vol. 2018-November). IEEE. https://doi.org/10.1109/ICDSP.2018.8631859

@inproceedings{29767d2e858841d7913e094fd28278b8,

title = "Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image",

abstract = "Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.",

keywords = "deep learning, Light field, super-resolution",

author = "Jing Jin and Junhui Hou and Jie Chen and Henry Yeung and Sam Kwong",

year = "2018",

month = nov,

doi = "10.1109/ICDSP.2018.8631859",

language = "English",

isbn = "9781538668122",

series = "International Conference on Digital Signal Processing, DSP",

publisher = "IEEE",

booktitle = "2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)",

address = "United States",

note = "23rd IEEE International Conference on Digital Signal Processing (DSP 2018), DSP 2018 ; Conference date: 19-11-2018 Through 21-11-2018",

url = "http://dsp2018.org/home.asp",

}

Jin, J , Hou, J, Chen, J, Yeung, H & Kwong, S 2018, Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. in 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)., 8631859, International Conference on Digital Signal Processing, DSP, vol. 2018-November, IEEE, 23rd IEEE International Conference on Digital Signal Processing (DSP 2018), Shanghai, China, 19/11/18. https://doi.org/10.1109/ICDSP.2018.8631859

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image. / Jin, Jing ; Hou, Junhui; Chen, Jie et al.
2018 IEEE 23rd International Conference on Digital Signal Processing (DSP). IEEE, 2018. 8631859 (International Conference on Digital Signal Processing, DSP; Vol. 2018-November).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

AU - Jin, Jing

AU - Hou, Junhui

AU - Chen, Jie

AU - Yeung, Henry

AU - Kwong, Sam

PY - 2018/11

Y1 - 2018/11

N2 - Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

AB - Light field (LF) has proven to be promising in immersive representation of the real world. However, a major limitation of micro-lens array based LF camera is the low spatial resolution, due to the inherent trade-off between angular and spatial dimensions. In this paper, we propose a framework to show that a single high-resolution (HR) RGB image effectively improves the performance of LF spatial super-resolution. We adopt an end-to-end convolutional neural network, which takes a low-resolution (LR) light field image (LFI) and a single HR center view as inputs. The LFI provides the information about LF structure in angular domain, while the HR center view provides more details in spatial domain. Experimental results on 57 test LFIs with various challenging natural scenes demonstrate that our algorithm outperforms current state-of-the-art methods.

KW - deep learning

KW - Light field

KW - super-resolution

UR - http://www.scopus.com/inward/record.url?scp=85062776177&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85062776177&origin=recordpage

U2 - 10.1109/ICDSP.2018.8631859

DO - 10.1109/ICDSP.2018.8631859

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9781538668122

T3 - International Conference on Digital Signal Processing, DSP

BT - 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP)

PB - IEEE

T2 - 23rd IEEE International Conference on Digital Signal Processing (DSP 2018)

Y2 - 19 November 2018 through 21 November 2018

ER -

Light Field Spatial Super-resolution via CNN Guided by A Single High-resolution RGB Image

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this