Hyperspectral Image Super-Resolution via Deep Progressive Zero-Centric Residual Learning

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review

47 Scopus Citations
View graph of relations

Author(s)

Related Research Unit(s)

Detail(s)

Original languageEnglish
Article number9298460
Pages (from-to)1423-1438
Number of pages16
Journal / PublicationIEEE Transactions on Image Processing
Volume30
Online published17 Dec 2020
Publication statusPublished - 2021

Abstract

This paper explores the problem of hyperspectral image (HSI) super-resolution that merges a low resolution HSI (LR-HSI) and a high resolution multispectral image (HR-MSI). The cross-modality distribution of the spatial and spectral information makes the problem challenging. Inspired by the classic wavelet decomposition-based image fusion, we propose a novel lightweight deep neural network-based framework, namely progressive zero-centric residual network (PZRes-Net), to address this problem efficiently and effectively. Specifically, PZRes-Net learns a high resolution and zero-centric residual image, which contains high-frequency spatial details of the scene across all spectral bands, from both inputs in a progressive fashion along the spectral dimension. And the resulting residual image is then superimposed onto the up-sampled LR-HSI in a mean-value invariant manner, leading to a coarse HR-HSI, which is further refined by exploring the coherence across all spectral bands simultaneously. To learn the residual image efficiently and effectively, we employ spectral-spatial separable convolution with dense connections. In addition, we propose zero-mean normalization implemented on the feature maps of each layer to realize the zero-mean characteristic of the residual image. Extensive experiments over both real and synthetic benchmark datasets demonstrate that our PZRes-Net outperforms state-of-the-art methods to a significant extent in terms of both 4 quantitative metrics and visual quality, e.g., our PZRes-Net improves the PSNR more than 3dB, while saving 2.3× parameters and consuming 15× less FLOPs. The code is publicly available at https://github.com/zbzhzhy/PZRes-Net.

Research Area(s)

  • cross-modality, deep learning, Hyperspectral imagery, Hyperspectral imaging, image fusion, Image reconstruction, Optimization, Principal component analysis, Residual neural networks, Spatial resolution, super-resolution, Tensors, zero-mean normalization

Citation Format(s)

Hyperspectral Image Super-Resolution via Deep Progressive Zero-Centric Residual Learning. / Zhu, Zhiyu; Hou, Junhui; Chen, Jie et al.
In: IEEE Transactions on Image Processing, Vol. 30, 9298460, 2021, p. 1423-1438.

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journalpeer-review