Learning Based Hyperspectral Image Reconstruction and Discriminative Representation

Description

Hyperspectral imaging aims at densely sampling the light spectrum covering both visible and non-visible ranges, producing hyperspectral images (HSIs) with much richer spectral information than ordinary RGB images. As different materials have uniquely distinguishable light spectra, HSI enables/facilitates a wide range of applications, such as remote sensing, medical diagnosis, food quality control, and security. However, the advantage of HSIs over RGB images comes at the cost of a non-trivial acquisition process. That is, existing hyperspectral imaging systems are unsuitable for rapid acquisition, capturing scenes with moving objects, or generating HSIs with both high spatial and spectral resolutions. Moreover, they are bulky and expensive. These limitations severely hinder widespread deployment of HSI based applications. Besides, due to various atmospheric conditions and sensor interference, spectral variations always occur in HSIs for remote sensing purposes, i.e., measurements corresponding to the same material may vary dramatically, which pose significant challenges for subsequent applications, e.g., terrain classification. This project aims at studying novel and effective learning based frameworks for reconstruction of high-quality HSIs from single RGB images and finding discriminative representations of degraded HSIs for remote sensing. Such severely ill-posed problems are extremely challenging, and existing methods cannot address them well, due to insufficient modeling of HSI's unique properties/structures. To this end, we will leverage recent developments of deep learning, low-rank representation theorem, nonconvex optimization, and semi-supervised learning for possible theoretical and performance breakthroughs. Specifically, we will propose novel deep learning schemes to characterize the complicated process from RGB images to HSIs covering both visible and non-visible ranges. Novel components including incorporation of side information, multi-scale feature learning, and structure-aware loss functions will be investigated for fully modeling HSI's unique properties/structures. To restore degraded HSIs, we will propose novel low-rank representation based semi-supervised learning models, which are capable of simultaneously augmenting the supervisory information and finding the discriminative representations with improved intra-class similarity and inter-class dissimilarity. With our multidisciplinary backgrounds and promising preliminary validations achieved, it is highly expected that our investigations will provide effective solutions for rapid, convenient and affordable acquisition of HSIs, which will fundamentally contribute to the hyperspectral imaging field as well as applications based on HSIs. Besides, our investigations will significantly facilitate research in image based scene/object understanding and analysis, which will be greatly beneficial for applications in artificial intelligence and computer vision. Last, the theoretical innovations of our approaches will continuously motivate the research on high-dimensional signal modeling in the image/signal processing community.