Hyperspectral Image Super-resolution Via Deep Learning

Hyperspectral images(HSIs)are the main data type dealt in hyperspectral imaging.Compared to RGB images,hyperspectral images contain more spectral information and offer a more careful subdivision of electromagnetic spectrum,resulting in dozens or even hundreds of channel bands.This unique feature makes HSIs popular among various material analysis tasks.However,in real-world situations the quality of HSIs is circumscribed by the design of present hyperspectral cameras and photo’ing conditions,making the captured HSIs far from satisfactory.One key problem is that HSI imaging usually sacrifices the spatial resolution of HSIs for better logging of spectral information.Hence it is important and of practical use to design a post super-resolution algorithm to help improve the quality of HSIs so as to facilitate the follow-up tasks.This paper studies the super-resolution of HSIs via deep learning and comprises the following two parts:(1)Single hyperspectral image super-resolution.Considering present methods do not fully exploit the 3D nature of HSIs,we choose to build a spatial-spectral distinguishment mechanism by using a multi-scale image pyramid model,which is used later to design a pyramid module for feature extraction.The module could learn spatial and spectral features separately on different stages of its image pyramid,making the learning process of super-resolution much easier.Then we employ that module to build an end-to-end network model via residual learning.Experiments and ablation studies have proved that our model produced good objective and subjective results compared to present methods on hyperspectral remote sensing datasets.(2)Hyperspectral image super-resolution via fusing low-resolution HSI and highresolution auxiliary RGB image.As RGB-HSI pairs contain strong complementary information to each other,they could be fused together to achieve good superresolution results.However,present methods usually take too long to perform.Inspired by kernel learning methods,we here propose a lightweight deep learning model via learnable affine transforms,which directly maps input RGB image to HSI.We first justify the use of affine transform as an inverse spectral response mapping and then design a corresponding network model.The proposed model composes of two sub-networks which each learns a set of grid-based basis affine transforms and reweighting coefficients,respectively.These are then combined to produce pixel-wise affine transform matrices via a spectrally-designed grid upsamping operation.The design of the network reduces computation power and meanwhile maintains its learning ability.Extensive experiments have demonstrated that our purposed method is comparable to state-of-the-art methods with much faster running speed.