Deep Learning Based Spatial And Spectral Super-resolution Reconstruction Of Hyperspectral Images

Hyperspectral imaging usually records the spectrum of the scene in terms of tens or hundreds of bands,each corresponding to a specific narrow wavelength range.The collected hyperspectral images contain rich spectral information compared to color images,and the spectral characteristic of the scene has been demonstrated to be conducive to many research fields.Traditionally,hyperspectral imaging has been used in remote sensing imaging,drug detection,and mineral exploration.Recently,hyperspectral im-ages have been applied in many computer vision tasks,such as image segmentation,face recognition,and object tracking.In the traditional acquisition of hyperspectral images,two-dimensional sensors are usually used to capture three-dimensional information,so it is inevitable to make a trade-off between spectral resolution and spatial or temporal resolution.For example,when capturing the spectral information of a dynamic scene,a snapshot spectral imaging sacrifices spatial resolution for spectral resolution.Therefore,hyperspectral spatial super-resolution technology is often adopted as the post-processing.In addition,since the hardware system and reconstruction algorithms in hyperspectral imaging still have high complexity problems,researchers have proposed hyperspectral imaging technology based on color cameras,and reconstruction algorithms from color images to hyperspectral images(spectral super-resolution)have been the research focuses.This dissertation mainly explores the spectral reconstruction algorithm based on deep learning.And it can be roughly divided into spectral super-resolution and hyperspectral image spatial super-resolution.We also utilize the complementary information between high spatial resolution color images and low spatial resolution hyperspectral images to complete the hyperspectral image spatial-spectral joint reconstruction task.Furthermore,we also introduce the domain adaptation into the spectral super-resolution task,so we can adapt the previous proposed models to color images in different image domains.Specifically,the principal work and innovations of this dissertation can be summarized as:1.We develop a unified deep learning method for different degradation models,such as spatial down-sampling and spectral down-sampling.Then we study the performance difference between different modules like residual learning module and dense connection module in details,and we propose a channel attention mechanism based on residual learning module.We also propose a fusion network for the joint spatial-spectral super-resolution task.Compared with the other methods,the proposed methods in this paper supported by the existing advanced network structures have achieved the best results in the spectral challenges of NTIRE2018 and PIRM2018.2.We introduce the domain adaptation framework into the hyperspectral image spectral super-resolution task.It first determines if the color image belongs to the original domain or the target domain through the discriminator and then uses the adversarial learning method to transfer the two domains.This method can resolve the problem that the training data in the target domain is unavailable.Experimental results show that the proposed method is enabled to transfer effectively between color images synthesized by two different spectral sensitivity functions.