Deep Learning Reconstruction From RGB Image To Hyperspectral Image

Compared with traditional RGB images,Hyperspectral Image(HSI)contains richer and more detailed spectral information,which is often used in military remote sensing,intelligent detection,medical diagnosis and other challenging fields.However,hyperspectral imaging equipment is bulky,complex and very expensive,resulting in limited applications.In recent years,with the rapid development of artificial intelligence and image processing technology,it has become a research focus to achieve low-cost and rapid acquisition of hyperspectral images through spectral reconstruction of RGB images.At present,the spectral reconstruction of RGB images based on deep learning has achieved certain research results.However,because the reconstruction of high-dimensional spectral data from three-channel RGB images is a,pathological mathematical inverse problem,resulting in its solution difficulty still exists,and the recorstructed spectral images still face the problems of structural detail loss and poor spectral reconstruction accuracy.Therefore,how to accurately infer the missing spectral information through RGB images so as to carry out high-quality spectral rcconstruction is still of important research significance.In order to effectively reduce spectral reconstruction errors and achieve high-quality spectral reconstruction in a real sense,two end-to-end metwork models based on deep learning are proposed to learn the thrcc-to-many mapping relationship between RGB images and the corresponding hyperspectral images in two different scenarios,"Clean" and "RealWorld".The main work is as follows:(1)In the "Clean" scenario,aiming at the shortcomings of feature extraction and fusion methods in the current spectral reconstruction tasks of RGB images,this paper proposes an algorithm for spectral reconstruction of RGB images based on dual attention mechanism.First,the shallow feature information is extracted from the input images by convolution.Then,the correlation between the channels in the feature layer is calculated by using the effective multifrequency channel attention mechanism.Secondly,the weighted fusion attention mechanism of layer features is introduced to learn the dependencies between features of different layers.Finally,the deep features extracted by effective multi-frequency channel attention and layer feature weighted fusion attention are convolved to generate 31-channel spectral images.Experimental validation of the proposed algorithm was conducted on NTIRE 2020,CAVE and TokyoTech datasets,and the results show that the proposed algorithm is advanced in both subjective and objective indicators.(2)In the "RealWorld" scenario,when the current spectral reconstruction algorithms of RGB images based on Convolutional Neural Network(CNN)to extract feature information,the network is often deepened to expand the field of perception and extract deeper feature information,ignoring the problem of network degradation caused by too deep a network and the difficulty of convolution to effectively perceive global feature information.Therefore,this paper proposes a spectral reconstruction algorithm based on CNN and Transformer fusion.The algorithm is designed with a CNN and Transformer parallel structure,using the Transformer to compensate for the disadvantages of the CNN in global interaction,while compensating for the shortcomings of the Transformer in local processing through the CNN.Combining the fusion module to pass the feature information extracted by the CNN and the Transformer in both directions,the network has both local and global modelling capabilities,thus improving the performance of the network model and the accuracy of the spectral reconstruction.In addition,the algorithm is experimentally compared with seven RGB image spectral reconstruction algorithms on four evaluation metrics,and the experimental results show that the proposed algorithm reconstructs the spectra with less error and higher spectral reconstruction accuracy.