Hyperspectral Image Recovery Based On Decomposable 3D Convolution

Hyperspectral image is widely used in mineral exploration,environmental monitoring and other fields.However,in the process of hyperspectral image imaging,noise is inevitably added due to the effects of acquisition and transmission and other factors.Furthermore,due to the the limited of incident energy during image acquisition,there is a trade-off between spatial resolution and spectral resolution.The high spectral resolution of hyperspectral images will inevitably lead to a lower spatial resolution.These shortcomings have placed great limitations on the application of hyperspectral images.Recently,deep learning has been successfully applied in restoration tasks such as noise reduction and defogging of natural images.However,existing deep network frameworks cannot make full use of the spatial spectral correlation in 3D hyperspectral data,and directly extending it to 3D deep networks will significantly increase the computational complexity of the algorithm.In addition,the lack of training data for hyperspectral images is also a problem that cannot be ignored.Secondly,it is unrealistic to directly use a deep convolutional network to achieve a low spatial resolution hyperspectral image to a high spatial resolution hyperspectral image.The main reasons are that the GPU capacity of network is large and the existing network structure in the case of large scale super resolution reconstruction results decline obviously.To meet these challenges,we propose a new deep learning-based 3D convolution noise reduction network for hyperspectral images.Furthermore,based on the degradation process of hyperspectral images,we propose a hyperspectral image super-resolution method which uses the same scene RGB image with high resolution and fuses the prior features extracted by the noise reduction network.The main work and innovation points of this paper are as follows:1.The existing deep learning models do not make full use of the spatial correlation of hyperspectral images,so this paper proposes a multiscale coding and docoding denoising network based on decomposable 3D convolution strategy.We decompose the 3D convolution into2 D spatial convolution and 1D spectral convolution,which can greatly extract the spatial spectral features,save the number of network parameters and reduce the computational complexity.The experimental results show that the proposed noise reduction method achieves good results under different types of noise.2.Due to the lack of published hyperspectral datasets,the proposed deep denosing network lacks sufficient training data.We propose a method of using RGB images as auxiliary training data to generate hyperspectral images.Based on the idea of transfer learning,the proposed denosing network was pretrained by the synthetic pseudo-hyperspectral images,and then the network were fine-tuned on real hyperspectral images.Experimental results show that the network based on pretraining can obtain more detailed images in most cases than training directly on the public hyperspectral data.3.Existing convolutional neural networks cannot reconstruct a better result under large scale super resolution.The regularization designsing of method based on image degradation model is relatively complex,so we combine these two methods,considering the degradation process of high-resolution spectral images to low-resolution spectral images and high-resolution RGB in the same scene to build a mathematical model of image super-resolution.We use the half quadratic splitting method to convert the super resulotion model into an iterative and alternating optimization process,in which the regularization is extracted by the denoising network based on the decomposable 3D convolution to extract the hybrid prior features.Experimental results show that the proposed method has better performance in preserving spatial details and spectral information.