Research On Intensity Correlation Multispectral Image Reconstruction Algorithm Based On Deep Learning

With the development of target detection technology,multispectral and polarization image reconstruction techniques can achieve accurate identification of targets by making full use of the spectral and polarization information,and therefore have received more and more attention and research from scholars.Among them,Ghost imaging via sparsity constraint(GISC)spectral camera has received special attention because it simplifies the spectral imaging process.The existing GISC multispectral image reconstruction algorithm has the problems of slow reconstruction speed and low reconstruction quality,and the in-depth research on GISC multispectral image reconstruction algorithm will further promote the development of ghost imaging technology in optical imaging system.In this paper,we focus on the reconstruction of GISC multispectral images by traditional algorithms(correlation operation reconstruction,compression-aware reconstruction),reconstruction based on deep learning networks,and joint reconstruction by combining traditional algorithms and deep learning networks,respectively,to achieve high quality and fast reconstruction of GISC multispectral and polarization images.The main work and innovation points of this paper are as follows:1.A GISC multispectral image reconstruction algorithm based on spatial and inter-spectral Transformer is proposed.To address the problems of low quality of reconstructed images due to underutilization of image data by traditional reconstruction algorithms,an SSTU-Net3+ based on spatial and inter-spectral Transformer is proposed.A deep learning network is used to finely map the spatially relevant information and inter-spectral relevant information of multispectral images to enhance the multidimensional feature representation of the images.The reconstruction results of Differential Ghost Imaging(DGI)are used as the input of the deep learning network to reduce the sensitivity of the network to the system measurement matrix and to enhance the flexibility of the network.Experimental results show that based on the GISC spectral imaging system and ICVL hyperspectral image data,the PSNR of the image reconstructed by the method proposed in this paper is improved by more than 1.1 d B compared with the U-Net series network;it is improved by more than 14 d B compared with DGI and compressed sensing(CS)algorithms.2.A GISC multispectral image reconstruction algorithm based on compressed sensing and Co T-Unet is proposed.To address the problems of low quality of GISC multispectral image reconstruction at low sampling rate and non-lightweight network,a lightweight deep learning network Co T-Unet with improved U-Net structure is proposed using convolutional neural network and contextual self-attention.The network utilizes Res2Net-SE module in order to fully extract spatially relevant information of multispectral images and effectively represent multiscale features of images;Meanwhile,the network employs a design of contextual self-attention on different spatial orientations to accurately represent image detail features.In addition,this paper proposes to use the results of the compressed sensing algorithm as the input of the Co T-Unet network,combine the advantage that the compressed perception algorithm can reconstruct images with low sampling rate,and use the deep learning network to further refine the image detail features in order to improve the quality of the reconstructed images.The experimental results show that the joint reconstruction method combining CS and Co T-Unet can achieve high quality image reconstruction at low sampling rate.3.A Co T-Unet-based reconstruction method for GISC spectral polarization images is proposed.Aiming at the problem of low quality of GISC multispectral polarization image reconstruction,the paper extends the reconstruction method combining compressed sensing and Co T-Unet to the GISC spectral polarization image reconstruction task based on the powerful feature representation capability of Co T-Unet and the advantage of the compressed sensing algorithm to reconstruct images at low sampling rate.In the image reconstruction process,the spectral images at each polarization state initially reconstructed by the CS algorithm are used as the input of the training model of the Co T-Unet to reconstruct high-quality polarization spectral images.The quality of the reconstructed target object images by the method in the paper is analyzed in four linear polarization directions(0°,45°,90° and 135°),respectively.The experimental results show that compared with the traditional algorithm and the Co T-Unet,the method in this paper reconstructs the spectrally polarized image with the best quality.