Research On Image Super-Resolution Based On Convolutional Neural Network And Transformer

Images are an important carrier to record people’s lives and reproduce historical scenes,especially in traffic flow prediction,remote sensing,criminal investigation and forensics and other aspects of urban governance with a lot of image data.With the emergence of highresolution display devices,images with low-resolution cannot get good visual effects on high resolution devices.Image super-resolution technology can not only recover the high-resolution pixel from the image with low resolution,but also save the cost of replacing or upgrading the camera components because the super-resolution method is to process the photos.So,image super-resolution has high research value.This paper mainly carries out the following research.Natural image super-resolution algorithms based on deep learning generally have better reconstruction effect than the traditional learning-based super-resolution algorithms.However,the deep learning algorithms usually has many parameters and a large amount of calculation,so it cannot be applied to mobile terminal devices with low computing power.Aiming at this problem,this paper studies the lightweight dense residual connection block and the improvement of information distillation modules.In addition,since Transformer structure brings larger receptive fields and enhances the correlation between features to improve image processing,we try to use Transformer structure in hyperspectral super-resolution field to obtain better reconstruction effect.The thesis has made the following three achievements:Natural image super-resolution algorithms based on deep learning generally have better reconstruction effect than traditional super-resolution algorithms based on sparse representation.However,deep learning algorithms usually have problems such as large number of parameters and large amount of calculation.It cannot be applied to mobile terminal devices with small computing power.In addition,since Transformer structure brings larger receptive field and the correlation between enhanced features has an improved effect on image processing,most hyperspectral super-resolution algorithms are still implemented by convolution traditional learning with smaller receptive field.In view of these two problems,this paper carries out research.The paper mainly achieves the following three achievements:(1)In order to obtain better image super-resolution effect with fewer parameters,a Feedback Ghost Residual Dense Network(FGRDN)is proposed.First,feedback mechanism is used as the framework of the network to refine low-level features through high-level features.Then,Ghost Module is used to replace the convolution of feature extraction in the residual dense block(RDB)to remove redundant channels and reduce the number of parameters caused by the deepening of network depth.Finally,at the end of GRDB modules,spatial and channel attention mechanisms are used to learn more useful information in feature mapping from spatial and channel.The improved algorithm has faster convergence speed and smaller number of parameters,better edge,and texture reconstruction effect,and better Peak Signal to Noise Ratio(PSNR)and Structure Similarity(SSIM)than other lightweight algorithms.(2)Further,lightweight network structure,and proposed Enhanced Information Multiple Distillation Network(EIMDN).The network is based on a feedback mechanism,with higher refinement to obtain lower features.Then,we optimize the existing lightweight Information Multiple Distillation Network,The Information Multiple Distillation Block(IMDB)replaced the character-extraction convolution operations with Ghost Module.The enhanced Information Multiple Meeting Block(EIMDB)modules are proposed to reduce the number of computations and parameters.Finally,we use Coordinate Attention(CA)mechanism in IMDB and EIMNB to improve the ability of extracting important information of space and channel.Compared with other lightweight algorithms,our algorithm can achieve convergence faster with fewer parameters and less calculation,and significantly improve the performance of network reconstructed image texture and target contour with higher PSNR and SSIM.(3)The convolutional neural network has a small receptive field and cannot obtain global spatial information,resulting in poor spatial resolution reconstruction effect.A3D-Transformer Hyperspectral Super-Resolution(3D-THSR)method is proposed.Transformer structure is used to obtain global spatial information through self-attention mechanism.Specifically,we introduce Transformer structure as feature extraction part to enhance the learning ability of global spatial information.Meanwhile,in order to extract the correlation between different spectral bands,3D convolution modules are added into the feedforward part of Transformer to extract high-level features by integrating spectral and spatial information.Finally,we redesign the loss function,using Mean Absolute Deviation(MAE)loss,Spatial Spectrum Total Variation(SSTV)loss,and Spectral Angle Mapper(SAM)loss can reduce the distortion between spectra and ensure the purity between various spectral bands.Compared with other single image hyperspectral methods based on convolutional neural network and the ablation experiment of modules,the proposed method is proved that the proposed method has more spatial texture details and lower spectral distortion,and achieves better reconstruction effect.