Research On Hyperspectral Image Fusion Based On Model-driven Deep Learning

High-resolution hyperspectral images have both rich spatial information and rich spectral information.Therefore,high-resolution hyperspectral images were widely used in the field of target detection,recognition and tracking.However,due to the insufficient performance of current imaging equipment,we cannot directly obtain high-resolution hyperspectral images.Instead,we can only perform a certain degradation in space and spectrum,and instead obtain a set of low-resolution hyperspectral images and high-resolution multispectral images for the same scene,and then process them through a certain algorithm.Then we reconstructed this set of images as a high-quality,high-resolution hyperspectral image.This process is so called hyperspectral image fusion.In the process of hyperspectral image fusion,the key point is to make the fused target image as close as possible to the low-resolution hyperspectral image in spectrum,and as close as possible to the high-resolution multispectral image in space.In this thesis,a series of hyperspectral image fusion algorithms are proposed based on the traditional mathematical modeling optimization method combined with deep learning.From the results,these algorithms have achieved good results on both simulated and real datasets.At the same time,combined with the project engineering practice,this thesis also proposes an easy-to-use hyperspectral image processing platform.The main work of this thesis is summarized as follows:At present,there are mainly two kinds of solutions to the fusion of hyperspectral images.The first is the method of solving by using deep neural network.These methods mainly focus on the fusion of feature level and mainly improve the network structure.Although they show excellent results after a large number of sample training,they lack adaptability and interpretability as a whole.The other is the fusion algorithm based on image feature analysis.This kind of algorithm is mainly based on the properties of hyperspectral images,which has strong adaptability and interpretability.To introduce the advantages of fusion algorithm based on image feature analysis into deep hyperspectral image fusion methods,we analyze the representative Bayesian fusion methods and multi-resolution fusion methods in traditional hyperspectral image fusion.The key steps of these methods are discussed,and relevant experiments are carried out.Furthermore,to improve the applicability of the algorithm and reduce the dependence on external conditions,combined with the prior knowledge of the image itself,this thesis proposes a model-driven algorithm based on depth image prior.Like fusion algorithm based on image feature analysis,we first transform the fusion problem into a class of mathematical model optimization algorithms.We start with this optimization problem and convert the overall optimization formula into an iterative gradient descent solution through the semi-quadratic splitting method.Different from the traditional hyperspectral image fusion algorithm here,we use the depth image prior as the spatial regularization term of the algorithm.Because of this,we take advantage of the trend to interpret the entire optimization solution process in the form of a deep neural network.In the network,to utilize the spatial and spectral information in each iteration,we add an aggregation module at the end of the network to integrate the information in each stage.Finally,the experimental results on various datasets prove that the model-driven deep network has good adaptability and fusion ability.Finally,we design and implement a platform that can be used to synthesize hyperspectral images.With the increasing application of hyperspectral image processing in the work,the traditional manual or hard-coding method to complete the hyperspectral image fusion has the disadvantages of high threshold and low efficiency.In addition,in the face of complex data,it is difficult for practitioners to obtain corresponding information intuitively.The platform provides a one-click fusion operation,which greatly facilitates the processing of hyperspectral images.In addition,the platform also provides visual hyperspectral image processing operations to facilitate users to obtain hyperspectral image information.