A Spectral Image Fusion Method Based On Depth Representation Prior And Multivariate Regression

With the rapid development of satellite sensors,multispectral remote sensing images have been widely used.However,due to technical limitations of sensors and other factors,existing remote sensing sensors must compromise between spatial and spectral resolution.Panchromatic sensors capture panchromatic?PAN?images with high spatial resolution,while multispectral sensors capture low resolution multispectral?LRMS?images with rich spectral information.Complementary information fusion of PAN images and LRMS images can be used to integrate spatial details of PAN images and spectral information of LRMS images to obtain high resolution multispectral?HRMS?images,which commonly referred as"Pan-sharpening".This paper focuses on the Pan-sharpening problem.The main research results are as follows:?1?A low-rank regularization Pan-sharpening method based on multivariate regression is proposed.Firstly,low rank regularization term and row sparse prior are used to characterize the intrinsic structure of HRMS images.Secondly,a data fidelity term based on band-dependent multivariate regression is constructed under the detail injection framework,and then a low-rank regularization model based on multivariate regression is established.Finally,the model was solved using the Augmented Lagrangian Multiplier?ALM?to obtain the HRMS image.Both the simulation data experiments and the real data experiments proved that the algorithm performs better than many component substitution methods,multi-resolution analysis methods and typical variational methods.?2?Combining multivariate regression with deep learning,this paper proposes a Pan-sharpening method based on deep representation prior and multivariate regression.The advantage of deep neural networks over traditional component substitution,multi-resolution analysis,and variational methods is they can combine multiple layers of nonlinear mapping functions to model complex relationships between multiple variables.In this paper,not only the deep residual network is applied to Pan-sharpening,but also combined with multivariate regression to modify the loss function of the deep residual network.At the same time,considering the sparsity of the target,the mean absolute error?MAE?based on the L₁-norm is used as a loss function for the training network,instead of the mean squared error?MSE?based on the L₂-norm,to make sure the network learns more precise details.Simulation data experiments and real data experiments show that this method can obtain better fusion quality,reduce spectral distortion and spatial distortion,and has better generalization ability than deep residual network Pan-sharpening?PanNet?.?3?A pan-sharpening fusion system for remote sensing images is implemented.The system mainly includes two functional modules.One is the remote sensing image fusion processing toolbox under the general framework,and the other is a software package based on deep learning and multivariate regression methods.The former implements nine different Pan-sharpening methods based on component substitution under the general framework,integrates a toolbox for remote sensing image fusion,and calculates the evaluation indexes of fusion results.The latter implements the two methods proposed in this paper.It also achieves remote sensing image visualization and image quality evaluation.