Deep Structural Prior Learning For Spectral Image Reconstruction

Compared to RGB images,hyperspectral images have more spectral bands,enabling finer discrimination of material.Therefore,hyperspectral images have significant applications in industries and daily life,including national defense and security,remote sensing,smart agriculture,healthcare,and food analysis.Although conventional spectral imaging methods can obtain high-quality hyperspectral images,they are slow and not suitable for rapidly capturing dynamic scenes.In recent years,snapshot hyperspectral imaging technology has gained widespread attention due to its capability to rapidly capture dynamic scenes.Snapshot hyperspectral imaging systems encode three-dimensional spectral information through hardware,generating two-dimensional observed images.These images are then decoded and reconstructed using algorithms to obtain high-quality three-dimensional hyperspectral images.Therefore,researching spectral image reconstruction methods is of great significance.Spectral image reconstruction methods can be divided into two categories: knowledge-driven methods and data-driven methods.Knowledge-driven mathematical optimization methods involve the design of degradation models based on the physical imaging process and the construction of image prior models to construct mathematical models for spectral image reconstruction.These models are then solved using optimization methods,typically requiring multiple iterations to obtain the final spectral images.While these hand-crafted priors effectively reduce the solution space,they often have limited representation capabilities,neglecting the rich spectral correlations of spectral images,which results in suboptimal reconstruction performance.In contrast,data-driven deep learning methods rely on extensive training data to directly learn image priors from the data.Therefore,they show impressive reconstruction performance,outperforming knowledge-driven methods.However,such methods often overlook the physical imaging model,omitting constraints of the likelihood model,making it challenging to further improve reconstruction results.To combine the constraints of both likelihood and prior models,model-guided deep learning methods unfold the traditional optimization iterative solutions into a multi-stage network,learning priors and target spectral images in an end-to-end manner,yielding advantages in terms of reconstruction performance.The key to spectral image reconstruction lies in modeling and learning spectral image priors.However,existing deep learning methods typically employ implicit models to model image priors.These implicit prior networks mainly focus on local spatial correlations of spectral images,leading to inaccuracies modeling and insufficient learning of priors.To address the above issues,this dissertation focuses on spectral image reconstruction task and centers around the learning of spectral image priors.This dissertation proposes three spectral image reconstruction method based on deep structural prior learning,achieving promising results in spectral image reconstruction performance.The main research content and innovations of this dissertation are as follows:1.To address the issues that existing deep learning-based spectral image reconstruction methods often overlook likelihood constraints and insufficiently explore the rich spatial-spectral correlations of spectral images,this dissertation proposes a spectral image reconstruction method based on deep autoregressive model.It is the first to combine autoregressive models with deep learning for spectral image reconstruction.Firstly,this dissertation unfolds the iterative algorithm of maximum a posteriori estimation model for spectral image reconstruction into a multi-stage deep network.Secondly,spectral image priors are learned through autoregressive models and spatial-spectral correlations of spectral image at each stage.Finally,this dissertation proposes to use a deep network to adaptively learn the spatial-spectral correlations between each pixel of the spectral image and its local neighborhood pixels,ensuring the full utilization of the spatial-spectral correlations of spectral image.Extensive experimental results demonstrate that the proposed method outperforms existing spectral image reconstruction methods in terms of objective quality metrics and subjective visual comparisons.2.To address the issue that existing deep learning-based spectral image reconstruction methods often tend to learn deterministic priors and overlook the varying challenges and accuracies in prior learning across different regions of spectral images,this dissertation proposes a spectral image reconstruction method based on a deep Gaussian scale mixture prior.Firstly,considering the differing accuracy in prior estimation across different regions,this dissertation proposes to employ Gaussian scale mixture model to characterize the parameterized distribution of the spectral image priors,where the scale factor of the Gaussian scale mixture model represents the uncertainty in prior learning.Secondly,this dissertation combines the maximum a posteriori estimation framework with the Gaussian scale mixture model and proposes a deep unfolding spectral image reconstruction network.Finally,to address the limited long-range dependencies learning capability of existing convolutional prior networks,this dissertation proposes a Gaussian scale mixture prior learning network based on Swin Transformer.This network simultaneously learns the means and variances of the Gaussian scale mixture prior.Extensive experimental results on both synthetic and real datasets demonstrate the proposed method outperforms other state-of-the-art spectral image reconstruction methods.3.To address the issue that existing spectral image reconstruction methods tend to focus on local feature learning and overlook global feature learning,this dissertation proposes a spectral image reconstruction method based on joint global-local feature learning.Firstly,to address the suboptimal performance of the initialization of existing spectral image reconstruction methods and the introduction of additional degradation,this dissertation proposes a novel spectral image initialization method based on Fourier transform.The proposed method effectively preserves spectral image texture details while removing partial mosaic artifact.Secondly,the dissertation proposes an efficient joint global-local feature learning module that learns the correlations of spectral image global-local features in the frequency and spatial domain,respectively.Finally,by combining the proposed spectral image initialization method with the joint global-local feature learning module,this dissertation proposes a spectral image reconstruction network,enabling efficient learning of spectral image priors in feature domain.The proposed spectral reconstruction network has significantly fewer parameters and lower computational requirements compared to existing deep learning spectral reconstruction networks.Experimental results on synthetic and real spectral image reconstruction tasks demonstrate that the proposed spectral image reconstruction method outperforms other state-of-the-art methods.