Research Of Hyperspectral Remote Sensing Image Classification Based On Convolutional Long Short-term Memory Neural Network

Hyperspectral images(HSIs)are obtained by imaging the land covers with the spectrome-ter along dozens to hundreds of narrow and continuous spectral channels in the range of visible light,near infrared,mid infrared,and thermal infrared bands.Different from other remote sensing data,HSIs can effectively describe the spatial and spectral information of land cover-s,presenting a data structure with“combination of image and spectrum”,and their nanoscale spectral information can provide good data support for the identification of subtle differences between different land covers,and the observation and research in various application fields,which have made good progress in urban planning,precision agriculture,environmental mon-itoring,military reconnaissance,biomedicine,wetland monitoring,and other fields.In these applications,HSI processing technology is the most key research content,where HSI classifi-cation,as an important information acquisition technology,has become a hot research topic.HSI classification is mainly to identify the categories of different land covers according to the differences of their spectral information.Although the existing classification algorithms have obtained good research progresses,there have been some problems,such as insufficient utilization of data structure information,high model complexity,insufficient network training,and limited classification performance.As such,combined with the latest research results of deep learning theory,this thesis carries out the research on HSI classification based on deep convolutional long short-term memory(Conv LSTM)neural networks(CLNNs),and plenty of simulation experiments and comparative analysis illustrate the rationality and effectiveness of the proposed algorithms.The main research work involve the following hour aspects:1.A deep CLNN-based algorithm is proposed for HSI classification.Firstly,due to the special imaging mechanism,there are rich spectral information and strong spectral correla-tion.As such,the 2-D Conv LSTM(Conv LSTM2D)cell is taken as the basic unit,and a spatial-spectral Conv LSTM2D neural network(SSCL2DNN)is designed,which can extract the spatial-spectral features by learning the spatial information band by band along the spectral dimension for HSI classification.Furthermore,a spatial-spectral Conv LSTM cell(namely Con-v LSTM3D)is developed,and a spatial-spectral Conv LSTM3D neural network(SSCL3DNN)is constructed,capturing the strong correlation and maintaining the intrinsic structure of HSIs,so as to extract more discriminative spatial-spectral features and improve the classification ac-curacy.In particular,the special gate structure makes our deep CLNN-based algorithm suppress the influences of noise and abnormal data in HSIs on the classification results.2.A dual-channel deep CLNN-based model is constructed for the fusion and classification of HSI and Li DAR image.In order to make up for the problem of low spatial resolution of H-SIs and improve the ability of earth observation,this thesis integrates the respective advantages and complementary information of HSI and Li DAR image,and proposes a dual-channel deep CLNN(dual-channel spatial,spectral,and multiscale attention CLNN,dual-channel A~3CLNN).Firstly,on the basis of attention mechanism,a spectral attention block,a spatial attention block,and a multiscale residual attention block are designed,with which a multiscale spectral atten-tion neural network(MSSe A)and a multiscale spatial attention neural network(MSSa A)are built as the HSI and Li DAR channels to extract the spatial-spectral features enhanced by the spectral correlation and the spatial features enhanced by the elevation information,respective-ly.After that,a three-level fusion strategy is designed to realize the mutual enhancement and effective fusion between these two channels.Finally,inspired by the ideas of transfer learning and multitask learning,a novel optimization algorithm guided by the stepwise training strategy and the multiloss function is further designed to fully train the whole model and effectively improve its classification performance.3.A deep pseudo complex-valued(CV)deformable CLNN model based on the multi-modality feature fusion is established for HSI classification.Aiming at the problems of poor interpretability of deep semantic features and limited classification performance in the case of small samples for deep neural networks,the multimodality feature fusion is introduced into HSI classification for the first time,and a deep Pseudo CV Deformable Conv LSTM2D Neural Network with Mutual Attention Learning(APDCLNN)is designed.By utilizing the advan-tages of complex operation and deformable convolutional in improving the feature extraction ability of deep neural networks,the Conv LSTM2D cell is firstly extended to the complex do-main,and a PD Conv LSTM2D(PDConv LSTM2D)cell is designed,with which a spatial-spectral PDConv LSTM2D neural network(SSPDCL2DNN)is proposed to extract the scale-and correlation-enhanced deep features.Then,the mutual attention learning is introduced to construct a multimodality feature learning and fusion(MAMLF)module,where the traditional features of HSIs are extracted by the 3-D Gabor filter and dynamically fused with the deep features.Finally,a novel attention loss subnetwork is built and combined with the classifica-tion loss subnetwork to design a multitask loss function.Therefore,the traditional features are effectively integrated into the whole model to dynamically optimize the training process of the whole model,thus reducing the training difficulty and improving the feature extraction and classification performance in the case of small samples.4.The HSI classification based on the lossless lightweight deep tensor CLNN is studied.Aiming at the problems of many parameters and high complexity of deep Conv LSTM2D neural network,by combining with the tensor theory and attention mechanism,this thesis proposes a lossless lightweight tensor attention-driven Conv LSTM2D neural network(TACLNN).Firstly,the tensor-train(TT)decomposition is applied to extend the Conv LSTM2D cell to the tensor domain,and a TT Conv LSTM2D(TTConv LSTM2D)cell is designed,with which a spatial-spectral TTConv LSTM2D neural network(SSTTCL2DNN)is constructed for HSI classifica-tion under low complexity.Furthermore,to recover the lost classification performance due to the reduction of network parameters,with the help of tensor local preserving projection(TLPP)algorithm which has the advantages of reducing the dimension of multidimensional data while maintaining their inherent structure and avoiding the loss of local information,the tensor rep-resentation of HSIs is learned to construct a tensor attention residual block(TARB).Finally,by integrating the above two structures,our TACLNN model can take into account the spec-tral correlation and intrinsic structure of HSIs to realize the effective compression of the whole model without classification performance loss.