Research On Hyperspectral Image Classification Algorithm Based On Transformer

With the rapid development of sensor technology,it is now possible to obtain high resolution hyperspectral images,which provides a technical guarantee for the research and analysis of hyperspectral image data.It is important to make reasonable use of these data to achieve effective identification and classification of features in certain fields through the development of appropriate networks.In the analysis of remote sensing data,hyperspectral image classification is a relatively critical part.The extracted features contain useful information of the images,and the classification results can not only meet the practical needs,but also provide favorable data information and technical support for other applications.Among the existing deep learning methods,Convolutional Neural Networks(CNNs)are widely used in hyperspectral image classification because of their excellent local context modelling capability,where the method based on joint spatial-spectral feature extraction has achieved better results in classification.In hyperspectral image(HSI)spectral data,the correlation of information between adjacent bands is important for the analysis of different features with similar spectral characteristics;however,in the traditional CNN-based methods for processing hyperspectral image spectral data,CNNs fail to well mine and represent sequence attributes with several hundreds of inter-spectral band features due to the limitations of the size of convolution kernels and the depth of convolution in the network.Based on these problems,the following research work is done in this paper.(1)In order to extract as many discriminative features as possible,this paper proposes a spatial-spectral joint feature extraction network(SST＿Like)fusing Transformer and VGG networks for hyperspectral image classification,which uses a VGG network with 3D convolution kernel to extract spatial spectral features,a Transformer network with a sparse constraint strategy is introduced to extract continuous inter-spectral information,finally,the resulting joint spatial-spectral feature information is used in a multilayer perceptron(MLP)to complete the feature classification task.The SST＿Like network model was tested on three open datasets of hyperspectral images,and it can be seen from the results that it can extract deeper,discriminative features and greatly improve the classification performance compared with the traditional CNN-based hyperspectral image classification algorithm.(2)Considering that the VGG network can only extract features layer by layer and the global self-attention-based computation of the Transformer network,which is computationally intensive and has poor classification timeliness when the image resolution is high and there are many pixel points,this paper proposes a new network(Re STrans)based on Swin Transformer and 3D residual multilayer fusion network for hyperspectral image classification.In the Re STrans network,the 3D residual multilayer fusion network is used to extract the deep-level joint spatial-spectral features,and then the Swin Transformer network module based on a self-attentive mechanism is used to capture the relationships between consecutive spectra.The experimental results shows that the network has superior classification performance compared with other single feature extraction networks,and also show that taking into account the correlation between each spectral band based on the joint spatial-spectral feature extraction can lead to higher classification accuracy.