Research On Hyperspectral Remote Sensing Image Classification Method Based On Sparse Transformer

Convolutional neural networks have strong image spectral-spatial feature extraction capabilities,which can effectively capture the spectral information and spatial context information in images,and have been widely used in the field of hyperspectral remote sensing image classification,achieving good results.However,CNN extracts spectral-spatial features by means of multi-layer local convolution,which is not direct in modeling global long-distance dependence,and is difficult to capture the global dependence relationship in images with a large spatial span.For this reason,Transformer is introduced into the field of hyperspectral remote sensing image classification,using its multi-head self-attention mechanism to capture the long-distance dependence relationship in the spectral sequence,which can enhance the model’s expressive ability and generalization.However,when facing class-imbalanced hyperspectral remote sensing images,traditional Transformer is difficult to train fully,and has limitations in mining local spatial detail information,and is prone to misclassification when facing same-spectrum different-object and different-spectrum same-object problems.To address these problems,this paper explores based on sparse Transformer,and the main contributions are as follows.(1)In order to solve the problem of class imbalance in hyperspectral images,a dual-branch sparse Transformer hyperspectral remote sensing image classification model is proposed,which introduces sparse multi-head self-attention in Transformer,selectively focusing on key features that are conducive to class discrimination,reducing interference information,and enhancing the model’s representation learning ability.In addition to the traditional sparse Transformer branch,a rebalancing branch is added,which takes anti-sampling data as input,balances data distribution,and improves classification performance.Finally,a dual-branch learning strategy is introduced to fuse the dual-branch that takes into account representation learning and classifier learning.(2)In order to fully extract local-global spectral-spatial information,alleviate the misclassification problem caused by same-spectrum different-object/different-spectrum same-object,a hyperspectral remote sensing image classification model based on CNN-Transformer hybrid feature extraction is proposed.Among them,the CNN branch mainly focuses on multi-scale local detail information,and introduces Ghost module to reduce computation and memory consumption.The Transformer branch uses sparse multi-head self-attention mechanism to focus on global information and model long-distance dependence of spectra.Through the multi-level multi-scale feature complementarity of these two branches,the model can fully mine local-global spatial information,improve classification accuracy and alleviate the misclassification problem of same-spectrum different-object or different-spectrum same-object.(3)Design and implement a hyperspectral remote sensing image classification system based on Transformer,and verify the proposed method on four common datasets,achieving stable and accurate classification.