Research On Fine-Grained Seed Image Classification Based On Self-Attention Mechanism And Local Saliency Feature

With the increasing strategic position of seed safety in China,seed classification as a crucial means to ensure the purity of seeds and seed safety has aroused widespread concern.Traditionally,seed identification is mostly completed by professionals,which is costly,inefficient,and unstable.While RGB images can be acquired at a low cost,using data-driven method with seed images to achieve rapid seed classification is favored.Seed image classification belongs to the category of fine-grained image classification due to its small size,wide variety,and small differences in some seeds.This type of classification is more difficult compared to conventional image classification tasks.In addition,it is important to discover an efficient,fast,and low-cost nondestructive seed classification method for the development of wisdom and modernization of agriculture and forestry.Therefore,based on seed RGB images,this paper proposes novel methods for seed image classification using deep learning technology with a high accuracy of seed classification.The main research contents are as follows:(1)Seed characteristics are analyzed and a seed image dataset,LZUPSD is established.The seed images in the dataset were captured by a mobile phone-based recording device and pre-processed with cropping and filtering.This dataset contains 88 species of 4496 seed images,belonging to 23 different families of seeds.Experiments are conducted on the LZUPSD dataset using Convolutional Neural Networks(CNN)and Transformers.The issues such as the number of parameters of the model and the classification effect of different families of seeds are discussed.The experimental results show that the classification of seed images using the techniques mentioned above is effective in identifying seeds.(2)To address the problems of traditional Convolutional Neural Networks being unable to locate saliency discrimination regions and the insufficient extraction of local features by Transformer,a fused network of Transformer and CNN,Weight Transformer Network(WTN),is proposed for seed classification.First,the model uses the Weight Generator(WG)module to generate weights through a self-attentive mechanism to locate and select the regions and features with significant discriminative properties.Then these regions will be fed into Convolutional Neural Network where the features will be captured,improving the disadvantage of Transformer in extracting weak local information.The output vectors from the Convolutional Neural Network and Transformer are combined to one,which reflects the fusion of local and global features.The experimental results showed that the model WTN achieved an improved classification accuracy of96.7% on LZUPSD.(3)To address the shortcomings of traditional Convolutional Neural Networks in acquiring channel and spatial remote interrelationships,the Multiple Self-Attention(MSA)module is proposed.This module uses the weights of the self-attention mechanism to obtain the remote interrelationships between the channels and the space where the hybrid attention mechanism is constructed.The MSA module compresses the space or channels before and after the fused network model,obtaining different channel and spatial weights and assigning different weights to the feature map.This improves the ability to obtain local significant information.VGG and Res Net are used as the backbone networks to construct VGG-MSA and Res Net-MSA respectively,which verify the effectiveness of the module.The results on the Stanford Cars and FGVC-Aircraft datasets show that the network accuracy is also improved to varying degrees with the addition of the MSA module.(4)An intelligent seed recognition system software was developed.Using the proposed WTN,the algorithm is implemented to facilitate user operation.Users can quickly and accurately classify seeds by simply uploading images.It is of great practical importance to meet the needs for the seed classification system software.