| The jujube fruit is rich in nutrients and minerals,which has great practical value in Chinese medicine.Jujube has been cultivated for thousands of years in China,and more than 900 germplasm resources have been discovered so far.Due to the similar characteristics of different varieties of fresh jujube,identifying accurately jujube varieties can be challenging.With the development of image processing technology,it is more and more convenient to obtain the image of jujube fruit by using mobile phones and other mobile devices,which lays a good foundation for intelligent recognition of jujube varieties.At present,there are relatively few studies on the recognition of multiple jujube varieties in the natural environment.In order to meet the demand for automatic,rapid and accurate identification of fresh jujube varieties,this paper takes 20 jujube varieties as the research object and makes an in-depth on automatic recognition of jujube varieties in the natural environment based on deep learning technology.The major research contributions are as follows:(1)In order to solve the problem of large intra-class differences and subtle inter-class variation of jujubes in the natural environment,a jujube variety recognition model based on metric learning is proposed,which is named JCN.JCN model takes a positive set(images of the same jujube variety)and a negative set(images of the other varieties)as input,and uses VGGNet-16 network as the basic network to design a dual-flow structure.This structure learns rich jujube image features through shape and fine-grained features learning branches.It is optimized by using Coupled cluster loss and Focal loss functions,which can not only improve the intra-class cohesion and expand differences between classes,but also reduce the interference of unbalanced sample distribution.Through experiments on jujube data sets,the overall accuracy of JCN model is 84.16%,which is 5.9～23.32 percentage points higher than SVM,AlexNet,VGGNet-16 and ResNet-18 models.The average accuracy of cylindrical and circular jujube varieties with high shape similarity is 87.33%and 81.33%,respectively.The results show that JCN model can effectively improve the identification accuracy of jujube varieties.(2)In order to solve the problem of unsatisfactory recognition result caused by the similar shape of jujube fruit in the image set,a jujube variety recognition model based on feature fusion is proposed,which is named JC-FFN.The model takes jujube fruit image,jujube leaf and its texture image as input,and ResNet-18 network is used as feature extractor to design a three-branch network structure.Leaf shape and texture features are added together,and then they are concatenated with jujube fruit features.The fusion features obtained from the three branches are input to the full connection layer and Softmax classification layer to output classification results.The comparative experiments results show that the overall recognition rate of JC-FFN constructed by feature fusion at the convolution layer reaches 91.68%.It increased by 7.52%compared with JCN model,especially the average accuracy of round and oblong shape jujube varieties improved by 9.5 and 10.72 percentage points respectively.It is proved that JC-FFN can effectively avoid the interference of different jujube varieties with similar shape and improve the classification accuracy of jujube varieties.(3)In order to address the problem that the general neural network cannot accurately locate salient discrimination regions in jujube fruit and leaf images and cannot effectively extract the features of these regions.Based on JC-FFN model,the JC-MAN model is constructed by embedding spatial attention and channel attention mechanism.Furthermore,the convolution layer fusion of JC-MAN model is changed to classification layer fusion to build the ensemble network model.Through comparative experiments,the results shows that JC-MAN model can accurately locate the significant discrimination areas such as jujube fruits and leaves in the image,and the recognition accuracy reaches 94.77%.Compared with JC-FFN model,the overall recognition rate improved by 3.09 percentage points,especially the Yuanlingzao has increased by almost 8 percentage points.The comparative experiments of different integration strategies show that the accuracy of the weighted average method achieves 95.21%when the weighted value is(α1,α2,α3)=(1,0.7,0.4),which indicates the validity of the ensemble model. |
