Deep Learning Driven Crop Species Identification Method With Dual Viewpoints

Fast and accurate crop classification is the basis for estimating crop information and improving farm management.Applying deep learning models based on time-series remote sensing data has become the main method for crop classification.The cloud computing platform like Google Earth Engine(GEE)has powerful computing power to facilitate the completion of huge deep learning model operations.To address the problems of difficulties in combining cloud computing platform and deep learning and the unknowable role of each component in deep learning models on classification performance,this study establishes a deep learning method(DuPLODA)combining attention mechanism under the dual perspective of convolutional neural network(CNN)and recurrent neural network(RNN)for crop classification.Taking the river-loop irrigation area in Inner Mongolia,China as an example,time series data from Sentinel-2 satellite images are fused based on the GEE cloud computing platform,and then DuPLODA is used for crop classification,and the impact of each component in the model on the classification performance is quantified using ablation experiments.This study provides a feasible crop classification method combining cloud computing and deep learning at the irrigation district scale.The main findings are as follows.(1)The major crops grown in the river-loop irrigation area can be classified into 2 major categories based on the similarity of time series curves.Sunflower,maize and green pepper are in one category,and wheat,tomato,zucchini and others are in another.The crops in category 1 had similar trends and could be distinguished only by differences in spectral characteristics during the lush growth period and minor changes in other time periods.In category 2,wheat and other crops have different trends and are easier to distinguish,while tomato and zucchini have similar time series curves.(2)The classification performance of using RF and SVM classifiers for different features and time series increases as the number of selected features increases.The classification performance is optimal when all features are selected.the overall accuracy of RF is 92.49% and the Kappa coefficient is 89.20%;the overall accuracy of SVM is 93.24%and the Kappa coefficient is 90.29%.the classification performance of SVM is better than RF for all features.and the classification performance of RF is better when less data are used.(3)The DuPLODA model showed better classification performance than the existing mainstream classification models,with the highest OA and Kappa coefficients(OA = 95.86± 0.4%,Kappa = 94.11 ± 0.57%)and the second highest F1 score(F1 score = 93.08 ± 0.86%)for the DuPLODA model.In comparison with the 5-class comparison model in spatial distribution details,DuPLODA can better classify multi-class crops in cross-cropping areas and has the same excellent classification performance for large cropping areas,which can effectively avoid misclassification and pretzel phenomenon.DuPLODA provides a deep learning classification method for a single pixel for learning generalizable features from time-series remote sensing data.(4)The CNN branch and RNN branch in DuPLODA have complementary properties,and the organic combination of them improves the crop classification performance.The ablation experiment quantifies the impact of each component of the deep learning model on the classification performance,and the t-SNE visualization reveals the feature learning process of the deep learning model.the CNN branch in DuPLODA has a decisive role in the classification performance,while the RNN branch will assist the CNN branch to improve the classification performance of the model,and the dual-view architecture can be used to optimize the ability of the model to extract features.the attention of the CNN branch The attention mechanism in the CNN branch can effectively improve the classification performance of the model,while the addition of the attention mechanism in the RNN branch does not promote the classification.Adding auxiliary classifiers to the dual-view model can also improve the classification performance of the model.t-SNE visualization results illustrate that the learning process of features is different for different models.DNN models have formed distinct clusters at shallow layers,RNN branches form distinct clusters only at fully connected layers,and CNN branches can observe the overall process of forming clusters.