Research And Implementation Of Fruit Detection And Tracking System Based On Deep Learning

With the rapid development of artificial intelligence and computer vision technology,intelligent agriculture has ushered in unprecedented development opportunities,among which target detection and tracking technologies have shown great application value.The technologies are of great practical significance for realizing fruit yield statistics,automatic fruit picking,fruit growth analysis and automated orchard management.Among them,accurate estimation of orchard yield before harvesting fruits helps growers to better plan fruit harvesting operations and marketing strategies,which helps farmer-growers to bring more economic benefits.In the future,it will help to deploy picking robots working strategies for efficient harvesting tasks and intelligent orchard production.This study uses field oranges as the research object,and the research target is to use fruit detection and tracking technologies to process the collected field fruit video sequences and count the number of oranges fruits.However,in the actual field environment,the orange fruits are small in scale,dense in growth,obscured by leaves as well as severely obscured between orange fruits,which makes it difficult to detect each fruit target accurately,thus affecting the accuracy of yield estimation.To address the above problems,this paper designs a fruit detection and tracking counting system based on deep learning,and the main research contents are as follows.（1）The data terminal-oriented field fruit detection algorithm Orange Yolo.The algorithm is mainly applied to large data terminal platforms with high computing power,such as NVIDIA Geforce GTX 1080 Ti.First,based on the principle of matching the feature map receptive field with the target scale,a network structure for detecting small-scale targets was designed by performing K-means statistical analysis on the field orange dataset with Dark Net53 as the backbone network,on the basis of ensuring the multi-scale target detection function.Second,a dual-attention multi-scale fusion module based on channel attention and spatial attention is designed to efficiently fuse the feature key information of different convolutional layers.This module can enhance the channel features and spatial characteristics of the fused features,adaptively fuse the semantic features of the deep network with the shallow texture detail features,and improve the small-scale fruit detection accuracy.（2）Lightweight fruit detection algorithm Orange Yolo-Light for edge devices.The algorithm is mainly applied to lightweight,portable computing platforms with less computational power,such as NVIDIA Jetson Xavier NX.First,a lightweight network model Light-CSPNet is designed based on the CSPNet network structure,which can substantially improve the detection speed while ensuring good accuracy and realizing real-time inference operations on the lightweight device Jetson Xavier NX.Second,a deep and shallow feature fusion module is designed to fuse shallow,medium and deep features,and use the three different scales to fuse features for fruit detection to substantially improve small target localization accuracy.（3）Field fruit multi-target tracking algorithm Orange Sort.First,by analyzing the movement characteristics of fruit targets in field fruit video sequences,a target position estimation method based on motion displacement similarity is proposed to effectively reduce the frequent change of target ID in the tracking process.Second,a specific tracking region counting strategy is proposed by analyzing the complex occlusion of orange fruits in the global video sequence.By tracking and counting the fruits in the counting region to overcome the fruit double counting problem caused by the complex occlusion of fruits in the global video sequence.The paper utilizes the collected field orange video data for the experiments of detection algorithm and tracking counting algorithm,respectively.The average detection precision of Orange Yolo reached 0.938.The average detection precision of Orange Yolo-Light reached 0.930 on Jetson Xavier NX,with an inference speed of21.3 fps.The average counting error of the fruit multi-objective tracking and counting algorithm Orange Sort in six video sequences is only 0.081.The results show that the detection accuracy and inference speed of the fruit detection algorithm proposed in this paper have reached a high level,which can quickly and accurately complete the fruit detection task in the field environment,and the fruit counting accuracy has also reached a high level,which has practical application value.