Research On The Visual Detection And Localization Technology Of Tea Harvesting Robot

In recent years,tea industry has been developing in the direction of ecological planting,product standardization,brand management,market internationalization,and industrial integration.Although the overall trend is good,there is still a big gap between the production methods and technical level of the tea industry and the requirements for the advancement of agricultural modernization.One of the representative problems is that the plucking of high-quality tea is mainly labor-intensive manual plucking,and it is faced with the contradiction of increasing output but increasing shortage of labor,which seriously restricts the development of high-quality tea industry.Therefore,it is of great significance for the development of tea industry to solve the problem of difficulty in plucking tea through mechanization and intelligence plucking.However,unlike the picking of other crops,the field automatic plucking of high-quality tea still faces many challenges: the dramatic changes in the ambient light of the tea garden in the field,and the similarity of the target color and the background of the old leaves bring challenges to the automatic detection task of the high-quality tea;The shape of the tips is different,and the small plucking parts of the tea stems make it difficult to locate the plucking point,and the inevitable occlusion problem in the field also poses a huge challenge for the localization of the plucking point.What’s more,the characteristics of tea shoots with different heights and high growth density put forward demands on the harvesting motion efficiency of the robotic arm.This paper takes the field high-quality tea intelligent harvesting robot as the research object,and focuses on the object detection,target localization and plucking planning technology in the tea harvesting process.The subject research involves theoretical research,technical implementation and experimental verification in image processing,deep learning,point cloud processing,motion planning,robotics,etc.The main research contents and research results of the paper are as follows:(1)Research on rapid detection technology of tea shoots in an unstructured environment.To achieve accurate detection of near-colored shoots under natural light,the field tea images were collected to establish a tea shoot data set,and the tea shoot areas were detected based on the YOLO deep learning framework.To facilitate the deployment of deep learning models on low computing power platforms for rapid detection of tea shoots,a deep learning model compression method of channel pruning and layer pruning is used to prune and optimize the tea shoot detection model.According to the γ coefficient distribution of the BN layer,the channels and layers with a low contribution to the network are eliminated to reduce the model size and improve the model detection speed under the condition that the model accuracy does not drop sharply.Finally,the model was deployed on a low-computing device(Jetson NX)for field shoot detection experiments.The experimental results show that the m AP and recall are 90.01% and 83.99%,respectively,for the detection of tea shoots in the field,and the detection frame rate is15.92 fps,which meets the accuracy and speed requirements for fast and efficient detection of high-quality in the field.(2)Research on the localization technology of special-shaped small targets based on a cylindrical envelope in a tea garden environment.To solve the localization problem of special-shaped small targets in complex agricultural environment,a fast localization algorithm of tea shoots plucking position based on a cylindrical envelope was proposed in combination with the tea harvesting method.Firstly,by comparing and evaluating the effect of RGB-D camera shooting images of different sensing schemes,the sensors with suitable sensing schemes are screened.Then combined with the target detection area information,the RGB-D camera is used to obtain the local point cloud of the target area for point cloud processing.Then,a cylindrical envelope algorithm for tea plucking is proposed,and the calculation of the plucking pose of tea shoots is realized by combining the principal component analysis method and the growth characteristics of tea leaves.Finally,the experimental results of the field tea shoot localization show that the proposed algorithm can realize the rapid plucking position calculation and attitude estimation of the tea shoot,which meet the requirements of accuracy,rapidity,and robustness of the high-quality tea plucking localization in the field.(3)Research on the spatial localization and reasoning technology of tea shoots based on topological structure analysis.To improve the localization accuracy and localization robustness of occluded shoots in complex environments,a new localization method was proposed based on the topological shape analysis of the tea shoots.Firstly,based on 2D image skeleton analysis,a 3D shoot skeleton extraction algorithm for leaf crops was proposed,and then combined with the binary tree data structure,a localization method based on 3D skeleton binary tree topology analysis was constructed.To achieve accurate localization in the case of visible intersections and accurate inference in the case of invisible intersections.Finally,the multi-view topological analysis and localization experiments were carried out by using the rotating slide table,and the results show that the proposed algorithm has high localization accuracy and robustness,which can achieve the correct inference of the invisible plucking points of tea shoots under occlusion conditions.(4)Research on the motion planning technology of the manipulator during the plucking operation.To ensure smooth and fast movement during the plucking operation,the plucking sequence problem was converted into a 3D traveling salesman problem for optimization solutions according to the efficiency requirements of tea shoot plucking.Then,the planning of the position and attitude of the end motion trajectory is proposed based on Bezier and spherical linear interpolation.Finally,the simulation results show that the parallelogram motion based on a genetic algorithm can effectively improve the harvesting efficiency of high-quality tea,and the pose interpolation based on Bezier position interpolation and spherical linear interpolation can effectively ensure the fast and stable operation of the end effector during the moving process.(5)Localization experiments evaluation and plucking experiments analysis of highquality tea plucking robot.To verify the effectiveness of the plucking algorithm and evaluate its localization performance,a robot hand-eye system was built,and calibration experiments were carried out.Then,the shoot localization experiment and evaluation were carried out based on the cylindrical envelope localization algorithm and the binary tree topology localization algorithm,respectively,and the comparative analysis was carried out for the cylindrical envelope based on position localization and pose localization.Then combined with the actual harvesting needs of the tea garden,a highquality tea harvesting robot prototype was built to carry out the field harvesting experiment,and the harvest results were statistically analyzed.The field experiments showed that the success rates of detection,localization,and motion plucking are 85.16%,85.15%,and 80.23%,respectively,the whole process harvesting success rate is 53.91%,and the average harvesting time of each shoot is 2.229 s.This shows that the proposed method can realize the accurate and rapid harvesting of tea shoots and can provide guidance for intelligent plucking of high-quality tea.