Research On The Arm Guidance System Of Tomato Pollination Robot Based On 3D Vision

China is the world’s largest producer and consumer of tomatoes,and tomato production is one of the important ways for farmers to increase income and get rich and earn foreign exchange through export.At present,tomato artificial pollination methods have problems such as strong subjectivity,low work efficiency,and high labor intensity,and improper hormone treatment can easily cause hormone residues or produce deformed fruits.Pollination robots can effectively reduce field labor and improve tomato yield and quality.Most of the existing pollination robots are in the experimental or theoretical stage;therefore,the development of tomato pollination robots is important for reducing production costs,reducing labor intensity,and improving operating efficiency.significance.This paper takes tomato flowers in a large glass greenhouse as the research object,and develops a set of three-dimensional vision-based tomato pollination robot arm guidance system.The main research contents are as follows:(1)Identification and detection of tomato bouquet and flowering period in a greenhouse environment.In order to improve the accuracy of flowering period recognition,a tomato flowering period recognition and detection method based on cascaded convolutional neural network is proposed,taking greenhouse tomatoes as an example.First,the improved feature-based pyramid bouquet extraction neural network(Flower Extraction Feature Pyramid Networks,FE-FPN)is used to extract the local area of the tomato bouquet,and the Prim minimum spanning tree is used to identify and prioritize the extracted bouquet area images,and then press the sequence is input into the improved YOLOv3 network to realize accurate identification and detection of tomato flowers in different flowering periods.Experiments were performed on a tomato bouquet image dataset containing 4 types of flowering periods and a total of 1 600 samples.The method in this paper has a good detection performance for different flowering periods of tomatoes,with an average detection accuracy of 82.79% and an average single sheet detection time of 12.54 ms.The accuracy of each flowering period is 85.71% in the bud period,95.46% in the full bloom period,62.66% in the flowering period,and 88.34% in the early fruit period;compared with Mask R-CNN and Spatial Pyramid Pooling Networks(SPP-Net),The average accuracy is increased by 3.67% and 2.39%,and the recognition error rate is reduced by 1.25% compared with the basic YOLOv3 network.Finally,this paper deploys the proposed method to a tomato pollination robot in a large glass greenhouse environment for actual verification.The method recognizes an accuracy rate of 76.67% in a complex environment and an accuracy rate of 85.18% for removing the missing bouquets.(2)Positioning of the tomato flowers by the pollination robot.In order to provide reliable technical guidance for the pollination of the tomato pollination robot,a method for positioning the tomato pollination flower based on 3D vision is proposed.The RGBD structured light camera is used to quickly obtain the color map and Depth map information of the tomato plants in the greenhouse,and the rapid small target detection YOLOv3 neural network realizes the recognition of the tomato bouquet on the plant,and extracts the area occupied by the pollination bouquet in the image;The active alignment method combines the PCL to align the color map and the Depth image to obtain the spatial point cloud information of the tomato plant;the area outlier filtering method is used to denoise the point cloud,and the bouquet area is finely registered to obtain a high-precision point cloud.Combine the two-way average algorithm to calculate the pollination centroid coordinates of the bouquet 3D box.The positioning test results show that the method has an average positioning success rate of 86.25% in a greenhouse environment,and an average pollination success rate of 71.25%,which can meet the needs of tomato pollinator robots for locating pollination points.(3)Develop a set of tomato pollination robot pollination system based on Open CV and QT framework under Linux environment.The system includes six parts: image acquisition,use of recognition models,point cloud coordinate calculation,arm guidance control,execution of the powder end,and control of robot ground movement.The image acquisition uses a 3D structured light camera to collect the RGB image and the Depth image,and connects to the core controller via USB3.0.The recognition model is based on a convolutional neural network model,and the model is used under the Open CV framework.Point cloud computing includes image alignment,registration,and denoising to obtain spatial point cloud information from PCL.The arm guidance control moves according to the coordinate value of the target bouquet after calibration,which is programmed and generated by QT.After the arm moves to the target position,the core controller sends serial commands to the ARM development board to drive and execute the components for pollination at the powder end.After pollinating all the bouquets in a single field of view,the system sends commands to the robot site system according to the TCP protocol to control the robot to move the plant forward or backward to the planting distance.Then repeat operations such as identification,positioning,and pollination.This thesis uses greenhouse tomato flowers as materials,and applies machine learning and three-dimensional vision technology to design a set of tomato pollination robot arm guidance system based on three-dimensional vision.From the test results,the system can quickly and accurately pollinate tomato flowers in the greenhouse,effectively reducing labor output,and can provide a set of efficient technical support and accurate operations for the tomato intelligent pollination robot to provide an important basis for accurate pollination.