Research On Spatial Location And Grab Of Sugarcane In Field Based On Binocular Vision

Due to the landform characteristics of Guangxi,lodging characteristics of sugarcane and the high requirements of sugar industry for sugarcane harvesting,traditional sugarcane harvesters are often difficult to apply because of a series of problems such as high impurity content and broken head rate.Therefore,it is of great significance to abandon the traditional extensive sugarcane harvesting mode and study the intelligent sugarcane harvesting robot for refined sugarcane harvesting.In this paper,aiming at the problem of simulating human’s spatial location and grasping of sugarcane in the field during intelligent sugarcane harvesting,a binocular vision sugarcane spatial location and grasping harvesting method based on improved YOLOv5 m is proposed.Taking sugarcane in the complex natural environment in the field as the research object,the stem node recognition model is obtained by training the data set of sugarcane stem nodes in the complex field and improved to recognize the complex morphological stem nodes.Binocular vision and stereo matching extraction technology based on SIFT features are used to obtain the location of sugarcane stem nodes in threedimensional space,and the posture of the whole sugarcane is restored through the three-dimensional coordinates of multiple stem nodes.Finally,the coordinates of grabbing points in the world coordinate system are transformed and provided to ROS system to control the mechanical arm to finish sugarcane grabbing.The main research work of this paper is as follows:(1)Using Zhang Zhengyou’s camera calibration method,the left camera and the right camera are calibrated separately,and the internal reference matrix and distortion coefficient of the two cameras are obtained,and the error analysis of the calibration results is carried out by re-projection;The left and right cameras are calibrated jointly,and the rotation matrix and translation vector of the left camera relative to the right camera are obtained.(2)A high-quality dataset of 9,000 images of sugarcane stem nodes with complex form was established by collecting images of sugarcane stem nodes with different time,different form and different numbers in the actual field environment.At present,the latest YOLOv5 m algorithm is used to conduct deep learning training on this data set,and the recognition model of sugarcane stem nodes with complex form in the field environment is obtained.Aiming at the problem of high resource utilization rate of this model,the channel pruning technology in network slimming is used to improve it.Under the condition of keeping the AP basically unchanged,the complexity of the model is greatly reduced,including model Params,FLOPs and model size,so that the improved YOLOv5 m model can be successfully deployed on the embedded chip jetson nano of harvesting robot.In addition,through the data precision calibration and interlayer fusion in Tensor RT,the deployed model is accelerated,the acceleration effect is more than 6 times,and the position information of sugarcane stem nodes in pixel coordinate system is obtained.(3)By stereo-correcting the binocular camera,the line alignment of pixels of images taken by left and right cameras is completed.In the selection of stereo matching algorithm,the stereo matching algorithm based on regional gray level and the stereo matching algorithm based on feature points are compared.The results show that the stereo matching algorithm based on SIFT feature points is more suitable for stereo matching of stems with complex forms.RANSAC algorithm and epipolar constraint are used to eliminate the wrong matching point pairs,and the homography matrix between the left and right camera images is obtained from the optimized matching point pairs.By combining the improved YOLOv5 m sugarcane stem node recognition model with homography matrix,the spatial location of sugarcane stem node is realized by binocular vision principle.Experiments were designed to verify the spatial location accuracy of sugarcane nodes with complex form in the field environment,and the location accuracy in X/Y/Z directions was verified respectively.(4)A reduction method of stem node position based on various constraints is put forward.By combining the obtained spatial coordinate information of stem node,the growth posture of the whole sugarcane plant is reduced by this reduction method,and the fitting expression of the growth posture of sugarcane plant in camera coordinate system is calculated.(5)Through the Robotics Toolbox toolbox of MATLAB,the D-H modeling and forward and inverse kinematics analysis of the simulated grasping manipulator are carried out.Through the hand-eye calibration experiment between the camera and the mechanical arm,the hand-eye calibration matrix of the sugarcane harvesting robot is obtained to complete the transformation of sugarcane plant growth attitude from the camera coordinate system to the world coordinate system.The "Move It!" of the robot operating system(ROS)is used.The function package completes the motion planning and trajectory output of the grasping point of the manipulator,receives the joint angle information of Jetson nano from the upper computer through the lower computer Arduino,and uses the steering gear controller to control the steering gear motion simulation to realize the grasping of sugarcane by the manipulator.