Research On Point Cloud Recognition Network And Scene Analysis System In Zanthoxylum Bungeanum Picking Task

In recent years,with the rapid development of machine vision,the application of machine vision to fruit and vegetable picking has been promoted.It can not only achieve accurate fruit and vegetable identification and location but also increase the flexibility and intelligence of the picking robot in time and effectiveness.At present,researchers mainly rely on twodimensional images to achieve Zanthoxylum bungeanum recognition.However,the recognition of Zanthoxylum bungeanum based on the color image will be affected by uneven illumination in the natural environment,resulting in low recognition accuracy and robustness.Moreover,the fruit of Zanthoxylum bungeanum is small and dense,and the phenomenon of mutual occlusion is very serious,which makes it very difficult to segment and locate the fruit of Zanthoxylum bungeanum based on two-dimensional image.The accuracy of recognition and locating can be insured only when the camera is close to Zanthoxylum bungeanum.Compared with two-dimensional images,in three-dimensional space,there is still a certain distance between the fruits of these occlusion,which can be separated by depth information.Thus,it is necessary to use depth images to identify and locate Zanthoxylum bungeanum in a complex environment.Combined with the advantages of camera and three-dimensional laser scanner in Zanthoxylum bungeanum recognition,and the study in this thesis was applied on Hanyuan Zanthoxylum bungeanum in outdoor environment.It proposed the distant-shot and close-shot combined vision system for Zanthoxylum bungeanum recognition,and the work focused on the distant-shot one.First of all,based on the 3D point cloud data obtained by the Leica BLK360 3D laser scanner,the realization process of Zanthoxylum bungeanum prospect recognition and positioning was designed,and the hand-eye calibration,point cloud data preprocessing and Zanthoxylum bungeanum point cloud data set production involved in the process were studied.The singular value decomposition method was used to solve the hand-eye relation matrix,the manual segmentation method was utilized to cut the point cloud data,and the statistical filtering method was used to eliminate the noise in the point cloud.Finally,the voxel grid method was used to downsample the data to complete the point cloud data pre-processing.Based on the collected Zanthoxylum bungeanum point cloud data and subsequent data enhancement,the production of the Zanthoxylum bungeanum point cloud data set was realized to provide data support for subsequent in-depth learning.Secondly,aiming at the defect of fruit and vegetable segmentation only using shape and color features in the current fruit and vegetable recognition based on point cloud data,and considering the particularity of fruit shape characteristics of Zanthoxylum bungeanum,the deep learning method was used to segment the point cloud of Zanthoxylum bungeanum.Through the in-depth study of the Point Net++ network,、the input structure of the network was improved and the input feature dimension was extended to 9 dimensions.In addition,in order to reduce the impact of uneven illumination on the segmentation accuracy in the natural environment,the RGB space where the point cloud color information was located is converted to the HSV space.The improved Zanthoxylum bungeanum point cloud data set was used to verify the effectiveness of the network improvement.the experimental results showed that the average accuracy of the improved network segmentation based on normal vector and HSV was 92.97%,which was 18.69% higher than that of normal vector-based semantic segmentation and 9.35% higher than that of HSV-based semantic segmentation.For the average intersection ratio m Io U,the improved network m Io U based on normal vector and HSV was 83.77%,which was 13.59% higher than that of normal vector based m Io U and 8.06%higher than that of HSV based m Io U.Finally,through the analysis of the requirements of Zanthoxylum bungeanum distantshot identification and positioning system,based on Qt,PCL point cloud database,Leica SDK,other related tools and open-source algorithm libraries,a set of Zanthoxylum bungeanum point cloud data processing software was designed and developed,which realized the functions of hand-eye calibration,point cloud data preprocessing,communication with the scanner and the manipulator in the process of picking Zanthoxylum bungeanum.The functional modules of the software were verified by field experiments,in which the overall error of hand-eye calibration was less than 5 mm.In the positioning accuracy of Zanthoxylum bungeanum,the error range was-7 to 9 mm in the X coordinate,-6 to 8 mm in the Y coordinate,and-8 mm to10 mm in the Z coordinate.The overall experimental results of the software system showed that the designed system could initially meet the requirements of completing the visual recognition and positioning task of Zanthoxylum bungeanum.