The Method Of Obtaining Spatial Coordinates Of Tomato Cluster’s Picking Point Based On Binocular Vision

The process of tomato harvesting is extremely labor-intensive,in order to reduce the labor demand of tomato harvesting,some successful tomato automatic harvesting robot is put into use,but it is all for single fruit.Comparing manual labor,The advantage of single fruit picking efficiency is not obvious.Taking tomato cluster for picking can improve efficiency.The tomato picking robot in the process of picking tomato clusters needs space coordinate of picking point,in response to this demand,the contents of this research are as follows:(1)The study compared the component diagram of tomato cluster images in HSV and RGB models.The image of fruit cluster was divided into three regions: green leaf,fruit stalk and fruit.The mean of each region’s color component was calculated,it was found that the optimal image segmentation was by R-G component difference.R-G histogram was double peaks distribution,so the Otsu algorithm was used in image segmentation.The morphological method was taken to remove interference and the contour of fruit clusters were extracted by using Canny algorithm.Finally,a series of recognition features such as the centroid of the fruit cluster,the aspect ratio of the external rectangle,the contour perimeter and the pixel area were calculated and counted.(2)According to the features of fruit clusters,the ROI of fruit stem was determined and the region of fruit stem was changed into HSV image.Under the H channel,the fruit stem image become gray images.The binary image was obtained by fixed threshold method.By using threshold method,morphological method,transverse scanning pixel to get single stem.Compared the effect of extracting skeleton by distance transformation thinning algorithm,Zhang thinning algorithm and morphological thinning algorithm.The results showed that the Zhang thinning algorithm had the least skeleton burr,and the connectivity was better.The Harris algorithm was used to extract the corner points of the skeleton,and the corner points were sorted from large to small according to the vertical axis coordinates,Extracting pixel coordinates of feature points by the formula of the relation between corner and feature point.(3)The experiment of extracting space coordinate was made by binocular camera,the internal and external parameters of camera were calibrated by the Matlab’s calibration tool,and the stereo matching was performed by the picking feature points combined with epipolar constraint and disparity range constraints,etc.The depth information was obtained by formula and compared with the measured value.The results show that the depth calculation error is within 10 mm in the distance of 250~900mm.(4)Analysis and study of the reasons for the big error between the calculated value and the measured value,in order to further improve the accuracy,the error compensation mechanism was introduced.Linear interpolation was performed by building a database,the experimental results show that the error depth is improved by 2.92 mm after error compensation.Finally,the three-dimensional coordinate was calculated by geometry principle and compared the coordinate value measured by distance measuring instrument.The experiment shows that the coordinate error in 250~900mm range is less than 6.8mm,which meets the requirement of end effector’s picking.In this research,Visual platform and method flow for spatial positioning experiment of tomato cluster’s picking point were set up and designed,and the space coordinate of tomato cluster’s picking point was acquired under the natural light.