Researchon Close-shot Identificationfor On-Branch Citrus Fruit Basedon Depth Dataof RealSense Camera

The accurate and rapid identification of fruit targets is a key prerequisite for realizing the practical use of harvesting robots.Existing methods of fruit recognition based on visible light image have inevitable defects of both light sensitivity and difficulty to segment overlapping or occlusion objects.While present methods based on depth information are mainly realize fruit identification by 3D reconstruction of point cloud,image feature extraction or fusion with RBG information,they are much more complex.In this paper,taking citrus fruits that widely cultivated in our country as research objects,a method of fruit identification based on single depth information with RealSense was proposed,and the close-shot identification and experimental verification of citrus fruits were carried out.The main research contents are as follows:(1)In order to achieve the close-shot identification of citrus fruit,the collection and preprocessing of the depth information of citrus objects on-branch were carried out firstly.To provide high-precision,real-time depth information,we selected the RealSense F200 camera as the depth acquisition device after comparing the parameters of several mainstream RGB-D camera,and determined the suitable close-shot detection range for citrus picking robot on recognition operations.According to the original depth point cloud of citrus on-branch obtained by RealSense F200,the redundant background,cavity noise and some unstable foreground objects are removed effectively by the distance threshold segmentation.(2)Considering the three-dimensional geometric characteristics of the fruit,the leaf and branch objects of the citrus are the real sphere,the slice and the thin cylinder,together with the advantage of depth sensors can collect the depth point cloud of the object.A method to identify citrus fruits based on depth-sphere transversal was proposed in this paper.The use of point cloud clustering and regional division method to obtain isolated and adhering area,and the feature extraction algorithms of isolated fruits and adhering fruits were put forward to fruit and leaf in isolated areas in fruits or leaves in touching areas,respectively.In addition,in-depth data processing and fruits recognition strategy of a complex environment were obtained.As the depth-sphere transversal method directly used the original depth value of transversal extraction,which could effectively play RealSense F200 high-precision data advantage and greatly reduce the complexity of the fruit feature extraction process.(3)The result of the close-shot recognition of the fruit on the branches of mature seedless tangerine: by the depth-sphere transversal method,the average success rate was 98.4% in isolated area among 184 tests,the recognition rate was 75.0% and 77.1% among 120 one-fruitand two-fruit tests,while the fruit recognition rate of the complex branches on the leaves of the three fruit leaves was 63.8% among 80 tests.The causal leaf space caused by the difference between the occlusion and a small amount of touch the situation can also guarantee a higher recognition rate which can effectively improve the overlap,occlusion of fruit recognition caused by the great distress.And the fruit leaves can be detected when the depth of the data can be detected by the depth of the data to reduce the truncated circular features plummeted,even if the cutting is still difficult to meet the cross-line criteria,resulting in fruit failure is not successful.(4)In order to further verify the depth-sphere transversal method for fruit recognition has a good applicability for a variety of citrus,five kinds of citrus fruits(Gannan navel orange,Egyptian orange,Yunnan rock candy orange,Sichuan ugly orange and Yongchun citrus)were selected for the experiment by considering the size and shape characteristics of citrus fruit.Followed by 60 groups of light and 60 groups of no-light conditions tests of close-shot identification for single-fruit two-leaves object on-branch.The results showed indicated that the respective recognition rates of Gannan navel orange,Egyptian orange,Yunnan rock candy orange,Sichuan ugly orange and Yongchun citrus were 95.0%,100%,82.3%,90% and 92.5%.And severe adhesion under the causal phenomenon of false negatives lead to all kinds of fruit recognition rate has declined,the respective recognition rates were 68.3%,71.6%,58.0%,70.0%and 68.8%.According to the regression analysis,the effect of fruit size and shape factor on the recognition effect was decreased,and the higher the fruit size,the higher the recognition rate,the closer to the standard circular recognition rate is also higher;the higher recognition rate is also higher.There was no obvious effect on the fruit recognition with or without lighting conditions,which could effectively overcome the light sensitivity.Therefore,the method of fruit identification based on depth-sphere transversal has good applicability to many various citrus fruits,and also provides a new idea for the all-weather fruit identification and positioning operations.In summary,RealSense was taken as a depth of information access equipment for the advantages of low-cost,high-precision and small integration.Taking advantages of less object and simple structure in close range vision,the depth-sphere transversal method of fruit recognition for on-branch citrus fruit was realized.The whole algorithm is much simpler,which can eliminate the adverse influence of partly overlapping occlusion and light change significantly.It has good feasibility to many kinds of citrus.This research has important reference value for the identification of apple,tomato and other near-spherical fruits.