Research On Identifying And Locating Apple In Orchard Based On Neural Network And 3D Vision

Agriculture in China has a strong competitiveness in fruit cultivation,but according to the development features and trends of China's fruit industry,we still have some urgent problems to be solved,among which fruit picking is one of the most time-consumine and most expensive steps in the whole industry.In China,the picking process is accomplished by human resources.However,with the aging of the population and the increase of production costs,there is an urgent need to introduce automation based on Artificial intelligence into fruit and vegetable picking.Based on the specific goal of orchard apple,two key steps of identification and location in the process of harvesting were studied in this paper.The main work and conclusions are as follows:(1)The target detection network based on neural network is used to recognize apples.The algorithm of SSD(Single Shot Multibox Detector)algorithm is analyzed in depth.It is pointed out that the characteristics required are different due to different emphasis of classification tasks and location tasks.In view of this contradiction,this paper aims at improving the network structure of SSD,and proposes the FSF-SSD algorithm,which separates the two tasks of category prediction and location prediction to a certain extent,and carries out feature fusion operation on the feature map of location prediction task,combining the upper semantic information with the lower appearance features,so as to obtain more locations and detailed information.(2)Based on the binocular stereo vision theory,the spatial position of apples is located on the basis of recognizing apples.Based on the in-depth study of 3D vision theory,a stereo matching algorithm combined with FSF-SSD algorithm is designed.The algorithm calculates the IoU(Intersection over Union)of the regression box in the image to be matched and the IoU maximum regression box number in the reference image and the image to be matched.If the IoU maximum regression box number obtained twice is the same,the matching is successful.On the basis of successful matching,the parallax method is used to locate the apple,and the three-dimensional coordinates of the apple are obtained to provide data support for the grasping of the manipulator.(3)Based on the above research,the experiment is carried out in this paper.For orchard apple recognition,the average detection accuracy of FSF-SSD algorithm is 95.25%,which is improved by 2.31%on the basis of SSD.In order to prove the migration ability of the model,this paper also validates the Pascal VOC 2007 public data set,with an average detection accuracy of 74.92%,and an improvement of 1.03%on the basis of SSD.For apple localization,the matching algorithm is validated firstly.The experimental results show that the matching accuracy of multi-fruits scenes with less occlusion is 97.8%,and that of partially occluded scenes is 96.8%.On the basis of successful matching,the disparity positioning experiment is carried out.The experimental results show that the accuracy of disparity positioning method based on center point coordinate matching can meet the error requirements of the manipulator.