Detection And Localization Of Fruits On The Apple Tree Based On Computer Vision

With the rapid development of urbanization,the existing mode of apple production has not satisfied the increasingly scale of apple planting due to the labor shortage and the rising costs in rural area.In order to enhanced robot’s accuracy rate and efficiency of fruit operation,technologies of computer vision,digital image processing,pattern recognition and machine learning were used to study the methods of apples detection and localization in natural scene.The main research works were listed as follows:(1)Information acquisition and characteristic analysis of apple images were studied.First,growth characteristics of apples in young fruit stage and fruit maturity stage were discussed.Second,features of commonly used color spaces in image process were analyzed,and then color signature and difference of apple images in color components were compared after processed that with 15 color components from 5color spaces.Third,quantitative statistical approaches were used to analyze the gray scale of apples,branches,leaves,stems and sky in images.(2)Methods of robotic detection and localization of young green apples in orchard environment were studied.First,adaptive green and blue chromatic aberration(AGBCA)map was designed and combined with the iterative threshold segmentation algorithm(ITS)to detect region of interest(ROI)contains potential apple fruits pixels.Then potential fruits were identified by using an improved circular Hough transformation(CHT)after morphological operation and blob analysis of the results obtained from AGBCA and ITS which kept as many potential apple fruits pixels as possible.Finally,a kernel support vector machine(SVM)classifier optimized by using grid search optimal algorithm was built to remove false fruit objects based on histogram of oriented gradient(HOG)feature descriptor.(3)Methods of robotic detection and localization of overlapped apples in natural scene were studied.First,the region of apple objects in image was detected by using the improved GrabCut algorithm based on visual saliency.Second,the key corners on the apple outline were found among the Harris corner points by calculating the extremum of distance curve between every corner point and the centroid of the overlapped fruits,then the contour of unblocked apple in overlapped area was extracted by utilizing the Canny algorithm in the detection window located by key corners.Third,Y-junctions detection algorithm was utilized to separated contour of individual apple from overlapped apples,and localized the complete contour of unblocked apple.Finally,the missing contours of apples were reconstructed by using distance least square algorithm.(4)Methods of robotic detection and localization of apples occluded by branches and leaves in natural environment were studied.First,ROI of the occluded apples image were segmented by improved K-means algorithm based on the searching of heuristic clustering center.Second,QuickHul algorithm and convex hull theory were introduced to construct the continuous convex hull of apples,the adaptive control algorithm for boundary extraction were used to remove fake boundaries of apples and acquired true boundaries.Third,according to detected contours of apples,the missing apple contours that have different curvature and length in occluded area were reconstructed by using the Euler arcspline.(5)The software system of apple detection and localization were designed and developed.Based on MATLAB and GUI toolkit,the software system was developed for apple detection and location.The multiple step process mode and the one-click process mode were designed respectively to meet different needs of users.The main innovations were summarized as follows:(1)According to the characteristics of young green apples in natural illumination,the segment method that integrated with AGBCA and ITS was designed,which significantly enhanced detection performance of ROI of apple image.The HOG-SVM model was introduced into young apple recognition,which improved the identification ability of young apples.(2)The automatic GrabCut algorithm improved by visual saliency was designed,that make up for the deficiency of traditional GrabCut that requires manual operation.The proposed method of key-corner detection was compensated the imperfection of real contours obliteration in overlapped area when separated overlapped apples and avoided the noise interference caused by global edge detection.(3)The QuickHull algorithm and the Euler arc spline were introduced for detection and localization of apples that occluded by branches and leaves,which preliminarily solved the problem of adaptive localization and reconstruction of missing apple contours that have different curvature and length.