| In the process of citrus production, harvest picking is about 40-50% of the amount of whole work. However, in our country, the degree of automation is very low because of the complexity of picking operation. In order to enhance the productivity of labor and ensure the real-time picking of the fruit, it has an important significance to explore and develop citrus picking robot. As the core system of a robot, stereo machine vision system plays a significant role in the robot research, the main work of it is to recognize and locate the fruits accurately, which is the key to accomplish the automatic, fast and correct technology of fruits picking.This paper observed the vision system of citrus picking robot focusing on the citrus on the tree, combining the theory of binocular stereo vision, and aiming at the fruit recognition and location. The main works are as follows:(1) A hardware platform of binocular stereo vision was constructed. According to the experimental place and shooting distance, this research chose the baseline distance in the range of 100～200mm, the depth of measure in the range of 0.5～1.5m, and the angle between two cameras in the range of 30°～45°as the parameters of the platform. Based on the two dimensional DLT method, it calibrated the camera to obtain the internal parameters (including the focal length f, the principal point (u0,v0) and the radial distortion K1,K2) and external parameters (including the rotation matrix and the translated vector which are relative to the world geodetic system observed from the camera). Throughout the above-mentioned parameters, the research has done the anti-calculation towards part of the typical space coordinates. The differences between the calculated results and the coordinate component abstracted from each lattice point are in the range of 0.03～0.41 pixel. Those proved the high precision of the calibrated data from the camera.(2) The preprocessing method towards the image of citrus has been studied. Experimental results showed that R-B color space could divide the fruit from the background better. Therefore, it’s regarded as the best color space. Besides, there are image preprocessing methods such as binarization, removing noise, hole and burr based on median filtering and morphology, recovering the gray value information of the foreground objects based on multiplication, and correction processing towards the couple of binocular images. The corrected images couples are coplanar. They parallel the baseline. Their polar lines parallel each other with the scan line. Furthermore, the matching points have the same value of vertical coordinate.(3) The currently methods of conventional segmentation, similarly circular target segmentation and obstacle segmentation were analyzed. It particularly designed and analyzed the algorithms about the segmentation of the overlapping region, detection and recognition of the target fruit. Based on the region segmentation algorithm, this paper suggested an improved K-means clustering algorithm. It can calculate the number of clusters automatically and iterate rapidly. It also can segment the overlapping regions of fruits better than the traditional K-means algorithm. That conclusion reflected in the contrast between regions which could achieve 95% and the consistence of the internal region and the target count (i.e. segmentation accuracy) which could achieve 98% and 85% respectively. Considered that the processing time to a 696×464 pixel image could achieve 109.85ms, our algorithm is suitable for real-time processing. However, it may lead to over segmentations. On the foundation of the edge-based segmentation algorithm, this paper proposed a self-adaptive Canny algorithm. This algorithm can automatically calculate the parameters of the Gaussian scale and the high/low threshold value. The edge connectivity of it is good. It also has the capacity of weak edge detection, the error of which is 16.6%. Both of the segmentation capacity and the effect are better than the classical edge detection operator. Owing to the limitation of single-scaled edge detection, this paper designed a multi-scaled detection method based on the wavelet and combined it with the Canny operator. That method optimized the segmentation results so that the error of it is only 7.79%. According to the advantages and disadvantages of each segmentation algorithm based on region or edge, an integrated algorithm combined the two above. It limited the over segmentations of regions by edge detection, improved the integrity of the edge and the detection capacity of the weak edge where the fruits were overlapping. Its regional segmentation precision could achieve 92% while the error of edge detection could reduce to 7.06%. This method obtained a preferable effect on the segmentation of overlapping fruits. The divided region was closed to the one of single fruit, and the detected edge of the fruit was complete, continuous and closed. Then on the foundation of fruit segmentation, the author designed a method aiming at the shape character of circular similarly fruit itself. This method is based on RHT method of the subgraph resolution. It could identify and extract the fruit so completely that its correct recognition rate was 83.3%. Its advantage could obviously reflect on the efficiency of time and recognition rate when there were a large number of fruits. Finally, this paper proposed a fruit segmentation algorithm based on disparity map for the first time. The algorithm extended the segmentation basis and information from the color, texture of 2D images to the value of depth in 3D space. Its regional precision could achieve 90% while edge detection error was 5.74%. Experiments proved that the method had a strong capacity on segmenting overlapping regions.(4)The common stereo matching algorithms were analyzed and a new one based on SURF operator and epipolar constraint was designed. Then, the fruit positioning was done throughout matching the fruit regions and the point couples. When the baseline is 105mm, and the depth from target to baseline is 1500±150mm, the error between the calculated and measured distance is less than 3%, which can meet the requirement of picking accuracy.In summary, this paper constructed a binocular stereo vision system for citrus picking, observed reasonable methods of image preprocessing, fruit region segmentation, recognition, extraction and location for citrus on the trees. All of these can not only provide a reference to the recognition and location of vision system, but also make a foundation for picking robot. |
PDF Full Text Request