Vision Information Acquisition For Fruit Harvesting Robot And Development Of Robot Prototype System

The fruit and vegetable harvesting robot worked in unstructured environment of greenhouse. Under the condition of complex light, the robot should position the target accurately and avoid touching the stem and leaf. And then the fruit would be grasped and cut nondestractively. Information acquisition of fruit was the difficulty and key of harvesting robot. According to the corlor difference between target and background, the study object could be classified into similar-color fruit and different-corlor fruit. Two representative fruits——strawberry and mini-watermelon were selected as study objects, and the havesting information acquisition of fruit was studied, and a fruit havesting robot system was developed. The techniques of machine vision, image processing, spectral analysis were applied to solve fruit recognition, spatial matching and three-dimensional coordinate location. At last, a modular prototype of havesting robot system was developed, and a trail of the robot was done. The main research contents and conclusions were as follows:(1) The information acqusition method of havesting strawberry based on color and morphology was studied. Firstly, according to the color distribution characteristics of fruit and background in four common color spaces, color components (R and G) were selected to image segmentation. Secondly, the region occluded by fruit calyx was inpainted through contour compensation, and the whole fruit region was recognized. Thirdly, the near-color background interference was filtered by image mask processing, and green region of immature fruit was extracted to judge the strawberry immaturity level. Then, according to the structure of robot end-effector and the spatial pose characteristics of strawberry stem, the picking line and rectangle region of interest were set to extract the image coordinate of picking point. At last, the experiment resut showed that the correct recognition rate of fruit was94.2%, as well as the rates of the picking point was93.0%.(2) The information acqusition method of havesting mini-watermelon based on near infrared image was studied. Firstly, the spectral characterisics of mini-watermelon’s fruit, stem and leaf were compared, and optimal wavebands near850nm were selected to capture near-infrared gray image. Secondly, According to the characteristics of fruit and backgroud’s gray pixel distribution, an improved Otsu algorithm was developed to segment image. Thirdly, a matching template likes as circle was proposed to detect fruit region concentrated, and the region adhesion and small area interference were reduced effectively. Then, according to the characteristics of spatial pose and relative position of the fruit and stem, the image coordinate of cutting point was located by "block-location method". Finally, a trail was done to test the algorithm of acqusiting mini-watermelon’s harvesting information, and the result show that the average correct recognition rate of fruit was86%under different illumation condition, as well as the rates of the picking point and the cutting point were93.0%and88.4%respectively. Meanwhile it provides a new technical idea for acqusiting the infromation of harvesting similar-color fruit.(3) The spatial matching strategy of picking point based on double layers constraints was studied and the three-dimensional coordinate location of picking point was explored. Firstly, according to strawberry’s morphology features and the corresponding relation between fruit region and picking point, the picking points were selected preliminary by a region matching method based on global features and relationship features. Secondly, the picking points were determinad by epipolar geometry constraint. Thirdly, the hardware system of binocular stereovision was developed, and then the intrinsic parameters and external parameters of camera and the hand-eye parameters were calibrated. After coordinate transformation model of the image, camera and manipulator was developed, the flow of locating picking point’s three-dimensional coordinate was obtained.(4) A modular prototype of havesting robot system was developed. The robot was constructed of binocular stereovision system, manipulator system, central controller, self-guided moving platform, battery system, and other appendix. Firstly, the image was segmented by normalized color difference (2r-g-b), and then the guide line was extracted. Secondly, the inverse kinematics of4-freedom joint type manipulator was analysised, and then the rotate angle of every joint at target position was obtained. Thirdly, the flow of the robot picking operation was planned, and a trail to test the robot performance was done, and the result show that the success rate of picking fruit was86%, and the execution time of a harvesting cycle was28s. Every fuctional modular of the robot run effectively and well adept to the working environment in greenhouse.