The Recognition And Localization Of Ripe Dragon Fruit Based On Binocular Vision

China is the second largest Dragon fruit producer in the world.In the process of Dragon fruit production picking is still done manually with high harvest costs and low picking efficiency.At present,agricultural mechanization is a must,and it is very necessary to develop a device that can automatically pick ripe Dragon fruit for the development of the Dragon fruit industry.The identification and positioning system of ripe Dragon fruit is an important part of the intelligent Dragon fruit picking hand,and its performance directly determines the picking rate and work efficiency of the Dragon fruit only picking equipment.In this paper,the ripe red Dragon fruit produced naturally in the field environment was taken as the research object,and the identification and location of the ripe Dragon fruit were studied.A recognition method based on ripe Dragon fruit is proposed,and on this basis,it is located,the coordinates of the center point of the ripe Dragon fruit are output,carried out related experiments on dragon fruit identification and localization.The main research contents and conclusions of this article are as follows:(1)Building a ripe dragon fruit image acquisition system.The Depth Quality Tool was used to calibrate the system,and the internal and external parameters of the ripe dragon fruit identification and positioning system were obtained.The system was used to take a total of 530 images of ripe dragon fruit under different light intensities,and the light intensity at the time of shooting was recorded using a light meter.When shooting the distance between the D415 binocular lens and the ripe dragon fruit is in the range of 0.5m-3.0m.To increase the number of ripe dragon fruit in the samplethe data augmentation method is used to augment it to 1344 images.(2)Identify ripe dragon fruit based on color model.The images of ripe dragon fruit were preprocessed by comparing various schemes to remove the image noise generated in the field environment.The peak signal-to-noise ratio method was used to evaluate the collected dragon fruit images and the pre-processed post-ripe dragon fruit images.Through comparison the appropriate preprocessing method of the dragon fruit images was determined.The distribution range of each parameter in the color model of ripe dragon fruit was counted,and according to the statistical results,the ripe dragon fruit was identified and the ripe dragon fruit identification algorithm under the RGB color model and the HSV color model was designed and tested.Among them,the recognition success rate of ripe dragon fruit under the HSV model can reach up to 90%.(3)Detecting ripe dragon fruit based on deep learning.The augmented ripe dragon fruit samples were labeled using LabelImg software.The YOLO V4 neural network was trained using the labeled ripe dragon fruit data.Target detection of mature dragon fruit in the training set can achieve a recognition efficiency of 65 frames per second.The recognition rate of unobstructed mature dragon fruit can reach up to 99%,and the recognition efficiency of shaded mature dragon fruit can reach up to 92%.((4)Based on binocular vision to locate ripe dragon fruit.By using the D415 binocular lens this paper calculates the coordinates of the center point of the smallest circumscribed rectangle of the ripe dragon fruit in the identified area through identification results.Through the bench experiment the relative movement between the picking machinery and the ripe dragon fruit is simulated.When the relative movement speed is less than 1.2m/s,the recognition rate is greater than 75%.Finally the human-computer interaction interface of the ripe dragon fruit identification and positioning system is designed using Python.The experimental results show that the algorithm for identifying and locating ripe Dragon Fruit based on binocular vision proposed in this paper can meet the requirements of picking machinery for the location of the center point of ripe Dragon Fruit,and it is the basis for the design and control of the subsequent intelligent picking machinery for ripe Dragon Fruit.The acquisition of the location of the action execution point to be picked provides an effective technical reserve.