Detection And Localization Of Thin-shelled Pecan Based On YOLOv5s

During the process of fruit planting and production,fruit picking constitutes a critical step,which needs to invest a lot of labor.It is becoming important to reduce the use of labor in fruit picking and to automate fruit picking to improve operational efficiency.At present,the research on thin-shell pecan mainly focuses on the growth status,quality improvement,and biological characteristics,but there are few studies on the detection,localization,and picking of the fruit.Current research on fruit detection focuses on identifying fruits with distinct characteristics such as shape,color,and texture,which is applicable to apples,citrus,and pomegranates.For the varieties with small fruits,unclear color characteristics,and complex planting environments,conventional fruit detection methods could not meet the requirements of the orchard operating environment.Therefore,in this thesis investigates the application of deep learning-based thinshell pecan detection in fruit picking and harvesting,which is important to reduce the labor intensity of fruit picking,improve operational efficiency,and enable automatic picking.The primary focus of this thesis is the identification algorithm for pecans with thin shells utilizing a convolutional neural network for classification.Based on the YOLOv5 s target detection model,the algorithm is improved by combining the characteristics of thin-shelled pecan fruit.The main research achievements and contents of this thesis as follows:(1)Establishing a target fruit dataset.The existing thin-shelled pecan dataset was established independently to address the problems of small numbers and single species of thin-shelled pecan dataset.To enable accurate target detection,the diversity of samples must be ensured in the thinshelled pecan data collection.Finally,the collected 1307 samples were manually labeled to complete the dataset.(2)Improving network structure to improve detection accuracy.To address complex environments,small thin-shelled pecan targets,mutual occlusion of fruits and high color similarity between target fruits and background in thin-shelled pecan target detection tasks.This improvement is done by adding an attention mechanism and a Stage Block module to improve the performance of small target object detection.Experimentation reveals that the enhanced network yields a 5.4% improvement over the original YOLOv5 s network,with a fruit target detection speed of 54.3 Fps/s.This enhancement provides high fruit target detection accuracy that meets the requirements for both accuracy and real-time fruit target picking detection.(3)Calibration of the vision system of the robotic arm.To obtain the location of the target fruit in 3D space,the picking system needs to be calibrated first.In this thesis,the Kinect V2 camera is selected as the vision input device,and the Matlab software toolbox is used to calibrate the camera and perform aberration correction,and then the parameter model of the camera is obtained,and the eye-in-hand approach is selected for the hand-eye calibration of the camera and the robotic arm base.(4)Robotic arm forward and reverse kinematics analysis and trajectory planning.A modified D-H parameter method is adopted to construct the robotic arm model for meeting the requirements of its picking tasks.To ensure the correctness of the robotic arm motion,both forward and inverse kinematic analyses are conducted to obtain relevant equations.Using the Matlab software,the robot arm workspace is mapped out,and simulations of the forward and inverse kinematics of the robot arm model are performed to authenticate the accuracy of the modeling.Trajectory planning of the robot arm is implemented through three-polynomial interpolation and five-polynomial interpolation in the joint space,as well as linear and circular arc interpolation in the Cartesian space,based on different motion spaces.Consequently,by examining the alteration of each joint’s angle motion of the robot arm,the feasibility of its motion is verified by simulating the trajectory planning in different joint spaces.(5)Thin-shelled pecan fruit grasping experiments.By building a mechanical grasping platform,the grasping experiment of thin shell pecan fruit was carried out to simulate the fruit picking situation.Experiments show that the modified algorithm can accurately detect,locate and grasp thin-shelled pecan fruits with small experimental errors.It can meet the actual picking requirements and prepare the theory for future real robotic arm picking.