Research On Non-destructive Picking Method Of Cherry Tomatoes Based On A Visual-tactile Sensor

Many fruit picking procedures used in agricultural production are time-consuming and labor-intensive.Fruit picking that is done manually is not only labor-intensive but also ineffective.Therefore,picking fruit wisely is crucial.However,there are two inevitable difficulties in the intelligent picking process:(1)How to judge the ripeness of the fruit based on its firmness before picking,and then decide whether to pick and the grading and classification after picking;(2)Can the fruit be harvested without damage during the picking process,that is,can the minimum clamping force be used while ensuring the completion of the picking operation.Therefore,a fruit picking device and its method are proposed to achieve non-destructive fruit picking while detecting and predicting fruit firmness.To solve this problem,a non-destructive cherry tomato picking technique based on a visual tactile sensor has been put forth.With cherry tomatoes as the research subject,quantitative detection and prediction of their firmness are accomplished,and by closed-loop slip detection,the minimal picking clamping force was steadily approached.The following are the key research findings and related accomplishments of this article:(1)Designed a visual-tactile sensor and achieved sensing of the deformation height of its contact surface.Designed and manufactured visual-tactile sensors through a series of steps;Measured the deformation height of the sensor surface using photometric stereo vision technology;A calibration method based on a lookup table was proposed,and the sensor reflectance function R_k was calibrated by pressing a steel ball.In order to reduce the calculation error of deformation height at different contact positions,a calibration method based on neural network was proposed;At the same time,a performance comparison was conducted between the two sensor calibration methods to facilitate selection in subsequent applications.(2)Realized non-destructive testing and prediction of fruit firmness.Proposed to describe the firmness of cherry tomatoes based on the deformation height information of their surface under fixed displacement pressure;Verified the rationality of using the sum of deformation heights of each pixel on the sensor contact surface to measure fruit firmness;The experimental results of firmness tracking and prediction showd that the relative error of firmness prediction after one week of storage is mostly within±15%,while after two weeks of storage,it is within±25%.(3)A non-destructive fruit picking method was proposed.Obtain pressing status information through marker point detection algorithm and contact area segmentation method;After comparing the optical flow method,the slip detection based on template matching was proposed to control the gripper to approach the minimum clamping force that can complete the picking;In the steel ball release experiment,it was determined whether the clamping force used was the minimum,and the results showed that 67%of the experimental samples were picked up with the minimum clamping force.(4)The whole machine operation experiment of the fruit firmness ripening and minimum clamping force picking device has been achieved.We conducted a physical cherry tomato picking experiment and analyzed the experimental results.The experimental results showed that the success rate of the picking experiment exceeds90%,and the completion of picking is good,with high stability and safety.The final experimental results indicated that the method and device proposed in this article can effectively complete the non-destructive picking task of cherry tomatoes and meet the needs of modern picking production.