Research On Dominant/invisible Defects Detection Of Apple Skin Based On Deep Learning And Spectral Imaging

Our country is the largest apple producing country in the world,and its cultivation area and total output rank first in the world,but the average export rate of apples in the past ten years is not higher than 4%.At present,the quality of fruits in my country is uneven,and the level of post-harvest commercialization of fruits is low,which is caused by the different levels,low efficiency,strong subjectivity,and high labor intensity of manual sorting apples.Therefore,rapid online sorting of defective apples and grading apples are of great significance for expanding exports and reducing postharvest losses of apples.With the development of big data and deep learning,object detection technology,as an important task in computer vision,can gradually solve the problems existing in traditional machine learning algorithms.The paper mainly used hyperspectral imaging,multispectral imaging,machine learning,and deep learning algorithms.In order to solve the problem in mild injuries and short-term injuries and the influence of poor image quality caused by the light source system on the detection effect caused by machine learning algorithms detecting apples,object detection algorithms were used to detect apple defects in spectral images online.The main research contents of this paper are as follows:(1)Invisible defect detection based on hyperspectral data and YOLOv3 algorithm.For hyperspectral images with huge amount of data and dimensions,this paper used the method of selecting characteristic bands to reduce the amount of data while maintaining effective information and deep learning algorithm to detect apple defect in images of characteristic bands.Taking Shandong "Fuji" apples as the research object,bruised apples with different formation times,degrees,and numbers of bruises were produced.Long-wave near-infrared hyperspectral images(935-2548 nm)of bruised apples were collected and corrected using black and white board,and then bruised and normal tissues were taken as its regions of interest to analyze their spectral trends.According to the analysis results,multiple spectral regions were selected for characteristic band selection,and segmental PCA was used to finally select 1054 nm,1260 nm,and 1442 nm as characteristic bands.In order to verify the advantages of the original hyperspectral data and its preferred PC image respectively in the deep learning algorithm,the datasets composed of the characteristic wavelength grayscale image(Dataset I)and the characteristic wavelength PC grayscale image(Dataset II)were respectively input into the YOLOv3 algorithm to establish Bruise detection model.The test results showed that the test set based on Dataset I obtained a F1 score of 100% and a FPS of 68,while the YOLOv3 model and traditional detection method based on Dataset II have high confidence in sample detection,but they cannot eliminate the bright spot interference caused by uneven illumination.In addition,the detection ability is not strong for those with short formation time and light degree.The experiments verified that the YOLOv3 model based on characteristic wavelength has great potential in the online detection of apple bruises.(2)Explicit/invisible defect detection based on multispectral data and improved Center Net algorithm.In order to simultaneously detect external and invisible defects online,and reduce the detection impact of fruit stem and calyx,four targets of apple external defects,bruise,stem and calyx were simultaneously detected based on the multi-spectral data obtained in the online inspection production line.An optical system for characteristic band selection was built.On the basis of previous work,a characteristic wave band with obvious external defects and bruise was selected in 6wave bands,and used for the development of online production lines.Spectral data of various samples based on the online production line were obtained and divided into train,validation and test set;In order to meet the real-time requirements of online detection,a no-anchor box target detection algorithm,Center Net algorithm was selected as the detection model.Dla34 was used as the basic feature extraction network,and spp modules are added in multiple different positions to determine the detection effect of the Center Net-spp algorithm in different positions.Experiments results showed that adding the spp module between the basic feature extraction network and the idaup upsampling can most effectively improve the detection accuracy.The overall detection AP of the improved model for the sample reaches 89.6%,which is 2.4% higher than that before the improvement,and the detection speed is improved to 340 fps.In order to further determine the classification effect of defective apples by the model,a test experiment under balanced samples is designed.The results show that the detection model can classify defective apples with a F1-score of 96.6%,and the classification speed reaches the goal of online detection and meets real-time performance.This paper used "Fuji" apples as experimental materials,and adopted spectral imaging technology,object detection algorithm,and combines the existing online detection and production sorting lines in the laboratory to study the problem of online detection of external/invisible defects of apples by machine learning algorithms.Characteristic bands were extracted and used for object detection experiment.The improved object detection algorithm enhanced the detection effect of apple defects and the classification accuracy of defective apples,and provided support for the actual online sorting of defective apples.