Research On Phenotype Detection Of Single Cucumber Seedling Based On RGB-D Camera

As an important vegetable crop,cucumber is widely cultivated worldwide.Cucumbers are generally planted in factory plug trays,and the monitoring of morphological parameters during the growth process can provide a reference for seedling cultivation and growth environment parameters regulation.The current seedling phenotype detection methods mostly rely on traditional manual measurement,which has problems such as low efficiency,subjective judgment,large measurement error,and easy damage to seedlings.The lag in phenotypic measurement technology has seriously hindered the modernization of the nursery field.Therefore,it is necessary to study a high-throughput automated seedling phenotype detection technology to achieve non-destructive measurement of phenotypic parameters during the growth of seedlings and meet the needs of smart factory-based seedling breeding.This paper takes cucumber seedlings as the research object,builds a seedling image acquisition platform,and proposes a cucumber seedling organ segmentation detection and phenotype measurement algorithm based on RGB-D camera.The main contents of this paper are as follows:(1)An automated RGB-D image acquisition system is designed.According to the needs of large-scale seedling phenotype detection,combined with the analysis of the performance of the Azure Kinect depth camera,an automatic image acquisition system suitable for a single cucumber seedling was designed and built.The system consists of hardware structure and system software.The hardware structure includes 4040 aluminum profile frame,conveying mechanism,linear guide rail,end effector,STM32 control circuit,etc.The system software is partly developed based on Lab VIEW,which integrates the algorithm code,controls the movement of the device and the work of the camera,and realizes the intelligent detection of cucumber seedling phenotypic parameters and automatic data archiving.At the same time,the system can adjust the camera at multiple heights,angles and distances,adapting to seedlings of different growth periods and sizes.The flux reaches 60 plants/hour,and the shooting success rate is greater than 93%,which not only reduces the need for image data collection It also improves the efficiency of cucumber seedling phenotype detection.(2)Organ segmentation of cucumber seedlings based on two-dimensional images.Use the Azure Kinect camera to capture infrared images,color images,and depth images of cucumber seedlings from top and side views.The U-Net network is used to segment the infrared image of cucumber seedlings from a top view and extract the cucumber seedlings.Compared with the traditional threshold segmentation algorithm,the segmentation accuracy is better and the robustness is stronger.Mask the binary map obtained by U-Net segmentation and the alignment map(the color map is converted to the depth map)to generate an alignment map with only cucumber seedlings,and use the Mask-RCNN network for leaf segmentation to obtain individual leaves,IOU(segmentation)The cross-combination ratio of the result and the real result area)is above 0.91;the YOLOv5s network is used to detect the growth point of cucumber seedlings.When the confidence level is 0.5,the target prediction of the growth point of a single cucumber seedling is all correct.On the basis of the same operation,the infrared image and color image of the side view were used to obtain an alignment map containing only cucumber seedlings,and the Mask-RCNN network was used to segment the cucumber seedling canopy and hypocotyl.Split effect.(3)Calculation method of cucumber seedling phenotype.Using the depth map,combined with the images of each organ,a 3D point cloud of the organs of a series of cucumber seedlings was generated.The number of leaves is obtained through the Mask-RCNN instance segmentation network;the ellipse fitting is used to determine whether the cotyledons are occluded,and the IP-basic algorithm is used to restore the missing depth information for the occluded cotyledons to complete the leaf point cloud.Calculate leaf area and leaf perimeter through point cloud triangulation with the complete point cloud;calculate leaf inclination and leaf curl by fitting leaf point cloud with RANSAC algorithm;calculate hypocotyl length,hypocotyl uprightness and stem through minimum circumscribed rectangle Coarse;design a cucumber seedling growing point recognition algorithm to detect the position information of cotyledon nodes to calculate the plant height;use the AABB minimum bounding box of the point cloud to calculate the seedling height.The leaf area,leaf circumference,hypocotyl length,stem diameter,plant height and seedling height of cucumber seedlings were measured by phenotype.The root error RMSE is 0.245mm~2,1.002mm~2,1.143 mm~2,and the mean absolute percentage error MAPE is4.224%,9.619%,11.098%,respectively.Leaf circumference R~2 was above 0.92,RMSE was lower than 3.83mm,and MAPE was less than 5%.The R~2 of the hypocotyls were all higher than 0.91,the RMSE was lower than 1.12 mm,and the MAPE was lower than 4.62%.Stem diameter R~2 was 0.83,0.87,0.92,RMSE was lower than 0.16 mm,MAPE was 6.61%,5.24%,4.15%,respectively.The seedling height R~2 was above 0.92,the MAPE was within3%,and the RMSE was within 1.5mm.The plant height R~2 was above 0.90,the MAPE was within 4%,and the RMSE was within 2mm.In response to the three-dimensional phenotype detection requirements of cucumber seedlings,this paper proposes to collect cucumber seedling image data from two perspectives based on RGB-D camera,and combine sensor defects and cucumber seedling biological characteristics to segment cucumber seedling images,complete point cloud and intelligent phenotyping.The detection carried out a new research exploration to achieve a comprehensive and accurate measurement of the key phenotypes of cucumber seedlings.This study can provide an effective solution for the acquisition of key phenotypic parameters of other individual seedlings,and provide important basic data for seedling related research,which has certain application and promotion value.