3D Reconstruction Of Rapeseed Plants And Detection Algorithm Of Moss Phenotype Based On RGB-D Camera

As the largest oil crop in China,rapeseed phenotype nondestructive identification technology is still very backward,which has been stuck in the neck of related research.Therefore,high-throughput,automated and high-resolution phenotypic measurement techniques for rapeseed are urgently needed.The structure of rapeseed plants is complex and changeable,and the shelter is serious,so it is necessary to study a new 3D reconstruction technology to fully and accurately restore the three-dimensional configuration of rapeseed plants,and lay a foundation for subsequent analysis and measurement.In this paper,a 3d reconstruction and phenotypic measurement method of rapeseed based on RGB-D camera was proposed.The main research contents and conclusions are as follows:(1)Designed a set of automatic RGB-D sequence image acquisition device.The hardware equipment is mainly composed of an aluminum frame,movable lifting platform,electric rotating platform,STM32 control circuit,Azure Kinect camera sensor and rotating arm.The system software is developed based on Lab VIEW,which integrates algorithm code,control device and camera work,making image acquisition and 3D reconstruction simple and easy to operate.The device can realize Azure Kinect multi-angle,multi-position and multi-distance adjustment,which is suitable for image acquisition of crops of different sizes.Under the coordination of software and hardware,the system can control Azure Kinect images including color images,depth images and near infrared images,and send them back to the host wirelessly.The device can quickly and easily control the Azure Kinect camera to capture images with a throughput of 50 trees per hour.Compared with manual image shooting,it can save more than 70% of the time and reduce the intensity of image data acquisition.(2)3d reconstruction algorithm based on RGB-D camera.The Azure Kinect sensor was used to capture color,depth,and near-infrared images of the target from six viewing angles.The 8-bit infrared image binarization is multiplied by the general RGB-D image alignment results provided by Microsoft to remove the gouging and also remove most of the background noise.In order to filter out the floating point and outlier noise of the point cloud,a neighborhood maximum filtering method is proposed to filter out the abrupt point in the depth map.The floating point in point cloud is removed before the point cloud is generated,and then the outlier noise is removed by a straight-through filter.Compared with the traditional point cloud denoising algorithm,the processing speed is improved by 11.15 times and the denoising effect is better.Aiming at the shortcomings of the classical ICP algorithm,an improved method is proposed,by continuously reducing the sampling grid size and threshold,the distance between the corresponding points to the perspective of two point cloud registration for three times in a row,until the full color point cloud,compared with the traditional ICP registration algorithm,the success rate of the improved ICP registration algorithm is 2.85 times that of the traditional algorithm,and the speed is only30.8% of the traditional algorithm.A large number of experiments on full-growth rapeseed plants showed that the accuracy of point cloud obtained by this method was about 0.789 mm,and a complete scanning time was about 302 seconds,with high color restoration degree.Compared with the laser scanner,the method presented in this paper has comparable reconstruction accuracy and much higher reconstruction speed,but the hardware cost is much lower and the scanning system is easy to be automated.(3)3D measurement algorithm of rapeseed phenotype.A 3d phenotypic detection algorithm for rapeseed was studied.U-net neural network was used to segment the 2d stems and leaves of rape,and the segmented stems and leaves were masked with point clouds.The 3d reconstruction method mentioned above was used to reconstruct the leaves and stems of rape respectively to obtain a complete point cloud.Rape leaf number,through the calculation of european-style clustering using point cloud triangles in greed algorithm is rape surface area,through calculation of the point cloud bounding box largest operator plant height,seedling height measurement into the stem and leaf part,stem part by region growing segmentation algorithm segmentation branches,three leaves evenly divided,the two parts get seedling height length accumulation.The point cloud at the root neck was extracted by straight-through filtering,and the vertical distance from all points of the point cloud to the central point was calculated.The distance was twice the mean value as the root neck thickness.Branch Angle The stem segmentation diagram based on regional growth segmentation is obtained by the Angle transformation between the normal vector of each branch stem and the normal vector of the main stem.3D phenotypic detection demand of the indoor environment,to rape plant as the object,this paper puts forward to use RGB-D(color-depth)images of high resolution 3D reconstruction method,and further research in combination with characteristics of rapeseed biological and deep learning of rape plant point cloud processing,segmentation and phenotypic digital calculation method,the comprehensive accurate measurement in the phenotype of moss period rapeseed key.This study will provide an efficient means of data acquisition for research on physiological development,genetics and breeding,cultivation,transplanting machine design and other agricultural practices in maturity period.