Self-calibration Of A 3d Morphological Measurement Method For A Fruit Tree Canopy Based On Kinect Sensor

Perception of canopy formation is an important technology to standardized orchard intelligent management and control,the extraction of canopy morphological parameters from two dimensional images or three dimensional by single view images is not comprehensive enough,because of the complex three-dimensional structure of fruit canopy and the mutual occlusion of canopy branches and leaves.In order to effectively and accurately perceive canopy information in complex orchard field environment,it is necessary to extract and fuse three-dimensional information of fruit tree canopy from different perspectives.The registration fusion algorithm of three dimensional information from different perspectives and the extraction method of canopy’s parameters become a key to the problem.This study proposed a 3D morphological measurement method for a fruit tree canopy based on Kinect sensor self-calibration,includes three-dimensional point cloud generation,point cloud registration and canopy information extraction of apple tree canopy.and relevant experiments are carried out to verify and error analysis.The experimental statistics show that the method can register point clouds and measure fruit tree parametershas accuracy and stability.The following are the main research work contents and conclusions of this paper:(1)Compared with the current 3D point cloud generation and fruit tree parameter acquisition methods,this study decided to use the Kinect sensor to obtain the original color image and depth information of the target fruit tree.Generating corrected fusion color image according to the spatial relationship conversion between the RGB camera coordinate system and the infrared camera coordinate system.Then three-dimensional point clouds are generated from depth images.Finally,RGB values are assigned to corresponding points to generate color point clouds containing both XYZ coordinate information and RGB information.(2)In order to follow up the point cloud and extract relevant parameters,This study did(3)the following point cloud pretreatment and related experiments:Author analyzed and compared several 3D point cloud and 2D image filtering methods,selected the bounding box to filter out the background point cloud,and used statistical filter to remove outlier noise.comparing point balls of different sizes and colors to generate point clouds by experiments,Finally,red yellow and blue spheres with diameter of 10 cm were selected as calibrators.(3)In order for the point cloud to contain more comprehensive canopy information,this study designed a coarse registration method can self-calibrate by calibration balls,and the principle and steps of point cloud registration for red,yellow and blue calibration balls at different viewing angles are introduced in detail.On the basis of rough registration,the point cloud density is reduced by downsampling,and an improved iterative closest point(ICP)method used to accurately register the point cloud was proposed,which can show more comprehensive canopy morphology information of the fruit tree.(4)A method for calculating height H,maximum width W,canopy thickness D and volume V of fruit trees from reconstructed point clouds was proposed.Taking apple tree(Yanfu 3)as the research object,the experiment of extracting canopy information of apple tree based on Kinect sensor was carried out in the orchard.The results show that based on the information fusion on both sides of the fruit tree,this method can accurately extract the canopy feature information.The average relative error RAD of the calculated values and measured values of the fruit tree morphological parameters H,W and D in the V1 measurement mode are 3.8%,12.71%and 5.04%;the RAD of the V2 single viewing angle is 3.25%,9.48%and 4.9%;V1 and V2 The RAD of fusion was 2.54%,3.62%and 3.15%.It can be seen that the information fusion of the two viewpoints has significant advantages over the single viewpoint in the measurement accuracy of the maximum width W.The experimental verification shows that the three-dimensional point cloud reconstruction method based on Kinect’s self-calibration has the advantages of high precision and stable performance,and the auxiliary calibration materials are easy to carry and easy to install.The canopy can be realized under different experimental scenarios.The extraction of three-dimensional information is of great significance for the realization of intelligent control of standardized orchards.