Mobile Tomato Growth Information Acquisition System Based On Kinect

Tomato,as one of the vegetable varieties with the largest planting area in China,has a direct impact on farmers’ economic benefits and is also an important basis for guiding scientific field management.At present,there are a lot of researches on how to obtain crop images and predict crop yield by using sensors by domestic and foreign scholars,but they are mainly based on two-dimensional planar images,which inevitably result in overlapping and occlusion of fruits.With the development of science and technology,3D reconstruction technology has been widely used in agriculture.Aiming at these problems,this study designed a mobile greenhouse tomato growth information acquisition system based on Kinect sensor,including tomato plant point cloud formation,reconstruction,fruit recognition and counting.The experimental data show that the method is accurate and stable in calculating the number of tomato fruits.The main research contents and conclusions of this paper are as follows:(1)Established a mobile greenhouse tomato growth information acquisition system.According to the structural characteristics and environmental requirements of the greenhouse,ROS robot platform based on SLAM vision technology was selected to realize autonomous positioning and navigation,and the platform was loaded with sensors to dock at fixed points to extract plant information.(2)To extract the color image and depth image of tomato plants as the target,KinectV2.0 sensor was used in this study to compare the current 3D point cloud generation and related acquisition methods.According to the spatial relationship between the RGB camera coordinate system and the infrared camera coordinate system,the corrected fused color diagram is generated by the process of fusing color diagram,three-dimensional point cloud and color point cloud containing RGB information.Then the depth image is used to generate three-dimensional point cloud.Finally,the value of RGB is assigned to the corresponding point to generate a color point cloud containing both spatial coordinate information and RGB information.(3)In order to conduct subsequent processing of the point cloud and extract relevant parameters,analyze and compare several three-dimensional point cloud and two-dimensional image filtering methods,select direct filtering to remove ground noise,and select statistical filter to remove outlier noise.Next,in order to perform subsequent point cloud registration,10cm blue,yellow and orange balls were used as calibration objects in this study to perform initial image registration,and then the three-dimensional point cloud information of tomato plants after fusion was obtained through ICP precise registration.(4)In order to achieve accurate segmentation of tomato fruits,tomato fruits were used as the experimental object to analyze and compare the sensitivity of different color Spaces to light.Strong,weak and dark light were used to compare the numerical changes of different color Spaces.Finally,the information of channel a in the Lab color space was determined to be used for fruit segmentation.After obtaining the spatial position and color information of tomato fruits,k-nearest neighbor search based on grid descending sampling and mass center fusion based on k-means clustering algorithm were adopted to obtain tomato point cloud clusters based on the number of fruits.The k-means algorithm was further optimized,and the tomato diameter information was combined to solve the problem of counting the fruits of neighboring tomatoes.The results of all fusion graphs were summed to finally achieve the statistics of ripe tomato yield in the overall greenhouse scenario.(5)The reliability of the mobile tomato growth information acquisition system was verified by experiments.By calculating the height and maximum width of the fusion image and combining with the actual measurement values,the correlation determination coefficient R^2 was obtained to be 0.99 and 0.83,respectively,so as to verify the effectiveness of 3D image fusion.Data were collected under different light intensities,and the optimal collection time was verified from 8 AM to 10 AM,and from 4 PM to 6 PM.When Kinect was in the perspective of horizontal view,up view,down view,the corresponding mean relative error RAD was 14.34%,5.68%and 21.61%,respectively.Thus,the optimal acquisition perspective was the perspective of up view.The results of the fusion from different perspectives were compared,and finally the output was counted by using the two-perspective fusion.The number of plants collected by Kinect in a single collection was compared,which were single plant,two plants in a single row and four plants in two rows.Finally,the best experimental method is to collect data from Kinect’s up view,collect information from two perspectives in the same scene,and count two plants in the same row at a single time.At the same time,a field experiment was added to compare the error between the calculated value and the measured value of the maximum height of the plant,the maximum canopy width and the number of mature fruits,so as to verify the applicability and effectiveness of the system in complex scenes.In this study,3D scene reconstruction and growth information collection of mobile greenhouse tomato were proposed,which are of great significance for intelligent information collection and fruit yield prediction of modern greenhouse in China.