The Research On Fruit Tree Recognition Based On Fusion Of Multi-source Information

In recent years,orchard target variable spray technology has been developed more and more accurate.From infrared sensors,ultrasonic sensors to cameras,Lidars and Kinects,the feature information about the fruit tree canopy from these detection equipment is more precise and diversified.It is not only possible to know whether there are fruit trees next to the sprayer,but also to know the distance between the fruit trees and the sprayer,the height of the fruit trees,even the density of the leaves and the volume of the canopy.However,each techniques has its own shortcomings,resulting the unsatisfactory performance of the target variable spray in actual spray work.Based on the analysis of the advantages and disadvantages of various sensors,this paper proposes the method of combining the camera and the Lidar to collect the information of the canopy of fruit trees and fuse their data to compensate for the shortcomings of each other.This method can achieve the effect of removing the second row fruit trees in the fruit tree image,which will guide the precision spray.1)The design of multi-source information collection scheme and platform.In order to design algorithms based on real environment information,a multi-source information collection scheme for simulating orchard environment is designed,including the static and dynamic collection of image data,Lidar point cloud data and attitude data under different illumination and platform in different postures.According to requirements,a multi-source information collection platform based on Raspberry Pi is designed and built.The platform can capture image data,Lidar point cloud data,platform attitude data and illumination data in standstill or in uniform linear motion state at the same time.Meanwhile,the data can be saved in a file.2)The algorithm design and experiment of extracting canopy image of fruit tree.In the process of image processing,the key is to segment the fruit tree from the background.And the color characteristics of the fruit tree canopy are relatively stable in the process of collecting images outdoors.Based on this,K-means clustering is used to preliminary segmentation and segmentation based on color features is used to obtain the final fruit tree crown area.This method is less affected by the light intensity and it is no need to set different segmentation thresholds for different images.So it is suitable for practical applications.Compared with the actual measurement values,the relative errors of tree crown data obtained by image processing are below 15%.3)The algorithm design and experiment of reconstructing three-dimensional information of fruit trees based on Lidar.When the platform walks forward,the Lidar scans a series of fruit tree sections,so the three-dimensional information about the fruit tree relative to the platform can be obtained.In the process of reconstructing three-dimensional information of the fruit tree,the point cloud data needs to be corrected,and the correction algorithm is designed,which includes the correction when the platform has a pitch angle or a roll angle,and the correction of the platform traveling state.Finally,the three-dimensional information of the fruit tree has been reconstructed,and numerical characteristics of the tree canopy are obtained.Compared with the actual measurement values,the relative errors are below 14%,which verified the validity of the reconstruction of the fruit tree based on Lidar point cloud data.4)The fusion algorithm design and experiment between image and Lidar point cloud data.After obtaining the segmented image of the fruit tree crown and the distance between the crown and the platform,the effective Lidar point cloud data are matched with the pixel regions in the image,which can remove the interference of the second row trees in the image and the accurate target fruit tree image can be obtained.The experiment results show that fusion effect is good and the second row fruit trees are basically removed.The relevant algorithm designed in this paper provides a reference for agricultural target detection technology.The algorithm of this paper can be transplanted to Raspberry Pi or other hardware,and be tested the recognition efficiency in actual work.According to the actual situation,the algorithm can be optimized.If the efficiency meets the requirements,the corresponding relationship between the canopy characteristics and spray volume,air-assist devices can be explored to achieve the precision spray,which can reduce the pesticides and improve the quality of fruits.