Research Of Automatic Leaf Segmentation And Analysis On Dense Plant Point Clouds

Greenhouse cultivation,which is a kind of highly integrated facility agriculture,is becoming increasingly important in raising the efficiency of agricultural production and solving the world's food shortage.Despite its significance,modern greenhouse cultivation still faces challenges of high investment,frequent investment of professional manpower,and high energy consumption.For this purpose,an economic,efficient,and intelligent control method for greenhouse environment is needed to guarantee a temperate growing environment for each plant during its whole growth period,and to accomplish high yields and economic benefits.The implementation of the environmental control algorithm needs to take the growth status of crops as reference for regulating.Currently,most of the methods in this field rely on the temperature,humidity,illumination and other environmental measurement data inside and outside the greenhouse as feedback information to control the sunshade net,ventilation and irrigation.However,they are indirect control methods,and vulnerable to carry out automatic observation and analysis on the actual growth of crops(such as spatial plant type,leaf morphology,color,and other phenotypic features).At the same time,hybridization and gene breeding,another promising technology in agricultural engineering,have faced bottleneck constraints.The breeding technology now still needs professional agronomists to compare and analyze the difference of crop phenotypes in different trial planting areas.This process is time-consuming and laborious,and requires the guidance of professional knowledge.Therefore,high-throughput automatic collection and analysis of crop phenotypic characteristics have become the key to improve breeding technology.Conclusively,in order to suffice for intelligent facility agriculture and reduce tedious manual labor in gene breeding,the research of automatic analysis algorithm for crop phenotype based on computer image and graphics technology is particularly important.Leaves account for the largest proportion of all organ areas for most kinds of plants and comprise the main part of photosynthesis and respiration in a plant.Leaves contain important information about the surface morphology and structure of a plant;therefore,observation of leaves can reveal its growth status.Changes in leaf morphology,texture,or color normally reflects biotic stress(plant diseases and pests)or abiotic stress(drought).Therefore,our work will focus on the research of the phenotypic analysis algorithm for leaves,realize automatic identification and segmentation of leaves,and serve for calculating phenotypic parameters of individual leaf,such as leaf length,leaf width,leaf area and leaf inclination.Traditional leaf segmentation and analysis algorithms are mostly based on 2D images.However,the images have inherent defects in representing the canopy of plants distributed in clusters,and hardly own the occluded leaf information.To solve the problem,we scan crops for dense 3D point cloud with 3D imaging technology,and then design novel individual leaf segmentation and phenotype analysis methods based on the point cloud of crop canopy.The main research contents of the paper are as follows.1.We propose an individual leaf segmentation approach for dense plant point clouds using facet over-segmentation and facet region growing.The main contributions of this work are summarized as follows.1)Inspired by the super-pixel segmentation in the image,our method uses middle-level features from facets.First divide the canopy point cloud into a set of facets,and then automatically grow facets with similar features into leaves.The method avoids the disadvantage that points based feature segmentation technique is easily affected by noise,solves the problem of the sensitivity of parameter in 3D segmentation algorithm,which greatly improves the accuracy of individual leaf segmentation.In the quantitative performance assessment,the method reaches higher than 80% in average Precision and higher than 90.81% in average F-measure for four different kinds of greenhouse ornamental plants.2)Although the original goal of our method is to segment leaves from a point cloud of crop canopy,which has an irregular 3D structure,we find out it has potential to be applied to many applications that aim to segment regular surfaces and objects from a point cloud of scene;for example,remote sensing,Building Information Model(BIM),Simultaneous Localization and Mapping.2.Based on the the first research,we propose a novel overlapping-free leaf segmentation approach.The main contributions of this work are summarized as follows.1)Inspired by the fact that eroding and removing overlapped parts of the objects,and then segmenting them successfully in 2D image processing,we develop a novel 3D joint filtering operator,a 3D eroding operator namely,to filter the point cloud of leaves with overlapping situations.The operator can effectively separate the leaves that touch and occlude each other at various angles and positions.After joint filtering,the central areas of leaves can be presegmented by clustering the remaining points.2)The points filtered by 3D joint filtering operator are added back to the presegmented central part of original leaves by growing the index of leaf centers at 3D facet level from inside to outside.And then an effective individual leaf segmentation for dense point cloud of plant canopy with occlusion is completed.3)The experimental results show that the proposed method is effective in segmenting individual leaves from crowed point clouds of different plant species,and is also applicable on point clouds scanned from three kinds of 3D imaging systems,and is more universal than similar algorithms.In the performance assessment,our method reaches an average cover rate at 96%,Precision at 99.33%,and an average F-measure at 99.66% for four different kinds of greenhouse ornamental plants,respectively.Furthermore,the average speed of the segmentation costs only 12.92 seconds per plant on a desktop PC.4)The proposed method can help to automatically calculate phenotypic traits of each single leaf(such as the area,length,and width),which shows the potential to become a highly effective tool for plant research and agricultural engineering.Experiments show that the average estimation errors of leaf area,length,width,and inclination angle for the point cloud of Calathea makoyana are merely 0.47%,2.89%,4.64%,and 2 degrees,respectively.