A Study On 3D Phenotype Acquisition And Automatic Analysis Technology Of Plants Based On Multi-view Images

High-throughput acquisition of high quality and reproducible phenotypic traits of plants can greatly facilitate the systematic and in-depth mining of the intrinsic relationship of"gene-phenotype-environment"and comprehensively reveal the formation mechanism of specific biological traits,accelerate the breeding process,and provide decision data support for precise agricultural cultivation.At present,there are technical barriers such as low precision,low throughput and high cost in plant phenotype technology.It is an urgent need and significance to break through the key technology of high-throughput crop phenotype acquisition and analysis to promote the development of digital breeding modernization in China.Multi-view image-based plant phenotype acquisition technology,which can acquire plant 3D point clouds with high-precision RGB information at low cost and fine detail,has received wide attention from researchers in recent years,but the traditional multi-view reconstruction system has problems such as low operational efficiency and reconstruction scale bias.In addition,accurate organ segmentation of plant point clouds is crucial for their 3D phenotypic analysis,and the research progress of deep learning methods on point cloud data has enabled organ segmentation of plant point clouds to show great potential but there are problems such as difficult construction of training sample sets and low robustness of segmentation.Therefore,based on the above research needs,this paper presents the key technology system research from the development of plant data acquisition equipment to the development of 3D phenotype automatic analysis system for low-growth wheat and multi-branched maize male ears,with the content of 3D phenotype acquisition and automatic analysis at plant single plant scale.The main research contents are as follows.(1)Research on the reconstruction and correction methods of plant point clouds based on SFM and MVS.Based on the SFM and MVS algorithms,the open source Open MVG and Open MVS libraries were integrated to develop a batch automated reconstruction system of plant plant point clouds based on multi-view images from image pre-processing,multi-view reconstruction,reconstruction point cloud scale and orientation correction,and realized the high precision reconstruction from plants.(2)A portable plant 3D phenotype high-throughput acquisition device and system.A portable wheat plant 3D phenotype high-throughput acquisition system was designed for wheat with multiple tillers,many leaves,and compact leaf growth facing upward.The system consists of a plant rotating multi-view acquisition device for image capture,which is portable and low-cost compared to the camera rotating multi-view acquisition device.A pipeline data processing system integrated by multi-view 3D reconstruction,point cloud processing and phenotype resolution was developed and selected for application to different wheat strains at the spike stage.The experimental results showed that the RMSEs of leaf length,leaf width and plant height extracted from the 3D point cloud of image reconstruction obtained by the plant rotating multi-view acquisition device were 0.79 cm,0.13 cm and 0.53 cm,respectively,and the MAPEs were 3.26%,7.63%and 0.74%,indicating that this method has high accuracy of point cloud reconstruction and phenotype extraction,and provides a low-cost solution for wheat plant phenotype acquisition It provides a low-cost solution for wheat plant phenotype acquisition.(3)Deep learning-based organ segmentation and phenotype resolution of maize male ears.Deep learning methods require a large number of datasets as support,however,the lack of manual annotation datasets for plants has become an important factor hindering the organ segmentation of plant point clouds.For maize male ears point cloud data,an innovative method of incomplete annotation is proposed to easily produce the dataset of maize male ears,a random function is used to enhance the dataset,a shortest path growth algorithm is proposed and a Maize Tassel Seg segmentation network is built to achieve the organ segmentation of maize male ears from the point cloud.The Io U,precision,and recall of the segmentation results were 96.29,96.36,and93.01,respectively,and the statistical analyses of the branch length,branch angle,and number of branches of maize male ears were performed with R~2and RMSE of 0.9897and 0.529 cm(branch length),0.9317 and 4.516°(branch angle),and 0.9587 and 0.875(branch count),respectively.