Morphological And Structural Phenotypes Extraction Of Maize Using Multi-source Data And Deep Learning

Using big data of plant phenotypics can systematically and deeply explore the essential relationship between"gene-phenotype-environment"from the perspective of omics level,and comprehensively clarify the mechanism of plant specific biological traits,which will greatly promote the process of modern breeding and high-efficiency cultivation research.However,the complex characteristics of plant phenomics data,such as multi-dimensional,multi-environment and multi-source heterogeneity,have led to bottlenecks such as poor precision and low applicability of standardized and high-throughput processing methods.With the development of computer technology,computer vision techniques and deep learning methods have provided new ideas to solve the above problems.It is important to integrate agronomic knowledge to find key traits,non-destructive data collection using computer vision techniques and deep learning algorithms to achieve high throughput processing of heterogeneous plant phenotypes from multiple sources at different scales.As one of the important grain varieties in China,maize is a large grain crop grown in the north of the country.Maize is a valuable object of study because of its tall plants and relatively simple morphological structure.Take maize as the research object to collect images and Li DAR data of individual maize and populations in the field environment.A three-dimensional reconstruction of the maize plant point cloud was carried out based on multi-view image sequences.High-throughput phenotypic resolution techniques that can extract phenotypic parameters from multiple sources of heterogeneous raw data of maize plants in the field at different scales were developed,and the main research of the article is as follows:1?Phenotypic Analysis of Top-view Images of Maize Shoots in Field.Image processing technologies are available for high-throughput acquisition and analysis of phenotypes for crop populations,which is of great significance for crop growth monitoring,evaluation of seedling condition,and cultivation management.However,existing methods rely on empirical segmentation thresholds,thus can have insufficient accuracy of extracted phenotypes.Taking maize as an example crop,we propose a phenotype extraction approach from top-view images at the seedling stage.An end-to-end segmentation network,named Plant U-net,which uses a small amount of training data,was explored to realize automatic segmentation of top-view images of a maize population at the seedling stage.Morphological and color related phenotypes were automatic extracted,including maize shoot coverage,circumscribed radius,aspect ratio,and plant azimuth plane angle.The results show that the approach can segment the shoots at the seedling stage from top-view images,obtained either from the UAV or tractor-based high-throughput phenotyping platform.The average segmentation accuracy,recall rate,and F1 score are 0.96,0.98,and 0.97,respectively.The extracted phenotypes,including maize shoot coverage,circumscribed radius,aspect ratio,and plant azimuth plane angle,are highly correlated with manual measurements(R~2=0.96-0.99).This approach requires less training data and thus has better expansibility.It provides practical means for high-throughput phenotyping analysis of early growth stage crop populations.2?Analysis of Point Cloud Phenotype for Field Maize Population.The extraction of high-throughput,time-series phenotypic data from plant populations in the field for Li DAR data suffers from the problem of point cloud data alignment.The study uses a field orbital phenotyping platform with Li DAR and industrial cameras for high throughput,high time series raw data collection of maize populations in field.Alignment of Li DAR point cloud data using orthorectified images of maize populations in the field with the Direct Linear Transformation(DLT)algorithm.After registration,the ground point cloud is removed based on the cloth simulation filtering(CSF)algorithm.Aiming at the problem of extracting accurate phenotypic parameters from the population point cloud,a voxel slicing method was proposed to process the maize population point cloud,and finally the plant height was extracted by variety as the unit.The results showed that the height of 13 maize varieties obtained using the method proposed in this study was highly correlated with the manual measurement values for 45 consecutive days(R~2>0.95).The accuracy of plant height extraction demonstrates the reliability of the extraction methods presented in this section of the study and shows the feasibility of using a field track-based phenotyping platform for high-throughput,time-series phenotyping of plant populations.3?Point Cloud Phenotyping Analysis of Single Maize Plants.Automated plant point cloud segmentation methods are one of the bottlenecks in achieving big data processing of plant 3D phenotypes.To address this problem,the study proposes Deep Seg3DMaize,a plant 3D point cloud segmentation method integrating high-throughput data acquisition and deep learning,using maize as an example.The method addresses the needs of deep learning for large data volume,good data quality and high coverage of morphological and structural features in training datasets,and proposes the use of a crop single plant high-throughput platform to obtain point clouds for correlation analysis of maize population plants to construct datasets.Specifically,the high-throughput data acquisition and point cloud data labelling were done using the self-developed MVS-Pheno crop single plant high-throughput phenotyping platform and the maize plant point cloud labelling software Label3DMaize,respectively.Based on this,Point Net was introduced to achieve stalk and stem-leaf segmentation and organ segmentation of the 3D point cloud of maize plants.The results show that the average precision and F1-Score of stem and stem-leaf segmentation are0.91 and 0.85,respectively.The mean accuracy and F1-Score for organ segmentation were 0.94 and 0.93,respectively.The correlations of leaf length,leaf width,leaf inclination,leaf growth height,plant height and stem height extracted based on the segmentation results with the measured values were 0.90,0.82,0.94,0.95,0.99 and0.94,respectively.This study has achieved a complete process of high-throughput acquisition of maize plant point cloud-automatic plant point cloud segmentation-plant phenotype analysis,which provides a systematic solution reference for accurate identification of plant single-plant-scale 3D phenotypes.This study is dedicated to the development of a multi-scale method for high-throughput phenotyping of heterogeneous raw data from multiple sources in maize.Important for monitoring the growth and development of maize crops,evaluating seedling conditions and making decisions on cultivation management.It is expected to provide a valuable tool for modern breeding and efficient cultivation.