Research And Application Of Few-shot Image Segmentation Method For Complex 3D Material Microstructure

Computer vision is a science that can teach the computer how to "see" the world.Recently,with the rapid development of computational hardware and computer vision methods,as well as the in-depth cross and integration of multi-disciplinary,scientists try to apply artificial intelligence and computer vision technology,such as image segmentation,to the field of material science.It is aiming to quantitatively analyze the internal microstructure of material through image processing methods and concluding the relationship between the microstructure and macroscopic properties.However,polycrystalline iron image segmentation,which is a classical task in material microscopic image processing,is a binary segmentation task with complex shape and class imbalance.Each grain has different shape features,but there is no obvious difference in texture representation,which makes the algorithm design difficult.Besides,the grain is three-dimensional structure that a single grain will exist in the multi-layer cross-section.How to design algorithm to identify the same grain in multi-layer cross-section for three-dimensional segmentation is the key of structure reconstruction.To further improve the performance of the model,a large number of labeled data are needed to train the model.However,due to the complexity of the material preparation process,trivial and time-consuming labeling process,only a few samples of labeled data can be obtained,so it is necessary to improve the generalization of the model by data augmentation method.In this work,we propose some methods to handle the above problems.The main contributions are shown below:(1)We propose a skeleton aware loss function to lead the model to preserve shape information and improve the accuracy in two-dimensional material microscopic image segmentation.Compared with the classical loss functions,the proposed method has the following characteristics:1)adaptive.It can be weighted adaptively according to the shape of the target region;2)universality.The method proposed to weight on both the boundary area and the target area;3)generality.The method has no super-parameters and does not need to manually adjust the parameters according to the task,so it can be easily transferred to different data.Experiments show that the proposed method outperforms the currently 9 classical loss functions in the segmentation task of two data sets on with five baseline models.(2)We develop a region aggregation method to identify the same grain region in different sections,and then realize the three-dimensional segmentation or region marking.Through experimental comparison,the proposed method effectively utilizes the high-dimensional features extracted by deep learning method,and its error rate is lower than the classic handcrafted features such as minimum centroid distance and maximum area coincidence area.In addition,the proposed method achieves consistent improvement in both isotropic and anisotropic datasets.(3)In order to further improve the performance of the model,we present a data augmentation method based on style transformation.By fusing the grain structure information in the simulation model and the texture information in the real image,the synthetic image is created by the style transformation,which is used as data augmentation to expand the data set for training the image segmentation model.Experimental results show that the data augmentation method can bring performance gain for material microscopic image segmentation.The average synthesis time of single image is about 1%of that of real data.Moreover,only using 35%real data and synthetic data,its performance is better than that of using only 100%real data to train the model,which shows that the method can save 65%of the workload of real data preparation.Through ablation experiments,we prove that the proposed method is better than the traditional data augmentation method and the transfer learning method based on pre-training and fine-tuning.Finally,we develop a material microscopic image segmentation software based on deep learning,which provides a visual operation tool for material researchers.Meanwhile,we propose a modified overlap-tile strategy,which separates the core region clipping process from the network structure design process,improves the accuracy of the edge of segmentation results through post-processing operations,and reduces the dependence of deep learning model on high memory hardware equipment in practical application.