Annotation And Segmentation Of Few-shot Medical Scanning Electron Microscopic Particle Images

Instance segmentation of scanning electron microscopic images provides important quantitative information for exploring particle size distribution,morphology and surface texture,which helps various biomedical studies,such as phenotyping analysis in drug delivery systems.The great success of deep learning algorithms has made them widely used in medical images and rapidly advancing compared with the traditional methods.However,due to the expensive and laborious manual annotation,it becomes quite challenging to obtain sufficient amount of manual annotation as training data.For a specific medical scanning electron microscopic image segmentation task,overlapping particles,blurred edges,small image samples and difficult image annotation make it a hot issue in the field of medical vision with great value and challenges.In this paper,we design a weakly supervised few shot image annotation and segmentation method.The main work of the paper consists of three parts as follows.1)To address the problems of image data scarcity and the difficulty of annotation,this paper proposes a Monte Carlo-based domain randomization image synthesis method.The method relies on only one input from the user,i.e.,labeling one particle in the original image.Considering the characteristics of the images,their semantic information is abstracted into variable parameters in the simulation space,and a large number of synthetic images and their corresponding labels are synthesized by parameter randomization.Considering the structured information of the particles in the image,three different placement rules are designed in the paper: physical simulation,radius constraints,and coordinate randomization,which reduce the computational effort brought by complete randomization while ensuring the realism and diversity of the synthesized images.To verify the feasibility and effectiveness of the method,histograms and perceptual hash are used as evaluation metrics to assess the similarity of the synthetic images to the corresponding original images.2)To address the problem that segmentation performance of the original image in the network cannot be quantified directly,this paper proposes a labeling method with auxiliary edge segmentation.Inspired by greedy algorithm,three labeling strategies are adopted to maximize the labeling efficiency.Firstly,image filtering and edge detection methods are performed to extract the edge features of all particles.secondly,a modified U-shaped edge segmentation network against on hand selected synthetic samples is trained to handle the samples that cannot be accurately segmented in the previous strategy.Then the segmentation results of the above two strategies are used for visual assistance to simplify the labeling process;Thirdly,the remaining difficult samples can make full use of prior knowledge and are annotated manually.The final annotated information will be transformed into image labels used to train the instance segmentation network,which will be used in the quantitative analysis later.3)To address the problem of manual selecting the synthetic images as training set which is equally tedious and time-consuming to labeling,this paper proposes to design an unsupervised contrastive learning network using the idea of maximizing mutual information.Firstly,the global and local features of the original and synthetic images are mapped to a high-dimensional latent space by encoder network.Secondly,the contrastive network is trained by maximizing the mutual information between feature vectors and the corresponding images in the high-dimensional space.Then the trained network learns the features of each image while optimizing the representation of features in the hidden space.Finally,the synthetic images that are closest to the original image features are selected as the final training set.Through the powerful presentation ability of the neural network,the filtered training set should have visual features close to the original image,which provides the guidance to the segmentation of the original image.Finally,to verify the usability and feasibility of the proposed method,several experiments are conducted on the labeled dataset to explore the effects of networks and training sets on segmentation accuracy.Also,we test the segmentation performance of the proposed model on different types of samples.The results show that the algorithmic process proposed in this paper achieves better results in the segmentation task of small sample medical electron microscopy images.