Research On Few-Shot Semantic Segmentation Method

In recent years,thanks to the powerful automatic feature extraction capability of deep convolutional network,the segmentation model based on deep convolutional network has greatly improved the accuracy of image semantic segmentation.However,these models have defects,they all rely on a large number of training image samples of pixel-level marks and difficult to generalize to the new object segmentation class.In order to solve the above problems,few-shot semantic segmentation task is proposed,which aims to train with a small amount of sample data and test on a new class.The difficulty of few-shot semantic segmentation task is mainly reflected in two aspects:First,how to extract features from small sample data,distinguish commonness and characteristics of various features,and make small sample data compatible with high complexity model without causing network overfitting;Second,how to make the network have good generalization performance,achieving segmentation on the new classes.Many scholars have carried out research on this task and have put forward some few-shot semantic segmentation algorithms.Aiming at the defects of the existing few-shot semantic segmentation algorithm,this paper carries out research from the following two aspects:1)Existing few-shot semantic segmentation model are usually only to extract single level semantic feature,but due to the small sample data of less sample size and different attributes of semantic features at each level,it is difficult for the network model to extract semantic features at a single level to ensure the segmentation ability and generalization.To solve this problem,a few-shot semantic segmentation based on inter-set semantic complementarity of two levels is proposed.In this method,the high-level semantic features of the support set with strong category are used to weight the generalization middle-level semantic features of the query set,and the generalization capability of the query set semantic features is preserved while the features of the query set target categories are enhanced.In addition,the model enhances the interaction between the two sets of information by maximizing the potential semantic information of the support set and constructing non-parametric learning prior information for the query set,so as to obtain richer discriminant information.Using mean-Io U evaluation index on the PASCAL-5~idata set,the segmentation accuracy of the model can reach 44.6%in 1-way 1-shot setting and48.8%in 1-way 5-shot setting.Binary-Io U evaluation index can be used to test the segmentation accuracy of 64.3%in 1-way 1-shot setting and 67.9%in 1-way 5-shot setting.The test results of the above simulation experiments prove the feasibility and validity of the few-shot semantic segmentation based on inter-set semantic complementarity of two levels.The proposed model has better segmentation effect for categories with more complex features,because the network can make full use of the potential information contained in the middle-level semantic features to provide auxiliary help for the foreground,thus improving the generalization ability of the network.In addition,we conducted ablation experiments for the two-level semantic attention mechanism module and the priori information module of non-parametric learning,and the experimental results prove the role of the above two modules in the process of few-shot semantic segmentation.2)Due to the small sample size of small sample data,the pre-training model is usually used for feature extraction of small sample data in the experimental process.However,the source domain of the data set of the pre-training model is different from the target domain of the task,which leads to the error prediction of the task target by the few-shot semantic segmentation model under the influence of pre-training knowledge.In order to solve this problem,few-shot semantic segmentation model based on causal learning is proposed.The model is based on the theoretical modeling of structural causal model,and the causal relationship between pre-training model and query set is established by intervention.In order to avoid loss too much context information due to the causal model,i AFF module is used to supplement the relevant target information.The model was tested using mean-Io U evaluation index on the PASCAL-5~idata set,the segmentation accuracy was 43.4%on the 1-way 1-shot split1.Binary-Io U evaluation index can be used to test the segmentation accuracy of 64.9%in 1-way 1-shot setting and 66.0%in 1-way 5-shot setting.The test results show that the few-shot semantic segmentation model based on causal learning proposed in this paper has a better segmentation effect on the foreground of the relatively complex background information categories.The model can effectively grasp the causal relationship between various features and improve the expression ability of the model.However,for the categories with extremely complex background features,the segmentation effect of the proposed model is not good,because causal learning weakens the context relationship between various features and loses a lot of auxiliary information that is conducive to foreground segmentation.In addition,we conducted ablation experiments for the causal learning module,and the experimental results prove that the causal learning module can effectively eliminate the influence of pre-training knowledge.