Weakly Supervised Semantic Segmentation

Semantic segmentation is a kind of fine-grained scene understanding task,which plays a vital role in applications such as autonomous driving and computing images.With the development of deep learning,semantic segmentation has also made significant progress with the deep convolutional neural network.However,since semantic segmentation requires accurate pixel-level labeling,compared to classification models,it greatly increases the difficulty and cost of labeling,further limiting the application and expansion of semantic segmentation tasks in new scenarios.For this reason,this paper explores how to ensure the effectiveness of the segmentation model in the case of weak supervision,that is,in the case of reducing the number or quality of data annotations.For the semantic segmentation task with inexact supervision,this paper proposes a Collaborative Segmentation Network for semantic segmentation(CSS)with weak supervision inspired by the collaborative learning.The CSS consists of a localization sub-network and a segmentation sub-network.The localization sub-network takes only image-level labels as supervision and generates attention maps.The segmentation sub-network is supervised by the pseudo masks from both the segmentation sub-network and the localization sub-network due to the lack of fine labels.Because of the model architecture and supervision type are different,the pseudo masks from the localization sub-network provides rough localization information of objects and those from the segmentation sub-network provide the shape information.Considering at the initial stage of model training,the prediction of the segmentation subnetwork is less accurate than that of the localization subnetwork,this paper designs an adaptive mask mixup strategy,which gradually increases the mixup ratio of the pseudo masks from the segmentation subnetwork.The two sub-networks share the same backbone and are expected to mutually enhance each other during training.To validate the proposed method,this paper experiments with the PASCAL VOC 2012 Semantic Segmentation benchmark.Experimental results have shown that our method reaches 65.7% and 65.8% m Io U scores on val and test sets respectively and outperforms the state-of-the-art methods.For the segmentation task with incomplete supervision,this paper introduces the consistency regularization into semi-supervised segmentation and proposes a High-Order Consistency Regularization Based Semi-Supervised Segmentation(HCRSS)method.By enforcing the consistency of the segmentation network for unlabeled samples under different perturbations,the purpose of using the data distribution of unlabeled samples is achieved.In this paper,image-patch level augmentation with Cut Mix is first applied to semi-supervised semantic segmentation,which provides an efficient perturbation method.Based on this,this paper proposes the use of adversarial learning for capturing the relation between pixels.By designing a discriminative network to determine the true degree of the segmentation network result,the segmentation network is forced to produce more realistic predictions after perturbations.This enforces the high-order consistency between the prediction results before and after perturbations.The experimental results on the Cam Vid dataset show that under the premise of using different proportions of labeled data,by using additional unlabeled data,the method in this paper can improve by 1.3% to 2.9% over the supervised learning algorithm.In the case of using only half of the labels,the algorithm in this paper reaches 72.18% and 62.42% in the val and test sets,which is close to or even better than the performance of the baseline algorithm when using all labels.