Research On Image Semantic Segmentation Method Based On Interactive Complementation And Verificatio

Semantic segmentation is considered as one of the most basic but challenging tasks in the field of computer vision,and also one of the three major sub-directions of image segmentation.It has been applied in many fields,such as automatic driving,medical image segmentation and remote sensing image segmentation.Semantic segmentation aims to assign a predefined class label to each pixel in the image,which can be considered as a pixel-level classification task.For scene parsing tasks such as semantic segmentation,context information is crucial.Many models use methods such as attention,dilated convolution to capture the contextual information and long-range correlation between pixels,and use a single neural network to promote the network to automatically extract discriminative features through back-propagation of loss.But in these methods,the definition of discriminative features are vague and broad.For this,we decompose the discriminative features into two parts: the information of the pixel itself and the correlation information between pixels,and correspondingly use two branch networks to focus on the extraction of their own information.To ensure that the extracted features contain correct information,the convolutional segmentation head based on multi-scale and dilated convolution,and the fully-connection segmentation head based on fully-connection layer are used in the two branches respectively.In order to fully utilize the stability of pixels’ own information,this thesis uses Mask Average Pooling method to extract class vectors for each class in dataset and fully explore their possible uses.In order to leverage the bridging effect of class vectors between two branches,Mutual Complementation module is proposed,which combines attention,feature fusion,and other mechanisms to promote the interaction and complementation of features between two branches.Based on the correctness of class vector,this article innovatively uses class vector as template and the prediction probability map as weight to reconstruct features in Mutual Validation module,so that the prediction of two branches can have additional verification methods and optimization standards.In order to fully leverage the advantages of dual branches,Discrepancy-Guided Fusion module is proposed,which allows the model to use the differences between features as a reference and more effectively fuse the features of two branches.In addition to using class vector to achieve cross-stage residual connections for features,it’s also used in Class Vector Complementation module,which uses class vector as supplementation for context vector,so that features can be updated based on richer contextual information.As for loss functions,the traditional Cross Entropy loss is modified in this thesis.In order to extract pixel information more accurately,this thesis not only introduces Contrasted loss,but also proposes Contrasted Cross Entropy loss to better constrain features.In order to make full use of the characteristics of dual segmentation heads,Point-wise Discrepancy Cross Entropy loss is proposed,which uses the differences of two prediction as weight for Cross Entropy loss to put more attention on hard samples.Compared with the existing entropy-based weighted Cross Entropy loss,this method is more suitable for the model with dual segmentation heads.On the Dark-Zurich dataset and the Cityscapes dataset,a large number of experiments were conducted to prove the effectiveness of this method.