Research On Semantic Segmentation Based On Optimal Transport And Domain Adaptation

Image semantic segmentation aims to classify images at the pixel level,allowing computers to understand each scene in an image in greater detail.This technique occupies an extremely important position in the field of computer vision and has been widely used in areas such as autonomous driving,medical image analysis,and video content analysis.With the rapid growth of deep neural networks,most existing semantic segmentation methods require the help of large amounts of manually labeled data to train a deep model for prediction.However,the human as well as monetary costs for such manual annotation of pixel-level labels are very expensive.To reduce the burden of image annotation,a common approach is to use computer-generated synthetic images as training data.In this way,an almost inexhaustible amount of labeled data can be obtained,but the models trained by synthetic data usually do not generalize well to real scenes due to the presence of domain shift.To solve this problem,this paper tackles the semantic segmentation task using an unsupervised domain adaptation technique,which aims to migrate the knowledge from the synthetic image(source domain)to the real image(target domain),thereby enabling the model to more accurately classify the pixels of the target domain images.In this paper,two network models with different structures are built based on two training strategies,adversarial training and self-training,respectively,for the domain adaptive image semantic segmentation task,and the distribution difference between the two domains is reduced using optimal transport theory.The main research of the paper includes the following two aspects:(1)In order to reduce the domain shifts that often occur in semantic segmentation tasks,the domain adaptive semantic segmentation model OTGADA based on adversarial training is proposed.In order to make the probability distribution of the target domain closer to that of the source,OTGADA applies the concept of optimal transport to narrow the distribution distance between two domains,thus improving the segmentation accuracy of the model.In addition,OTGADA also applies the MM Segmentation semantic segmentation framework,which makes the training time of the model significantly reduced.Finally,the effectiveness of the proposed OTGADA model is verified with the help of a series of adaptive experiments.(2)Since the trade-off between the generative and discriminative structures may be imbalanced in the adversarial training-based model,another self-training-based domain adaptive semantic segmentation model,OTCLDA,is proposed.OTCLDA applies the idea of contrastive learning to improve the intra-class compactness and inter-class separability of the target domain image pixels,thus improving the performance of the model.The optimal transport theory is also applied to OTCLDA,which makes the distribution between two domains more similar.Finally,a series of adaptive experiments are applied to verify the effectiveness and accuracy of the proposed method.