Research On Unsupervised Domain Adaptation Based On Optimal Transport

With the rapid growth of datasets and the improvement of hardware computing resources,machine learning techniques have rapidly developed and achieved notable progress in various applications.Traditional supervised learning methods have been used in training models for specific domains(source domain)using the annotated data.However,the performance shows significant degradation when applying the previously trained models to other domains(target domain)that do not satisfy the independent and identically distributed assumption.Unsupervised domain adaptation aims to leverage the abundant knowledge from the source domain to the unlabeled target domain and alleviate the impact of distribution divergence between domains.Optimal transport provides a theoretical foundation for effectively measuring the distribution discrepancy,under the expectation of establishing a globally optimal mapping between the source and the target by exploring the geometric structure of the latent measurable space,aims to minimize the distribution discrepancy by constructing optimal transport between samples,thereby enhance the performance of the model in the target domain.This dissertation delves into unsupervised domain adaptation based on optimal transport within the context of deep learning,concentrating on two typical scenarios,single-source and multi-source domain adaptation,and proposes several novel methods to address specific challenges.The main contributions of this dissertation are summarized as follows.(1)To alleviate the negative transfer of single source unsupervised domain adaptation,a Decomposed-Distance Weighted Optimal Transport(DDW-OT)method is proposed.DDW-OT constructs category prototypical representations of the target domain by utilizing the spatial clustering information.The decomposed distance for reweighting the original transport cost matrix captures the uncertainties of target categories and the inter-class correlations simultaneously,improving the measurement of sample relationships and optimizing inter-domain transportation.In addition,DDW-OT uses fully connected neural networks to parameterize the Kantorovitch potentials when solving the dual form of the optimal transport problem,thereby promoting the fitting to the optimal transport distance.The experiments are conducted on four commonly used publicly available domain adaptation datasets.The results demonstrate that DDW-OT can outperform existing advanced methods,and the effectiveness of each proposed module is also validated.(2)To enhance the class-conditional distribution alignment of unsupervised multi-source domain adaptation,a Class-Aware Sample Reweighting(CASR)optimal transport method is proposed.CASR uses a class-aware sampling strategy to sample a specified category of samples from multi-source and target at each training iteration with the pseudo labels of the target samples predicted by the classifier,which contributes to establishing inter-domain transport within the same class.The transportation is conducted between unified multi-source and the target to alleviate the impact of distribution biases between multi-source.In addition,CASR computes sample weights to reweight the mass of the original uniformly distributed samples,according to the prediction uncertainty and the complex spatial relationships for both target and multi-source,thus achieving optimized adaptation.The experiments are carried out on four commonly used scene datasets.The experimental results demonstrate the superiority of the CASR,and each module proposed is proven to enhance the performance significantly.(3)To enhance the ability to model complex distribution relationships between samples of unsupervised multi-source domain adaptation,a learnable Sample-Weighting Partial Optimal Transport(SWPOT)method is proposed.SWPOT utilizes a shallow neural network as learnable parameters to parameterize the samples’ weights from multiple source domains,which are optimized alternately during training,encouraging assigning higher weights to source domain samples that contribute to minimizing the transport distance.Additionally,SWPOT utilizes partial optimal transport to measure the distribution discrepancy between batch samples,thereby promoting global optimal transport.Furthermore,a mass fraction optimization technique is proposed to adaptively adjust the transport proportion during each training iteration,devoted to alleviating inaccurate inter-class transportation.The experiments are conducted on three commonly used publicly available datasets,and the results validate the superiority of the SWPOT and the effectiveness of each module.In summary,this dissertation focuses on the unsupervised domain adaptation methods based on optimal transport,involving single source and multi-source data scenarios.Attempting to transfer the source knowledge to the target by minimizing the transport cost to reduce the distribution discrepancy,which provides new methods for solving the negative transfer,class-conditional distribution alignment,and complex distribution relationship modeling in unsupervised domain adaptation.