Improved Semantic With Multiple Clustering Based And Self-supervised Learning Based Cross-domain 3D Model Retrieval

With the development of 3D technique,the 3D models have experienced an explosive increase.Thus,how to retrieve the desired model from the huge 3D database becomes an urgent task.Due to the easy accessibility and information completeness of2 D images,this paper uses 2D images as query to retrieve the relevant 3D models from3 D database.However,the large gap between 2D images and 3D models damages the retrieval performance.Many methods first utilize deep adversarial network to eliminate the domain gap.Meanwhile,to achieve class-level alignment,they directly or indirectly employ the classifier that is well-trained on labeled image dataset to assign the pseudo labels for unlabeled 3D models.Finally,the discriminability of network can be successfully transferred into 3D domain by relying on the pseudo labeled 3D models.But the classifier is biased towards the image domain characteristics,which inevitably leads to false pseudo labeled 3D models,misguiding the training of network.To avoid the above disadvantage,we have proposed two algorithms for image-based 3D model retrieval task: 1)Multiple Clustering based 3D Model Retrieval: We first apply the multiple K-means clustering on the whole 3D features to capture the similarity information between 3D models from different semantic classification level,and then combine 3D features with similarity information to obtain the final pseudo labels for3 D domain.Finally,we align the centroid of the same category in 2D and 3D domains,which makes the discriminability of the network on 2D domain transfer to 3D domain.2)Self-supervised based 3D Model Retrieval: We perform contrastive learning via constructing the anchor,positive and negative samples on the view representations of3 D models to make the network capture the discriminative properties about unlabeled3 D models.Since the quantity and quality of the negative samples are essential for the contrastive learning,we design a repository containing all features of 3D models.In each iteration,the content of the repository will be updated by the most representative view feature of the current input 3D models based on entropy minimization principle.Experimental results on the public image-based 3D model retrieval datasets,i.e.,MI3 DOR and MI3DOR-2,show that our methods perform better than other existing methods.