Cross-Domain 3D Model Retrieval Based On Sketches

3D model retrieval with inputting sketches is convenient for users to express their retrieval demand,and has become a research hotspot.Using sketches to retrieve3 D model has attracted extensive attention in the fields of computer graphics,image retrieval and computer vision.Compared with other retrieval methods,sketches have great advantages,such as intuition and convenience,but it still has some disadvantages.For example,sketch is only a rough description of 3D model from one viewpoint,with high abstraction and strong subjectivity.Moreover,sketches and the 3D models are cross-modal data in different domain spaces,and there are great differences between them.This leads to the low accuracy of the current sketches retrieval system.In order to solve this problem,we propose a 3D model retrieval method based on sketches,which mainly includes the following three aspects:1.We propose a 3D model projection method based on viewpoint groups and a multi-level pseudo-sketches representation method based on Canny operators.This method solves the problem that the sketch viewpoint is different from the 3D model viewpoints and the sketch and view information details are too different,while reducing the cross-domain difference between the two,which is convenient for the later retrieval.Firstly,we propose a 3D model projection method based on viewpoint groups,we can project the 3D model according to the viewpoint of the sketch to obtain the view and form a one-to-many sketch-view pairs.Therefore,we preprocess the views and make the representation information of the views more similar to the sketches.We use Canny operator to extract the pseudo sketches of each view.By setting different thresholds,different levels of pseudo sketches will be generated.Multi-level pseudo sketches participate in training,which not only enriches the amount of data and data diversity,but also further reduces the difference between domains.Experiments show that the proposed method can effectively use the 3D model view based on sketches viewpoints and improve the accuracy of 3D model retrieval.2.We propose a feature extraction method based on modified Sketch-A-Net.It solves the problem that it is difficult for ordinary Convolution Neural Network to extract effective features because of the small amount of information of sketches and pseudo sketches.By introducing Sketch-A-Net into this task,and improving and pre training according to the actual needs of the task and the experimental effect.Experiments show that the improved sketch-a-net has stronger feature expression ability,and speeds up the convergence speed of subsequent network training.3.We propose a sample selection method based on Triplet metric learning network.The problem of low accuracy of cross-domain matching and large difference of retrieval accuracy of different classes is solved.The sample selection method based on the Triplet Metric Learning Network proposed in this paper is a sampling strategy,including the average random selection method and the average screening combination method.The average random selection method uses the number of models as a benchmark to realize a consistent number of views,and the average screening combination method is to consider only the number of views without considering the number of models.Both methods aim to achieve a balanced training data.After sample selection and feature extraction,the features are embedded into the common feature embedding space through Triplet network.Experiments show that the proposed method effectively reduces the difference of retrieval accuracy between different classes and improves the average retrieval accuracy.