Research On Sketch-image Retrieval Method Based On Multi-feature Fusion

In recent years,with the blooming development of Internet technology,touchable electronic devices such as mobile phones and tablet computers have gradually become an indispensable part of people’s lives,people can easily draw the outline of an object in the form of hand-drawn sketch on the mobile terminals anytime and anywhere,therefore,sketch-image retrieval has gradually become one of the hot spots in the research field of computer vision.In the field of intelligent building,it is an urgent problem to search CAD drawings by sketch,which can reduce the restriction of the existing drawing file management system to the retrieval condition and increase the flexibility of the retrieval method.In the intelligent security system,the witness can greatly reduce the search scope and save the search time by comparing and analyzing the sketch of the suspect in the criminal image database of the public security system.Hand-drawn sketches only contain a small amount of outline information and lack color and texture information,while natural images contain rich texture and color information.Therefore,how to effectively extract the features of sketches and natural images,cross-domain retrieval between two different kinds of images has become the focus and difficulty in sketch retrieval field.Considering that the traditional manual features have the advantages of clear purpose,strong explanatory power and high efficiency,and the depth feature can effectively obtain the feature representation which is closer to the semantic level,therefore,this paper will combine the advantages of the two methods mentioned above,and combine the characteristics of simple outline and abstract semantics in hand-drawn sketches,in this paper,the traditional manual feature of edge contour and the depth feature of semantic information are fused,and a new sketch/image feature representation is obtained to narrow the gap between sketch and natural image.The main research contents are as follows:（1）We choose two different network models-Alex Net and VGG16 Net,to extract the features of sketches and natural images,and obtain sketch retrieval results,which show that Alex Net is better at retrieving sketches.Then,by changing the number of convolutional layers of Alex Net network,the experiment results show that the shallow layer network is more suitable for sketch feature extraction because of the lack of sketch information.（2）We present a new sketch-image retrieval method,which combines the traditional manual feature HOG with the depth feature based on Alex Net to form a new feature representation,and then carries out similarity retrieval.This method not only has the advantages of traditional manual features,but also can overcome the influence of light factors in images.At the same time,it combines the advantages of deep learning,is insensitive to image deformation and rotation,and can learn better semantic information.By fusing the features obtained by the two methods,the performance of sketch retrieval is improved.A comparative experiment on the published data set shows that the sketch retrieval method based on the fusion of HOG and depth features is superior to the retrieval method using single feature,which proves the feasibility and effectiveness of this method.（3）We present a new feature fusion method,which combines the traditional manual feature HOG with the depth feature based on Alex Net based on full connection non-linear feature fusion to form a new feature representation,and then similarity search is carried out.The fully connected non-linear feature fusion method not only synthesizes the advantages of traditional manual feature and depth feature,but also can effectively depict the edge contour information of the image and obtain the feature which is closer to the semantic level,the performance of sketch retrieval is improved by fully connected non-linear fusion.Compared with other typical feature fusion methods on the open data sets,the results show that the proposed method is superior to other feature fusion methods.Figure [31] table [9] reference [53]...