Research On Sketch-based Fashion Image Retrieval Method

Due to the rise of e-commerce platforms,online shopping has become a trend.However,the current mainstream retrieval methods are still limited to using text or exemplar images as input.For huge commodity databases,it remains a long-standing unsolved problem for users to find the interested products quickly.Different from the traditional text-based and exemplar-based image retrieval techniques,sketch-based image retrieval(SBIR)provides a more intuitive and natural way for users to specify their search need.Due to the large cross-domain discrepancy between the free-hand sketch and fashion images,retrieving fashion images by sketches is a significantly challenging task.This paper studies the method of sketch-based fashion image retrieval,and mainly completes the following tasks:In order to implement sketch-based fashion image retrieval,we have collected a fine-grained Fashion Image dataset.The dataset contains fashion images of 4 categories and 26 sub-categories,and a total of 36,074 sketch-photo pairs.As far as we know,it is the first comprehensive fashion image dataset.A new algorithm for sketch-based fashion image retrieval based on VAE-GAN is proposed.In our method,first,the sketch is transformed into photo.Second,the deep features of the transformed photo and all photos in the dataset are extracted respectively.Third,the similarity distance between the feature of the transformed photo and the features of all photos in the dataset is calculated.Finally,the search result is returned to the user according to the similarity distance.A new algorithm for sketch-based fashion image retrieval based on cross-domain transformation is proposed.The algorithm consists of three modules,which are crossdomain transformation module,cross-domain feature extraction module,and crossdomain similarity measurement module.In our approach,first,the sketch and the photo are transformed into the same domain.Second,the sketch domain similarity and the photo domain similarity are calculated respectively and fused to improve the retrieval accuracy of fashion images.Extensive experiments conducted on our Fashion Image dataset and two fine-grained instance-level datasets,i.e.,QMUL-shoes and QMULchairs,show that our model has achieved a better performance than other existing methods,and effectively improve the accuracy of user to retrieval fashion image: When using this algorithm to retrieve the collected Fashion Image dataset,the correct result of clothing retrieval is ranked top-1 with an accuracy of 96.6%.The accuracy of the correct result of the pants search ranked top-1 is 92.1%.The accuracy of top-1 for skirt retrieval is 91.0%,and the accuracy for top-1 retrieval of shoes is 90.5%.