Research On Shoeprint Retrieval Algorithm Based On CNN

Shoeprint retrieval is an important research direction in forensic science.Shoeprint retrieval refers to the retrieval of the same shoeprint as the suspected shoeprint from the shoeprint database,that is,to be retrieved.The challenge of shoeprint retrieval mainly includes the following two aspects: one is the difficulty of pattern recognition caused by background interference and pattern blurring;the other is the incomplete shoeprint.This paper studies the shoeprint retrieval algorithm based on convolutional neural network.Aiming at the difficulties of shoeprint retrieval,the following work is carried out.Firstly,we researched the related literature about shoeprint retrieval over the past 30 years,combed the development history by artificial retrieval to the development of automated retrieval,introduced the previous research work from four aspects: transformation domain based method,SIFT based method,deep learning based method and other methods,and summarized the development trend.Secondly,a shoeprint dataset for training containing 2827 images of 432 categories was established and further expanded to 228987 images.In order to test the performance of the algorithm,CSS-200 shoe-print test data set was established and two public test data sets,FID-300 and CS-Database,were introduced.We screened the partial shoeprints in FID-300,and sorted out a test dataset Part-FID containing 139 partial shoeprints of 85 categories.Thirdly,the pre-trained VGG-16 was fine-tuned using the training dataset,and the network was used as a feature extractor to retrieval shoeprints.The difference of retrieval accuracy between the pre-trained model and the fine-tuned model was compared,and it was found that the retrieval accuracy of the fine-tuned model was greatly improved.The features of conv5-1,conv5-2,conv5-3,fc6 and fc7 layers were extracted,and it was found that the accuracy of feature retrieval using convolution layer was higher than that using full connection layer.Among different convolutional layers,the feature retrieval accuracy of Conv5-3 layer is lower than that of Conv5-1 and Conv5-2 layer,and Conv5-1 layer has the highest accuracy.In CSS-200 dataset,the recognition rate of Top1 and Top10 improves from 27.5% to 52%,and from 51% to 75.5%.Fourthly,in order to solve the problem of poor results of incomplete shoeprints using convolutional feature,selective convolutional descriptor aggregation method was introduced.We used this method to select the convolutional feature of broken shoeprints and the results shows that the accuracy has been improved significantly.Using conv5-1 layer in CSS-200,the recognition rate of Top1 is increased from 52% to 62.5%,and the recognition rate of Top10 from 75.5% to 85.5%.Experimental comparisons on datasets FID-300,part-FID and CSDatabase show that the proposed algorithm still has a gap with the highest level of shoeprint retrieval at present.This gap is mainly reflected in those shoeprints with very low quality(high blur,serious background interference and large defect area).The results point out the direction for further research.