Key Technologies Of Low Quality Shoeprint Image Retrieval

As a kind of forensic evidence,shoeprints play a vital role in forensic investigations.For a shoeprint image collected at a crime scene,it is a challenging task to retrieve the most similar shoeprints in the dataset with large amounts of low quality shoeprint images.With regard to low quality shoeprints collected at crime scenes,the existing shoeprint retrieval algorithms couldn't achieve expected performance.The main reasons are that,(1)shoeprints derived from crime scenes are usually incomplete and degraded,it is difficult to represent and match shoeprints;(2)low-level features can't represent semantic information contained in images effectively.In this dissertation,three key technologies about low quality shoeprint image retrieval are studied.To solve the problem that it is difficult to represent the low quality shoeprint images,a multi-grained representation method is proposed.The method represents low quality shoeprint images by extracting features from holistic perspective,local perspective and detail perspective.Firstly,the holistic-grained feature and local-grained feature are extracted to represent the shoeprint image texture from holistic perspective and local perspective.Then,the fine-grained feature is extracted to describe shoeprint details from detail perspective.In terms of multi-grained feature fusion,an adaptive feature fusion method is proposed to fuse the multi-grained features,and the method computes the fusion weight coefficients adaptively.Experiment results show that the proposed multi-grained representation method can represent shoeprint images effectively.The results also show that the proposed adaptive feature fusion method can take advantages of each level of features and improve the performance of shoeprint retrieval.To solve the problem that it is difficult to match low quality shoeprint images,a neighbor graph based similarity measure method is proposed.Firstly,the similarity of neighbor graph is used to measure the similarity of images,and the similarity between two shoeprint images is defined as the average similarity of the maximum-matched similar neighbor pairs.Then,multi-feature graphs are constructed and merged,and images on the fused neighbor graph can be viewed as candidate images.Finally,similarity measures are fused at the score level,and the ranking scores can spread effectively between those candidate images on the fused neighbor graph,and the ranking scores of those candidate images can be assigned high values.Experiment results show that the proposed neighbor graph based similarity measure method can measure similarities of low-quality shoeprint images effectively,and further improves the performance of shoeprint retrieval.To solve the problem that the gap between low-level visual representation and semantic information of an image makes the retrieval results not accurate,an experts'opinions guided shoeprint image ranking score computation method is proposed,which includes an offline model training stage and an online retrieval stage.At the offline model training stage,the opinion scores are labeled by experts according to semantic similarities of images,and theopinion score model is constructed to establish relationship between the opinion scores andfeature similarities.At the online retrieval stage,the opinion scores predicted by the learnedopinion score model are used to refine the user's opinion scores,and the refined opinion scores are embedded in the ranking score computation method to guide ranking score computation.Experiment results show that the proposed ranking score computation method can narrow the gap between low-level visual feature similarity and semantic similarity and achieve much higher retrieval performance.To verify the effectiveness of the proposed method,experiments were conducted on the crime scene shoeprint image dataset MUES-SR10KS2S and the public available shoeprint image dataset FID-300.The cumulative match scores of the proposed method are more than 96.4%and 82.5%in the top 1%of the ranked lists,respectively,which outperform those of the state-of-the-art shoeprint retrieval methods by 6.3 percentage points and 2.8 percentage points,respectively.The experimental results show that the proposed method can effectively address the key problems that low quality shoeprint retrieval faces,which include representation of degraded shoeprint images,similarity measure of low quality images,and the gap between low-level visual feature similarity and semantic similarity,and as a result,the retrieval performances have been improved.