Towards The Unsupervised Instance Search Based On The Pre-trained Neural Network

Instance search aims to hunt for images in which the query instance appears,furthermore localize the instances in the images.In recent years,the application of instance search arises in different scenarios,such as product searching in online shopping and video object searching.Due to the variety of instance categories,searching and localizing instances from the generalized categories become the main challenges of instance search task.Thanks to the promising performance of deep convolutional network in visual understanding,existing solutions based on deep convolutional neural network become increasingly popular,where object detection frameworks are leveraged to enable instance localization.However,most of them are supervision approaches,which require full annotation on the training set.This in turn makes the trained model insensitive to unseen object categories.The aim of this thesis is to explore solutions that require very few supervision for both instance-level feature representation and localization tasks.Firstly,the existing approaches require demanding training conditions and are only sensitive to the known instance categories.To address this issue,an instance-level feature representation based on the pre-trained neural network is proposed in the thesis.The instance localization is achieved by detecting deep salient regions through a probability back-propagation model.Namely,the probabilities are obtained with convolutional weights and responses layer-by-layer,which allows to highlight the class-agnostic instances.The instance-level features are extracted based on the detected instance regions.Experiments show that the proposed instance-level feature representation outperforms the existing solutions on several instance search benchmarks.While only based on the pretrained classification model,the proposed approach demonstrates competitive localization performance with the existing approach based on weakly supervised object detection.Secondly,to further alleviate the issue of imprecise localization,an unsupervised localization refinement method for instance search based on co-localization is proposed in this thesis.Specifically,a co-localization method is carried out on each group of retrieval results for refining.Along with the pre-trained network model,features are extracted from the responses of high convolutional layers,from which the principal component is calculated for co-localization.Based on the co-localization results,precise bounding boxes of instances are produced using a clustering algorithm.These boxes could be used to refine the original boxes of retrieval results,or to update the candidate instance search database for better localization.Experiments show that,the proposed method achieves considerably better performance in terms of both localization and search quality in comparison to state-of-the-art approaches.