An Object Retrieval And Localization System Based On Structure-Aware Compositional Sketches

Scalable instance-level image search has recently attracted considerableattention: images are retrieved for containing the same object instance as theone present in a given query photo. Several hashing methods have been developedto allow approximate but very efficient retrieval, but they often suffer fromlow recall rate due to the insufficient discriminability of imagerepresentations (e.g., low-level features and linear models). In this paper,we integrate the compositional structure-preserved object representation withthe min-Hash method, leading to a simple yet effective retrieval framework, inwhich the object localization is further solved alone with image search.The contributions are three-fold as follows.(i) A new image feature, namelyPair of Geometric Coupled Words (PGCW), is presented to impose spatial contextinto visual words, and PGCW are used to produce very discriminative hashfunctions.(ii) We further select a batch of hashing functions into groups, i.e.sketches in min-Hash, by learning with a number of supervised retrievals.Specifically, we first learn a conditional random field (CRF) model for a smallset of training examples. The sketches are then generated by selecting thehashing functions based on the learned object model. We thus regard each sketcha structural object configuration.(iii) In the step of hash matching with thesketches, we introduce an auxiliary offset space, into which the location ofeach available matching is projected, and the object localization can be thusestimated by clustering.We conduct the experiments on several public image retrieval databases, e.g.,Oxford5k, Paris and Kentucky. The experimental results demonstrate superior performance of our method over other popular state-of-the-art methods.