Image Retrieval Oriented Deep Representation And Coding

With the development of mobile Internet and multimedia technology,there has been a lot of images as the representative of the huge amount of multimedia data,and the amount of data increases rapidly every day.Images on the Internet are usually in the form of compressed bitstream to save storage.An important application for im-ages is content-based image retrieval(CBIR).A common practice is to extract image features,then convert these feature vectors to bitstream for storage.To store the data generated by image compression and image retricval separately will cost tremendous system resources.Can the bitstream of images and image features be unified and fur-ther condensed?If it is possible that the same binary code serves for compression and retrieval simultaneously,lots of system resources would be saved.There are many classical standards for image compression,such as JPEG.These standards usually keep main information of images and leave out the less important information.Similar method is used in image retrieval.The exacted feature vectors preserve main information of images,thus could be used for image retrieval.Since both compression and retrieval extract main information of images and store it in binary form,there must be information redundancy between them.This work aims to reduce,even eliminate the redundancy,and decrease the cost of system resources that storage of bitstream requires.There are two ways to measure whether information redundancy is eliminated.One of them is that the size of bitstream for coding is far less than the size of bitstream for solely compression plus feature vectors without reducing performance.The other one is that performance is improved while the size of bitstream for coding is equal to solely compression plus feature vectors.We choose the second measurement as our aim and experimental method.To address this problem,we proposed a unified image deep coding method.For the typical CBIR scene,image search engine,we choose to com-press and recover thumbnails with relatively small sizes.Input images are coded with deep neural network so that the codes could be used to reconstruct the original images and retrieve images simultaneously.Similarity between images while retrieval is de-fined with Hamming distance between binary codes.These codes could be reused with our coding system,information redundancy between image compression and retrieval is thus reduced.We first train a deep network for compressing thumbnails into bitstream.Original thumbnails could be reconstructed from the bitstream with decoder.We then train an-other deep network for extracting image features as binary vectors.These vectors could be stored in binary form,and could be used for image retrieval.We combine the above two networks,and fine-tune the combined network using triplets of images for the task of CBIR.The combined codes are used for image retrieval.Our experimental results show that the proposed scheme achieves a compression ratio of 5.3 for 32×32 thumbnails,outperforms JPEG at similar compression ratios,and the resulting code is directly available for CBIR.Besides,the retrieval results using codes from our unified coding system also outperforms retrieval results with binary vectors from image feature extractor.The performance of image retrieval is improved without extra memory cost.Information redundancy between image compression and retrieval is reduced.Our work indicates a promising direction of simultaneous image compression and retrieval.