Research On Remote Sensing Multimedia Retrieval Based On Attention And Re-ranking

With the rapid development of satellite remote sensing image technology and the popularity of the Internet,high-resolution remote sensing images become more and more easy to obtain,and at the same time,the number of remote sensing images has been growing explosively in recent years.In the face of large-scale remote sensing image data,how to retrieve the required remote sensing image quickly is an urgent problem to be solved in the current digital earth system construction process.In the task of data retrieval,the deep feature extracted by deep neural network has better retrieval effect than the manual feature extracted by traditional method.However,it takes a lot of time to calculate the similarity distance between high-dimensional continuous deep features.In order to retrieve quickly and accurately,the method based on deep hashing is proposed.According to the different modality of query data for retrieving remote sensing image,this paper mainly uses the deep hashing technology to study the singlemodal content-based remote sensing image retrieval and the cross-modal audio-remote sensing image retrieval tasks.The main work and contribution of this paper are as follows:(1)We introduce the working principle and application of deep neural network;summarize the general framework of deep hashing algorithm,and further summarize the classic single-modal deep hashing retrieval algorithm and cross-modal deep hashing retrieval algorithm;briefly describe the researches on single-modal retrieval and crossmodal matching of remote sensing image,and analyze the shortcomings and challenges in the current researches.(2)In order to solve the problems that the extracted features are easily disturbed by irrelevant background information and express insufficiently in remote sensing image retrieval,a hashing retrieval algorithm based on attention is proposed.The proposed algorithm contains an attention mechanism which takes into account the channel and spatial information of remote sensing image to extract more differentiated visual information;designs a hash layer that can return gradient back for optimizing discrete hash code;introduces category information as the weight of Hamming distance measurement to further improve the retrieval accuracy.Experimental results show the proposed algorithm can achieve state-of-the-art retrieval results.(3)In order to explore the relevance and reduce the semantic gap between audio and remote sensing image,a novel hashing audio-image retrieval method based on reranking is proposed.The proposed method maps audio and image into a uniform feature space;designs an audio-audio hashing retrieval network to match the related audio;and adopts an audio-image hashing retrieval network for every related audio to retrieve related images,and the most frequent image is voted as the final result.Extensive experiments on two remote sensing cross-modal data sets demonstrate that the proposed method can visualize the content of audio.