Remote Sensing Image Content Retrieval Based On Image Learning Representation And Reranking

Remote sensing(RS)image retrieval(RSIR)is one of the hot research topics in the remote sensing community.Conventional retrieval methods mainly depend on the metadata of RS products,such as the type of satellite,the information of geographic location,etc.This text-based retrieval architecture,however,cannot stratify the users' complicated demands increasingly with the number of RS images increases dramatically.Thus,the content-based image retrieval(CBIR)draws the researchers' attentions.Compared with the natural pictures,there are many special characteristics in the RS images.For example,the contents within an RS image are large in quantity and diverse in type.Consequently,the RS retrieval results obtained by many mature methods(proposed in the natural image processing community)are not satisfactory.Recently,researchers introduce an ocean of RSIR approaches to overcome different technical fortresses,ranging from the simple similarity metric learning to the complicated high-level semantic extraction and image annotation.Based on the background mentioned above,this thesis proposes a series of algorithms to accomplish the RSIR task,which are summarized as follows:1.Taking the special properties of Synthetic Aperture Radar(SAR)images into account,a general-purpose SAR image retrieval method is proposed on the basis of the semantic classification and improved region-based similarity measure.When users input a query image,our method first classifies it using the state-of-the-art semi-supervised learning.Then,to avoid the negative influence of inevitable classification error,we expand the query's semantic label to the label set using the confusion matrix and the empirical posterior probability.Third,the improved integrated region matching measure is developed to compute the similarities between the query and the target SAR images which have the same labels with the query,and the retrieval results are acquired according to the calculated similarities.The novelties of this work can be summarized as i)not only the matching space is reduced but also the semantic gap can be partly solved by the semantic classification,and ii)the new measure is developed based on special characteristics of the SAR image.2.Taking the speckle noise within the SAR image into consideration,we introduce another SAR image retrieval method based on the superpixel and fuzzy theory.In addition,the multiple relevance feedback scheme is proposed to refined the initial retrieval results.The reasons for choosing the superpixel are that it can depress the speckle noise within the SAR images,and that the superpixel can reduce the computational cost.Moreover,we use the fuzzy theory in our method to deal with the issue of segmented uncertainty and describe the blurry boundaries between segmented regions.The initial retrieval results obtained only by similarities cannot always satisfy the users,so that we adopt the relevance feedback technical to refine the initial results with the help of the users.Since the performance of single relevance feedback is restricted to certain kinds of images,we expand it to multiple scenario,which is achieved by the different active learning.3.Considering the characteristics of the SAR image,we propose an effective image reranking method for SAR image retrieval.Similar to the relevance feedback mentioned above,image reranking is a post-processing method which can improve the retrieval performance.Different from the relevance feedback,image reranking enhance the retrieval behavior via deeply mining the content relationships between the images rather than depending on the users' help.In addition,image reranking only takes the top ranked retrieved images into account to improve the retrieval performance.This work extracts the diverse SAR-oriented visual features first to roundly describe the SAR images.Then,different sets of relevance scores of retrieved images are estimated under different modalities(feature spaces).Meanwhile,the modal-image matrix is constructed according to the obtained relevance scores,and the fusion similarities between SAR images are calculated using this matrix.Finally,the reranked results are acquired by an existing reranking function.4.The conventional image reranking method always directly use the top ranked retrieved images to improve the retrieval behavior.Although this scheme can ensure the images for reranking are more relevant to the query,some mixed noisy images(which are not similar to the query)are ignored.In addition,the users' opinions are totally neglected during the reranking procedure,which can reduce the workload of users but increase the risk which the reranked results cannot reach the users' demands.To overcome the limitations mentioned above,we propose a two-stage reranking method and apply it to improve the performance of general RSIR.In the fist stage,an editing scheme is developed to find out and eliminate as many noisy images as possible,which ensures the remaining RS images are more relevant to the query and satisfy the users.In the second stage,we propose a multi-similarity fusion reranking algorithm to rerank the images,and an alterant optimization method is developed to solve the different parameters within the reranking objective function.The effectiveness of this method is verified on different RS image databases.5.To reduce the workload of users and ensure the reliability of the reranked results,we propose the joint reranking method to improve the RSIR.This work splits the reranking into coarse and subtle process to complete the reranking.First,we analyze the reranking problem from the Bayesian theory and propose the late fusion scheme to rerank the top ranked retrieved results coarsely,which leads to the remaining images for subtle reranking are more similar to the query.Second,we use the proposed multi-similarity fusion reranking algorithm to rerank the retaining images.To decrease the computational cost of solving the reranking function,we propose a new combination optimization algorithm to optimize the different parameters.The experimental results prove that the combination optimization algorithm can achieve the convergence faster.6.Compared with most of handcrafted low-level visual features,the deep feature obtained by the deep leaning can represent image in an even better fashion.The huge successes obtained in the image classification and object recognition prove the superiority of deep feature once again.Under this background,we attempt to apply the deep convolutional neural network(CNN)on the SAR image retrieval.First,we introduce the basic CNN architecture in this work,and display the role of each component within the CNN.Second,we adopt 4 popular deep CNN to extract the feature from SAR images(the output of the last fully connection layer in general),and obtain the retrieval results in line with those deep features.Finally,we analyze the experimental results comprehensively,including the influence of different distance measures,different parameters,etc.