Deep Hashing For Multi-Label Medical Image Retrieval

With the development of medical imaging technology and the increasing convenience of image transmission and storage,medical images which can reflect the pathological conditions of patients have been grown rapidly.These images are an important basis for doctors to make clinical diagnosis and arrange surgical plans.Content-based image retrieval(CBIR)has attracted increasing attention in the field of computer-aided diagnosis,for which learning-based hashing approaches are popular among researchers in large-scale image retrieval because of the fast retrieval speed and low storage cost.How to optimize the hash function to better mine and exploit the multi-label information in medical images for modeling multilevel semantic similarity is a key issue in the research of image retrieval algorithms based on deep hashing.To solve this issue,the main work of this thesis is as follows:In this work,we propose a Supervised Hashing method with EnergyBased Modeling(SH-EBM)for large-scale multimorbidity image retrieval,where concurrence of multiple symptoms with subtle differences in visual feature makes the search problem quite challenging,even for sophisticated hashing models built upon modern deep architectures.In addition to similarity-preserving ranking loss,multi-label classification loss is often employed in existing supervised hashing to further improve the expressiveness of hash codes,by optimizing a normalized probabilistic objective with tractable likelihood(e.g.,multi-label cross-entropy).On the contrary,we present a multi-label EBM loss,which is more flexible to parameterize and can model a more expressive probability distribution over multimorbidity image data.We further develop a multi-label Noise Contrastive Estimation(ml-NCE)algorithm for discriminative training of the proposed hashing network.On two multimorbidity dataset constructed by the ChestX-ray14 and the CheXpert,our SH-EBM outperforms most supervised hashing methods by a significant margin,implying that the energy-based supervised hashing possesses better expressiveness for representation of multi-label medical images,facilitating multilevel similarity preservation in multimorbidity image retrieval.In addition,we build a medical image retrieval system on web with a concise interactive interface,which intuitively demonstrates the effectiveness and convenience of our method and can assist doctors to diagnose the disease of the query image.