Multimodal Retrieval Algorithm Based On Hash Learning

Owing to the rapid development of social networks and self-media platforms,crossmodal retrieval is gradually becoming a frontier and hot spot for academic research at home and abroad.Multimodal retrieval algorithm based on hash learning are becoming a popular research direction due to their low storage footprint and high computational efficiency.The key problem of cross-modal hashing is the "semantic gap",which means that the feature representation and high-level semantics of different modalities are inconsistent.To solve this problem,an efficient multimodal retrieval algorithm based on hash learning is designed in this paper to further improve the retrieval accuracy.Firstly,this paper proposes Joint-modal Semantic Alignment for Unsupervised CrossModal Hashing.The algorithm uses self-encoders for hash code generation and alignment,and combines distribution-based similarity decision and weighting method with deep hash code alignment,making full use of inter-modal information to learn more discriminative hash codes.It is experimentally demonstrated that the hash codes obtained by this algorithm can retain more similarity relationships in the original feature space for cross-modal data.Secondly,this paper studies Multi-path Unsupervised Cross-Modal Hashing.The algorithm uses an unsupervised graph-based approach to capture the underlying flow structure of different modalities and generative adversarial networks by multipath to learn the flow structure and use adversariality to improve the retrieval performance,solving the problem that most existing works focus on pairwise relationship modeling and ignore the intrinsic correlation of multiple modalities.The algorithm achieves the highest retrieval accuracy in cross-modal retrieval experiments on MIRFlickr and PKU XMedia datasets.Finally,this paper proposes Semi-Supervised Knowledge Distillation for Cross-Modal Hashing.The algorithm uses the knowledge distillation method to guide the supervised model with the output of the semi-supervised model to further improve the retrieval accuracy.At the same time,by improving the ternary sorting loss,the ability of the discriminant model to distinguish between real pairs and generated pairs is further enhanced,so that the hash function learned from the discriminant model can better reflect labeled samples semantic information and the data distribution of unlabeled samples labeled.Experiments show that the algorithm can play an important role in reducing the "semantic gap" problem,and the retrieval accuracy of the algorithm is higher than that of the original algorithm on both bimodal datasets and multimodal datasets.