| With the rapid development of the Internet and the rise of various social media software,the access to information has begun to change in the direction of multi-sourcing.As a result,a huge amount of data information is generated in the network environment every moment,which is usually in the form of multi-modal data such as text,picture,video and sound.Multi-modal data usually have different degrees of complementarity in understanding the same thing,and by fusing and unifying the representation of different modal data,the demand for cross-retrieval between multi-modal data can be satisfied.Therefore,the research of retrieval methods across different modal data has important research value and broad application prospects.Cross-modal retrieval research mainly focuses on establishing semantic associations by mining semantic information among multi-modal data,so as to achieve more accurate cross-information retrieval among data.The current research on cross-modal retrieval mainly faces three problems: firstly,the underlying feature structure of different modal data is inconsistent,leading to a heterogeneous gap between the data;secondly,there is a significant difference between the underlying feature representation of information extracted by computers and human perception of high-level semantic information,making a significant semantic gap between the two;in addition,existing cross-modal retrieval techniques are usually not scalable to In addition,existing cross-modal retrieval techniques are usually not scalable to large-scale datasets,which greatly limits the size and diversity of the multi-modal data being retrieved.For the study of cross-modal retrieval methods,it is important to learn the homogeneous common representation from the heterogeneous features of large-scale multi-modal data,to narrow the heterogeneous gap between cross-modal data,and to overcome the semantic gap within different modalities while ensuring the semantic relevance of the homogeneous common representation to achieve an effective metric for cross-modal data.In this paper,we take text and image modalities as the research objects,and the main research works are as follows.(1)To address the problems of heterogeneous distribution of different modal data feature representations and incomplete and insufficient mining of potential semantic information,a cross-modal hash retrieval method based on semantic auto-encoder learning is proposed.The method makes use of the labels corresponding to different modal data features,deeply mines the rich semantic information in the labels,guides the correlation modeling between the underlying data structure within different modalities and the high-level semantics between modalities,uses the rich feature and semantic label information to narrow the heterogeneous gap between different modalities,finds the optimal hash code with similarity metric,and then uses linear self-encoder to learn the hash function,making the cross-modal retrieval tasks can be performed directly between heterogeneous data of different modalities that have large differences in structure and dimensional.(2)To address the problems of poor correlation and high storage overhead between the data structure within and between modalities,we propose a cross-modal hash retrieval method based on matrix decomposition and self-encoding learning,which is divided into two parts:matrix decomposition and self-encoding learning.In the first part,since the same thing can be described by completely different modalities,i.e.,the "heterogeneous gap" between modalities,in order to explore and establish the semantic association between different modalities,this algorithm uses matrix decomposition to learn the consistency representation of potential semantics among modalities and construct their potential factor models.In the second part,combined with matrix decomposition,the hash function is learned by linear auto-encoder,and the potential semantic representation matrix is used as a constraint to narrow the heterogeneous gap between modalities and fully explore the potential semantic association information between different modalities.The two algorithmic models proposed in this paper are implemented on three benchmark datasets,WIKI,MIRFLICKR,and NUS-WIDE,respectively,and the experimental results fully demonstrate that the algorithmic models designed in this paper have certain advantages compared with other comparative algorithmic models. |
