Research And Implementation Of Entity Alignment Technology Based On Multi-modal Knowledge Graph

With the continuous development of artificial intelligence technology,knowledge graph has become an indispensable component in the field of artificial intelligence because of its powerful knowledge representation and reasoning ability,which has attracted much attention from academia and industry.Knowledge graphs have been widely used in semantic search,question and answer,and knowledge management,but the research on multimodal knowledge graphs focuses more on the construction of entities and their multimodal semantic relationships under different modalities.However,the existing multimodal knowledge graphs can only obtain information from limited data sources,and their domain coverage is low.In order to improve the coverage of multimodal knowledge graphs,a feasible approach is to integrate useful knowledge from other multimodal knowledge graphs.In this process,it is crucial to identify equivalent entities in different knowledge graphs,because entities are the link that connects these different knowledge graphs.This process is also referred to as multimodal entity alignment.In the entity alignment task,due to the heterogeneity of different modalities,many entity alignment methods based on knowledge representation learning only use data from a single modality(e.g.,text)and ignore the entity feature information in other modalities(e.g.,images),resulting in the data in other modalities not being effectively utilized.In order to make full use of the information in different modalities,this paper takes multimodal entity alignment as the main research content to compensate the shortage of single-modal data.Specific work includes.(1)The multimodal information features are fused in the entity alignment task to increase the information of knowledge graph image entities,which makes up for the limitations of single modality and increases the complementarity between modalities.(2)In feature extraction,different modalities use different models for feature extraction.For the knowledge graph text modality uses graph convolutional neural network(GCN)to extract text entity features,and the image modality uses Image Net pre-trained residual network(Res Net-50)to extract image entity features.(3)For feature fusion,feature fusion is performed by hyperbolic graph convolutional neural network(HGCN),which maps the embeddings of image features into hyperbolic space and uses hyperbolic multimodal entity alignment(HMEA)to predict entity alignment results using aggregated embeddings in hyperbolic space.Experiments are conducted on three datasets of multimodal knowledge mapping FB15 K,DB15K,and YAGO15 K,and the experimental results demonstrate that multimodal over single-modal entity alignment results in 80% of seed entity pairs,HGCN-Align improves nearly 15% in Hits@1 and Hits@10,and adding features of image entities can greatly improve the accuracy of entity alignment and efficiency,which provides new ideas for fields such as multimodal link prediction or cross-modal search.